多端场景文件上传到对象存储的最佳实践

收藏
我的收藏

介绍​

业务场景中涉及到上传文件的场景,直接上传到服务端会受到带宽限制,大量用户并发上传会导致服务稳定性降低。​
抖音端我们提供了SDK Cloud.uploadFile 可直接上传到对象存储,对于业务不仅在抖音运营,还有 Web、APP 的场景,最佳实践是客户端通过预签名URL直接上传文件到对象存储。​
预签名URL中包含AK、有效期、资源、操作、签名等信息,任何使用该URL的人在有效期内都可以执行该URL对应的操作。这种方法称作预签名。​
客户端首先从服务端获取需要上传的对象的预签名URL,然后客户端通过Http请求,将对象传至对象存储桶。​

操作指引​

获取对象存储 SDK访问密钥​

在「对象存储-配置-密钥」生成对象存储SDK的访问密钥,标题栏为对象存储桶ID​

预签名URL上传文件到对象存储​

Golang代码示例​

支持预签名的sdk详细使用说明文档如下​
package main import ( "encoding/json" "fmt" "github.com/volcengine/ve-tos-golang-sdk/v2/tos" "github.com/volcengine/ve-tos-golang-sdk/v2/tos/enum" "log" "net/http" ) func main() { http.HandleFunc("/get_pre_sign_url", GetPreSignUrlHandler) listenPort := ":8000" if listenPort == "" { log.Fatal("failed to load _FAAS_RUNTIME_PORT") } fmt.Println("http ListenAndServe ", listenPort) log.Fatal(http.ListenAndServe(listenPort, nil)) } var ( //填写从抖音云--对象存储--配置获取的AK和SK accessKey = "xxxxxxxxxx" secretKey = "xxxxxx" // 如果部署在抖音云服务中,建议替换成内网域名 endpoint = "tos-cn-beijing.volces.com" region = "cn-beijing" // 填写从抖音云--对象存储获取的桶的ID bucketName = "xxxxxxxxxx" httpClient = &http.Client{} ) func GetPreSignUrlHandler(w http.ResponseWriter, r *http.Request) { // 初始化对象存储 client client, err := tos.NewClientV2(endpoint, tos.WithRegion(region), tos.WithCredentials(tos.NewStaticCredentials(accessKey, secretKey))) if err != nil { fmt.Fprint(w, "client init error") return } objectName := r.URL.Query().Get("object_name") // 调用Tos SDK 生成上传对象预签名 url, err := client.PreSignedURL(&tos.PreSignedURLInput{ HTTPMethod: enum.HttpMethodPut, Bucket: bucketName, Key: objectName, }) if err != nil { fmt.Fprint(w, "get pre sign url error") return } data := make(map[string]string) data[objectName] = url.SignedUrl msg, err := json.Marshal(data) if err != nil { fmt.Fprint(w, "json marshal error") return } w.Header().Set("content-type", "application/json") w.Write(msg) }

利用预签名 URL 在 Web 端进行上传示例​

<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Document</title> </head> <body> <input id="uploadFile" type="file"> <button onclick="upload()">上传</button> <script> function upload() { const uploadFile = document.querySelector("#uploadFile"); const files = uploadFile.files; let xhr = new XMLHttpRequest(); const fileReader = new FileReader(); fileReader.readAsArrayBuffer(files[0]); fileReader.onloadend = function() { // 转换为二进制流 const binaryData = fileReader.result; // 从服务端获取预签名URL, 本文档中获取预签名URL的接口名称为/get_pre_sign_url xhr.open("PUT", "XXX"); xhr.onload = function(response) { console.log('reponse', response); } xhr.send(binaryData); }; } </script> </body> </html>

UGC 内容审核后文件权限从私有读转为公开读​

文件上传到对象存储后,默认为私有读。UGC 内容建议审核后再修改文件权限为公开读,否则存在内容违规风险。​
服务端Golang代码示例:修改文件权限为公开读​
package main import ( "context" "fmt" "github.com/volcengine/ve-tos-golang-sdk/v2/tos" "github.com/volcengine/ve-tos-golang-sdk/v2/tos/enum" "log" "net/http" ) func main() { http.HandleFunc("/put_object_public_read", PutObjectPublicReadHandler) listenPort := ":8000" if listenPort == "" { log.Fatal("failed to load _FAAS_RUNTIME_PORT") } fmt.Println("http ListenAndServe ", listenPort) log.Fatal(http.ListenAndServe(listenPort, nil)) } var ( //填写从抖音云--对象存储--配置获取的AK和SK accessKey = "xxxxxxxxxx" secretKey = "xxxxxx" // 如果部署在抖音云服务中,建议替换成内网域名 endpoint = "tos-cn-beijing.volces.com" region = "cn-beijing" // 填写从抖音云--对象存储获取的桶的ID bucketName = "xxxxxxxxxx" httpClient = &http.Client{} ) func PutObjectPublicReadHandler(w http.ResponseWriter, r *http.Request) { //初始化请求对象存储的client client, err := tos.NewClientV2(endpoint, tos.WithRegion(region), tos.WithCredentials(tos.NewStaticCredentials(accessKey, secretKey))) if err != nil { fmt.Fprint(w, "client init error") return } objectName := r.URL.Query().Get("object_name") // 调用Tos SDK将对象访问权限变更公开读 acl := enum.ACLPublicRead _, err = client.PutObjectACL(context.TODO(), &tos.PutObjectACLInput{Bucket: bucketName, Key: objectName, ACL: acl}) if err != nil { fmt.Fprint(w, "PutObjectACL error") return } w.Header().Set("content-type", "application/json") w.Write([]byte("success")) }