**文件上传** ############################## 在NOS中用户的基本操作单元是对象,亦可以理解为文件,NOS C# SDK提供了丰富的上传接口,可以通过以下的方式上传文件: - 流式上传 - 单块上传 - 分块上传 流式上传、单块上传最大为100M;分块上传除最后一块外,分块不能小于16k,每一个分块不能大于100M,最多能上传10000个分块,即分块上传的文件最大支持1T。 流式上传 ============================= 您可以使用NosClient.PutObject上传一个一个Stream中的内容,具体实现如下:: // 初始化NosClient var nosClient = new NosClient(endpoint, accessKeyId, accessKeySecret); /// /// 流式上传 /// /// 桶名 /// 对象名 /// 需要上传的流 public void PutObject(string bucket, string key, Stream content) { try { nosClient.PutObject(bucket, key, content); Console.WriteLine("Put object:{0} succeeded", key); } catch (NosException ex) { Console.WriteLine("Failed with HTTPStatus: {0}; \nErrorCode: {1}; \nErrorMessage: {2}; \nRequestID:{3}; \nResource:{4}", ex.StatusCode, ex.ErrorCode, ex.Message, ex.RequestId, ex.Resource); } catch (Exception ex) { Console.WriteLine("Failed with error info: {0}", ex.Message); } } .. attention:: 1. 上传的流内容不超过100M 单块上传 ============================ 您可以使用NosClient.PutObject上传文件内容,具体实现如下:: // 初始化NosClient var nosClient = new NosClient(endpoint, accessKeyId, accessKeySecret); /// /// 上传文件 /// /// 桶名 /// 对象名 /// 上传的文件 public void PutObject(string bucket, string key, string fileToUpload) { try { nosClient.PutObject(bucket, key, fileToUpload); Console.WriteLine("Put object:{0} succeeded", key); } catch (NosException ex) { Console.WriteLine("Failed with HTTPStatus: {0}; \nErrorCode: {1}; \nErrorMessage: {2}; \nRequestID:{3}; \nResource:{4}", ex.StatusCode, ex.ErrorCode, ex.Message, ex.RequestId, ex.Resource); } catch (Exception ex) { Console.WriteLine("Failed with error info: {0}", ex.Message); } } .. attention:: 1. 上传的文件内容不超过100M 分块上传 =========================== 除了通过putObject接口上传文件到NOS之外,NOS还提供了另外一种上传模式-分块上传,用户可以在如下应用场景内(但不限于此),使用分块上传模式,如: - 需支持断点上传 - 上传超过100M的文件 - 网络条件较差,经常和NOS服务器断开连接 - 上传文件之前无法确定文件的大小 分块上传一般流程如下所示: - 初始化一个分块上传任务(InitiateMultipartUpload) - 上传分块(UploadPart) - 完成分块上传(CompleteMultipartUpload)或者取消分块上传(AbortMultipartUpload) 初始化分块上传 ------------------------ 您可以使用NosClient.InitiateMultipartUpload初始化分块上传,具体实现如下:: // 初始化NosClient var nosClient = new NosClient(endpoint, accessKeyId, accessKeySecret); /// /// 初始化分块上传 /// /// 桶名 /// 对象名 public string InitiateMultipartUpload(String bucket, String key) { string uploadId = ""; try { var request = new InitiateMultipartUploadRequest(bucket, key); var result = nosClient.InitiateMultipartUpload(request); uploadId = result.UploadId; Console.WriteLine("Init multipart upload succeeded"); } catch (NosException ex) { Console.WriteLine("Failed with HTTPStatus: {0}; \nErrorCode: {1}; \nErrorMessage: {2}; \nRequestID:{3}; \nResource:{4}", ex.StatusCode, ex.ErrorCode, ex.Message, ex.RequestId, ex.Resource); } catch (Exception ex) { Console.WriteLine("Failed with error info: {0}", ex.Message); } return uploadId; } 上传分块 ------------------------ 您可以使用NosClient.UploadPart上传分块,具体实现如下:: // 初始化NosClient var nosClient = new NosClient(endpoint, accessKeyId, accessKeySecret); /// /// 上传分块 /// /// 桶名 /// 对象名 /// 上传的文件 /// 分块上传的ID /// 分块大小 public List UploadParts(String bucket, String key, String fileToUpload, String uploadId, int partSize) { // 计算片个数 var fi = new FileInfo(fileToUpload); var fileSize = fi.Length; var partCount = fileSize / partSize; if (fileSize % partSize != 0) { partCount++; } // 开始分片上传 var partETags = new List(); try { using (var fs = File.Open(fileToUpload, FileMode.Open)) { for (var i = 0; i < partCount; i++) { var skipBytes = (long)partSize * i; //定位到本次上传片应该开始的位置 fs.Seek(skipBytes, 0); var size = (partSize < fileSize - skipBytes) ? partSize : (fileSize - skipBytes); var request = new UploadPartRequest(bucket, objectName, uploadId) { Content = fs, PartSize = size, PartNumber = i + 1 }; var result = nosClient.UploadPart(request); partETags.Add(result.PartETag); } } Console.WriteLine("Put multipart upload succeeded"); } catch (NosException ex) { Console.WriteLine("Failed with HTTPStatus: {0}; \nErrorCode: {1}; \nErrorMessage: {2}; \nRequestID:{3}; \nResource:{4}", ex.StatusCode, ex.ErrorCode, ex.Message, ex.RequestId, ex.Resource); } catch (Exception ex) { Console.WriteLine("Failed with error info: {0}", ex.Message); } return partETags; } 完成分块上传 ------------------------ 您可以使用NosClient.CompleteMultipartUpload完成上传分块,具体实现如下:: // 初始化NosClient var nosClient = new NosClient(endpoint, accessKeyId, accessKeySecret); /// /// 完成分块上传 /// /// 桶名 /// 对象名 /// 分块上传的ID /// PartETag 对象,它是上传块的ETag与块编号(PartNumber)的组合 public void CompleteUploadPart(String bucket, String key, String uploadId, List partETags) { try { var completeMultipartUploadRequest = new CompleteMultipartUploadRequest(bucket, key, uploadId); foreach (var partETag in partETags) { completeMultipartUploadRequest.PartETags.Add(partETag); } nosClient.CompleteMultipartUpload(completeMultipartUploadRequest); Console.WriteLine("Complete Upload Part succeeded"); } catch (NosException ex) { Console.WriteLine("Failed with HTTPStatus: {0}; \nErrorCode: {1}; \nErrorMessage: {2}; \nRequestID:{3}; \nResource:{4}", ex.StatusCode, ex.ErrorCode, ex.Message, ex.RequestId, ex.Resource); } catch (Exception ex) { Console.WriteLine("Failed with error info: {0}", ex.Message); } } 分块上传示例 ------------------------ 下面通过一个完整的示例说明了如何进行分片上传操作,可以参考下面代码:: // 初始化NosClient var nosClient = new NosClient(endpoint, accessKeyId, accessKeySecret); /// /// 分块上传示例 /// /// 桶名 /// 对象名 /// 上传的文件 public void UploadPart(string bucket, string key, string fileToUpload) { try { /* * 初始化一个分块上传事件 * */ var initRequest = new InitiateMultipartUploadRequest(bucket, key); var initResult = nosClient.InitiateMultipartUpload(initRequest); /* * 上传分块 * */ // 设置每块为 1M const int partSize = 1024 * 1024 * 100; var partFile = new FileInfo(fileToUpload); var fileSize = partFile.Length; // 计算分块数目 var partCount = fileSize / partSize; if (fileSize % partSize != 0) { partCount++; } // 新建一个List保存每个分块上传后的ETag和PartNumber var partETags = new List(); //upload the file using (var fs = new FileStream(partFile.FullName, FileMode.Open)) { for (var i = 0; i < partCount; i++) { // 跳到每个分块的开头 long skipBytes = partSize * i; fs.Position = skipBytes; // 计算每个分块的大小 var size = partSize < partFile.Length - skipBytes ? partSize : partFile.Length - skipBytes; // 创建UploadPartRequest,上传分块 var uploadPartRequest = new UploadPartRequest(bucket, key, initResult.UploadId) { Content = fs, PartSize = size, PartNumber = (i + 1) }; var uploadPartResult = nosClient.UploadPart(uploadPartRequest); // 将返回的PartETag保存到List中。 partETags.Add(uploadPartResult.PartETag); /* * 完成分块上传事件 * */ var completeRequest = new CompleteMultipartUploadRequest(bucket, key, initResult.UploadId); foreach (var partETag in partETags) { completeRequest.PartETags.Add(partETag); } nosClient.CompleteMultipartUpload(completeRequest); } fs.Close(); } Console.WriteLine("Upload Part succeeded"); } catch (NosException ex) { Console.WriteLine("Failed with HTTPStatus: {0}; \nErrorCode: {1}; \nErrorMessage: {2}; \nRequestID:{3}; \nResource:{4}", ex.StatusCode, ex.ErrorCode, ex.Message, ex.RequestId, ex.Resource); } catch (Exception ex) { Console.WriteLine("Failed with error info: {0}", ex.Message); } } .. attention:: 1. 上面程序一共分为三个步骤:1. initiate 2. uploadPart 3. complete 2. UploadPart 方法要求除最后一个Part以外,其他的Part大小都要大于或等于16K。但是Upload Part接口并不会立即校验上传Part的大小(因为不知道是否为最后一块);只有当Complete Multipart Upload的时候才会校验。 3. Part号码的范围是1~10000。如果超出这个范围,NOS将返回InvalidArgument的错误码。 4. 分块上传除最后一块外,分块不能小于16k,每一个分块不能大于100M。 5. 每次上传Part时都要把流定位到此次上传块开头所对应的位置。 6. 分片上传任务初始化或上传部分分片后,可以使用abortMultipartUpload接口中止分片上传事件。当分片上传事件被中止后,就不能再使用这个Upload ID做任何操作,已经上传的分片数据也会被删除。 7. 每次上传Part之后,NOS的返回结果会包含一个 PartETag 对象,它是上传块的ETag与块编号(PartNumber)的组合。在后续完成分片上传的步骤中会用到它,因此我们需要将其保存起来,然后在第三步complete的时候使用,具体操作参考上面代码。 取消分块上传 ------------------------ 您可以使用NosClient.AbortMultipartUpload取消上传事件,具体实现如下:: // 初始化NosClient var nosClient = new NosClient(endpoint, accessKeyId, accessKeySecret); /// /// 取消分块上传 /// /// 桶名 /// 对象名 /// 分块上传的ID public void AbortMultipartUpload(string bucket, string key, string uploadId) { try { var abortRequest = new AbortMultipartUploadRequest(bucket, key, uploadId); nosClient.AbortMultipartUpload(abortRequest); Console.WriteLine("Abort Multipart Upload succeeded"); } catch (NosException ex) { Console.WriteLine("Failed with HTTPStatus: {0}; \nErrorCode: {1}; \nErrorMessage: {2}; \nRequestID:{3}; \nResource:{4}", ex.StatusCode, ex.ErrorCode, ex.Message, ex.RequestId, ex.Resource); } catch (Exception ex) { Console.WriteLine("Failed with error info: {0}", ex.Message); } } 查看已经上传的分片 -------------------------- 查看已经上传的分片可以罗列出指定Upload ID(InitiateMultipartUpload时获取)所属的所有已经上传成功的分片,您可以通过NosClient.ListParts接口获取已经上传的分片,可以参考以下代码:: // 初始化NosClient var nosClient = new NosClient(endpoint, accessKeyId, accessKeySecret); // /// 列出已上传的分块 /// /// 桶名 /// 对象名 /// 分块上传的ID public void ListParts(string bucket, string key, string uploadId) { try { var listPartsRequest = new ListPartsRequest(bucket, key, uploadId){ MaxParts = 10, PartMumberMarker = '10' } var result = nosClient.ListParts(listPartsRequest); Console.WriteLine("List parts succeeded"); foreach (var part in result.Parts) { Console.WriteLine(part.ToString()); } } catch (NosException ex) { Console.WriteLine("Failed with HTTPStatus: {0}; \nErrorCode: {1}; \nErrorMessage: {2}; \nRequestID:{3}; \nResource:{4}", ex.StatusCode, ex.ErrorCode, ex.Message, ex.RequestId, ex.Resource); } catch (Exception ex) { Console.WriteLine("Failed with error info: {0}", ex.Message); } } 查看已经上传的分片可以指定以下参数: +---------------------+-----------------------------------------------------------------+ |参数 | 描述 | +=====================+=================================================================+ |MaxParts | 响应中的limit个数 类型:整型 | +---------------------+-----------------------------------------------------------------+ |PartMumberMarker |分块号的界限,只有更大的分块号会被列出来。 类型:字符串 | +---------------------+-----------------------------------------------------------------+ 查看当前正在进行的分片上传任务 --------------------------------------- 查看正在进行的分片上传任务可以罗列出正在进行,还未完成的分片上传任务,您可以通过NosClient.ListMultipartUploads接口当前的上传任务,可以参考以下代码:: // 初始化NosClient var nosClient = new NosClient(endpoint, accessKeyId, accessKeySecret); // /// 查看当前正在进行的分片上传任务 /// /// 桶名 public void ListMultipartUploads(string bucket) { try { var listMultipartUploadsRequest = new ListMultipartUploadsRequest(bucket){ KeyMarker = "", MaxUploads = 10 }; var result = nosClient.ListMultipartUploads(listMultipartUploadsRequest); Console.WriteLine("List multi Uploads succeeded"); foreach (var mu in result.MultipartUploads) { Console.WriteLine("Key:{0}, UploadId:{1}", mu.Key , mu.UploadId); } } catch (NosException ex) { Console.WriteLine("Failed with HTTPStatus: {0}; \nErrorCode: {1}; \nErrorMessage: {2}; \nRequestID:{3}; \nResource:{4}", ex.StatusCode, ex.ErrorCode, ex.Message, ex.RequestId, ex.Resource); } catch (Exception ex) { Console.WriteLine("Failed with error info: {0}", ex.Message); } } 上述代码中使用的options参数如下: +------------------------+-----------------------------------------------------------------+ |参数值 | 描述 | +========================+=================================================================+ |KeyMarker |指定某一uploads key,只有大于该KeyMarker的才会被列出 | +------------------------+-----------------------------------------------------------------+ |MaxUploads |最多返回MaxUploads条记录,默认1000 | +------------------------+-----------------------------------------------------------------+ 设置文件元信息 ============================ 文件元数据(object meta),是上传到NOS的文件属性描述信息,分为http标准属性和用户自定义元数据。文件元信息可以在各种上传方式(流式上传、单块上传、分块上传)时进行设置。 设定http header ----------------------------- NOS允许用户自定义http Header。http header相关信息请参考 RFC2616,几个常用的header说明如下: +--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------+ |名称 | 描述 | 默认值 | +================================+========================================================================================================================================================+===============================+ |Content-MD5 |文件数据校验,设置了该值后NOS会启用文件内容MD5校验,把您提供的MD5与文件的MD5比较,不一致会抛出错误 |无 | +--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------+ |Content-Type |文件的MIME,定义文件的类型及网页编码,决定浏览器将以什么形式、什么编码读取文件。如果用户没有指定则根据Key或文件名的扩展名生成,如果没有扩展名则填默认值 |application/octet-stream | +--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------+ |Content-Disposition |指示MINME用户代理如何显示附加的文件,打开或下载,及文件名称 |无 | +--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------+ |Content-Length |上传的文件的长度,超过流/文件的长度会截断,不足为实际值 |流/文件实际长度 | +--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------+ |Expires |缓存过期时间,NOS未使用,格式是格林威治时间(GMT) |无 | +--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------+ |Cache-Control |指定该Object被下载时的网页的缓存行为 |无 | +--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------+ 下面的源代码实现了上传文件时设置Http header:: // 初始化NosClient var nosClient = new NosClient(endpoint, accessKeyId, accessKeySecret); /// /// 上传文件 /// /// 桶名 /// 对象名 /// 上传的文件 public void putObject(string bucket, string key, string uploadFile) { try { using (var fs = File.Open(fileToUpload, FileMode.Open)) { var metadata = new ObjectMetadata { CacheControl = "no-cache", ContentDisposition = "abc.zip", ContentEncoding = "gzip", }; //user metadata metadata.UserMetadata.Add("x-nos-meta-my-metaMyKey1", "MyValue1"); metadata.UserMetadata.Add("x-nos-meta-my-metaMyKey2", "MyValue2"); nosClient.PutObject(bucket, key, fs, metadata); Console.WriteLine("Put object succeeded"); } } catch (NosException ex) { Console.WriteLine("Failed with HTTPStatus: {0}; \nErrorCode: {1}; \nErrorMessage: {2}; \nRequestID:{3}; \nResource:{4}", ex.StatusCode, ex.ErrorCode, ex.Message, ex.RequestId, ex.Resource); } catch (Exception ex) { Console.WriteLine("Failed with error info: {0}", ex.Message); } } .. attention:: 1. 各种上传方式(包括流式上传、单块上传、分块上传)都可以设置元数据信息。流式上传设置元数据的方式与单块上传相同,分块上传在构造ListMultipartUploadsRequest对象时传入ObjectMetadata参数,如:ListMultipartUploadsRequest.ObjectMetadata; 2. 文件的元信息可以通过NosClient.GetObjectMetadata获取; 3. user meta的名称以"x-nos-meta-"开头,例如设置为:’x-nos-meta-name’,读取名字为x-nos-meta-name的参数即可; 上传时校验MD5 ============================== 下面的代码实现了上传时校验MD5:: // 初始化NosClient var nosClient = new NosClient(endpoint, accessKeyId, accessKeySecret); /// /// 上传文件 /// /// 桶名 /// 对象名 /// 上传的文件 public void putObject(string bucket, string key, string uploadFile) { try { string md5 = ""; using (var fs = File.Open(fileToUpload, FileMode.Open)) { md5 = NosUtil.GetMD5FromStream(fs); var metadata = new ObjectMetadata { ContentMD5 = md5 }; nosClient.PutObject(bucket, key, fs, metadata); Console.WriteLine("Put object succeeded"); } } catch (NosException ex) { Console.WriteLine("Failed with HTTPStatus: {0}; \nErrorCode: {1}; \nErrorMessage: {2}; \nRequestID:{3}; \nResource:{4}", ex.StatusCode, ex.ErrorCode, ex.Message, ex.RequestId, ex.Resource); } catch (Exception ex) { Console.WriteLine("Failed with error info: {0}", ex.Message); } } .. attention:: 1. 校验MD5会有一定的性能损失;