问题描述
我正在尝试向 Azure Blob 存储发送 HTTP PUT 请求,但不允许使用 ChunkedStreamingMode。我正在从未知大小的 InputStream 中读取数据。我可以将 PUT Blob 请求分成多个 PUT Block 请求(Azure Blob Storage 提供了一个 PUT BLOCK 操作来存储单个块,最后,我可以将所有块构建到一个 Blob)。在内存中缓冲 1 MiB 并像块一样发送它是一个很好的解决方案吗?还是从输入流中读取并保存到本地文件系统中的一个临时文件中,然后读取该文件并将其作为Block发送是更好的解决方案?
解决方法
根据我的理解,您想分块上传一个大文件。我认为您的两个解决方案都可以工作,我将为您的第二个解决方案提供一些示例代码:将输入流另存为临时文件并按块上传,只需尝试 Azure Blob SDK 下面的代码:
import java.time.Duration;
import com.azure.storage.blob.BlobClient;
import com.azure.storage.blob.BlobServiceClientBuilder;
import com.azure.storage.blob.ProgressReceiver;
import com.azure.storage.blob.models.AccessTier;
import com.azure.storage.blob.models.BlobHttpHeaders;
import com.azure.storage.blob.models.BlobRequestConditions;
import com.azure.storage.blob.models.ParallelTransferOptions;
public class StorageTest {
public static void main(String[] args) {
//skip the process about saving a temp file,just leaving its path
String tempFilePath = "";
String connString = "<azure storage connection string>";
String containerName = "<container name>";
String destBlobName = "<blob name with path>";
BlobClient blobClient = new BlobServiceClientBuilder().connectionString(connString).buildClient()
.getBlobContainerClient(containerName).getBlobClient(destBlobName);
// 1MB per request in case of consuming too much jvm memory while uploading
long blockSize = 1024 * 1024;
ParallelTransferOptions parallelTransferOptions = new ParallelTransferOptions().setBlockSizeLong(blockSize)
// 2 Concurrency requests as max,you can set more than it to accelerate uploading
.setMaxConcurrency(2)
.setProgressReceiver(new ProgressReceiver() {
@Override
public void reportProgress(long bytesTransferred) {
System.out.println("uploaded:" + bytesTransferred);
}
});
BlobHttpHeaders headers = new BlobHttpHeaders().setContentLanguage("en-US").setContentType("binary");
blobClient.uploadFromFile(tempFilePath,parallelTransferOptions,headers,null,AccessTier.HOT,new BlobRequestConditions(),Duration.ofMinutes(30));
}
}
我已经进行了测试,我可以上传 5GB 的文件。 如果您还有其他问题,请告诉我。