Java Manipulate DigitalOcean Spaces using S3TransferManager

Library
Create an S3TransferManager object
Upload files with S3TransferManager
Download files with S3TransferManager
More examples with S3TransferManager

1. Library

<!-- https://mvnrepository.com/artifact/software.amazon.awssdk/s3 -->
<dependency>
	<groupId>software.amazon.awssdk</groupId>
	<artifactId>s3</artifactId>
	<version>2.21.10</version>
</dependency>

<!-- https://mvnrepository.com/artifact/software.amazon.awssdk/s3-transfer-manager -->
<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>s3-transfer-manager</artifactId>
    <version>2.21.10</version>
</dependency>

<!-- https://mvnrepository.com/artifact/software.amazon.awssdk.crt/aws-crt -->
<dependency>
    <groupId>software.amazon.awssdk.crt</groupId>
    <artifactId>aws-crt</artifactId>
    <version>0.28.0</version>
</dependency>

<!-- Log Library -->
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-api -->
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>2.0.9</version>
</dependency>

<!-- Log Library -->
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-simple -->
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-simple</artifactId>
    <version>2.0.9</version> 
</dependency>

2. Create an S3TransferManager object

Creating an S3TransferManager object to interact with DigitalOcean Spaces Bucket is a little different from creating an S3TransferManager object to interact with an Amazon S3 Bucket.

You need to check which data center your Spaces Bucket is located in, thereby knowing its "SLUG".

https://docs.digitalocean.com/products/platform/availability-matrix/

DigitalOcean Data centers

Slug
A human-readable string used as a unique identifier for each region.

Below are utility methods that create an S3TransferManager object that allows you to interact with the Spaces Bucket.

MyS3TransferManagerUtils.java

package org.o7planning.java_14253_do_spaces.utils;

import java.net.URI;

import org.o7planning.java_14253_do_spaces.MyAccessKey;

import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3AsyncClient;
import software.amazon.awssdk.transfer.s3.S3TransferManager;
import software.amazon.awssdk.transfer.s3.SizeConstant;

public class MyUtils {

	// 
	// @digitalOceanSlug: nyc1, nyc2, fra1,..
	// @see: https://docs.digitalocean.com/products/platform/availability-matrix/
	//
	public static S3TransferManager createS3TransferManager(String digitalOceanSlug) {
		final Region region = Region.of(digitalOceanSlug);
		final String endpointUrl = "https://" + digitalOceanSlug + ".digitaloceanspaces.com";
		//
		AwsCredentials credentials = AwsBasicCredentials.create( //
				MyAccessKey.DO_ACCESS_KEY_ID, //
				MyAccessKey.DO_SECRET_ACCESS_KEY //
		);

		StaticCredentialsProvider credentialsProvider = StaticCredentialsProvider.create(credentials);

		S3AsyncClient s3AsyncClient = S3AsyncClient.builder() //
				.credentialsProvider(credentialsProvider) //
				.region(region) //
				.endpointOverride(URI.create(endpointUrl)) // IMPORTANT for DigitalOcean
				.build();

		return S3TransferManager.builder().s3Client(s3AsyncClient).build();
	}

	// 
	// @digitalOceanSlug: nyc1, nyc2, fra1,..
	// @see: https://docs.digitalocean.com/products/platform/availability-matrix/
	//
	// This method is currently not working (Need to wait for CRT to be supported by DigitalOcean)
	// NOTE: 2023 November, CRT still is not supported by DigitalOcean. (*****)
	// CRT: AWS Common Runtime (CRT)
	// https://aws.amazon.com/blogs/developer/introducing-crt-based-s3-client-and-the-s3-transfer-manager-in-the-aws-sdk-for-java-2-x/
	public static S3TransferManager createCrtS3TransferManager(String digitalOceanSlug) {
		final Region region = Region.of(digitalOceanSlug);
		final String endpointUrl = "https://" + digitalOceanSlug + ".digitaloceanspaces.com";
		//
		AwsCredentials credentials = AwsBasicCredentials.create( //
				MyAccessKey.DO_ACCESS_KEY_ID, //
				MyAccessKey.DO_SECRET_ACCESS_KEY //
		);

		StaticCredentialsProvider credentialsProvider = StaticCredentialsProvider.create(credentials);

		S3AsyncClient s3AsyncClient = S3AsyncClient.crtBuilder() // (*****)
				.credentialsProvider(credentialsProvider) //
				.region(region) //
				.endpointOverride(URI.create(endpointUrl)) // IMPORTANT for DigitalOcean
				.targetThroughputInGbps(20.0) //
				.minimumPartSizeInBytes(10 * SizeConstant.MB) //
				.build();

		return S3TransferManager.builder().s3Client(s3AsyncClient).build();
	}
}

No ADS

Note: The MyUtils.createCrtS3TransferManager method does not work with DO-Spaces at the time I write this article (November 2023). It is a new way to create S3TransferManager objects based on the Amazon CRT (AWS Common Runtime) library, which makes uploading and downloading 70% faster. It may take time for it to be supported by DigitalOcean.
https://aws.amazon.com/blogs/developer/introducing-crt-based-s3-client-and-the-s3-transfer-manager-in-the-aws-sdk-for-java-2-x/

See also other ways to create AwsCredentialsProvider objects with "accessKeyId/secretAccessKey" set in more secure locations instead of hardcoding in Java code. Note: You also need a few minor changes to make sure it works with Spaces Bucket.
Java Awssdk Common Credentials Providers

3. Upload files with S3TransferManager

Simple example, upload a file to Spaces Bucket using S3TransferManager.

S3TransferManagerUploadExample1.java

package org.o7planning.java_14253_do_spaces.example;

import java.nio.file.Paths;
import java.util.concurrent.CancellationException;
import java.util.concurrent.CompletionException;

import org.o7planning.java_14253_do_spaces.utils.MyUtils;

import software.amazon.awssdk.transfer.s3.S3TransferManager;
import software.amazon.awssdk.transfer.s3.model.CompletedFileUpload;
import software.amazon.awssdk.transfer.s3.model.FileUpload;
import software.amazon.awssdk.transfer.s3.model.UploadFileRequest;
import software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener;

public class S3TransferManagerUploadExample1 {
	// DigitalOcean SLUG List:
	// https://docs.digitalocean.com/products/platform/availability-matrix/

	private static final String MY_DO_SLUG = "fra1";
	private static final String MY_DO_SPACES_NAME = "my-do-spaces";

	public static void uploadFile(S3TransferManager transferManager, String bucketName, //
			String key, String filePath) {
		// Create UploadFileRequest.
		UploadFileRequest uploadFileRequest = UploadFileRequest.builder()
				.putObjectRequest(b -> b.bucket(bucketName).key(key)) // bucketName & key
				.addTransferListener(LoggingTransferListener.create()) // Listener
				.source(Paths.get(filePath)) // filePath
				.build();

		FileUpload fileUpload = transferManager.uploadFile(uploadFileRequest);
		try {
			CompletedFileUpload completedFileUpload = fileUpload //
					.completionFuture() // CompletableFuture<CompletedDirectoryUpload>
					.join(); // Wait until completed, and return value.
			//
			System.out.println("SuccessfullyCompleted");
			System.out.println(" --> " + completedFileUpload.toString());
		} catch (CancellationException e) {
			System.out.println("CancellationException");
			System.out.println(" --> (It could be by the user calling the pause method.)");
		} catch (CompletionException e) {
			System.out.println("Completed with error: " + e);
		}
	}

	public static void main(String[] args) {
		// There are several ways to create an S3TransferManager
		// @See my article 14231.
		S3TransferManager transferManager = MyUtils.createS3TransferManager(MY_DO_SLUG);
		//
		uploadFile(transferManager, //
				MY_DO_SPACES_NAME, // DigitalOcean Spaces Name.
				"static/videos/sample-15s.mp4", // key
				"/Volumes/New/Test/download/sample-15s.mp4" // filePath to save.
		);
	}
}

Output:

[main] WARN software.amazon.awssdk.transfer.s3.S3TransferManager 
- The provided DefaultS3AsyncClient is not an instance of S3CrtAsyncClient, and thus multipart upload/download feature is not enabled and resumable file upload is not supported. To benefit from maximum throughput, consider using S3AsyncClient.crtBuilder().build() instead.
[main] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - Transfer initiated...
[main] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - |                    | 0.0%
[Thread-0] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - |====================| 100.0%
SuccessfullyCompleted
 --> CompletedFileUpload(response=PutObjectResponse(ETag="044d7d8ed37e78d6537c8ab5d68b2d78"))

4. Download files with S3TransferManager

For example, download files with S3TransferManager.

S3TransferManagerDownloadExample1.java

package org.o7planning.java_14253_do_spaces.example;

import java.nio.file.Paths;
import java.util.concurrent.CancellationException;
import java.util.concurrent.CompletionException;

import org.o7planning.java_14253_do_spaces.utils.MyUtils;

import software.amazon.awssdk.transfer.s3.S3TransferManager;
import software.amazon.awssdk.transfer.s3.model.CompletedFileDownload;
import software.amazon.awssdk.transfer.s3.model.DownloadFileRequest;
import software.amazon.awssdk.transfer.s3.model.FileDownload;
import software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener;

public class S3TransferManagerDownloadExample1 {
	// DigitalOcean SLUG List:
	// https://docs.digitalocean.com/products/platform/availability-matrix/

	private static final String MY_DO_SLUG = "fra1";
	private static final String MY_DO_SPACES_NAME = "my-do-spaces";
 
	
	public static void downloadFile(S3TransferManager transferManager, String bucketName, //
			String key, String filePath) {
		DownloadFileRequest downloadFileRequest = DownloadFileRequest.builder()
				.getObjectRequest(req -> req.bucket(bucketName).key(key)) // bucketName & key
				.destination(Paths.get(filePath))
				// Attaching a LoggingTransferListener that will log the progress
				.addTransferListener(LoggingTransferListener.create()).build();

		FileDownload fileDownload = transferManager.downloadFile(downloadFileRequest);
		try {
			// Wait for the transfer to complete
			CompletedFileDownload completedFileDownload = fileDownload //
					.completionFuture() // CompletableFuture<CompletedFileDownload>
					.join(); // Wait until completed, and return value.
			//
			System.out.println("SuccessfullyCompleted");
			System.out.println(" --> " + completedFileDownload.toString());
		} catch (CancellationException e) {
			System.out.println("CancellationException");
			System.out.println(" --> (It could be by the user calling the pause method.)");
		} catch (CompletionException e) {
			System.out.println("Completed with error: " + e);
		}
	}

	public static void main(String[] args) {
		// There are several ways to create an S3TransferManager
		// @See my article 14231.
		S3TransferManager transferManager = MyUtils.createS3TransferManager(MY_DO_SLUG); 
		//
		// Test download a video file 11.9MB
		//
		downloadFile(transferManager, //
				MY_DO_SPACES_NAME, // bucketName
				"static/videos/sample-15s.mp4", // key
				"/Volumes/New/Test/download/sample-15s.mp4" // filePath to save.
		);
	}
}

Output:

[main] WARN software.amazon.awssdk.transfer.s3.S3TransferManager 
- The provided DefaultS3AsyncClient is not an instance of S3CrtAsyncClient, and thus multipart upload/download feature is not enabled and resumable file upload is not supported. To benefit from maximum throughput, consider using S3AsyncClient.crtBuilder().build() instead.
[main] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - Transfer initiated...
[aws-java-sdk-NettyEventLoop-1-2] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - |                    | 0.0%
[aws-java-sdk-NettyEventLoop-1-2] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - |=                   | 5.0%
[aws-java-sdk-NettyEventLoop-1-2] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - |==                  | 10.0% 
..
[aws-java-sdk-NettyEventLoop-1-2] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - |==================  | 90.0%
[aws-java-sdk-NettyEventLoop-1-2] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - |=================== | 95.0%
[aws-java-sdk-NettyEventLoop-1-2] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - |====================| 100.0%
SuccessfullyCompleted
 --> CompletedFileDownload(response=GetObjectResponse(AcceptRanges=bytes, LastModified=2023-11-04T12:38:49Z, ContentLength=11916526, ETag="aa355e698dca652d60c0bd39931714a4", ContentType=video/mp4, Metadata={}))
[sdk-async-response-0-0] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - Transfer complete!

5. More examples with S3TransferManager

As mentioned DigitalOcean Spaces is compatible with the Awssdk S3 API so the Awssdk S3 articles below will give you lots of other examples. Such as:

List objects on the Spaces Bucket.
Upload a directory.
Copy and move objects between Spaces Buckets.
...

Java Awssdk S3 S3TransferManager download object
Java Awssdk S3 Upload object with S3TransferManager
Amazon Web Services Tutorials