Java Awssdk S3 S3Client Upload object
In Java Awssdk S3, there are two ways for you to upload files to S3 Bucket: using S3Client or S3TransferManager.
S3Client
S3Client provides a basic way to transfer objects to and from Amazon S3 with high performance and reliability, using the "Multipart Upload API" and "byte-range fetches" of Amazon AWS.
S3TransferManager
The S3 Transfer Manager is a high-level transfer utility built on top of the S3Client. It provides a simple API to allow you to transfer files and directories between your application and Amazon S3.
The S3TransferManager also enables you to monitor a transfer’s progress in real-time, as well as pause the transfer for execution at a later time.
No ADS
In this article, I will guide you using S3Client to upload files to S3 Bucket and other related examples.
First we create the MyUtils class with a utility method to create the S3Client object, which is used in the examples of this article.
MyUtils.java
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
public class MyUtils {
public static S3Client createS3Client(Region region) {
AwsCredentials credentials = AwsBasicCredentials.create("accessKeyId", "secretAccessKey");
AwsCredentialsProvider credentialsProvider = StaticCredentialsProvider.create(credentials);
S3Client client = S3Client.builder() //
.credentialsProvider(credentialsProvider) //
.region(region) //
.build();
return client;
}
}
To create an S3Client object you need to create an AwsCredentialsProvider object, this object provides credentials that allow you to interact with AWS. See the article below to create an AwsCredentialsProvider suitable for your purposes.
1. Library
<!-- https://mvnrepository.com/artifact/software.amazon.awssdk/s3 -->
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
<version>2.21.10</version>
</dependency>
<!-- https://mvnrepository.com/artifact/software.amazon.awssdk/s3-transfer-manager -->
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3-transfer-manager</artifactId>
<version>2.21.10</version>
</dependency>
<!-- Log Library -->
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-api -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>2.0.9</version>
</dependency>
<!-- Log Library -->
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-simple -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>2.0.9</version>
</dependency>
2. Upload file to S3Bucket with S3Client
No ADS
A simple example using S3Client to upload a file to S3 Bucket.
S3ClientUploadExample1.java
package org.o7planning.java_14201_awssdk_s3;
import java.io.File;
import org.o7planning.awssdks3.utils.MyUtils;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
public class S3ClientUploadExample1 {
// EU (Frankfurt) - Germany.
private static final Region myRegion = Region.EU_CENTRAL_1;
private static final String bucketName = "test1.o7planning.org";
public static void main(String[] args) {
S3Client s3Client = MyUtils.createS3Client(myRegion);
PutObjectRequest putObject = PutObjectRequest.builder() //
.bucket(bucketName) //
.key("statics/images/sample.png") //
.build();
File localFile = new File("/Volumes/New/Test/pngFile.png");
// This example uses RequestBody.fromFile
// to avoid loading the whole file into memory.
s3Client.putObject(putObject, RequestBody.fromFile(localFile));
System.out.println("Done!");
}
}
In case your Bucket is ACL enabled, you can manage access to each individual object. For example, set public read permission for an object:
PutObjectRequest putObject = PutObjectRequest.builder() //
.bucket(bucketName) //
.key("statics/images/sample.png") //
.acl(ObjectCannedACL.PUBLIC_READ) // (**)
.build();
See more about ACL to see how it works.
- AWS S3 ACL
3. Upload Stream, String, byte[]
You can also upload String, byte[] or a Stream of predetermined length to S3 Bucket. Take a look at the static "Factory" methods of the RequestBody class below:
RequestBody static methods
public static RequestBody fromInputStream(InputStream inputStream, long contentLength)
public static RequestBody fromString(String contents, Charset cs)
public static RequestBody fromString(String contents)
public static RequestBody fromBytes(byte[] bytes)
public static RequestBody fromByteBuffer(ByteBuffer byteBuffer)
public static RequestBody fromRemainingByteBuffer(ByteBuffer byteBuffer)
public static RequestBody empty()
public static RequestBody fromContentProvider(ContentStreamProvider provider, long contentLength, String mimeType)
public static RequestBody fromContentProvider(ContentStreamProvider provider, String mimeType)
No ADS
For example, upload a String into an object on the S3 Bucket:
PutObjectRequest putObject = PutObjectRequest.builder() //
.bucket(bucketName) //
.key("readme.txt") //
.build();
RequestBody dataToUpload = RequestBody.fromString("Some Content...");
s3Client.putObject(putObject, dataToUpload);
4. Object Metadata
When uploading an object to S3 Bucket with S3Client you can set Metadata for this object.
By convention, user-defined Metadata must have the prefix "x-amz-meta-", for example:
- x-amz-meta-myval-1
- x-amz-meta-myval-2
No ADS
S3ClientUploadMetadataExample1.java
package org.o7planning.java_14201_awssdk_s3;
import java.io.File;
import java.util.HashMap;
import java.util.Map;
import org.o7planning.awssdks3.utils.MyUtils;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.HeadObjectRequest;
import software.amazon.awssdk.services.s3.model.HeadObjectResponse;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
public class S3ClientUploadMetadataExample1 {
// EU (Frankfurt) - Germany.
private static final Region myRegion = Region.EU_CENTRAL_1;
private static final String bucketName = "test1.o7planning.org";
public static void main(String[] args) {
S3Client s3Client = MyUtils.createS3Client(myRegion);
Map<String, String> userMetaData = new HashMap<>();
userMetaData.put("x-amz-meta-myVal", "A Test Value");
PutObjectRequest putObject = PutObjectRequest.builder() //
.bucket(bucketName) //
.key("statics/images/sample2.png") //
.metadata(userMetaData) //
.build();
File localFile = new File("/Volumes/New/Test/pngFile.png");
// This example uses RequestBody.fromFile
// to avoid loading the whole file into memory.
s3Client.putObject(putObject, RequestBody.fromFile(localFile));
System.out.println("Upload Done!\n");
showMetadatas(s3Client);
}
private static void showMetadatas(S3Client s3Client) {
HeadObjectRequest headObjectRequest = HeadObjectRequest.builder() //
.bucket(bucketName).key("statics/images/sample2.png")//
.build();
HeadObjectResponse headObjectResponse = s3Client.headObject(headObjectRequest);
System.out.println("Content Type: " + headObjectResponse.contentType());
Map<String, String> userDefineMetadatas = headObjectResponse.metadata();
userDefineMetadatas.forEach((k, v) -> {
System.out.println(k + " : " + v);
});
}
}
Output:
Upload Done!
Content Type: image/png
myval : A Test Value
Results seen on S3 Bucket:
5. Object Tags
"TAGS" are additional information attached to an object, each TAG is a pair of values and each object is allowed to have a maximum of 10 such value pairs.
- What are object tags in Amazon S3?
No ADS
S3ClientUploadTagsExample.java
package org.o7planning.java_14201_awssdk_s3;
import java.io.File;
import java.util.ArrayList;
import java.util.List;
import org.o7planning.awssdks3.utils.MyUtils;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.GetObjectTaggingRequest;
import software.amazon.awssdk.services.s3.model.GetObjectTaggingResponse;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.model.Tag;
import software.amazon.awssdk.services.s3.model.Tagging;
public class S3ClientUploadTagsExample {
private static final Region myRegion = Region.EU_CENTRAL_1;
private static final String bucketName = "test1.o7planning.org";
public static void main(String[] args) {
S3Client s3Client = MyUtils.createS3Client(myRegion);
Tag authorTag = Tag.builder() //
.key("author").value("Smith").build();
Tag languageTag = Tag.builder() //
.key("language").value("English").build();
List<Tag> tags = new ArrayList<>();
tags.add(authorTag);
tags.add(languageTag);
Tagging tagging = Tagging.builder().tagSet(tags).build();
PutObjectRequest putObject = PutObjectRequest.builder() //
.bucket(bucketName) //
.key("java/java-tutorial.pdf") //
.tagging(tagging) //
.build();
File localFile = new File("/Volumes/New/Test/Java-Tutorial.pdf");
s3Client.putObject(putObject, RequestBody.fromFile(localFile));
System.out.println("Upload Done!\n");
showTags(s3Client);
}
private static void showTags(S3Client s3Client) {
GetObjectTaggingRequest request = GetObjectTaggingRequest.builder() //
.bucket(bucketName) //
.key("java/java-tutorial.pdf") //
.build();
GetObjectTaggingResponse response = s3Client.getObjectTagging(request);
List<Tag> tags = response.tagSet();
tags.forEach(tag -> {
System.out.println(tag.key() + " : " + tag.value());
});
}
}
Output:
Upload Done!
author : Smith
language : English
Results seen on S3 Bucket:
6. Redirect URL
If the Static Website feature is enabled in your Bucket, objects can be accessed via a URL. You can also set up an object's URL to redirect to another URL.
- URL: https://yourdomain.com/article/java-tutorial-old
- ---> https://yourdomain.com/article/java-tutorial-new
No ADS
S3ClientUploadRedirectExample1.java
package org.o7planning.java_14201_awssdk_s3;
import org.o7planning.awssdks3.utils.MyUtils;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
public class S3ClientUploadRedirectExample1 {
// EU (Frankfurt) - Germany.
private static final Region myRegion = Region.EU_CENTRAL_1;
private static final String bucketName = "test1.o7planning.org";
public static void main(String[] args) {
S3Client s3Client = MyUtils.createS3Client(myRegion);
PutObjectRequest putObject = PutObjectRequest.builder() //
.bucket(bucketName) //
.key("article/java-tutorial-old") //
.websiteRedirectLocation("/article/java-tutorial-new") //
.build();
// Upload Empty Data
s3Client.putObject(putObject, RequestBody.empty());
System.out.println("Done!");
}
}
After running the above example, you can see the results on the S3 Bucket:
- Object Key: article/java-tutorial-old
- Metadata Key: x-amz-website-redirect-location
- Metadata Value: /article/java-tutorial-new
The object URL can also redirect to an external URL (Of another website).
PutObjectRequest putObject = PutObjectRequest.builder() //
.bucket(bucketName) //
.key("go/digitalocean/promotion") //
.websiteRedirectLocation("https://m.do.co/c/9fdc6836d334") //
.build();
7. S3Waiter Example
If you are uploading an object on Thread-1, and Thread-2 is waiting for this object to appear on the Bucket. Let's use S3Waiter in Thread-2.
No ADS
S3WaiterExample1.java
package org.o7planning.java_14201_awssdk_s3;
import java.io.File;
import org.o7planning.awssdks3.utils.MyUtils;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.core.waiters.WaiterResponse;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.HeadObjectRequest;
import software.amazon.awssdk.services.s3.model.HeadObjectResponse;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.waiters.S3Waiter;
public class S3WaiterExample1 {
// EU (Frankfurt) - Germany.
private static final Region myRegion = Region.EU_CENTRAL_1;
private static final String bucketName = "test1.o7planning.org";
private static String objectKey = "statics/videos/sample-15s.mp4";
public static void main(String[] args) {
MyUploadThread uploadThread = new MyUploadThread();
uploadThread.start();
SubscriberThread subscriberThread = new SubscriberThread();
subscriberThread.start();
}
static class MyUploadThread extends Thread {
@Override
public void run() {
S3Client s3Client = MyUtils.createS3Client(myRegion);
PutObjectRequest putObject = PutObjectRequest.builder() //
.bucket(bucketName) //
.key(objectKey) //
.build();
System.out.println("Upload a Video 11.7MB (MyUploadThread)");
// A Video 11.7MB
File localFile = new File("/Volumes/New/Test/sample-15s.mp4");
// This example uses RequestBody.fromFile
// to avoid loading the whole file into memory.
s3Client.putObject(putObject, RequestBody.fromFile(localFile));
System.out.println("Upload Done! (MyUploadThread)");
}
}
static class SubscriberThread extends Thread {
@Override
public void run() {
S3Client s3Client = MyUtils.createS3Client(myRegion);
System.out.println("Wait until video exists! (SubscriberThread)");
S3Waiter waiter = s3Client.waiter();
HeadObjectRequest requestWait = HeadObjectRequest.builder() //
.bucket(bucketName) //
.key(objectKey) //
.build();
WaiterResponse<HeadObjectResponse> waiterResponse = waiter.waitUntilObjectExists(requestWait);
waiterResponse.matched().response().ifPresent(System.out::println);
}
}
}
Output:
Wait until video exists! (SubscriberThread)
Upload a Video 11.7MB (MyUploadThread)
Upload Done! (MyUploadThread)
HeadObjectResponse(AcceptRanges=bytes, LastModified=2023-11-11T08:08:16Z, ContentLength=11916526, ETag="aa355e698dca652d60c0bd39931714a4", ContentType=video/mp4, ServerSideEncryption=AES256, Metadata={})
- Java Awssdk S3 S3Waiter
8. Object Retention
"Object Retention" is an object that is temporarily protected for a set period of time. During the time it is protected you cannot modify or delete it. In other words, "Object Retention" is a "Write-Once-Read-Many (WORM)" object.
S3Client allows uploading objects with "Retention" feature, provided S3 Bucket has "Object Lock" mode enabled. If you are interested, you can see more details in the article below:
- Java Awssdk S3 Object Retention
No ADS
Amazon Web Services Tutorials
- Introduction to Amazon Web Services (AWS)
- Introduction to Amazon S3
- Introduction to Amazon Cloudfront and its architecture
- How to reduce Amazon Cloudfront Costs?
- Amazon CloudFront Invalidation
- Introduction to DigitalOcean Spaces
- Create DigitalOcean Spaces Bucket
- Java Awssdk S3 S3Client Upload object
- Create AWS accessKeyId/secretAccessKey
- Java Awssdk S3 List objects
- Host a static website on Amazon S3
- Java Awssdk CloudFront Invalidation
- DigitalOcean Spaces Create Access Key
- Java Awssdk Common Credentials Providers
- Java Awssdk ProfileCredentialsProvider
- Java Awssdk Creating and using EnvironmentVariableCredentialsProvider
- Java Awssdk Creating and using SystemPropertyCredentialsProvider
- Java Awssdk S3 Upload object with S3TransferManager
- Java Awssdk S3 S3TransferManager download object
- Java Manipulate DigitalOcean Spaces using S3TransferManager
- Java Create, list and delete S3 Bucket
- Introduction to Amazon ACM
- Aws Console create IAM User
- Create Amazon S3 Bucket
- Configure custom domain for Amazon S3 static website
- Create a CloudFront distribution for S3 Bucket
- Configure Amazon CloudFront Error Pages
- Amazon S3 Bucket policies
- Amazon AWS Policy Generator - policygen
- Migrate DNS service to Amazon Route 53
- Transfer domain registration to Amazon Route 53
- Request an SSL certificate from Amazon ACM
Show More