Quickstart: Upload an Object Using the AWS SDK for Java

Tuesday, September 13, 2016

Quickstart: Upload an Object Using the AWS SDK for Java

Goal:

Sample code for "Upload an Object Using the AWS SDK for Java"

Env:

aws-java-sdk-1.7.4.jar
CentOS 6

Solution:

Say I want to upload one file "/root/hao/abc.txt" from CentOS node to S3 location "s3://haos3/test/a.txt". ("hao3" is the bucket name).

1. Create AWS Credentials File : ~/.aws/config

The current user is root, so the location for this file is /root/.aws/config.
The content is Access Key and Secret Access Key.
For example:

# cat /root/.aws/config
[default]
aws_access_key_id=AxxxxxA
aws_secret_access_key=axxxxxxxxxxxxxxxxxxxxa

2. Create sample java code

import java.io.File;
import java.io.IOException;

import com.amazonaws.AmazonClientException;
import com.amazonaws.AmazonServiceException;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.PutObjectRequest;
import com.amazonaws.services.s3.model.ObjectMetadata;

public class UploadObjectSingleOperation {
 private static String bucketName     = "haos3";
 private static String keyName        = "test/a.txt";
 private static String uploadFileName = "/root/hao/abc.txt";
 
 public static void main(String[] args) throws IOException {
        AmazonS3 s3client = new AmazonS3Client(new ProfileCredentialsProvider());
        try {
            System.out.println("Uploading a new object to S3 from a file\n");
            File file = new File(uploadFileName);
            PutObjectRequest putRequest = new PutObjectRequest(bucketName, keyName, file);

            // Request server-side encryption.
            // ObjectMetadata objectMetadata = new ObjectMetadata();
            // objectMetadata.setSSEAlgorithm(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION);     
            // putRequest.setMetadata(objectMetadata);

            s3client.putObject(putRequest);

         } catch (AmazonServiceException ase) {
            System.out.println("Caught an AmazonServiceException, which " +
              "means your request made it " +
                    "to Amazon S3, but was rejected with an error response" +
                    " for some reason.");
            System.out.println("Error Message:    " + ase.getMessage());
            System.out.println("HTTP Status Code: " + ase.getStatusCode());
            System.out.println("AWS Error Code:   " + ase.getErrorCode());
            System.out.println("Error Type:       " + ase.getErrorType());
            System.out.println("Request ID:       " + ase.getRequestId());
        } catch (AmazonClientException ace) {
            System.out.println("Caught an AmazonClientException, which " +
              "means the client encountered " +
                    "an internal error while trying to " +
                    "communicate with S3, " +
                    "such as not being able to access the network.");
            System.out.println("Error Message: " + ace.getMessage());
        }
    }
}

3. Compile it with needed libs

export CLASSPATH=/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/tools/lib/aws-java-sdk-1.7.4.jar:.:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/*
javac UploadObjectSingleOperation.java

4. Test

# java UploadObjectSingleOperation
log4j:WARN No appenders could be found for logger (com.amazonaws.internal.config.InternalConfig).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Uploading a new object to S3 from a file