AWS S3 v2

This library is a built-in class that provides functions to communicate with Amazon Simple Storage Service (Amazon S3).

Library usage requirements
Type	Integrations
Name	AWS_S3_V2
Version	2020-11-04
Code size used	8.2KB
Resources used	HTTPS x 1, Timers x 1

Abstracts

Methods()/Properties	Summary
`new AWS_S3()`	Creates an `AWS_S3` instance.

{AWS_S3} Instance

Methods()/Properties	Summary
`.createXAmzHeaders()`	Converts any AWS API styled `params` into `x-amz-*` headers. Ignores `bucket` and `key`.
`.createHttpHeader()`	Creates the authentication headers for the given arguments according to the AWS Signature v4 documentation.
`.put()`	Sends a PUT request, outputting the result to the callback function.
`.get()`	Sends a GET request, outputting the result to the passed callback.
`.abortRequest()`	Aborts the currently ongoing request.
`.errors`	An array containing the invalid configuration key names. Exists only when an invalid configuration is passed to the constructor.

Details

`new AWS_S3(config)`

Initializes the AWS_S3 object with the passed configuration.

Configuration

When instantiating the AWS_S3 object, the following configurations are required to be set by the user:

Name	Type	Default	Summary
`region`	string	MANDATORY	Region to be used when accessing a bucket
`accessKeyId`	string	MANDATORY	Access key id for authorization from AWS IAM
`secretKey`	string	MANDATORY	Secret key for authorization from AWS IAM
`bucket`	string	MANDATORY	Name of the S3 Bucket to operate on
`ca`	string	MANDATORY	The root CA to communicate with AWS
`timeout`	number	90000	The timeout of the `AWS_S3` Instance in ms ver. 2020-07-20+

The configuration must be a JavaScript object, as the following:

var config = {
    region: "<String>",
    accessKeyId: "<String>",
    secretKey: "<String>",
    bucket: "<String>",
    ca: "<String>"
};

Once the configuration is set, it can be passed to AWS_S3 during instance creation. If any invalid values are passed, the result from setConfig is stored in an errors property, and should be handled appropriately.

var s3 = new AWS_S3(config);
if ('errors' in s3) {
    // TODO: handle errors
}

Setter Methods

After creating the AWS_S3 object, each of the aforementioned configurations may be changed by its corresponding setter, with a passed in value of String type. The values for accessKeyId and secretKey may be retrieved from AWS IAM.

Setter	Summary
`.setRegion(value: string)`	Set the region for this `AWS_S3` instance.
`.setBucket(value: string)`	Set the bucket for this `AWS_S3` instance.
`.setAccessKeyId(value: string)`	Set the Access Key ID for this `AWS_S3` instance, used for authentication.
`.setSecretKey(value: string)`	Set the Secret Key for this `AWS_S3` instance, used for authentication.
`.setRootCA(value: string)`	Set the root CA for this `AWS_S3` instance, used for connecting to AWS. The suggested value can be seen below.
`.setRequestTimeout(value: Number)`	Set the timeout for this `AWS_S3` instance. ver. 2020-07-20+	`

The configuration setters can also be chained, if they were all successful. Each setter returns an Error when given an invalid value, instead of an AWS_S3 instance for method chaining.

`.setConfig(config)`

The setConfig function can be used to set or change multiple config options at the same time, and takes in a typical JavaScript object of key:value pairs as an argument.

s3.setConfig({
    region: "value",
    accessKeyId: "value",
    secretKey: "value",
    bucket: "value",
    ca: "value"
});

Instance Methods

`.createXAmzHeaders(params)`

Creates the x-amz-* headers from the passed params.

Name	Type	Default	Summary
`params`	object	MANDATORY	The AWS API `params` object to convert into `x-amz-*` headers.
return	object	-	`params` converted into `x-amz-*` headers if they exist as `params` in the API docs.

`.createHttpHeader(host,path,method,bodyToHash,additionalHeaders)`

Creates the authentication headers for the given arguments according to the AWS Signature v4 documentation.
This method can be utilized when directly communicating using an HTTP object.

Name	Type	Default	Summary
`host`	string	MANDATORY	HTTP `host`
`path`	string	MANDATORY	HTTP `path`
`method`	string	MANDATORY	HTTP Method
`bodyToHash`	string	MANDATORY	HTTP Body; `null` to pass UNSIGNED_PAYLOAD instead.
`additionalHeaders`	object	MANDATORY	An Object with additional `x-amz-*` headers for signing
return	object	-	`{ Authorization: [auth], x-amz-content-sha256: [contenthash], x-amz-date: [timestamp]}`

`.put(path,userHeaders,body,callback)`

Sends a PUT request, passing the result to the callback function. This function uses the AWS S3 API: https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html

Name	Type	Default	Summary
`path`	string	MANDATORY	The path to the desired resource. The `path` is always appended to the host `[bucket].s3.amazonaws.com`.
`userHeaders`	object	MANDATORY	An Object with a "Content-Length" header (mandatory) and any additional HTTP/2 or `x-amz-*` headers.
`body`	function	MANDATORY	Callback to retrieve the contents of the object. Should return data in up to 4KB [string/ArrayBuffer] chunks until exhausted, then return `null`.
`callback(err, resp)`	function	MANDATORY	The user’s callback function, to handle the response. Will pass an Error as `err` and a Response as `resp`.

`.get(path,callback)`

Sends a GET request, passing the result to the callback function. This function uses the AWS S3 API: https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html

Name	Type	Default	Summary
`path`	string	MANDATORY	The path to the desired resource. The `path` is always appended to the host `[bucket].s3.amazonaws.com`.
`callback(err, resp)`	function	MANDATORY	The user’s callback function, to handle the response. Will pass an Error as `err` and a Response as `resp`.

`.abortRequest()`

Aborts the currently ongoing request.

Name	Type	Default	Summary
return	undefined	-	-

Response Object

This library uses an object called a Response to handle AWS server responses. Responses are always in the following format:

{
    statusCode: <Number>,
    statusMessage: <String>,
    body: <ArrayBuffer> or <String>
}

More details on statusCode and statusMessage can be seen in the HTTPs documentation for the appropriate device (NEQTO Bridge | Spresense).

body is the body data from the HTTP response.

Errors

In this library, Errors are returned in two ways: as return values from the setters, and passed as err into the callback function for the HTTP methods.

Errors raised outside of an HTTP request take the following form:

{
    name: <String>,
    message: <String>
}

Errors raised from an HTTP request take the following form:

{
    errCode: <Number>
}

Errors which detect invalid configurations, such as a missing Content-Length header, will have a message property; Errors which are raised from the HTTP/S layer will not have this property. Errors raised in this way have more information available regarding their errCode parameter in the NEQTO Bridge and Spresense neqto.js documentation.

The following is an example of how to handle the various forms of Errors:

var invalidConfig = {
    badKey: "badValue"
};
var s3 = new AWS_S3(invalidConfig);
// s3 now contains an array of errors
if (Array.isArray(s3)) {
    //handle config errors
} else {
    //s3 is good in this block
    var callback = function (err, data) {
        if (err) {
            // handle errors
            JSON.stringify(err)
        } else {
            // handle data
        }
    }
    s3.get('/example.txt', callback);
}

Usage Examples

Click here to learn how to get the CA that can be used in the config option.

Verify that the root CA is appropriate for the target S3 endpoint.

var rootCa = '-----BEGIN CERTIFICATE-----\n...<CA>...\n-----END CERTIFICATE-----';

Example 1: Put an Object to S3

// IMPORTED LIBRARIES
// - AWS_S3_V2

// Logging setup
log.setLevel(0,2); //-1:NONE 0:ERROR 1:WARNING 2:DEBUG 3:TRACE, 0:DISABLE 1:LOG 2:CONSOLE 3:BOTH
log.printLevel(2); //0:DISABLE 1:LOG 2:CONSOLE 3:BOTH

// Set config
var config = {
    region: "<String>",
    accessKeyId: "<String>",
    secretKey: "<String>",
    bucket: "<String>",
    ca: awsS3Ca
};

var callback = function (err, data) {
    if (err) {
        print("Error!", JSON.stringify(err));
    } else {
        print(data.statusCode, data.statusMessage);
        print(data.body);
    }
}

var s3 = new AWS_S3(config);
if ('errors' in s3) {
    // TODO: handle errors
    throw 'Invalid configuration'
}


/**
 * Create a large buffer (representing data generated from sensors)
 * and send it to AWS S3 with a custom tag and ACL parameter.
 */

 var path = '/path/to/resource.txt';
 var dataSize = 12 * 1024;
 var params = {
     "Tagging": "example=tag", // Requires the s3:PutObjectTagging permission
     "ACL": "bucket-owner-full-control", // Requires the s3:PutObjectAcl permission
     "ContentLength": dataSize.toString()
 }

 var buffer = new ArrayBuffer(dataSize);
 var bufferView = new Uint8Array(buffer);
 for (var b = 0; b < bufferView.length; b++) {
     bufferView[b] = (b % 255); // Fill it with arbitrary data
 }

 var putObject = function(path, params, body, callback) {
     var headers = s3.createXAmzHeaders(params);
     var MAX_CHUNK_SIZE = 4 * 1024;
     var bodyIndex = 0;
     var getNextChunk = function() {
         var ret;
         if (bodyIndex >= dataSize) {
             ret = null;
         } else {
             ret = body.slice(bodyIndex, bodyIndex + MAX_CHUNK_SIZE);
         }
         if (ret) {
             bodyIndex += ret.length || ret.byteLength;
         }
         return ret;
     }
     s3.put(path, headers, getNextChunk, callback);
 }

 putObject(path, params, buffer, callback);

Example 2: Put Data from Storage to S3

// IMPORTED LIBRARIES
// - AWS_S3_V2

// Logging setup
log.setLevel(0,2); //-1:NONE 0:ERROR 1:WARNING 2:DEBUG 3:TRACE, 0:DISABLE 1:LOG 2:CONSOLE 3:BOTH
log.printLevel(2); //0:DISABLE 1:LOG 2:CONSOLE 3:BOTH

// Set config
var config = {
    region: "<String>",
    accessKeyId: "<String>",
    secretKey: "<String>",
    bucket: "<String>",
    ca: awsS3Ca
};

var callback = function (err, data) {
    if (err) {
        print("Error!", JSON.stringify(err));
    } else {
        print(data.statusCode, data.statusMessage);
        print(data.body);
    }
}

var s3 = new AWS_S3(config);
if ('errors' in s3) {
    // TODO: handle errors
    throw 'Invalid configuration'
}

/**
 * Store data on the filesystem, then send that stored data.
 */

var path = "/path/to/storage.txt";
var dataSize = 16384;
var params = {
    "ContentLength": dataSize.toString(),
}

var store = storage.open();
var head = "HEADER:";

// Create a test data string and save it to memory
var s = "";
// Up to 4KB of data, to be compatible with put()
while(s.length < 1024) {
    s += "testdata";
}
for (var i = 0; i < 24; i++) {
    store.writeFrame(head, s);
}

// If there was an error during memory save, recover it.
if (!store.readFrame(head, 0)) {
    var ret = store.recoverFrame(head);
}

var putStringFromStorage = function (strStore) {
    var headers = s3.createXAmzHeaders(params);
    var getData;
    if (strStore) {
        if (strStore.getReadableSize() < dataSize) {
            return null;
        }
        if (head) {
            // Get total amount of data in frames with this head
            var frames = strStore.searchAllFrame(head, 0);
            if (frames < dataSize) { // not enough stored data
                return null;
            }
            // Get total number of frames with this head
            frames = strStore.searchAllFrame(head, 1);
            var frameIndex = -1;
            var frameDataCollectedSize = 0;
            var getData = function() {
                if (frameIndex++ < frames && frameDataCollectedSize < dataSize) {
                    var ret = strStore.searchFrame(head, frameIndex);
                    frameDataCollectedSize += ret.length;
                    return ret;
                } else {
                    return null;
                }
            }
        }
    }
    s3.put(path, headers, getData, callback);
}
putStringFromStorage(store);

Example 3: Put Custom Data to S3

// IMPORTED LIBRARIES
// - AWS_S3_V2

// Logging setup
log.setLevel(0,2); //-1:NONE 0:ERROR 1:WARNING 2:DEBUG 3:TRACE, 0:DISABLE 1:LOG 2:CONSOLE 3:BOTH
log.printLevel(2); //0:DISABLE 1:LOG 2:CONSOLE 3:BOTH

// Set config
var config = {
    region: "<String>",
    accessKeyId: "<String>",
    secretKey: "<String>",
    bucket: "<String>",
    ca: awsS3Ca
};

var callback = function (err, data) {
    if (err) {
        print("Error!", JSON.stringify(err));
    } else {
        print(data.statusCode, data.statusMessage);
        print(data.body);
    }
}

var s3 = new AWS_S3(config);
if ('errors' in s3) {
    // TODO: handle errors
    throw 'Invalid configuration'
}

/**
 * Send data to S3 using a custom getData method.
 */

 var path = "/path/to/custom.txt";
 var dataSize = 256;
 var params = {
     "Content-Length": dataSize.toString()
 }

 var send = false;
 var getData = function () { // Must return String, ArrayBuffer, or null.
     if (send) {
         return null;
     } else {
         send = true;
         var i = 0;
         var ret = "";
         while (i++ < dataSize) {
             ret += String.fromCharCode((i % 95) + 32); // ASCII table
         }
         return ret;
     }
 }

 s3.put(path, params, getData, callback);

The company names and product names mentioned above are registered trademarks or trademarks of their respective companies.

Updated: 2023-01-20

NEQTO Apps

Hardware Specifications

Software Specifications

neqto.js

Hardware Specifications

Software Specifications

neqto.js

Hardware Specifications

Software Specifications

neqto.js

Software Specifications

neqto.js

UART

I2C

Utils

Integration

Related Documents

Abstracts

{AWS_S3} Instance

Details

`new AWS_S3(config)`

Configuration

Setter Methods

`.setConfig(config)`

Instance Methods

`.createXAmzHeaders(params)`

`.createHttpHeader(host,path,method,bodyToHash,additionalHeaders)`

`.put(path,userHeaders,body,callback)`

`.get(path,callback)`

`.abortRequest()`

Response Object

Errors

Usage Examples

Example 1: Put an Object to S3

Example 2: Put Data from Storage to S3

Example 3: Put Custom Data to S3