AWS S3 v2
This library is a built-in class that provides functions to communicate with Amazon Simple Storage Service (Amazon S3).
Library usage requirements | |
---|---|
Type | Integrations |
Name | AWS_S3_V2 |
Version | 2020-11-04 |
Code size used | 8.2KB |
Resources used | HTTPS x 1, Timers x 1 |
Related Documents
For information on the AWS S3 API used in this library, please refer to the documentation.
Abstracts
Methods()/Properties | Summary |
---|---|
new AWS_S3() | Creates an AWS_S3 instance. |
{AWS_S3} Instance
Methods()/Properties | Summary |
---|---|
.createXAmzHeaders() | Converts any AWS API styled params into x-amz-* headers. Ignores bucket and key . |
.createHttpHeader() | Creates the authentication headers for the given arguments according to the AWS Signature v4 documentation. |
.put() | Sends a PUT request, outputting the result to the callback function. |
.get() | Sends a GET request, outputting the result to the passed callback. |
.abortRequest() | Aborts the currently ongoing request. |
.errors | An array containing the invalid configuration key names. Exists only when an invalid configuration is passed to the constructor. |
Details
new AWS_S3(config)
Initializes the AWS_S3
object with the passed configuration.
Configuration
When instantiating the AWS_S3
object, the following configurations are required to be set by the user:
Name | Type | Default | Summary |
---|---|---|---|
region | string | MANDATORY | Region to be used when accessing a bucket |
accessKeyId | string | MANDATORY | Access key id for authorization from AWS IAM |
secretKey | string | MANDATORY | Secret key for authorization from AWS IAM |
bucket | string | MANDATORY | Name of the S3 Bucket to operate on |
ca | string | MANDATORY | The root CA to communicate with AWS |
timeout | number | 90000 | The timeout of the AWS_S3 Instance in msver. 2020-07-20+ |
The configuration must be a JavaScript object, as the following:
var config = {
region: "<String>",
accessKeyId: "<String>",
secretKey: "<String>",
bucket: "<String>",
ca: "<String>"
};
Once the configuration is set, it can be passed to AWS_S3
during instance creation. If any invalid values are passed, the result from setConfig
is stored in an errors
property, and should be handled appropriately.
var s3 = new AWS_S3(config);
if ('errors' in s3) {
// TODO: handle errors
}
Setter Methods
After creating the AWS_S3
object, each of the aforementioned configurations may be changed by its corresponding setter, with a passed in value of String
type. The values for accessKeyId
and secretKey
may be retrieved from AWS IAM.
Setter | Summary | |
---|---|---|
.setRegion(value: string) | Set the region for this AWS_S3 instance. | |
.setBucket(value: string) | Set the bucket for this AWS_S3 instance. | |
.setAccessKeyId(value: string) | Set the Access Key ID for this AWS_S3 instance, used for authentication. | |
.setSecretKey(value: string) | Set the Secret Key for this AWS_S3 instance, used for authentication. | |
.setRootCA(value: string) | Set the root CA for this AWS_S3 instance, used for connecting to AWS. The suggested value can be seen below. | |
.setRequestTimeout(value: Number) | Set the timeout for this AWS_S3 instance.ver. 2020-07-20+ | ` |
The configuration setters can also be chained, if they were all successful. Each setter returns an Error when given an invalid value, instead of an AWS_S3
instance for method chaining.
.setConfig(config)
The setConfig
function can be used to set or change multiple config options at the same time, and takes in a typical JavaScript object of key:value pairs as an argument.
s3.setConfig({
region: "value",
accessKeyId: "value",
secretKey: "value",
bucket: "value",
ca: "value"
});
Instance Methods
.createXAmzHeaders(params)
Creates the x-amz-*
headers from the passed params
.
Name | Type | Default | Summary |
---|---|---|---|
params | object | MANDATORY | The AWS API params object to convert into x-amz-* headers. |
return | object | - | params converted into x-amz-* headers if they exist as params in the API docs. |
.createHttpHeader(host,path,method,bodyToHash,additionalHeaders)
Creates the authentication headers for the given arguments according to the AWS Signature v4 documentation.
This method can be utilized when directly communicating using an HTTP object.
Name | Type | Default | Summary |
---|---|---|---|
host | string | MANDATORY | HTTP host |
path | string | MANDATORY | HTTP path |
method | string | MANDATORY | HTTP Method |
bodyToHash | string | MANDATORY | HTTP Body; null to pass UNSIGNED_PAYLOAD instead. |
additionalHeaders | object | MANDATORY | An Object with additional x-amz-* headers for signing |
return | object | - | { Authorization: [auth], x-amz-content-sha256: [contenthash], x-amz-date: [timestamp]} |
.put(path,userHeaders,body,callback)
Sends a PUT request, passing the result to the callback function. This function uses the AWS S3 API: https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html
Name | Type | Default | Summary |
---|---|---|---|
path | string | MANDATORY | The path to the desired resource. The path is always appended to the host [bucket].s3.amazonaws.com . |
userHeaders | object | MANDATORY | An Object with a "Content-Length" header (mandatory) and any additional HTTP/2 or x-amz-* headers. |
body | function | MANDATORY | Callback to retrieve the contents of the object. Should return data in up to 4KB [string/ArrayBuffer] chunks until exhausted, then return null . |
callback(err, resp) | function | MANDATORY | The user’s callback function, to handle the response. Will pass an Error as err and a Response as resp . |
.get(path,callback)
Sends a GET request, passing the result to the callback function. This function uses the AWS S3 API: https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html
Name | Type | Default | Summary |
---|---|---|---|
path | string | MANDATORY | The path to the desired resource. The path is always appended to the host [bucket].s3.amazonaws.com . |
callback(err, resp) | function | MANDATORY | The user’s callback function, to handle the response. Will pass an Error as err and a Response as resp . |
.abortRequest()
Aborts the currently ongoing request.
Name | Type | Default | Summary |
---|---|---|---|
return | undefined | - | - |
Response Object
This library uses an object called a Response to handle AWS server responses. Responses are always in the following format:
{
statusCode: <Number>,
statusMessage: <String>,
body: <ArrayBuffer> or <String>
}
More details on statusCode
and statusMessage
can be seen in the HTTPs documentation for the appropriate device (NEQTO Bridge | Spresense).
body
is the body data from the HTTP response.
Errors
In this library, Errors are returned in two ways: as return values from the setters, and passed as err
into the callback
function for the HTTP methods.
Errors raised outside of an HTTP request take the following form:
{
name: <String>,
message: <String>
}
Errors raised from an HTTP request take the following form:
{
errCode: <Number>
}
Errors which detect invalid configurations, such as a missing Content-Length
header, will have a message
property; Errors which are raised from the HTTP/S layer will not have this property. Errors raised in this way have more information available regarding their errCode
parameter in the NEQTO Bridge and Spresense neqto.js documentation.
The following is an example of how to handle the various forms of Errors:
var invalidConfig = {
badKey: "badValue"
};
var s3 = new AWS_S3(invalidConfig);
// s3 now contains an array of errors
if (Array.isArray(s3)) {
//handle config errors
} else {
//s3 is good in this block
var callback = function (err, data) {
if (err) {
// handle errors
JSON.stringify(err)
} else {
// handle data
}
}
s3.get('/example.txt', callback);
}
Usage Examples
Click here to learn how to get the CA
that can be used in the config
option.
Verify that the root CA is appropriate for the target S3 endpoint.
var rootCa = '-----BEGIN CERTIFICATE-----\n...<CA>...\n-----END CERTIFICATE-----';
Example 1: Put an Object to S3
// IMPORTED LIBRARIES
// - AWS_S3_V2
// Logging setup
log.setLevel(0,2); //-1:NONE 0:ERROR 1:WARNING 2:DEBUG 3:TRACE, 0:DISABLE 1:LOG 2:CONSOLE 3:BOTH
log.printLevel(2); //0:DISABLE 1:LOG 2:CONSOLE 3:BOTH
// Set config
var config = {
region: "<String>",
accessKeyId: "<String>",
secretKey: "<String>",
bucket: "<String>",
ca: awsS3Ca
};
var callback = function (err, data) {
if (err) {
print("Error!", JSON.stringify(err));
} else {
print(data.statusCode, data.statusMessage);
print(data.body);
}
}
var s3 = new AWS_S3(config);
if ('errors' in s3) {
// TODO: handle errors
throw 'Invalid configuration'
}
/**
* Create a large buffer (representing data generated from sensors)
* and send it to AWS S3 with a custom tag and ACL parameter.
*/
var path = '/path/to/resource.txt';
var dataSize = 12 * 1024;
var params = {
"Tagging": "example=tag", // Requires the s3:PutObjectTagging permission
"ACL": "bucket-owner-full-control", // Requires the s3:PutObjectAcl permission
"ContentLength": dataSize.toString()
}
var buffer = new ArrayBuffer(dataSize);
var bufferView = new Uint8Array(buffer);
for (var b = 0; b < bufferView.length; b++) {
bufferView[b] = (b % 255); // Fill it with arbitrary data
}
var putObject = function(path, params, body, callback) {
var headers = s3.createXAmzHeaders(params);
var MAX_CHUNK_SIZE = 4 * 1024;
var bodyIndex = 0;
var getNextChunk = function() {
var ret;
if (bodyIndex >= dataSize) {
ret = null;
} else {
ret = body.slice(bodyIndex, bodyIndex + MAX_CHUNK_SIZE);
}
if (ret) {
bodyIndex += ret.length || ret.byteLength;
}
return ret;
}
s3.put(path, headers, getNextChunk, callback);
}
putObject(path, params, buffer, callback);
Example 2: Put Data from Storage to S3
// IMPORTED LIBRARIES
// - AWS_S3_V2
// Logging setup
log.setLevel(0,2); //-1:NONE 0:ERROR 1:WARNING 2:DEBUG 3:TRACE, 0:DISABLE 1:LOG 2:CONSOLE 3:BOTH
log.printLevel(2); //0:DISABLE 1:LOG 2:CONSOLE 3:BOTH
// Set config
var config = {
region: "<String>",
accessKeyId: "<String>",
secretKey: "<String>",
bucket: "<String>",
ca: awsS3Ca
};
var callback = function (err, data) {
if (err) {
print("Error!", JSON.stringify(err));
} else {
print(data.statusCode, data.statusMessage);
print(data.body);
}
}
var s3 = new AWS_S3(config);
if ('errors' in s3) {
// TODO: handle errors
throw 'Invalid configuration'
}
/**
* Store data on the filesystem, then send that stored data.
*/
var path = "/path/to/storage.txt";
var dataSize = 16384;
var params = {
"ContentLength": dataSize.toString(),
}
var store = storage.open();
var head = "HEADER:";
// Create a test data string and save it to memory
var s = "";
// Up to 4KB of data, to be compatible with put()
while(s.length < 1024) {
s += "testdata";
}
for (var i = 0; i < 24; i++) {
store.writeFrame(head, s);
}
// If there was an error during memory save, recover it.
if (!store.readFrame(head, 0)) {
var ret = store.recoverFrame(head);
}
var putStringFromStorage = function (strStore) {
var headers = s3.createXAmzHeaders(params);
var getData;
if (strStore) {
if (strStore.getReadableSize() < dataSize) {
return null;
}
if (head) {
// Get total amount of data in frames with this head
var frames = strStore.searchAllFrame(head, 0);
if (frames < dataSize) { // not enough stored data
return null;
}
// Get total number of frames with this head
frames = strStore.searchAllFrame(head, 1);
var frameIndex = -1;
var frameDataCollectedSize = 0;
var getData = function() {
if (frameIndex++ < frames && frameDataCollectedSize < dataSize) {
var ret = strStore.searchFrame(head, frameIndex);
frameDataCollectedSize += ret.length;
return ret;
} else {
return null;
}
}
}
}
s3.put(path, headers, getData, callback);
}
putStringFromStorage(store);
Example 3: Put Custom Data to S3
// IMPORTED LIBRARIES
// - AWS_S3_V2
// Logging setup
log.setLevel(0,2); //-1:NONE 0:ERROR 1:WARNING 2:DEBUG 3:TRACE, 0:DISABLE 1:LOG 2:CONSOLE 3:BOTH
log.printLevel(2); //0:DISABLE 1:LOG 2:CONSOLE 3:BOTH
// Set config
var config = {
region: "<String>",
accessKeyId: "<String>",
secretKey: "<String>",
bucket: "<String>",
ca: awsS3Ca
};
var callback = function (err, data) {
if (err) {
print("Error!", JSON.stringify(err));
} else {
print(data.statusCode, data.statusMessage);
print(data.body);
}
}
var s3 = new AWS_S3(config);
if ('errors' in s3) {
// TODO: handle errors
throw 'Invalid configuration'
}
/**
* Send data to S3 using a custom getData method.
*/
var path = "/path/to/custom.txt";
var dataSize = 256;
var params = {
"Content-Length": dataSize.toString()
}
var send = false;
var getData = function () { // Must return String, ArrayBuffer, or null.
if (send) {
return null;
} else {
send = true;
var i = 0;
var ret = "";
while (i++ < dataSize) {
ret += String.fromCharCode((i % 95) + 32); // ASCII table
}
return ret;
}
}
s3.put(path, params, getData, callback);