NEQTO CloudSync for GCP
Introduction
NEQTO CloudSync for GCP is a tool that allows customers to easily manage and sync devices from NEQTO to their GCP environment. Complementary to the GCP libraries for neqto.js, this tool makes sending data to GCP's IoT Core a breeze.
NEQTO CloudSync is distributed as a container on the GCP Marketplace. It is free of charge and can be utilized on Google Kubernetes Engine, Compute Instances, or Google App Engine Flexible Environment (GAE Flex). Our recommendation is to deploy this tool on GAE Flex with the Custom Runtime Environment.
Prequisites
Deploying to GAE Flex (Custom Runtime) using Cloud Shell
For this tutorial we will utilize Cloud Shell to deploy CloudSync to App Engine.
First, start by curling the application zip file to your Cloud Shell instance, then unzipping it.
curl https://download.neqto.com/cloudsync/nqcloudsync-dist-gcp.zip -o nqcloudsync-dist-gcp.zip
unzip nqcloudsync-dist-gcp.zip
cd nqcloudsync-dist-gcp
After navigating to the nqcloudsync-dist-gcp folder, utilize the ls command to list the contained two files. The first file is app.yaml. This is a configuration file that gives App Engine directions on how to execute our application in addition to environment variables. Please edit the env_variables section based on your NEQTO and GCP environments.
Note: Do not modify
runtimeorenv.
runtime: custom
env: flex
env_variables:
COMPANY_CODE: "COMPANY CODE HERE"
NQ_REGION: "NEQTOREGION HERE"
PROJECT_ID: "GCPPPROJECTID HERE"
CLOUD_REGION: "GCPREGION HERE"
REGISTRY_ID: "REGISTRY ID HERE"
| Key | Value |
|---|---|
| COMPANY_CODE | Your NEQTO Company Code. (Used at login) |
| NQ_REGION | The NEQTO region you would like to sync with GCP. (e.g. asia-pacific-1) |
| PROJECT_ID | The ID of the GCP project you will sync to. |
| CLOUD_REGION | The region of your registry in GCP. |
| REGISTRY_ID | The ID of the IoT Core registry you wish to sync with. |
The second file in the nqcloudsync-dist-gcp folder is the Dockerfile. This file simply directs the app engine to the locations that it can get the CloudSync image. This file will not need to be edited, so please confirm its contents.
FROM marketplace.gcr.io/cloudsync-public/cloudsync
Once environment variables are set in the app.yaml, you may deploy the application.
gcloud app deploy
You may need to set your project and app engine environment first. You may also need to "Authorize" this via a prompt and input (Y) to confirm the details of this release is correct.
Warning: Please be aware that when your app is deployed with this step, it will be temporarily accessible to anyone on the internet. This app programmatically interacts with Google APIs and thus we highly recommend you secure the app to limit the parties who may access it. Please refer to the following section on measures to easily enable security.
Security
To secure the Web App, Google Cloud's Identity Aware Proxy (IAP) will be used. This will restrict access to allow only users which are logged-in to Google Cloud Console and given access to utilize CloudSync.
First, login to Google Cloud and select the project you are using. Next, use the search function to navigate to the "Identity-Aware Proxy".
From the Identity-Aware Proxy screen, view the list of applications and find your App Engine instance. Then slide the toggle to enable IAP for your instance.
In the list of applications, find your App Engine instance and slide the toggle for IAP to the on posiiton.
Finally, check the desired app instances and in the panel to the right, click "Add Member".
Enter the Google email for the member to grant access to, then click "Select a role". Click "Cloud IAP" from the list, then select the "IAP-Secured Web App User" option. Save your changes.
After a few moments, verify the security of your app by navigating to the app page in a new private browser and confirming that you are prompted to log in. Logging into an account which has not been explicitly granted access will block you out. Logging into an account which you have added as a member will grant access to the app.
This concludes the set-up for NEQTO CloudSync for GCP.
How to Use
Once deployed to GAE, you may access via the url app engine provides to you. By going to that URL you will be greeted by a login screen.
Before you can use CloudSync, you must first login.
Note: Company code is not required since it was set as an environment variable in the installation step.
After logging in, all groups that have been created in NEQTO Console are visible.
By clicking on one of the groups in the list you will be transported to the Node list screen.
A Node is synced when it exists both on GCP and NEQTO Console, and has all the necessary environment variables for communicating with GCP IoT Core.
In the Node list, the "synced" column shows the status by displaying one of 3 icons.
Nodes that are currently synced will have a Checkmark.
Nodes that are not synced will have an "X" icon.
Nodes that exist in just GCP or just NEQTO denote that a node may have become unintentionally unsynced. Hovering gives a hint.
To find out more you can click the info icon in the row to the right hand side of the list.
You can verify which environment has become out of sync as well as various information on your Node.
In order to fix a Node that has become out of sync, first "unsync" it by selecting the desired Nodes. In the upper right hand corner an "Unsync" button will appear. Click it to unsync the device.
Unsyncing a device will delete it from Google Cloud and remove the GCP environment variables created when syncing from the NEQTO Console Node.
To sync the device again, or sync any new devices, select everything then click the "sync" icon instead.
Clicking sync will make calls to the GCP API to register and provision a device. It will then take the private key and other necessary information and save it in the environment variables of the Node in NEQTO Console.
The environment variables created are as followed:
| Key | Value |
|---|---|
| gcp_cloud_sync_key | The private key needed for connecting to GCP IoT Core. |
| gcp_registry_id | The ID of the GCP IoT Core registry. |
| gcp_project_id | The ID of the GCP project you will sync to. |
| gcp_region_id | The region of your registry. |
| gcp_device_id | The device ID in GCP. This is a concatenation of "nq_" and the NEQTO Node uuid. |
Via the NEQTO GCP Library you may simply use these environment variables to begin sending data to GCP IoT Core.
Sample Script
// Ref: GCP IoT Core Library documentation
var gcpIotCa = "-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----\n";
var params = {
privateKey: ENV["gcp_cloud_sync_key"],
project: ENV["gcp_project_id"],
location: ENV["gcp_region_id"],
registry: ENV["gcp_registry_id"],
device: ENV["gcp_device_id"],
};
var iot_core = new GOOGLE_IOT_CORE(params);
iot_core.setRootCa(gcpIotCa);
var host = iot.buildGrpcUrl();
var payload = secure.base64Encode("Publishing via HTTP");
var callback = function(err, resp) {
if (err) {
print(JSON.stringify(err));
}
if (resp) {
print(JSON.stringify(resp));
}
}
var body = {
"binary_data": payload
}
var jwt = iot.makeDeviceJWT(3600); //update JWT each connection. 3600 == 1 hour.
iot.httpPost(jwt, `${host}:publishEvent`, JSON.stringify(body), callback);