Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
This documentation details the steps to use ReXgen Air for the alternate S3 API-supported storage service MinIO in Windows.
Basic of
(detailed explanation)
The following document lists the simple steps that allow the user to create an AWS bucket for storage.
Amazon Simple Storage Service (S3) is an object storage built to retrieve data anytime and from anywhere. It lets you store, distribute and manage your data in the Cloud. The need for more and more data storage continues to grow, and building your storage is not an option. AWS S3 lets you securely collect, analyse and store your data at a massive scale. It even enables you to distribute and manage your data by running multiple applications, hosting high-traffic websites, using 3rd party software, and delivering content.
Influx ReXgen Air now lets you store your data to AWS S3. This not only reduces cost but has no upfront investments. One can think of it as a paid taxi service, where you own nothing and pay for the service, allowing users to pay only for the storage and data transfer they use. One of the main benefits of using S3 is storing and accessing data from anywhere with an internet connection.
Amazon Simple Storage Services, as the name suggests, provides a web interface for storing files. Each file is stored as an object within the 'bucket', which serves as a container for storing objects that need to be accessed by multiple users or devices, such as documents, images, and videos. S3 also offers a range of features that make it easy to manage and organise data, including creating and managing folders, setting access controls, and tracking users' activity. It is not only low cost as it allows you to pay as you go, but it also has multiple advantages like no investment, scalability, flexibility, global availability, reliability, and security, making AWS S3 stand out. Data stored in S3 buckets is stored across multiple devices in multiple facilities across the specified region, making it highly resistant to data loss. S3 is also designed to scale automatically to meet users' needs, making it easy to store and access large amounts of data without worrying about capacity limits. Overall, AWS S3 is a robust and reliable cloud storage service that offers a range of features and pricing options to meet the needs of different users. Whether you are looking to store large amounts of data or need a place to store and share documents, S3 is an excellent solution.
The first step for creating an AWS S3 bucket is to have an Amazon Web Services (AWS) account. Signing up for AWS automatically signs you up for all services in AWS, including Amazon S3.
Open https://portal.aws.amazon.com/billing/signup#/start/email
Sign-Up (new user): fill in the email address and an account name, then click 'verify email address'.
A sign-up automatically creates a 'root user' account on AWS. The root user can access all AWS services and resources in the account.
Sign in to your Amazon Web Services (AWS) account.
Under Services, choose Storage to open the Amazon S3 console.
Choose Create Bucket.
Specify the bucket name in Bucket Name and your desired region in the AWS Region section, for example, Europe (London) eu-west-2.
Fill in all the other required fields as desired.
Click Create bucket.
Once the bucket is created successfully, it is displayed in the Buckets pane.
Sign in to your Amazon Web Services (AWS) account:
Under Services, choose 'All services' and open the 'IAM' (Identity and Access Management) console.
Under 'Access management', click Users and then 'Create user'.
Specify the IAM user name (read the other instructions properly and grant permissions as desired).
Click 'Next'.
The next step involves granting permissions to the user-specified.
One can add this new user to the existing groups by selecting 'Add user to group' or to copy the policies from an existing user, select 'Copy permissions' options.
OR
You can create a custom policy with the necessary permissions to restrict access to a single Amazon S3 bucket when creating an IAM policy for an IAM user using JSON. Select 'Attach policies directly' and click 'Create policy'.
Select 'JSON" from the displayed options.
This displays the 'Policy editor window' where you can define the policy.
Below is an example of a JSON policy that allows read access to a single S3 bucket named "my-example-bucket" (you can modify it to fit your specific requirements).
Sign in to the AWS Management Console.
Navigate to the IAM service.
Create or select the IAM user to restrict access to.
In the "Permissions" tab for the user, click "Add permissions."
Choose "Attach existing policies directly."
Click the "JSON" tab and paste the JSON policy into the text box.
Review and click "Next: Review" to proceed.
Review the permissions and the policy you attached to the user.
Click "Add permissions" to attach the policy to the user.
You can modify the JSON policy to include additional actions or restrict access according to your requirements.
Once you have defined the permissions accordingly, click 'Next'.
Review the details specified and click 'Create user'.
Once the user is created successfully, identify and click on the name to proceed.
Click Security credentials, and under Access Keys, click 'Create access key'.
This takes you to a screen which prompts you to consider use cases and alternatives.
Click 'Other' (read the best practices for managing the access key).
Click 'Next'.
Specify the description tag value and click 'Create access key'.
This generates your Access and Secure Access keys.
Once the access key and the secret access keys are generated, copy and paste them to a secure location, as you can view the Secret access key only once.
Click 'Done' to complete the process.
New Mobile
Allows the user to create a new mobile setting.
Edit Mobile
Allows the user to edit the newly created mobile settings.
Delete Mobile
Allows the user to delete the created mobile settings, if not required.
Label: This allows the user to define a name for mobile settings.
User: This allows the user to enter the mobile server user name.
Pass: This allows the user to define a password for the mobile settings.
APN: Enter the APN.
PIN: Enter the pin to the defined APN.
No Communication timeout (min): The user can enter the delay interval for communication time out in minutes.
Get Real-Time From Mobile: Users can enable this to get real-time from the server.
New Mobile
Allows the user to create a new mobile setting.
Edit Mobile
Allows the user to edit the newly created mobile settings.
Delete Mobile
Allows the user to delete the created mobile settings, if not required.
Label: This allows the user to define a name for mobile settings.
User: This allows the user to enter the mobile server user name.
Pass: This allows the user to define a password for the mobile settings.
APN: Enter the APN.
PIN: Enter the pin to the defined APN.
No Communication timeout (min): The user can enter the delay interval for communication time out in minutes.
Get Real-Time From Mobile: Users can enable this to get real-time from the server.
Click the ‘Tools’ tab in the Navigation bar and select ‘Cloud Settings’.
In the Cloud Settings window, select the ‘S3 ‘option and then select the ‘New S3’ option.
In the S3 Settings window, specify the Region from the drop-down menu.
Specify the type of storage services - AWS or Compatible (alternate S3 API-supported storage services).
Specify the S3 Connection type from the drop-down menu. Select Plain for HTTP connection and SSL for HTTPS (Certificate must be added to the XML file).
Specify ‘Port’ details: 80 if the connection type is Plain, 443 if the connection type is SSL.
Specify the bucket name for objects stored in Amazon S3.
Specify the ‘Access Key’ and ‘Secret Key’ for the AWS account
Click ‘OK’.
The new AWS S3 setting details have been defined using ReXdesk.
Click on ’x’ to exit the window.
Make the new configuration.
Enable AWS from the Device Configuration menu.
Save the Project with the name you require for the configuration.
Under the ‘Config’ tab, select Export from the drop-down menu.
Select ‘Export ReXgen Configuration (rxc)’ from the listed options.
Save the file with the desired name.
Click ‘Save’.
Once saved, rename the file as ‘config.rxc.’
Add to your configuration folder in your S3 bucket by adding it to the root folder of the logger.
Prepare the base configuration for the logger.
In the Device Configuration window, Click on the ‘Upload’ tab.
Enable the ‘checkbox’ for ‘Enable Aws’.
For a single setting, details are auto-filled in the subsequent sections.
Users can select the desired setting from the corresponding drop-down menus for multiple settings.
Click the ‘Run’ tab in the Navigation bar.
Save the project:
a) Save project: allows you to save the project with the user-defined name.
b) Continue as “New Project”: saves the configuration labelled ‘New Project.’
Click ‘OK’ once the configuration is uploaded successfully.
Before proceeding, make sure you have Visual Studio 2022 Installed, and the supporting components are installed as per the .vsconfig file in the link below
Clone or download the repository
https://github.com/InfluxTechnology/ReXlibrary/tree/master/Demo/AWS_Lambda
Open \Demo\AWS_Lambda\AWSLambdaFileConvert.csproj
Provide your credentials with permission to S3 and Lambda for your AWS SDK (Refer)
Right-click on the project name and select 'Publish to AWS Lambda…'
Select
'AWS Credential'
Choose 'Region' in which you have the S3 Bucket.
Enter the 'Description' for the function if required.
Leave everything else to default.
Choose an IAM role that provides the Lambda functions necessary permission to access all S3 Services and all Lambda services.
If uploading for the first time, ensure you have created an IAM role with the following permissions; refer to the link.
If you have already created a role, you can continue by selecting the existing role.
Click 'Upload' to Upload the function to AWS.
Continue to Adding S3 Trigger for Lambda
Click on 'Add trigger'.
Select the following
Source as S3.
Select the Bucket to which the logger uploads data.
Suffix add (.rxd).
Acknowledge the warning.
Continue to Configuring conversion settings.
Install docker:
Download the ReXcloud docker image:
Load the docker image with the following command in Windows power shell or cmd prompt (from the image folder):
Run the docker image with the following command in Windows power shell or cmd prompt (from the image folder):
Access Fleet management by typing http://localhost:1337/ in the browser.
Load the DBC into the corresponding folder created by the Data Logger.
The DBC name should be as follows
dbc_can0.dbc For CAN 0 Bus
dbc_can1.dbc For CAN 1 Bus
dbc_can2.dbc For CAN 2 Bus
dbc_can3.dbc For CAN 3 Bus
In the User Object store, click 'Access Keys'.
Once the access keys window opens, click 'Create access key'.
This opens a window with your access and a secret key. If necessary, enter other details such as expiry/name/description/comments and click 'Create'.
Copy and save the access and secret key details before proceeding. Additionally, after clicking 'Create,' min-Io prompts you to download and save the access key and security key details. This can be done by clicking the 'Download for import' tab.
The access key is successfully created.
Click the 'Buckets' option and then the 'Create a Bucket' tab.
Specify the Bucket details.
Bucket Name: Name the bucket (as desired) after reading the bucket naming rules.
Versioning: allows multiple versions of the same object to be kept under the same key.
Object Locking: prevents objects from being deleted. Required to support retention and legal hold. It can only be enabled at bucket creation.
Quota: limits the amount of data in the bucket.
Retention imposes rules to temporarily prevent object deletion. Versioning must be enabled to set bucket retention policies.
After entering all the details, click 'Create Bucket'.
Once the bucket is created, its properties can be accessed by simply clicking on its name.
Click on the bucket name to add/upload a file to the bucket.
Click the 'Upload' tab to upload any file/folder to the bucket.
Click "Run Configuration," save the project, and provide a name for it. The logger will create files with the project name.
The configuration will be loaded in the data logger, and logging will start immediately.
Click the 'Tools' tab in the Navigation bar and select 'Cloud Settings'.
In the Cloud Settings window, select the 'S3 'option and then select the 'New S3' option.
Region: updates accordingly once the S3 type is entered.
S3 Type: This allows users to select between AWS and compatible storage services (alternate S3 API-supported storage services).
Endpoint: This allows the user to preview the END point.
S3 Connection Type: Select Plain for HTTP connection and SSL for HTTPS.
Port: 9000 in case of min-Io.
Bucket: Specify the bucket name from Mini-Io.
Access Key: paste the access key for the min-Io account.
Secret Key: paste the secret key for the min-Io account.
*Always format the logger's internal storage after sending a configuration with AWS enabled.
Make the new configuration.
Connect the logger to the PC via USB. Status Bar will display the logger status and time information.
Download FileZilla
Complete the installation with default settings.
Log in to the Admin Interface.
Click on the server settings icon.
Click on general settings and configure the default port. It is advised to use ports other than 21. Example:54345
Configure the welcome message, optional.
Configure the port range (Starting from the port configured in step 3) and the external IP address to be used for FTP passive mode configuration. Example: 54345 - 54350
If the server is being set up on Azure Virtual Machine or an Amazon EC2 instance, use the static IP address of the Virtual Machine or EC2 instance.
If it is a local server with a static IP address, use that static IP Address.
Ensure the ports are open in the Windows firewall and the Firewall settings in the Azure or AWS portal.
Go to FTP over TLS Settings
Enable the FTPS option
Disallow plain FTP
Generate new certificate
Leave other settings as shown in the image below.
*PROT P is not supported
Click Ok
Before proceeding, please create the function in the region where you have the S3 Bucket.
Select 'Author from scratch.
In Basic information:
Define function name.
Select Runtime as '.NET 6 (C#/PowerShell) (from the drop-down options).
Select Architecture, x86_64 (from the given options).
In Permissions, set the default execution role for Lambda functions:
If you are uploading for the first time, make sure you have created an IAM role with the following permissions; refer to the link
If you have already created a role, you can continue by selecting the existing role.
Select 'Use an existing role'.
Select the Existing role as 'influx_lambda_role' from the drop-down options.
Select 'Create function'.
Add Runtime as .NET 6 (C#/PowerShell).
Change handler as AWSLambdaFileConvert::AWSLambdaConvert.Function::ConvertFiles
Chose Architecture.
Click 'Save'.
For example
Name the database and click 'Create database'.
Name the role.
Add description (if needed/as required).
Click 'Create role'.
Click on the 'CRON expressions' link to learn more about CRON syntax for assigning a required schedule.
Note:
The database will be populated only if the parquet files are already available in S3; otherwise, there is no need to run the job manually.
Note:
Happens only when the job is run manually after creation it will have to wait until the first run.
Click on the user management icon.
Click on Add and create a new user.
Enable the password option and enter a password.
Select the user and click the add folder icon.
Select the folder that you wish to use.
Enable all the permissions.
The server is ready for use.
ReXgen AIR FTP And Mobile Settings: This lets users view and define the and settings.
From Tools, select Encryption Manager
Click Mobile and add SIM settings
Click the FTP Setting tab to add FTP server details.
Click upload in “Device Configuration.”
Select the added Mobile settings and FTP settings.
Store data to FTP using mobile internet: Enable storing the data to FTP using mobile internet.
Mobile: Enter the name defined in the SIM settings.
FTP: Enter the name defined in the server settings.
Check Config Time (min): Checks for new config on the server (at a given delay).
Send status Time (min): Updated the status in the given time (at a given delay).
Check Firmware Update: Checks for new firmware on the server (at a given delay).*
Automatic Firmware Update: Enable updating the firmware automatically.*
*will be enabled with the upcoming versions.
Note:
Once the USB is removed, only then the logger will connect to the server and upload files.
Use Server Name: This allows the user to use the server name to execute the FTP process.
Use IP Address: This allows the user to use the IP Address to execute the FTP process.
Protocol: This allows users to select the protocol type between FTP and FTPs.
Mode: This allows users to select the mode type between Active and Passive(recommended).
User: This allows the user to specify himself.
Pass: Allows the user to define a password for the FTP settings.
Influx provides a free, open-source conversion function to convert the native RXD format to formats that the user uses for their analysis automatically.
Note:
Before proceeding, please ensure that you create the function in the same region where you have the S3 Bucket and the Timestream Database.
When used with AWS, ReXgen will utilize the following Services:
Amazon S3 Bucket - Data is being stored from the device.
Amazon Lambda - Serverless Compute Service Run Code Without Provisioning or Managing Infrastructure.
Supports – BLF, MF4 and CSV formats.
Supports DBC Decoding.
Supports writing data to external Time Series Databases.
Open the 'Windows Security' menu, and click 'Advanced settings'.
Click on 'Inbound Rules' and then click the 'New Rule' option. This opens the 'New Inbound Rule Wizard', where the user needs to define the new rule's properties.
For 'Rule Type', enable 'Port' and click 'Next'.
For 'Protocol and Ports', specify the port details for 'Specific local ports', i.e. 9000 for MinIO.
Leave the following steps as they are and click 'Next' for both 'Action' and 'Profile'.
For 'Name', specify a desired name and click Finish.
Login to your AWS portal and go to Lambda Services.
Click on 'Create Function'.
Once in the 'Create Function' window:
Select 'Author from scratch.
In Basic information:
Define function name.
Select Runtime as '.NET 6 (C#/PowerShell) (from the drop-down options).
Select Architecture, x86_64 (from the given options).
In Permissions, Change the default execution role:
If you are uploading for the first time, make sure you have created an IAM role with the following permissions; refer to the
If you have already created a role, you can continue by selecting the existing role.
Select 'Use an existing role'.
Select the Existing role as 'influx_lambda_role' from the drop-down options.
Select 'Create function'.
Once the function is created, click on 'Upload from'.
Click 'Upload'.
Select the function code's zip file, and click 'Open'.
Once the zip file is uploaded, click 'Save'.
Once on the function window, under the 'Code' tab, locate 'Runtime settings' and click 'Edit' if you wish to edit any credentials.
In Runtime settings:
Add Runtime as .NET 6 (C#/PowerShell).
Change handler as AWSLambdaFileConvert::AWSLambdaConvert.Function::ConvertFiles
Chose Architecture.
Click 'Save'.
To edit the general configuration, click' Configuration' and select' Edit'.
As desired, edit details such as memory, ephemeral (temporary) storage, and time out.
Click “Run” to update the configuration.
A function is available in , and customers can deploy the function quickly to enable auto conversion. Below are the guides to deploy the lambda function to AWS:
Continue to
New FTP
Allows the user to define the name for adding new FTP settings.
Edit FTP
Allows the user to edit the newly created FTP settings.
Delete FTP
Allows the user to delete the created FTP settings if not required.
Upload the FileConvert.json file into the root folder of the Bucket.
Enable the required flags.
Continue to Adding the DBC file for conversion.
To begin - access MinIO's website and click Download.
Open the command prompt in the folder (where minio.exe is stored.
Paste the following in the command prompt and press 'Enter'.
(Username - type your user's name for logging in to the minIO Console)
(Password- type your password for logging in to the minIO Console)
Open the 'MinIO Console' by typing 'localhost:9001' into your browser.
Once you have access to the login window, enter the username and password you used in the command prompt. And click 'Login'.