Skip to main content

OTA Firmware Upgrades

OTA firmware upgrades is one of the most important functions of any IoT System. It helps add features and fix bugs for devices in fields. ESP RainMaker offers a rich OTA Upgrades interface with many options and flexibility.

Features​

  • OTA firmware image upload, which includes parsing of image to fetch metadata from header and validate.
  • Secure signing of bootloader and firmware for secure boot.
    • Auto signing of images with appropriate keys during OTA.
  • Specifying Target Nodes as
    • One or more Nodes based on node ids.
    • One or more Node Groups.
    • Dynamic OTA groups (based on node type/mode/firmware version).
  • OTA trigger methods
    • Default: Nodes get OTA Job when queried for.
    • Force push: Immediately sent to all online nodes and subsequently to other nodes based on when they query for OTA.
    • User Approved: Requires end users to query and approve OTA. Also provides an option to trigger push notifications to notify users about OTA availability.
    • Timings
      • Validity (Perform OTA within specific dates).
      • Node time window (Perform OTA within specified hours as per the Node's time zone.
  • OTA Download methods
    • HTTPS (using pre-signed URL): Low cost but extra RAM usage.
    • MQTT : Higher cost but memory efficient.
  • Monitoring
    • OTA Job Status summary and node wise info.
    • Timestamped detailed progress status for each node.
  • Additional Features
    • Rollback support to ensure that a faulty OTA does not brick a node and the node can rollback to previous firmware.
    • Priority based OTA: Specify ordering of multiple OTA jobs.
    • Host MCU: Provide images for host MCU (if applicable).
    • Custom metadata: Any user case specific metadata that is expected to be parsed by the Node.

Enabling OTA Upgrades in the Firmware​

The recommended API to enable OTA functionality is to call this single API:

     esp_rmaker_ota_enable_default();

This internally maps to "OTA Using Topics", uses the ESP x509 Certificate Bundle for server authentication and sets the default OTA callback.

Setting up OTA Upgrade Image​

Compile an OTA upgrade firmware image as you would normally do, but ensure the following:

  • The project name should be the same as that used for the firmware already flashed on the node(s).
  • The firmware version should be different than what the nodes already have (check set(PROJECT_VER "1.0") in the example's CMakeLists.txt file).

If you miss out any of the above, the OTA upgrade will fail. You can change this behavior by setting the following configuration options (not recommended):

CONFIG_ESP_RMAKER_SKIP_VERSION_CHECK=y
CONFIG_ESP_RMAKER_SKIP_PROJECT_NAME_CHECK=y

Performing OTA​

Before you can push out an OTA using topics, you need to have the admin access to the node. Please check here to find out how to get the admin access. Once done, please visit the ESP RainMaker dashboard and follow the further steps.

Uploading the OTA Upgrade Image​

  • Go to the Firmware Images section in the sidebar on the dashboard and click Add Image.
  • Fill in the details in the pop-up window.
    • The "Advanced" options of Firmware Version and Model (which is the same as the project name) should be left blank as they will be read from the binary directly. They have been provided only for cases wherein the image is encrypted or is meant for some non-ESP Host MCU.
  • Select the upgrade image just compiled and add it.
  • The image should now be visible in the Firmware Images list.

Starting an OTA​

  • Check the OTA image that you want to use for your OTA upgrade in the Firmware Images list.
  • Click the Start OTA button for that image.
  • Enter the name for the OTA job and select the group(s)/node(s) which you want to upgrade.

For testing purpose, please select the Force Push option, so that the OTA job information is sent to the node right away. Else, the node(s) will get the OTA URL as per the OTA upgrade policy defined by auto fetch configuration.

Advanced Options​

While creating an OTA Job, you can also choose some advanced options as shown below

  • User Approval: End user has to approve the job before it is sent to the node.
    • Notify User: Send a push notification to the user indicating availability of OTA, which the user can then approve/decline.
  • Apply at specific Device Time: Trigger the Job only during specific hours (like 2:00am-4:30 am).
  • OTA Job Valid for: Allow the job to be performed only between the specified dates.
  • Host MCU: Use the OTA Image for upgrading the Host MCU instead of the ESP Wi-Fi MCU.

Apart from the user approval, the rest is sent as metadata to the node and is handled by the firmware logic. Please check out additional information about this metadata here.

Monitoring an OTA Job​

  • Once you start an OTA, if it is successful, you will directly get a link to Go to job details. Click it to see the status of your OTA job. You can also access the OTA Job page from the side panel to view the details.
  • The OTA Job status page gives an overall picture of the job.
  • You can also view status of individual nodes by clicking on their id. A sample is shown here:
  • It is allowed to cancel a job midway, but the nodes that had already got the URL will continue with the upgrade.

How Does This Work?​

As the name suggests, the "OTA using Topics" method uses dedicated topics for OTA upgrades.

  • When the node connects to the MQTT broker, it subscribes to the topic node/<node_id>/otaurl.
  • Whenever an OTA is triggered from the dashboard using the Force Push option, the cloud service sends this information to the node on node/<node_id>/otaurl.
{
    "url": "<ota_image_url>",
    "fw_version": "fw_version_string",
    "ota_job_id": "<ota_job_id>",
    "file_size": <num_bytes>,
    "stream_id": "<mqtt_ota_stream_id>"
}
  • The node then starts downloading the image from the received <ota_image_url> and publishes the progress on node/<node_id>/otastatus in the following format:
{
    "ota_job_id": "<ota_job_id>",
    "status": "<in-progress/success/fail>",
    "additional_info": "<additional_info>"
}
  • The OTA job ends when all nodes have either reported success or failed.

Advanced Options​

When any of the advanced options are selected, additional metadata is sent to the node alongwith the utl and other details. A sample:

{
    "url": "<ota_image_url>",
    "fw_version": "fw_version_string",
    "ota_job_id": "<ota_job_id>",
    "file_size": <num_bytes>,
    "stream_id": "<mqtt_ota_stream_id>",
    "metadata": {
        "download_window": {
            "start": 120,
            "end": 270
        },
        "esp.ota.target": "host_mcu",
        "validity": {
            "start": 1703788200,
            "end": 1703961000
        }
    }
}

As can be seen above

  • The download window starts at 120 minutes (02:00am) and ends at 270 minutes (4:30am).
  • Validity starts at 1703788200 and ends at 1703961000, which are 12:00am IST for 29 Dec and 31 Dec, which was chosen while creating the job.
  • "esp.ota.target": "host_mcu" indicates that this OTA is for Host MCU.

Note that the framework allows you to add even more custom data in the metadata field as per your requirements. The ones shown here are just the ones sent by the current RainMaker dashboard.

Auto Fetch Configuration​

Most of the time, it is desired that the OTA upgrades are sent to the nodes in a staggered way, so as to prevent overwhelming the server with too many image download requests at the same time. Moreover, it is also possible that the node is offline when the cloud is publishing to node/<node_id>/otaurl, so alternative methods to fetch the upgrade information again are required.

When CONFIG_ESP_RMAKER_OTA_AUTOFETCH=y is set , the node tries to fetch the OTA upgrade periodically as per the interval specified by CONFIG_ESP_RMAKER_OTA_AUTOFETCH_PERIOD. This is accomplished by publishing the following data to node/<node_id>/otafetch:

{
    “node_id": “<node_id>",
    “fw_version": “<fw_version>"
}

If the cloud service finds any OTA job pending for the node, it publishes it to node/<node_id>/otaurl as specified above and the job starts.

More details about the OTA APIs can be found here.

OTA using Paramaters (Deprecated)​

This is the simplest way of pushing out an OTA firmware upgrade by End Users and can be triggered from the RainMaker CLI. This can be enabled using

    esp_rmaker_ota_enable(NULL, OTA_USING_PARAMS);

For triggering the OTA using this method, you will need the RainMaker CLI. Please set it up as per the instructions here. Once done, after the node is provisioned from the phone apps, follow these steps:

$ cd /path/to/esp-rainmaker/cli
$ ./rainmaker.py login # Use the same credentials used in phone app for setting up the node
$ ./rainmaker.py getnodes # Just to verify that you see the node that you want to upgrade
$ ./rainmaker.py otaupgrade <node_id> <path_to_ota_fw_image>

As the firmware upgrade starts, you will see the progress as below:

$ ./rainmaker.py otaupgrade 7CDFA1XXXXXX ../examples/switch/switch-2.0.bin
Uploading OTA Firmware Image...
Checking esp.service.ota in node config...
Setting the OTA URL parameter...
OTA Upgrade Started. This may take time.
Getting OTA Status...
[19:49:50] in-progress : Downloading Firmware Image
[19:49:58] in-progress : Downloading Firmware Image
[19:50:07] in-progress : Downloading Firmware Image
[19:50:16] in-progress : Downloading Firmware Image
[19:50:25] in-progress : Downloading Firmware Image
[19:50:34] in-progress : Downloading Firmware Image
[19:50:43] success : OTA Upgrade finished successfully

How Does This Work?

This OTA service is built on top of the RainMaker Services concept. When you call esp_rmaker_ota_enable(NULL, OTA_USING_PARAMS), the OTA service gets added to the node configuration. It has three parameters:

  • url (esp.param.ota_url)
  • status (esp.param.ota_status)
  • info (esp.param.ota_info)

When you use the OTA upgrade command from the CLI, the following happens:

  • The CLI reads the firmware upgrade image from the path provided and uploads it to the RainMaker cloud service.
  • It gets back a pre-signed URL in return. Note that this URL is valid for a limited time (1 day currently, but may change in the future).
  • It sets the url parameter with the value of this pre-signed URL.
  • The node starts downloading the image and reports the status and additional information as it progresses.
  • The CLI keeps querying this (using getparams) and shows the status.
  • The OTA upgrade ends when the status changes to either failed or success.
  • If the status is reported as success, the node boots up into the new firmware 10 seconds later.
  • It then resets the values of all the three parameters to the defaults, i.e., blank.

Server Validation​

The OTA framework uses ESP x509 Certificate Bundle for server validation. This allows it to fetch the OTA image from various different servers, including the RainMaker backend (which is the default).

The OTA configuration optionally accepts a server certificate (server_cert) in case a specific server certificate is to be used, you can copy the appropriate server.crt file into your example's main/ folder and embed it into the binary by including the below line in main/CMakeLists.txt:

target_add_binary_data(${COMPONENT_TARGET} "server.crt" TEXT)

It can be accessed in your code using the following:

extern const uint8_t ota_server_cert[] asm("_binary_server_crt_start");

{
    ...
    esp_rmaker_ota_config_t ota_config = {
        .server_cert = (char *)ota_server_cert,
    };
    esp_rmaker_ota_enable(&ota_config, OTA_USING_PARAMS);
    ...

If you want to skip server authentication, or use HTTP server instead of HTTPS (not recommended), please set CONFIG_OTA_ALLOW_HTTP=y in your sdkconfig.