The Evolution of Edge AI and Cloud Computing

Before 2019, most IoT systems consisted of ultra-low-power wireless sensor nodes, usually battery-powered, providing sensing capabilities.

Their main purpose was to send telemetry data to the cloud for big data processing. As IoT becomes the new buzzword and market trend, almost every company is doing so to achieve a proof of concept (PoC). Cloud service providers have beautiful dashboards presenting data in attractive charts to help support PoC. The main reason for PoC is to convince stakeholders to invest in IoT and prove the return on investment to fund larger projects.

As this ecosystem expands, it is clear that there is the potential to send too much data back and forth through the cloud. This can clog bandwidth pipelines and make it difficult to get data in and out of the cloud quickly. It can also create at least annoying latency and, in extreme cases, may disrupt applications that need guaranteed throughput.

Despite the significant improvements in bandwidth and transmission speeds promised by standards such as 5G and Wi-Fi 6E, the number of IoT nodes communicating with the cloud has exploded. In addition to the sheer number of devices, the cost is increasing. Early IoT infrastructure and platform investments need to be monetized, and as more nodes are added, the infrastructure needs to be scalable and profitable.

Around 2019, the idea of edge computing became a popular solution. Edge computing enables more advanced processing in the local sensor network. This minimizes the amount of data that must reach the cloud through the gateway and back. This directly reduces costs and frees up bandwidth for other nodes when needed. Fewer data transfers per node also can reduce the number of gateways needed to collect and transfer data to the cloud.

Another technology trend that is enhancing edge computing is artificial intelligence (AI). Early AI services were primarily cloud-based. As innovation and algorithms have become more efficient, AI has moved very quickly to the end node, and its use is becoming standard practice. A notable example is the Amazon Alexa voice assistant. Detecting and waking up after hearing the trigger word "Alexa" is a common use of edge AI. In this case, the trigger word detection is done locally in the system's microcontroller (MCU). After a successful trigger, the rest of the command goes to the cloud over a Wi-Fi network, where the most demanding AI processing is done. In this way, wake-up latency is minimized for the best user experience.

In addition to solving bandwidth and cost issues, edge AI processing brings additional benefits to applications. For example, small sensors can be added to a motor in predictive maintenance to measure temperature and vibration. A trained AI model can effectively predict when a motor has or will have bearing damage or overload conditions. This early warning is critical to repairing the motor before it fails. This predictive maintenance significantly reduces line downtime because the equipment is proactively repaired before it fails. This provides tremendous cost savings and minimal loss of efficiency. As Benjamin Franklin said, "An ounce of prevention is worth a pound of cure."

As more sensors are added, the gateway may be inundated with telemetry data from the local sensor network. In this case, two options exist to alleviate this data and network congestion. More gateways can be added, or more edge processing can be pushed to the end nodes.

The idea of pushing more processing to end nodes (usually sensors) is underway and rapidly gaining momentum. End nodes typically run at power in the mW range and sleep at power in the µW range most of the time. Due to the end nodes' low power consumption and cost requirements, they also have limited processing power. In other words, they have very limited resources.

For example, a typical sensor node could be controlled by an MCU as simple as an 8-bit processor with 64 kB of flash and 8 kB of RAM, and a clock speed of about 20 MHz. Alternatively, the MCU may be as complex as an Arm Cortex-M4F processor with 2 MB of flash memory and 512 kB of RAM, and a clock speed of approximately 200 MHz.

Adding edge processing to resource-constrained end-node devices is challenging and requires innovation and optimization at both the hardware and software levels. Nevertheless, since the end nodes will be in the system anyway, adding as much edge processing power is economical.

As a summary of the evolution of edge processing, it is clear that end nodes will continue to become smarter. Still, they must also continue to respect their low resource requirements for cost and power consumption. Edge processing will continue to be popular, as will cloud processing. The option to allocate functionality to the right place allows systems to be optimized for each application and ensures the best performance and lowest cost. Efficient allocation of hardware and software resources is key to balancing competing performance and cost goals. The right balance minimizes data transfer to the cloud, minimizes the number of gateways, and adds as much functionality as possible to the sensors or end nodes.

Conclusion

The IoT is changing and will continue to be optimized for massive and cost-effective scaling. New connectivity technologies are constantly being developed to help address power, bandwidth, and capacity issues. Artificial intelligence continues to evolve and become more capable and efficient, enabling it to move to the edge and even to end nodes. The IoT is growing and adapting to reflect continued growth and prepare for future growth.