IoT with PYNQ: a tutorial for beginners
IoT is an acronym for the Internet of Things, it represents a new working model based on connected devices which is used in modern companies to create new revolutionary applications.
The term [1] was coined to Kevin Ashton nowadays co-founder of Auto-ID Labs who used it for the first time in 1999 in a proposal of a pioneering project to monitor the performance and yields of the assembly lines of Procter & Gamble (P&G) through the sharing of RFID devices on the network.
An indicator of being a trending topic is given by the high value of results for the entry "internet of things" in principal meta-search engines such as Google Scholar with 3,390,000 results.
Over the years the term has been used with different meanings [2] such as a paradigm, an abstract concept, to describe a specialized network, global infrastructure, and many other usages, however the following words can be considered a modern definition of IoT term :
“The Internet of Things (IoT) is a system of interrelated computing devices, mechanical and digital machines, objects, animals or people that are provided with unique identifiers and the ability to transfer data over a network without requiring human-to-human or human-to-computer interaction.”[3]
Taking into consideration the historical example of Ashton and the above definition, it is clear that IoT is the result of an organized union of different values that can be identified with the terms: devices, infrastructure and intelligence Figure 1.
The term device refers to all mechanical or non-mechanical entities capable of collecting data that will be later placed on an environment structured by an infrastructure. The third value is the intelligence required in any analytical and decisional tasks related to data handling in order to minimize human intervention for the management of the system.
In this representation as a tripartite system in which intelligence, infrastructure, and devices are the non-replaceable terms, it is possible to identify a fourth component: technological development. Indeed, the IoT remained a "paper project" until the technology allowed the evolution from isolated systems to low-cost networked intelligent devices.
According to data collected by IDC (International Data Corporation) [4] [5], 646 billion dollars were invested in 2018 with a forecast of reaching 1.05 trillion in 2022.
The movement of these capitals is justified by the fact that IoT applied to the Industrial known as IIoT is also used in ecosystems not directly developed or designed for the IoT.
One of the most important tasks is the analysis of productive asset to find the point of failure or bottleneck and more generally the fault prevention decreases logistical problems such as the asset downtime in which the study of [6] Aberdeen estimates that the hourly cost of unscheduled downtime is increased to $ 260,000 between 2014-1016.
A cutting edge aspect involving IIoT is the digital twin phenomenon which is a discovery approach based on Computer-aided design (CAD) realization or mathematical models to connect the physical and digital world for pre-production validation.
Model-based engineering has a positive influence on production costs because it allows us to generate several in-silico realizations of the object in order to select the best one to realize the physical prototypes.
With significant technological effort, this procedure is able to minimize the use of the material and simulate the final behavior even in environments that are difficult to replicate.
Among the excellences that are already using this approach, we can mention the Italian Dallara for the study and processing of carbon fiber and the American Boeing [7] which applies simulation processes to some components of its aircrafts.
After some historical and general themes, at this point in the discussion, we want to analyze more specifically the set of components necessary to build an IoT solution [8]. A recap of the required macro-categories is given in Figure 2 which indicates a "stack" of components in which the base is populated by devices and in the upper part, we can find the final application.
The devices are described with the term embedded systems and are characterized by being projected for operating in environments highly contextualized with the possibility of being connected directly or indirectly to the network.
There are two main types of devices: sensors and actuators. The sensors detect quantities around them (e.g. rotation degrees, color gradations, temperature, etc.), while the actuators are able to accept input and propagate an action (e.g. activate a retention system after a collision).
The ability to transfer information is strongly “device-based”, therefore the communication layer takes care of guaranteeing the connection to the Internet regardless of the properties of the underlying layer through communication strategies and application protocols.
The scenario in Figure 3 shows the devices (identified by D letter) that through various strategies reach the target platform. In the simplest case, the devices are equipped with modules that permit direct communication, but in other cases, it is necessary the use of intermediaries such as gateways or hubs that connect nodes that technically are incompatible. A specialization of this last point is given by the "device to device" communication typical of mesh networks in which each node is connected to another but only some elements are responsible for collecting and sending data to the IoT platform.
The presence of intermediaries makes the system more flexible but also highlights the problem of homogeneity of communication because there isn't a common standard used to convey traffic to the central unit.
At this point, we have explored the principal aspects of the bottom of the IoT stack or rather the layer most closer to the hardware sensor so we can now start considering the software layer. Generally, it is hosted in some cloud environment defined in Figure 3 with the entity called IoT platform. In this case, the term used has a very generic meaning because "platform" could be a physical machine or a distributed cloud environment on which are active in several kinds of services. The first consists of core applications, e.g. data storage, and data filter services for storing and recovering data from sensors.
The above layer is reserved for the analysis unit that can be classified into tools dedicated to big data with the ability to process a vast amount of elements and semantic analysis tools known as machine learning applications to extract models that describe and/or foresee the data trend.
The last layer linked to the cloud environment is occupied by cognitive systems capable of offering HMI Human Machine Interface who accept input as close as possible to human language such as spoken language or patterns of particular movements and generating their outputs on the base of their experience.
Finally, the component at the top of Figure 2 is the final application that can be used for a specific context or it can establish the basis for other services by itself.
The stack described above is a photograph that can be useful in many areas however, the numerous application layers introduce levels of complexity that negatively affect the performance of the final application. In fact, you can try to imagine an IIoT context with hundreds of productive assets that perform critical tasks and at the same time, they must guarantee maximum productivity with the minimization of execution latency.
In this new context, entrusting the data processing to a remote unit only is not the optimal solution and it is preferred to use an IoT edge computing strategy. This approach is based on some assets with the computational capabilities of executing some tasks that are usually performed in a cloud environment to reduce the server-side computation, improve response times and optimize the flow of data in transit.
An example of distributed intelligence in which IoT edge computing and the cloud resources are involved is AWS Greengrass [9] on the PYNQ environment [10]. AWS Greengrass extends the AWS lambda function to the edge devices in order to act on the data directly on the device and at the same time guarantee the possibility to work in offline mode.
The computational performance required for this type of task is high and can be naturally supported by workstations, however, thanks to modern systems-on-chip (SoC) it is possible to maintain high processivity with reduced consumption and form factor.
The Zynq platform of Xilinx is one of these and is the base of the PYNQ (Python on Zynq) framework schematized in Figure 4 on the right section. The components that characterize these boards are the Jupiter server, for interacting with the PL (Programmable Logic) and the overlay which is an FPGA design related to API that permits to use of hardware features by software. Finally, the boards also mount generic components that concern the management of I/O (input/output) such as PMOD which are proprietary connectors, the GPIO pins (general purpose Input/Output) consisting of buttons/switches/LEDs and the HDMI ports that represent in this case video/data inputs/outputs.
The use of a high-level language such as Python in an embedded environment permits the use of the platform in various application fields because it is possible to directly reuse standard libraries such as NumPy and others shown in Figure 4 on the left section.
An example of application is reported in note [11] in which is considered the topic of fault detection carried out with PYNQ technology for predictive monitoring of ball bearing in an asset for high precision work. For this purpose on the board, there is a machine learning model (ML) trained to describes the functioning of the component in conditions of correct usage and by the continuous comparing of the parameters generated by the model and the value detected by sensors it is possible to define the state of wear of the asset and then schedule a maintenance action.
In this discussion, we want to provide a small insight into the IoT universe by placing the emphasis on his main components and explain his impact in production processes with the theme of fault prediction.
All these aspects added value to the internet network because its role has changed from a communication tool to a central and irreplaceable role of the technology provider.
Bibliography
[1](RFID journal) Ashton, K (2009) K. Ashton, "That ‘Internet of Things’ Thing," RFiD Journal, vol. 22, pp. 97-114, 2009http://www.rfidjournal.com/articles/view?4986 [2] (some paper)
J. Gubbi, R. Buyya, S. Marusic and M. Palaniswami, "Internet of Things (IoT): A vision, architectural elements, and future directions," Future Generation Comput. Syst., vol. 29, pp. 1645-1660, 9, 2013.
H. Ning and H. Liu, "Cyber-Physical-Social Based Security Architecture for Future Internet of Things," Advanced in Internet of Things, vol. 2, pp. 1-7, 2012.
L. Butgereit, L. Coetzee and A. C. Smith, "Turn me on! using the “Internet of things” to turn things on and off," in Pervasive Computing and Applications (ICPCA), 2011 6th International Conference on, 2011, pp. 4-10.
G. R. Gonzalez, M. M. Organero and C. D. Kloos, "Early infrastructure of an internet of things in spaces for learning," in Advanced Learning Technologies, 2008. ICALT '08. Eighth IEEE International Conference on, 2008, pp. 381-383.
Yinghui Huang and Guanyu Li, "Descriptive models for internet of things," in Intelligent Control and Information Processing (ICICIP), 2010 International Conference on, 2010, pp. 483-486.
O. Vermesan, P. Friess, P. Guillemin, S. Gusmeroli, H. Sundmaeker, A. Bassi, I. Jubert, M. Mazura, M. Harrison and M. Eisenhauer10, "Internet of things strategic research roadmap," Internet of Things: Global Technological and Societal Trends, pp. 9, 2009 [3] What is IoT?
Whether it’s for your own understanding or to send to friends/family/customers, this explanation is meant for anybody.
https://medium.com/iotforall/what-is-iot-f825c7304368 [4](white paper)
The State of Industrial Internet of Things 2019: Spotlight on Operational Effectiveness. Joe Biron, CTO, Internet of Things Shawn Kelly, VP, Corporate Strategy David Immerman, Business Analyst Jon Lang, Lead Principal Business Analyst
https://www.ptc.com/en/resources/iiot/white-paper/state-of-the-iiot-ppc?cl1=IoT_General_Google_CLC-cpc_IoT-Test&cmsrc=Google&cid=7012A000001UvT4QAK&elqCampaignId=9998&utm_source=Google_Search&utm_medium=cpc&utm_content=IoT-cpc_Test&gclid=Cj0KCQjw9tbzBRDVARIsAMBplx9cjp4c4VKqK7BQGq1RGWpDuYXrTM4OsCJzajan_83yQj8BNV29aYgaAkjrEALw_wcB [6] Stat of the Week: The (Rising!) Cost of Downtime
https://www.aberdeen.com/techpro-essentials/stat-of-the-week-the-rising-cost-of-downtime/
https://blogs.gartner.com/andrew-lerner/2014/07/16/the-cost-of-downtime/ [7] Developing Airplane Systems Faster and with Higher Quality through Model-Based Engineering
https://www.boeing.com/features/innovation-quarterly/may2017/feature-technical-model-based-engineering.page [8] (book) Enterprise IoT Naveen Balani [9] AWS IoT Greengrass
https://aws.amazon.com/it/greengrass/ [10] PYNQ: Python productivity
http://www.pynq.io/ [11] The Value of Python Productivity: Extreme Edge Analytics on Xilinx Zynq Portfolio
https://www.xilinx.com/support/documentation/white_papers/wp502-python.pdf
