The Guide to NXP iMX8M Plus Processor

The essence of machine learning is to enable computers to more efficiently imitate human behavior, especially in the fields of image classification, image recognition, speech recognition, etc., machine learning can concentrate on millions of calculations in seconds. In the past, at reasonable cost, such processing power was unimaginable and could not be realized until recent years. This is mainly because if these processes are not implemented at the edge, they need to be put in the cloud. Data is uploaded to the cloud, so we worry about who will see it. Do we trust the operators and leaders of the cloud service period? Obviously, the answer is no, so in the local edge computing, user privacy can be better protected, the response speed will be faster, the reliability will be stronger, because even if the server is powered off, the local device can still work stably, and even the device can work normally in remote areas without networks.

iMX8M plus with machine learning

iMX8M Plus application processor

By iMX8M Plus, NXP brings machine learning to edge devices, and it's not just about reaching 2 trillion times per second of computing power. It is not difficult to achieve such computational performance, but what NXP achieves is to bring about the optimization of model compression and sparsity, sparsity refers to the fact that there are a large number of weights of 0 in the parameters or data of the machine learning algorithm, which can avoid paying the storage fee of these model coefficients when reasoning and the computational efficiency will be greatly increased. Model compression is also important, given the large amount of data exchange during the calculation process, and if sufficient compression is sufficient, the use of system memory is minimized. Another point involved in system memory is iMX8M Plus equipped with a DRAM controller capable of 4G processing per second, the highest DRAM speed ever recorded by NXP.

iMX8M Plus with Machine Learning, Machine Vision

Machine learning is based on image data, and the source of the data is usually a camera module in a system, and the camera module requires an image processing unit, that is, an ISP. Speaking of the ISP unit of iMX8M Plus, in many systems, the camera may be very far from the target, and in order to get the details you need, the resolution must be high enough, such as 4K resolution, that is 8 million pixels.

iMX8M Plus with Machine Learning, Machine Vision

Processing such high-definition image data requires the assistance of isps. For low-resolution video, an ISP may not be required, and in many systems, the user may not realize whether the camera module integrates an ISP. But at high resolutions, the ISP needs to be integrated into the processor, just like iMX8M Plus like this. Interestingly, the NXP iMX8M Plus actually integrates two ISPs with two camera inputs. Therefore, in most application scenarios, a more three-dimensional image will be obtained, just like the two eyes of a person, which can perceive the depth of the image. Another application is two cameras, one as a wide-angle or fisheye viewing angle, and the other focusing on a close-up of a scene. In addition, there are other applications that require such features.

Image processing requires strong computing power, and the function of isps can theoretically be implemented by CPUs or GPUs, but to convert such computations into forms and parameters recognizable by the system and then calculate and process them, the amount of work involved is too large. So in such a system, a unit that specializes in processing image tasks, the ISP, must be used, just as iMX8M Plus does.

iMX8M Plus High Dynamic Range (HDR) support

iMX8M Plus also has a focus on HDR, which is high dynamic range, which you may see on many phones, but HDR is very important in embedded applications as well as consumer products and even industrial applications. HDR is primarily a fusion of multiple exposure images, which are manifested as two images with different brightness, one with a long exposure and one with a short exposure, one of which can help get better detail in the highlight part and the other to get better detail in the low-brightness area. You can even take three or more pictures to fuse them into one image. In this way, both highlight and shadow detail can be preserved, which is very important in some specific scenes, such as at home, the window is very bright, some places are very dark, with HDR, all the details you can see clearly.

iMX8M Plus High Dynamic Range (HDR) support

Distortion correction results in better images

Distortion correction is a very important feature in camera-related applications. The image received by the camera may be distorted, similar to the scene of viewing a fish tank. You can see a lot of things, but they're all distorted. Distortion correction can restore the image to a normal shape that humans and machine learning can recognize, and the straight curve is clearly distinguished and there is no inversion, which is what the distortion correction engine is responsible for. But the engine can do more than that, in addition to being able to correct these unexpected interferences, it can also correct the image is not clear due to camera problems, distortion and so on.

The engine can also process pixels for better image quality. For iMX8M Plus' ISP architecture is designed to ensure that all processing takes place in the form of a stream, which means that the ISP receives images directly from the camera for processing and outputs them to system memory. ISPs rarely need to rebuild images from the system before processing the output, resulting in very low latency performance and very efficient image processing. iSPs do a lot of processing for image enhancement, such as noise reduction, which is very important in low-light environments. In this environment, many pixels will be smeared, but you can't paint too many areas, otherwise, your image will become very blurry. So you have to use your ISP to tell where it's okay to smudge and where it's going to be clearly presented. Other processing tasks include detecting dead pixels in the image sensor to ensure proper operation, as well as adjusting the white balance to distinguish between different lighting environments, such as daylight, light, and so on. There's also information to help with auto exposure and focus, all of which is done by ISP.

H.265 video encoding

In many applications, cameras are plugged into systems and data is fed into machine learning algorithms or stored directly on the network. No matter where images are sent, they still need to be stored in some place, such as a server or a local device. It is impossible to save the original file directly, because the original file is so large that the data must be encoded, and the video encoding of iMX8M plus is very efficient. We support the h.265 encoding algorithm, which is more advanced and efficient than h.264, which is widely used in the industry.

H.265 video encoding

iMX8M Plus's performance in industrial applications

iMX8M Plus has some features for industrial applications, one of which is ECC, that is the error correction code. It is used to detect memory errors and even repair them without the user's perception.

This is super important because memory errors in some applications can have serious consequences, and while they do not cause permanent damage to the hardware, they can crash when reading memory software and programs.

To avoid these situations, you need ECC to detect and fix them, so that the error rate of the software, which we often call the SER, will be greatly reduced. iMX8M Plus comes with ECC in both cache and on-chip RAM, while the DRAM bus also supports online ECC, which does not require any additional bits to be stored to send the ECC algorithm and the master data.

TSN is also the highlight of iMX8M plus, which refers to time-sensitive networks. In industrial applications you want to be able to ensure that data can be sent out at the right point in time, whereas usually anyone in Ethernet is likely to be sending data at any time. It may cause congestion, and you may need to send it again later, which in industrial applications will undoubtedly be problematic. TSN is a good solution to this problem. iMX8M Plus also supports some industrial-grade peripherals, and we even integrated an 800MHz Cortex-M7 coprocessor to assist with the mixing. Running the operating system in combination, M7 core can also perform some real-time control to ensure low latency. We also have two CANFDs for low-latency communication between the local and the system or with the network.

All of these features and peripherals offered by the iMX8M Plus enable industrial and application design to open up endless possibilities for industrial applications.

How to evaluate the iMX8M Plus

As the official gold partner of NXP, Forlinx Embedded has cooperated with NXP in depth and launched an iMX8M Plus processor development kit OKMX8MP-C development board and FETMX8MP-C SoM, in addition to inheriting the above CPU capabilities, Forlinx Embedded in iMX8M Plus SoM also integrates LPDDR4, eMMC, PMIC and other key components, forming a minimum system that can operate independently; at the same time, considering the different needs of low-cost basic applications and high-end NPU applications, DDR can choose 2GB, 4GB, 6GB, 8GB multiple versions; the advantage of this core board is not only in performance, its functions are also very rich, dual Gigabit Ethernet, dual CAN-FD, 2 USB3.0, PCIe3.0, LVDS, HDMI, MIPI-DSI, MIPI-CSI and other interfaces are all-encompassing.

imx8m plus system on module