Opinion: the balance between edge and cloud.

Share on linkedin
Share on twitter
Share on facebook
Share on reddit
Share on digg
Share on email

Simon Forrest explains how embedded chips can meet the challenge of delivering true local AI processing.

GPUs and NNAs are rapidly becoming essential elements for AI on the edge. As companies begin to harness the potential of using neural networks for various tasks such as natural language processing through to image classification, the number of products introducing some element of artificial intelligence is steadily increasing. Meanwhile, the balance of processing for these tasks is migrating from traditional cloud-based architectures into the device itself, with dedicated hardware-based neural network accelerators now embedded into the silicon chips that enable local AI processing. From advanced driver-assistance systems (ADAS) monitoring the road ahead through to voice-activated consumer electronics products such as virtual assistants, the opportunity for integrated neural network-based AI is expanding across several market segments.

Imagination’s business is in supplying essential core building blocks for silicon chips.  We’re known predominantly for our embedded graphics (GPU) and neural network accelerator (NNA) technologies, licensing these to the world’s leading silicon vendors.  Their silicon chips are deployed widely across multiple products and services, therefore providing Imagination with a unique position in the market, because we enable entire ecosystems to participate in AI.

Undeniably, AI is now considered to be vitally important in many applications. But there are many challenges. One of these is balancing the processing load between edge and cloud and finding where best to place the AI inferencing task itself. For example, edge AI is used for local speech recognition on consumer devices; for example, identifying “wake words” or simple instructions, but it’s then necessary to achieve a large proportion of voice AI processing in the cloud in order to tap into the vast knowledge stores that simply cannot be stored locally on the devices themselves. The upshot is that many products today are being marketed with AI capability, but in fact, are only doing simple pattern matching and recognition locally, before relying on the cloud to deliver the impression of intelligence.

This situation will change. As silicon processes shrink further and embedded neural network accelerators (NNAs) become almost as ubiquitous as CPUs, this creates opportunity to increase AI processing capability at the edge. For example, expect to see smart security cameras proficient in monitoring for specific events. No longer limited to simply recording video, these will use edge AI processing to identify features within the field of view, such as vehicles on a road or faces within a crowd. This will trigger specific activities such as identifying the make and model of vehicles or enabling access to an authorized individual. The output may not be a recognisable video feed; instead it could simply be streams of metadata describing those activities. Embedding AI into security cameras will even save costs by reducing the number of “false positives” because edge AI within the cameras themselves can identify the difference between normal and suspicious behavior.

Although the number of applications for AI is increasing, this doesn’t necessarily mean that individual SoCs with integrated neural inference engines are the way forward for all scenarios. If we are to consider AI reaching across most market segments, fragmentation will naturally occur due to the fact that products using the technology have vastly different processing requirements. Fragmented markets are difficult to serve with generic application processors, such as those with integrated NNA and GPU; indeed a ‘one size fits all’ approach isn’t always applicable.

While some markets promise high-volume opportunities for SoC vendors, such as mobile phones or automotive ADAS, many markets targeting the use of AI will naturally present as low-volume prospects. Notably some products might require AI for voice processing or image recognition, but not both: a smart lighting vendor is unlikely to use an SoC originally designed for smartphones simply to introduce AI capabilities into their application; it cannot be cost-effective. The solution to this conundrum is to create specialized AI chips which are used alongside the main application processor as a companion chip. These offload the AI inference tasks that would normally be handled by an NNA core on the main application processor itself. This offers distinct advantages: SoC vendors can provide a range of edge AI chips with different performance levels; additionally, OEMs are afforded several options to scale product solutions appropriately, dependent upon the AI processing overheads they expect to have to handle within their specific application.

AI and car

So where is the AI market heading? In 2019 I expect interest in and demand for AI to continue; indeed, the technologies underpinning this will begin to mature. Conversely there will almost certainly be a realisation that AI isn’t the answer to everything, the hype will probably disappear somewhat, and many companies will shift focus. They will harness the potential of AI to enhance system capabilities, but AI may not necessarily remain central to the operation of those systems.

Further out, true AI – where machines have awareness and can take decisions based upon cognitive reasoning – is still a decade or more away. This implies that cloud connectivity will remain crucial for many years, affording access not just to the massively parallel compute resource necessary – possibly via quantum machines – but also the immense knowledge store and databases that AI relies upon to make sense of the world around it. New communication technologies promising higher bandwidth are on the immediate horizon for 2019, notably 5G and 802.11ax, so expect cloud AI architectures and the connectivity bandwidth to scale accordingly.

PowerVR Series2NX architecture

For true AI at the edge, we shall need to conceive innovative methods to improve the packing density of transistors onto silicon chips, and entire new ways of constructing SoCs that have both the capacity to acquire knowledge through learning alongside the necessary reasoning skills to adapt.

Imagination designs essential core technologies for silicon vendors wishing to build world-leading edge AI solutions. PowerVR GPUs provide high-performance GPU-compute capabilities necessary for processing visual elements in AI, such as image recognition and sorting, gesture driven interfaces, or live video analytics. PowerVR NNAs (neural network accelerators) create the heart of any leading edge AI solution, supplying the requisite hardware acceleration for advanced inferencing and edge data processing. When combined in silicon chips, our GPU and NNA technologies provide everything necessary for high-performance AI processing at the edge.

The future of AI is becoming clear… but don’t be surprised when it takes longer than anyone anticipates to reach the destination.

Simon Forrest

Simon Forrest

A graduate in Computer Science from the University of York, Simon possesses over 20 years’ experience in broadcast television, radio and broadband technologies and is author of several patents in this field. Prior to joining Imagination, Simon held the position of Chief Technologist within Pace plc.

Please leave a comment below

Comment policy: We love comments and appreciate the time that readers spend to share ideas and give feedback. However, all comments are manually moderated and those deemed to be spam or solely promotional will be deleted. We respect your privacy and will not publish your personal details.

Blog Contact

If you have any enquiries regarding any of our blog posts, please contact:

United Kingdom

Tel: +44 (0)1923 260 511

Search by Tag

Search by Author

Related blog articles

building shot

Imagination: Now and Beyond

Last year I was privileged to join Imagination as its chief executive officer and was delighted to join a company that has not only a distinguished history, but also incredible potential. With over 25 years of experience in designing and licensing market-leading IP processor solutions, we are today found in over 30% of the world’s mobile phones and, in total, over 11bn devices globally. Imagination is widely acknowledged as a company that creates innovative, high-quality technologies that solve complex problems for its customers. Across compute, graphics, and AI, it is razor-focused on delivering IP that provides high-performance and low power consumption in the smallest silicon area, thus empowering its partners to succeed, be it in mobile, consumer, automotive or other markets.

Read More »
ai edge

Reduced Operation Set Computing (ROSC) for Flexible, Future-Proof, High-Performance Inference

Designers of neural network accelerator (NNA) IP have a Herculean task on their hands: making sure that their product is sufficiently general to apply to a very wide range of current and future applications, whilst guaranteeing high performance. In the mobile, automotive, data centre and embedded spaces targeted by Imagination’s cutting-edge IMG Series4 NNAs, there even more stringent constraints on bandwidth, area and power consumption. The engineers at Imagination have found innovative ways to address these daunting challenges and deliver ultra-high-performance and future-proof IP.

Read More »
James Liu keynote

Imagination China sees 2020 out in award-winning style with IMG Series 4 NNA

Earlier this month Imagination Technologies picked up two awards for its recently launch IMG Series 4 neural network accelerator (NNA). Our NNA is no stranger to awards, with the design team behind the Series2 NX and the NNA itself picking up gongs for its groundbreaking performance. The Series 4 takes things to new levels, with its scalable architecture offering 500 TOPS of performance, and even beyond.

Read More »


Sign up to receive the latest news and product updates from Imagination straight to your inbox.