Making surround-view for cars smarter and safer

Share on linkedin
Share on twitter
Share on facebook
Share on reddit
Share on digg
Share on email

With modern advancements in automotive safety, we are seeing advanced driving assistance systems (ADAS) such as surround-view become the new standard in modern cars. This demonstration shows how this application can be enhanced with the addition of accurate real-time reflections and AI to detect hazards that make cars safer. When this is combined with the new safety mechanisms in our new XS GPU family, it is clear why our GPUs are considered the safest in the world.

First, let’s start with the graphics problem. Have you ever looked at an existing surround-view system and thought “Why does the car look like something out of a PS1 game? – i.e. flat, dull shading. Why doesn’t it look shiny and cool and more real – and why can’t I see the surroundings?”

stock surround-view

Rest assured, we have a created an innovative solution that pushes the envelope of both visual quality and accuracy for an automotive application. Below is a screenshot from our smart surround-view demonstration that shows soft lighting and real-time reflections mapped onto the car from the surrounding environment, features which are currently non-existent in any surround-view application on the market.

To be clear, what this means is that not only does the driver sees a high-quality realistic 3D model of the vehicle, but its actual surroundings are mapped onto the car – in real time. As such, the driver gets accurate feedback about their surroundings, making the surround-view feature even safer and more capable.

surround-view 3D model

How is this all done I hear you ask? First, the car rig captures four feeds from the cameras simultaneously, stitches them together and maps them to a hemisphere. This provides a top-down view of the car and the surroundings, which enables the driver to park their car safely and effectively. We are then able to use this hemisphere to calculate real-time reflections, that adapt to the car’s surroundings and map them onto the model of the car for an extra layer of safety, whilst truly immersing the car and driver within the environment.

surround view bubble

The application in this demo runs on a relatively modest PowerVR GX6650 GPU inside the Renesas R-Car H3. When you pair this up with our dedicated neural-network accelerator (NNA), the PowerVR Series2NX AX2185, you start to see the benefits of our “AI Synergy”, where we offload complex neural network processing from the GPU onto our NNA, whilst increasing the compute utilisation of the GPU.

surround-view demo photo

This car rendering and camera stitching are performed on the GPU. A snapshot of the environment hemisphere is passed to the NNA, which then processes the image and detects objects and hazards. Once the detection pass has been completed the detected hazards are then fed back to the R-Car H3 and are composited on top of the hemisphere.

surround-view demo photo2
The hazards are detected using a neural network which is trained on a specific image set – in this case, we are using the GoogleNet Single Shot Detector (SSD) to detect people, cars and many other objects. This is only scratching the surface of what could be achieved on the AI side as we edge closer to seeing autonomous vehicles becoming the new standard. Neural networks will be pivotal in this as they can be used to enhance a range of ADAS systems such as driver monitoring and road-sign recognition.

With the introduction of our XS family of GPU for automotive, Imagination offers a range of functionally-safe GPUs with unique safety mechanisms that can be combined with powerful NNAs to deliver faster smarter applications across the board.

Let us know what you think of this demo in the comments and to keep up to date with everything related to Imagination you can follow us on Twitter @ImaginationTech@PowerVRInsiderFacebook, and LinkedIn.

Mike Barnes

Mike Barnes

Michael manages the Demo Engineering team at Imagination. Before joining Imagination in 2014 he gained experience in the games industry as a 3D artist. He has a keen interest in all things relating to graphics and AI.

Please leave a comment below

Comment policy: We love comments and appreciate the time that readers spend to share ideas and give feedback. However, all comments are manually moderated and those deemed to be spam or solely promotional will be deleted. We respect your privacy and will not publish your personal details.

Blog Contact

If you have any enquiries regarding any of our blog posts, please contact:

United Kingdom

[email protected]
Tel: +44 (0)1923 260 511

Search by Tag

Search by Author

Related blog articles

pvr automotive car electric

How Imagination is steering the automotive industry to a safer future

Imagination has been supplying its GPU IP into the automotive sector for over 15 years and has the number one spot for market share for GPUs for application processors. As such is has silicon-proven experience assisting automotive OEMs to build SoCs that conform to the requirement for ASIL B or higher, while also providing the long-term support that is demanded by the industry. This last point is critical. We continue to offer support to solutions that were introduced into the market 10 years ago, as these are still in cars on the road today, and customers can rest assured we will be there in long into the future to support our current IP as well.

Read More »
Neural network accelerator (NNA) for the automotive industry

Imagination and Humanising Autonomy Part 1: The path to safer roads

Welcome to the first in a series of articles where we explore how Imagination and Humanising Autonomy, a UK-based behaviour AI company, are teaming up to deliver practical, real-world AI-driven active safety. We have talked at length about the steps that Imagination takes every day to make our world safer with automotive AI. Today we see advanced driver assistance systems (ADAS) that have AI at their core. From simple, dynamic satnav route planning, through lane keep assist, to full autonomous driving stacks, drivers are gradually becoming equipped with more ways than ever to take the stress out of their journeys.

Read More »

Meet David Higham: Focusing on functional safety

The ISO 26262 Digital Conference is a not-for-profit functional safety event organised by safety practitioners heavily involved with ISO 26262 and is designed to provide opportunities for those in the industry to network, share knowledge and engage in discussions. For obvious reasons, the event this year is taking place entirely online and will run between 24th – 25th March 2021.

Read More »


Sign up to receive the latest news and product updates from Imagination straight to your inbox.