Creating Unreal Engine 360° panoramas the easy way with ray tracing

Share on linkedin
Share on twitter
Share on facebook
Share on reddit
Share on digg
Share on email

360° VR panorama videos have been trending on YouTube since the viewing feature was introduced in 2015. Even without a VR headset they offer a uniquely immersive experience. What I’m referring to are mostly live action movies shot with a growing list of 360° video capable cameras. These panoramas are actually two 360×180 equirectangular 2D images stitched together and projected onto a conceptual 3D sphere.

They are often labelled as 3D or VR but it’s not really accurate. Virtual reality (VR) is currently described as a computer-generated world with some level of interactivity. However, with live action panoramas you are a passive viewer rather than an interactive participant. The ambiguity is compounded as VR headsets are the optimal way of viewing these 360° videos.

Some also refer to 360° videos as 3D since you can look all around, either by spinning the sphere or moving a mobile device. They should not be confused with 3D content shown on theatrical screenings or 3D capable televisions, which are viewed with special glasses. Monoscopic panoramas provide no depth information. Stereoscopic panoramas do provide some depth perception but must be viewed with a VR headset.

However, in both cases panoramas add much entertainment value by giving each viewer the potential for a unique experience. The possible applications range from computer-generated animated feature films, to architectural fly-by renderings of yet to be built structures.

So that’s a long preamble to this without the words ray tracing. This post is about creating 360° animated panoramas the easy way using the Unreal Engine and Imagination’s Wizard GR6500 ray tracing hardware. But why is creating these panoramas so much easier with ray tracing?

Creating panoramas with a rasterizer

The Unreal Engine is considered to be a rasterizer. The difference between a rasterizer and a ray tracer is that with the former the geometry dictates which screen pixel is affected, but with the latter, the pixel decides which geometry affects its colour. Due to the limitations of a rasterizer, there are currently two ways to create 360° panorama stills or video sequences with the Unreal Engine:

  • Render six sides of a cube into a cubemap by rotating a 90° field of view camera in a central location of the scene. The cubemap image is then converted to an equirectangular spherical 360° panorama with some well-defined projection methods essentially, projecting or shrink wrapping the cube onto a sphere. This method is fast but can result in stretching artefacts at the cube corners. An interesting discussion on this method can be found on the Unreal Engine answerhub.
  • Render many sub panels of the full 360° by rotating a camera with a fixed field of view in a central location. You then stitch each panel together to create the 360° panorama. This method is more accurate but can be very time consuming depending on the required final resolution of the panorama. A detailed description of the process using render “slices” can be found on the Unreal Engine blog.

Creating panoramas with a ray tracer

With Imagination’s implementation of ray tracing in the Unreal Engine the panoramas are created in one single render. Using a lens shader, which applies a spherical projection establishing the camera direction for each pixel, we can create a complete 360° flat panorama image. Of course, not only do you get a 360° image or video in one easy step, you also get physically correct reflections, refractions and shadows. This could also be applied in creating stereoscopic panoramas by appropriately shifting the camera direction for each eye.

With Imagination’s implementation of ray tracing in the Unreal Engine the panoramas are created in one single render. Using a lens shader, which applies a spherical projection establishing the camera direction for each pixel, we can create a complete 360° flat panorama image. Of course, not only do you get a 360° image or video in one easy step, but you also get physically correct reflections, refractions and shadows. This could also be applied in creating stereoscopic panoramas by appropriately shifting the camera direction for each eye.

Creating panoramas in Unreal can be difficult at first, but we’ve embedded two in-depth videos to show you how to make a flat panorama in the engine via The Chairs and Table project, a well-recognised tutorial for learning Unreal.

Find out how to create 360° panoramas with our ray tracing technology below..

This movie is the flat panorama before it is wrapped on a sphere:

This movie is the panorama with the player in 360° mode:​

Helping You Make The Most of Unreal Engine Ray Tracing

In early March at GDC 2017 in San Francisco, Imagination demonstrated the implementation of ray tracing in the Unreal Engine running in real-time on our Wizard GR6500 development hardware platform. The demo, based on the Vulkan API, enables you to toggle between several modes, including raster, ray traced and panorama. We have another blog post where you can read a full, in-depth discussion of the making of the demo and the challenges involved.

Will Anielewicz

Will Anielewicz

Will has been playing with computers since 1967. After completing an Honors Computer Science degree at York University in 1974, he was a multi-discipline Masters candidate combining Fine Arts, Computer Science and Philosophy. The mandate was to create a computer graphics piano that could be used in live concert performances. In 1976 Will had one of the world's first exhibitions of computer graphic art, at a Toronto Canada art gallery. In 1982 Will was hired as the first employee of Alias Research the creator of Maya. Since then, Will has worked on 14 feature films (such as "Star Wars: The Phantom Menace") and several award-winning commercials.

1 thought on “Creating Unreal Engine 360° panoramas the easy way with ray tracing”

Please leave a comment below

Comment policy: We love comments and appreciate the time that readers spend to share ideas and give feedback. However, all comments are manually moderated and those deemed to be spam or solely promotional will be deleted. We respect your privacy and will not publish your personal details.

Blog Contact

If you have any enquiries regarding any of our blog posts, please contact:

United Kingdom

benny.har-even@imgtec.com
Tel: +44 (0)1923 260 511

Search by Tag

Search by Author

Related blog articles

b series imgic technology hero banner

PowerVR SDK and Tools 21.1 Out Now – including ray tracing simulator

The sun is starting to come out, people are returning to society, and potentially most exciting; we at DevTech are releasing a new version of the PowerVR SDK and Tools. The PowerVR SDK contains not only our graphics development framework, but also a handful of example applications that show you not only how to implement certain techniques, but also make the most out of your PowerVR platforms. Between the modules that help you deploy cross-platform or libraries that let you focus on the actual code rather than boilerplate setup; our SDK has everything you need to hit the ground running with making a high-performance application. We’re starting with the SDK, rather than the tools this time, because this first talking point is a pretty big one, we think. 

Read More »
ray tracing Detail Lighting

Imagination collects 5G Cloud Gaming Industry Alliance award for ray tracing

Imagination pioneered hardware-based ray tracing technology several years ago with a focus on an ideal blend of performance, quality power efficiency and the IP is available for licensing. Its efforts have been recognised by the 5G Cloud Gaming Industry Alliance (5GCGA), a new cross-industry collective that aims to develop the 5G cloud game industry worldwide.

Read More »
android background

The Android Invasion: Imagination GPU IP buddies up with Google-powered devices

Google Android continues to have the lion share of the mobile market, powering around 75% of all smartphones and tablets, making it the most used operating system in the world. Imagination’s PowerVR architecture-based IP and the Android OS are bedfellows, with a host of devices based on Android coming to market all the time. Here we list a few that have appeared in Q4 2020.

Read More »

Connect

Sign up to receive the latest news and product updates from Imagination straight to your inbox.