Google Describes A New Process For Creating 3D Models From 2D Images

As the internet has evolved, and connectivity with it, visuals have increasingly become the key element that stands out and grabs users’ attention in ever-busy social feeds.

It started with static images, then moved to GIFs, and now video is the most engaging type of content. But in essence, you really need attractive and interesting visuals to stop people halfway through, which for the most part is far more effective than trying to catch them with a title or a witty line.

That’s why it’s interesting – today Google introduced its latest 3D image creation process called ‘LOLNeRF’ (yes, really), which is capable of Accurately estimate 3D structure from single 2D images.

– Advertising –

As you can see in these examples, the LOLNeRF can take your usual 2D image and turn it into a 3D display.

What Facebook has also offered a version of for some time, but the new LOLNeRF process is a much more advanced model, allowing for more depth and interactivity, without the need to understand and capture full 3D models.

As explained by Google (in English only:

In “LOLNeRF: Learn from One Look”, we propose a framework that learns to model 3D structure and appearance from collections of single view Pictures. LOLNeRF learns the typical 3D structure of a class of objects, such as cars, human faces or cats, but only from unique views of a single object, never the same object twice.

The process is able to simulate color and density for every point in 3D space, using visual “cues” in the image, based on machine learning – essentially replicating what the system knows from similar pictures.

“Each of these 2D predictions corresponds to a semantically consistent point on the object (for example, the tip of the nose or the corners of the eyes). We can then derive a set of canonical 3D locations for the semantic points, along with estimates of the camera poses for each frame, so that the projection of the canonical points in the images is as consistent as possible with the 2D landmarks.

From there, the process is able to render more accurate multi-dimensional visuals from a single, static source, which could have a range of applications, from AR art to creating extended objects in VR, and future metaverse space.

Indeed, if this process is able to accurately create 3D representations of a wide range of 2D images, it could dramatically speed up the development of 3D objects to help build metaverse worlds. The concept of the metaverse is that it will be able to facilitate virtually any real-life interaction and experience, but to do so it needs 3D models of real-world objects, across the spectrum, as source material for feed this new creative approach.

What if you could just feed a catalog of web images into a system and then have it spit out 3D equivalents, for use in ads, promotions, interactive experiences, and more?

There are a range of ways this could be used, and it will be interesting to see if Google is able to translate the LOLNerf process into more convenient and accessible usage options for its own AR and VR ambitions.

Leave a Comment