Augmented reality company Niantic Inc. today announced the general availability of the first set of AR tools from the Niantic Lightship platform, which will enable developers from around the world to build the real-world metaverse.

Niantic also announced the formation of Niantic Ventures to invest in and partner with companies that will build the future of AR experiences. The fund will open with an initial $20 million and it will focus on companies building applications looking to build the real-world metaverse and the global ecosystem of Niantic apps.

“From the earliest days of Niantic, our goal has been to create augmented reality experiences that can be enjoyed by millions of people around the world, and the platform that can power them,” said John Hanke, founder and chief executive of Niantic.

The Lightship Platform acts at the foundation of all Niantic’s products and represents the company’s long experience building AR experiences and apps such as “Ingress,” “Pokémon Go” and the recently released “Pikmin Bloom.”

The developer tools released for Lightship will be available as part of the Augmented Reality Developer Kit at Lightship.dev. It is a cross-platform software development kit designed to run seamlessly on both Android and iOS devices and provides three AR features: real-time mapping through meshing, scene understanding through semantic segmentation and social sharing through multiplayer experiences.

Real-time mapping makes it possible for virtual objects to properly appear as if they’re mapped into the real world. In most AR use-cases this means having them appear as if a character or object is sitting on a surface. But what should happen if a virtual object falls behind a real-world object? Using depth-finding technology and smartphone cameras, objects can bounce off walls, roll off tables or hide behind the couch.

Semantic segmentation allows developers to write apps that can understand the world by understanding the difference between parts of the real-world environment such as the ground, sky, water, buildings and more. As a result, virtual objects can properly interact with them. For example, using segmentation, a developer could have a blimp hovering in the sky that would be blocked by buildings as it floated past.

Finally, because gaming and remote collaboration has become an extremely important part of AR experiences, Niantic wanted multiuser experiences to be as easy as possible for developers. As a result, ARDK comes with a toolset to make that possible with both games and collaborative apps feel realistic and seamless, thus keeping virtual content and user interactions in sync, all in real time.

“Transforming humanity’s relationship with technology by merging the physical and virtual worlds will require the ideas and perspectives of as many people as possible,” said Hanke.

Several apps are already being built using the Lightship ARDK launching in 2022. These apps include a display of the Tower of London from the Historic Royal Palaces, which filled its moat with flowers for an AR experience. Coachella, the music and arts festival, is putting Lightship and semantic segmentation to work to allow its rainbow butterfly to land visually in large scale atop the Spectra tower, a seven-story rainbow walkway tower that debut in 2018.

“AR enables us to engage with Coachella fans in ways unlike ever before,” said Sam Schoonover, lead of Coachella Innovation. “At the festival, attendees will be able to experience the show in new, interactive ways. Globally, fans will be able to immerse themselves in Coachella no matter where they live. Our vision is a future where anyone in the world can experience Coachella from anywhere and AR tools from Niantic give us the unique ability to make that happen.”

The Science Museum Group is also using ARDK to make science education more engaging and interactive outside its walls. With its app, the museum group will connect and educate people on science subjects through AR starting in May 2022.

The core interfaces of the Lightship ARDK are currently free for developers, including semantic segmentation and meshing, no matter how many people use their apps. The multiplayer interface is free for apps with fewer than 50,000 monthly active users, but a fee applies to apps above that amount.

All these pricing terms apply for developers who sign up for the Lightship ARDK before May, 2022. More details about Lightship, the ARDK and pricing are available at Lightship.dev.

Image: Niantic