If you go back 25 years, Adobe took the world from a predominantly analog view of media to digital. If you go farther back, Dolby created the standard for how audio is implemented — not only in movies, but in just about every consumer device that incorporates sound.

More recently, we’ve seen standard-setters like Unity, which is the platform that powers over 70% of today’s mobile, PC, and console-based games — including Pokémon Go. Each of these companies represents a fundamentally enabling technology for their time and medium.

But at a time when people are watching more programming on more devices than ever before, there hasn’t been a fundamental technological breakthrough in how filmed content is created in over 25 years. We’ve been treading water since the advent of computer graphics to create visual effects.

That’s about to change.

Take any popular series — from Game of Thrones to House of Cards — and you’ll find that it relies bringing together the live action world (e.g. actors and people captured in a real environment) with computer generated visual effects. If you look at how this process is done today, it’s based on a technology legacy that hasn’t evolved for decades.

Directors film actors awkwardly in front of a “Green Screen”. Teams of technicians then weave in complex, computer-generated backgrounds. The process is expensive, time-consuming, and tedious. The only way to make it scale is by adding people and money.

That’s where Lytro comes in.

With its groundbreaking “Light Field” technology, Lytro is introducing a new way to capture environments that bring together the physical world and the digital world. Rather than capturing a flat, two-dimensional image through a standard camera, Lytro captures every ray of light in a frame. It can map exactly where every ray of light is coming from, including the angle, reflection points, and where its going. (Disclosure: GSV owns shares in Lytro)

With this data captured, Lytro creates a highly accurate three-dimensional models of entire environments. It makes the real world work much more like computer graphics, giving artists unprecedented design capability and flexibility.

When you take a standard picture or film a movie, creators have to make a variety of fundamental decisions up front — like frame rate, focus, and exposure. Once you make them, you’re locked in. With Lytro’s technology, this dynamic completely changes. All of those creative decisions become an after-the-fact, controllable variable, and you can render content in three dimensions. Think of it as software eating cameras and imaging. It’s a story of better, faster, and cheaper.

WATCH REMARKS FROM LYTRO CEO JASON ROSENTHAL HERE

Jason Rosenthal Discusses the Future of Imaging and Augmented/Virtual Reality

Looking ahead, Lytro will effectively sell “picks and shovels” to AR/VR “miners,” avoiding the commoditization risks borne by consumer technology companies and the cyclical risks facing content creators (e.g., movie studios, video game creators), issues that currently plague high-profile AR/VR companies such as Facebook’s Oculus and Magic Leap.

AR/VR STATE OF PLAY

Virtual Reality (VR) had been pursued and promised since the 1950s, but the convergence of low-cost mobile hardware and powerful new software platforms is bringing it to life. In less than five years we are seeing ripple effects across digital media and beyond.

Oculus VR founder Palmer Luckey, a college dropout, created a Virtual Reality (VR) “prototype” in 2011 with a smartphone, two eyeglass lenses, duct tape, and a bucket. One year later, Luckey tried to raise $250,000 on Kickstarter and raked in $2.4 million. In 2014, Facebook bought his company for $2 billion and VR had seemingly arrived.

At the time, Mark Zuckerberg observed, “Every 10 to 15 years a new major computing platform arrives — we think that virtual and augmented reality are important parts of this upcoming next platform.”

A field of technology traditionally referred to as “Virtual Reality” has bifurcated into two core technology/design approaches: Virtual Reality (VR) and Augmented Reality (AR). VR is closed and fully immersive, while AR is open and partly immersive — you can see through and around it. Whereas VR puts users inside virtual worlds, AR puts virtual things into the real world.

In recent years, AR/VR has primarily been thought of as a medium for hardcore gamers and tech evangelists, but you can increasingly see how it will impact a variety of industries, from entertainment to education, commercial real estate, and industrial design.

As the World increasingly adopts technology that enables you to experience an enhanced reality, physical experiences will increasingly be replaced by dollar apps in an app store. To this end, Goldman Sachs has recently projected that the AR/VR market may balloon to $80 billion by 2025. Not surprisingly, investment activity is following.

Investment Activity

AR/VR has traditionally been a difficult space to make venture bets. Until recently, there were really only been two options. You could invest in first generation consumer technology, which has historically moved rapidly towards commoditization. Or you could invest directly in companies creating broad content for the medium — things like video games and cinematic experiences, which tend to be highly cyclical, hit-driven, and high risk.

Today, we’re seeing companies emerge with nuanced content and business models, creating compelling investment opportunities.

Notable 2017 deals include Improbable’s “improbable” $502 million Series B financing from SoftBank, Andreessen Horowitz, and Horizons Ventures. The London-based company has developed a platform that enables third parties to develop and deploy virtual worlds, primarily for gaming applications. The company is also working with enterprise clients and governments to utilize their virtual simulation platform on a massive scale to develop detailed real-world models. Other notable deals in 2017 include 3D motion-control startup Leap Motion ($50 million Series C financing) and light-field imaging company Lytro ($60 million Series D financing).

KEY APPLICATIONS

Media & Entertainment

The first visual screen was introduced in 1920 — the television set. People gathered 15 feet behind these boxes to watch broadcasts in black and white. The PC was introduced in the 1970s, and suddenly, the screen was only three feet away. Then came the smartphone in the last decade, and now users interact with screens about six inches from their face.

There has been a consistent evolution of bringing people closer to the experience on the screen. AR/VR technology will eliminate the distance altogether. It will make consumers feel like they are fully immersed in an environment. It’s the ultimate media platform.

In the race to create entertainment platforms built on new AR/VR technology, early leaders like Facebook (Oculus VR), Sony (Project Morpheus VR), and HTC (Vive), have mainly focused on perfecting headset hardware, and then partnering with professional software developers to create premium content (e.g. Movies, Games, Training Applications, etc.). Alphabet (Google) has opted for a different approach.

In 2014, Google introduced its low-budget virtual reality platform, “Cardboard”, designed to make VR more immediately accessible through $5, easy-to-assemble kits that turn common Android smartphones into makeshift VR headsets. Subsequently, it introduced a new version for iPhones, and announced a partnership dubbed “Jump” with GoPro to bring VR content creation to the masses.

As Clay Bavor, Google’s vice president of virtual reality has said, “If you want to experience VR video, all you need is the YouTube app, your smartphone, and some cardboard.” Cardboard may not be the longterm answer, but it has certainly seeded the market.

Additionally, another crop of companies are providing the powerful VR/AR software platforms to power virtual games and worlds.

Last Summer, the launch of Pokemon Go, powered by Unity Technology’s game engine, acted as the starting gun that signaled that Virtual Reality was here to stay. In the first two months, Pokemon Go was downloaded 500 million times and was the fastest game to reach $500 million in revenue. The game quickly put the physical and digital realities in the blender and sent people into streets and parks, onto beaches, and even into cemeteries, in pursuit of Pokémon.

WATCH REMARKS FROM UNITY CEO JOHN RICCITIELLO HERE

John Riccitiello Discusses the Power of Unity’s Platform at the 2017 ASU GSV Summit

Unity’s technology not only powered Pokemon Go, but countless other games, including Super Mario. After the launch of Pokemon Go, in one quarter alone, they had 5 billion downloads and 2.4 billion unique devices running “Made with Unity” games.

Education

AR/VR isn’t just a computing platform, or even a vehicle for mind-blowing entertainment (although this is a key piece of the puzzle). It was fundamentally a new medium for acquiring information and knowledge.

Language, arguably the most crucial technological advancement in our history, transformed learning from simple mimicry and emulation into the realm of complex ideas. Expression through language accelerated innovation. At the same time, it enabled us to develop abstract ideas that evolved into a shared culture.

Writing, and later the printing press, multiplied the power of language by creating access to the best ideas for a much broader audience. An outsourced device for “remembering,” the written word also expanded our inventory of ideas, which were no longer dependent on the storage capacity of the human brain. Film, and later video and video games, were important extensions of this concept. But all are still approximations of reality.

AR/VR represents a fundamental leap forward because it has the potential to shatter physical boundaries that separate the World of ideas and experience. It is going to be a powerful platform to learn anything… everything.

Ironically, while skeptics ponder how the new technology will be applied to education, we have a long tradition of learning by doing.

Since the Wright Brothers took their first flight in 1903, for example, the aviation industry has had a basic need to simulate flight experience before putting pilots in the air. With the advancement of computer, graphics, and gaming technology, aviation simulation has come a long way since 1910, when the best training method involved sitting in a half-barrel to simulate flight pitch.

The most effective way to learn is often through hands-on experience and immersion. For one, cognitive research demonstrates conclusively that the more actively involved a person is in their educational environment (i.e. not sitting back and watching passively), the more information they retain. Intuitively, the closer you get to the “real thing,” the better you learn.

It is easier to master Mandarin if you live in Beijing. Or framed differently, how would you like to be a surgeon’s first patient after they learned their craft exclusively by watching YouTube videos — or even the best instructional videos ever made?

Learning by doing is a great idea, but the “real thing” is expensive and often impractical. Immersive education at scale was a fantasy. But Virtual Reality technology is approaching “magical” quality at mass market prices.

Top Neuroscience researcher, Dr. Adam Gazzaley (Director, Neuroscience Imaging Center, UCSF) has created a window to the future of education with his creation of Body Brain Trainer. Body Brain Trainer (BBT), is a full-body motion capture game that improves brain function by simultaneously challenging the core aspects of cognitive control — attention, working memory, multitasking — while prompting increasingly strenuous physical activity.

The game monitors the participant’s heart rate and uses adaptive algorithms to consistently ensure that players are challenged both physically and mentally. It’s scientifically proven that exercise can significantly improve brain function. By coupling that with brain training, BBT has the potential to enhance cognitive abilities more than physical or brain training alone.

Design

AR/VR will increasingly enhance and accelerate design processes, from large corporations to emerging startups. According to PwC, product design and development is the most popular application of the technology amongst manufacturers.

Ford, for example, has been using virtual reality technology to develop car designs since the year 2000. But in the last seven years, the 111-year old business has made virtual reality central to its automotive development, using Oculus technology.

Ford’s Immersion Lab is an advanced facility where automotive design professionals can don a virtual reality headset and explore various aspects of a concept car while colleagues watch what they experience on a large screen. The company’s technical design process normally begins with simple sketches, which are developed into complex designs, involving highly specialized engineering. Virtual reality can be brought in at any stage to test an idea.

Ford’s leading virtual reality specialist, Elizabeth Baron, has observed in interviews that the integration of VR into the design process has enabled the team to catch critical flaws that otherwise would have been missed and corrected later at great cost: “In the past we would have had to build a physical model to see how change would impact our design,” she said. “We are now a lot more agile in what we can understand right away, so we can understand problems that do come up a lot quicker.”

GE is meshing artificial intelligence (AI) with virtual reality to create smart models that help them streamline product design processes and while improving safety for customers.

As part of the product development lifecycle, GE is beginning to create “digital twins” — simulations of products based on sensor analytics, AI-enabled scenario analysis, and historical performance data. These twins enable GE to more accurately predict performance and wear and tear on everything from aircraft engines, to locomotives, and turbines.

VR visualizations help engineers to understand and pinpoint design flaws, as well as structural elements that can be improved. At the same time, predictive performance models can pinpoint a machine’s life expectancy. For example, an aircraft engine in the air could have a digital twin on GE’s servers that accurately tracks the health of the engine to determine the most optimal service schedule for its parts.

PLATFORM WARS

While Oculus may be a disappointment for Facebook to date given a mixed public reception and uneven sales numbers, there’s still a big belief and a lot of momentum in the idea that first Virtual Reality, and ultimately Augmented Reality, are the next major platform in computing — in the same way that the World went from mainframe to PC to client server to web to mobile. You can already see the smartest companies in the world deploying billions of dollars on this bet.

Snapchat, in particular, has pioneered commercial Augmented Reality technology in the social media space with its “selfie lenses” that can seamlessly apply everything from puppy dog ears to flower crowns over user images.

This April, Snapchat announced a new set of features that allows users to place virtual objects in the real world using a combination of advanced camera and depth sensor applications. CEO Evan Spiegel recently observed that the platform’s dancing hot dog image was the world’s, “first augmented reality superstar.” It was viewed over 1.5 billion times on Snapchat in in the second quarter of 2017 alone.

In January, Google announced partnerships with Gap and BMW who are deploying its AR app Tango to create virtual showrooms. With BMW, Google is developing an app that displays BMW vehicles on smartphone screens, allowing shoppers to walk around superimposed vehicles, manipulating, colors, trims, and even backdrops (e.g., home, garage, etc.) for additional context.

Amazon is rumored to be exploring digital stores to sell furniture and other home products. These stores will reportedly integrate augmented and virtual reality technology, enabling people to see how things like sofas, desks, ovens and more look in their homes before they make a purchase. Home design platform Houzz is already using AR to show possible room configurations on its iPhone app.

Apple officially entered the AR/VR race this Summer, announcing ARKit at WWDC. This new platform allows developers to create AR experiences for the iPhone and iPad using the device’s camera. ARKit will open up mobile augmented reality experiences beyond Pokemon Go and immersive games, potentially bridging the technology into schools, hospitals, and retailers.

Already, Furniture retailer IKEA has shown off an ARKit powered mall that superimposes furniture into homes. Just as Apple paved the way for an “App Economy” with its iOS App Store, ARKit could herald a new category of apps that incorporate augmented reality to build new consumer experiences.