In December of 2009, the U.S. Joint Forces Command called together a group of computer designers, engineers and defense-industry executives to make a single, powerful request: Remember the troops on the ground.
The group had come to discuss not weapons or firepower but the need for a new kind of virtual reality simulation—a means of dropping troops into accurately rendered computer-generated worlds where they could encounter the complex mix of languages, cultures and military threats they would soon reckon with overseas.
USJFCOM’s request drove a renewed urgency around Lockheed Martin’s efforts to develop revolutionary new simulations designed to not only change the way soldiers train, but also the way engineers design advanced systems from satellites to aircraft.
The Rise of Immersive Engineering
In the mid-2000s, engineers at Lockheed Martin’s Human Immersive Lab in Fort Worth, Texas, had begun making design changes to the company’s F-35 Joint Strike Fighter in an entirely novel way.
Instead of physically creating new modifications and add-ons for the aircraft, an expensive and labor-intensive process, engineers crafted digital parts and tweaked designs in a virtual workspace.
They strapped on virtual reality goggles, slipped on gloves with special sensors and entered their own virtual garage, where they could test the F-35’s capabilities.
In this virtual space, there was no risk of damaging expensive aircraft components and no need to create expensive physical mock-ups. And when issues were discovered, fixes could be implemented in a fraction of the time needed in the real world.
Working on just one project, the F-35, HIL engineers using immersive technology saved Lockheed Martin more than $100 million—a 15-fold return on the company’s immersive engineering investment.
Lockheed Martin Goes Hollywood
Building a virtual engineering workspace was one thing, but in order to create more detailed virtual worlds, Lockheed Martin looked to integrate motion-capture technologies with the immersive capabilities of online gaming.
On August 20, 2007, the company acquired software developer 3DSolve. The company’s founders, Richard Boyd and David Smith, had made a name for themselves working with James Cameron and Bryan DePalma in the film industry and launching a series of computer-gaming companies with marquee authors like Tom Clancy and Michael Crichton.
It would be Boyd’s job to create virtual environments in which people could safely develop different skills, from combat training and vehicle maintenance to language development and cultural awareness. Smith focused on the next generation of virtual world interfaces to include wearable portable systems.
In 2008, Lockheed Martin organized Virtual World Labs, an informal group of the company’s engineers, designers and creative talent, to brainstorm on how to use immersive technology across the company.
Few could predict in just four years what Lockheed Martin’s virtual world pioneers would achieve.
Augmented Reality
An opportunity to test the company’s bold new ideas came just a year later when Lockheed Martin won a Department of Defense contract to help develop the Infantry Immersion Trainer, an augmented reality simulator that blended the digital and physical worlds.
Small squads of soldiers, wearing 3D-glasses and sensors, roamed through a detailed Hollywood-like set in California made to resemble an Afghan town, complete with actors and props. But as the squad surveyed the physical space through their glasses, they also saw interactive holographic avatars and simulated battlefield effects, all programmed to provide the illusion of actual combat.
It was a startling success, not only in helping recruits prepare for what awaited them overseas but in proving to Lockheed Martin’s clients that virtual world simulators could have real world benefits.
Virtual Training
By January 24, 2011, the lure of immersive technologies had reached Lockheed Martin’s Space Systems Division in Littleton, Col., which debuted a dynamic Collaborative Human Immersive Lab (CHIL) in which engineers could not only tweak satellite designs but moonlight as virtual architects.
By slipping on their 3D glasses and motion sensor suits, designers could enter a virtual mockup of the division’s new production facility and make key alterations to its layout, rearranging the space between workstations and limiting the number of times expensive satellite parts had to be physically moved during testing.
The creation of a more efficient work environment led to the development of virtual training seminars, which detailed step-by-step procedures on how to properly assemble multimillion-dollar hardware.
The effects of Lockheed Martin’s immersive training technologies—which allow experts of every stripe to digitally record their movements for trainees to follow—will be felt across the company’s many divisions, including new arenas, such as healthcare, where a program called ICE STORM will allow doctors to model proper medical procedures and create “what if” scenarios for emergency room and ICU training.
The Internet Reborn
But Lockheed Martin’s most ambitious immersive technology has yet to be fully built. In response to USJFCOM’s call for new military simulations, the company began development on a vast Web-based open-sourced 3D program, named the Virtual World Framework, which will allow users to craft their own virtual environments, share information, and create lifelike training scenarios.
Unlike other proprietary training programs, Lockheed Martin proposed to build only the basic architecture of the program—a starting framework—allowing others to add new components, such as 3D models for study or weather effects in a virtual world, much like contributors add photos and knowledge to existing Wikipedia entries.
The end goal is to create a kind of parallel 3-D Internet, offering CHIL-like workspaces where engineers can modify airplane designs from their own living rooms or skilled avatars train soldiers in everything from learning new languages to laying siege to an Al Qaeda stronghold.
Awarded a contract to develop the Virtual World Framework by the Department of Defense in January 2011, Lockheed Martin produced a beta version of its kernel framework in March, moving one step closer to meeting the vision of new technology that will change the nature of virtual simulations forever.