Applying Extended Reality to Control Environments

Most technical advances happen iteratively. A new-generation jetliner is a bit more fuel-efficient than its predecessor. Your new PC processes data faster than the one you’re about to recycle. But every once in a while, circumstances align in such a way that evolution doesn’t just inch ahead, it leaps forward. “This is where we are with command and control facilities. The convergence of several key technological advances and societal trends is ushering in a new era of information management,” says Jeff LaFrenz of software developer VizworX.

He should know. His company has decades of experience designing next-generation industrial solutions, including mission control room interfaces. Together with audio visual and control room specialists Mechdyne Corporation, the two organizations can deliver comprehensive control room solutions for centralized, remote, and field operations.

To appreciate what’s about to happen, explains LaFrenz, “Consider first what HASN’T happened to command and control facilities over the last thirty-or-so years. Even as computing and communication technologies have been utterly transformed, the operational centers of our utilities, public safety facilities, and military look pretty much the same. They’re big rooms with lots of screens.

They’re still massively expensive to build, and as costly as ever to maintain – requiring regular technology overhauls that keep the screens lit, but do little to advance overall functionality.”

Most command centers still look like they did in movies from decades past because until now they’ve been bound by familiar constraints. Challenge number one: limitations of space. “There are only so many screens you can cram into an existing environment, and effectively sharing that information remotely can be equally challenging,” explains Chad Kickbush of Mechdyne.

“But where we really run into a wall,” adds LaFrenz, “is in our heads. Traditional command and control centers feed us information in a manner that our brains struggle to process.”

He’s referring to a phenomenon called “cognitive loading.” We see the world in 3D, and we process information most effectively in three dimensions. But most information relevant to command and control is delivered to us in 2D, via graphs, charts, and other data delivery formats. The time and energy we expend to “cognitively load” 2D information into a 3D brain creates fatigue and strain, slowing our ability to act on the information we’re receiving.

The events of the past few years add an element of urgency to the task of rethinking command and control. From travel restrictions to room capacity controls and individual health concerns, a host of factors can reduce the availability of needed experts. Consequently, remote participation is now an expected part of any solution. That fact only underscores the importance of transforming the way complex information is communicated.

That was then. Here's what's happening right now.

We can’t alter the way our brains work, but we can change the way information is delivered to them. And that’s precisely what’s happening, thanks to the convergence of several technology trends that will dramatically impact the future of command and control centers:

The proliferation of sensing, monitoring, and diagnostic technologies, plus the advent of new communication channels like social media, have given us access to a flood of critically important information from the field – information critical to managing increasingly complex power grids, effectively deploying resources in disaster zones and mastering battlefield complexity.
Advances in high-bandwidth, low latency data transmission (5G and soon 6G) mean the information flood gates are about to open. They’re already at 10, and they’re about to go to 11.
Dramatic advances in data visualization have given us the power to present information three-dimensionally.
And perhaps most important…

Industry innovators like VizworX and Mechdyne have developed the means to create fully-functional virtual and extended reality (XR) control rooms that deliver on the promise of cutting-edge technology. Combining virtual reality (VR), augmented reality (AR), and mixed reality (MR) technologies, XR can be added to new or existing command centers, or developed as a stand-alone capability for field operations.

Learn More

Making Virtual Control Rooms a Literal Reality

How will VR and XR transform traditional command and control? First, imagine all the functionality of your current command and control facility, available to you on a VR headset. Every screen, every map, every data feed.

Now imagine way MORE functionality. Freed from the constraints of 2D displays, a VR headset enables important capabilities:

Data is presented in spatial 3D, enabling us not just to see, but to process and act upon more information than ever before by presenting data in the way our brains have evolved to absorb it. 3D modeling already exists, of course, but in most cases, it’s presented on 2D screens. Viewing dense and complex 3D data points in 2D severely restricts our ability to see spatial relationships between those data points and generates cognitive loading as we struggle to make sense of what we’re seeing. In the virtual world, 3D isn’t simulated; it’s fully realized. Geospatial data can be absorbed and understood rapidly and intuitively. You can view a battlefield as if you’re actually in theater.
In applications where a review of information over time is beneficial, it can be difficult to interpret time progression using 2D data on a map. In 2D, clever tricks (or lengthy animations) are needed to try to show time impacts. In a VR environment, time can be shown as the third dimension on a spatial map and grasped more quickly.
Traditional limits on the quantity of data decision-makers can receive are removed. “Imagine yourself in a room where every screen can be positioned and sized to meet the demands of the moment,” says Jeff LaFrenz of VizworX. “If you’re accustomed to seeing ten screens in your physical control facility, in the virtual world any number of additional screens can be accessed as easily as you might swivel your chair.” Information can be presented in spatial or layered ways that are easily navigable, not overwhelming.

Instead of entering a physical facility, you’ll simply don VR headset a linked to a network-tethered computer. In this environment, space, equipment, and capacity constraints virtually disappear. Organizations can populate the virtual control room with familiar assets, creating a metaverse-like professional environment for their teams to collaborate in. The ability to accommodate any number of participants – wherever they may be – opens up tremendous potential for collaboration during crisis events or simulations.

Get Started Today

For Utilities - Virtual Reality Brings Tangible Benefits

“It all boils down to having more complete and usable information, enabling a faster and more coordinated response in dynamic situations,” says Chad Kickbush of Mechdyne. Consider a timely example: a wildfire that’s knocked out a whole section of the power grid, requiring a rebalancing of the network. Every second counts, yet you need time to assess the situation, gather the appropriate people, and formulate a plan of action to protect a highly dispersed infrastructure. The technology is at hand to assemble decision-makers in a shared virtual environment, present critical data with geospatial accuracy, and facilitate an effective response.

“Or what if an oil spill demands an immediate response, but your experts aren’t on-site?,” asks Kickbush. “Extended reality technology can provide full command and control access wherever your people are. From maritime traffic to weather patterns and sub-surface information, decision-makers can access complex, essential information in a form they can readily process and act upon.” These VR capabilities can be connected to traditional control center activities to ensure complete access and collaboration.

The possibilities for coordinated action become particularly striking when VR is paired with AR and MR capabilities. With the aid of augmented reality technology, for example, decision-makers can have an enhanced view of their immediate environs and act quickly by accessing command and control technologies via their headsets.

What’s more, you don’t need to venture to the farthest reaches of your grid to realize the value of these technologies. A powerplant’s engineers can bring command capabilities straight to the energy-generating infrastructure – the rods, the pipes, the turbines – to diagnose and address issues with maximum efficiency.

“VR control rooms also prove their value well before crises like wildfires or pipeline ruptures occur,” adds LaFrenz. “In fact, they’re a game-changer for training, simulations, and drills.” Most operations centers maintain mockup control rooms for training purposes. These facilities have limited capacity, and are expensive to operate and maintain. VR control rooms have no capacity limits. “Moreover, it’s easy for a supervisor to jump into a VR environment to monitor and lead the training,” says Kickbush.

Similarly, drills and simulations can take place in a virtual environment without impeding the functioning of the physical control room. Freed from capacity constraints and other issues, VR control rooms provide richer and more frequent opportunities to hone response tactics and advance situational awareness.

Virtual Topography Map in Virtual Control Room

Enhancing Emergency Response

When a natural disaster is unfolding, lives depend on coordinated relief efforts. Yet emergencies such as hurricanes present unique challenges to aid providers. Just as waters can come in a flood, so does information – in complex forms, from multiple directions, in a ceaseless flow.

Authorities need functioning command centers ASAP, but setting them up in affected areas takes time. And no matter how well equipped they are, mobile command facilities – which are typically housed in modified shipping containers – will always suffer from limitations of screen real estate and physical space.

VR control centers provide full information management, with the flexibility to bring in the visualizations you need, when and where you need them. That translates into better decisions in fast-evolving situations. A control room in a box is now possible.

Learn More

Visualizing Ship Positioning in a Virtual Control Room

Meeting the Demands of the Military

Perhaps no environment presents greater obstacles to control than a military battlefield. In a ground troop context, command facilities – again occupying limited spaces – must be set up in often dangerous environments, then broken down, moved, and subsequently relocated to stay one step ahead of enemy targeting. “Such facilities are mobile, but they’re hardly portable,” says LaFrenz. “It requires tremendous logistical acumen to ensure a steady stream of actionable information in theater.”

Ship-based command centers carry their own unique challenges – from the highly confined nature of their spaces to the constraints imposed by their location deep in a ship’s bowels. XR command and control brings a wealth of resources straight to the command deck, or to a more remote, safer location. Wherever they’re deployed, the technologies can provide the critical information military leaders need to maintain battlefield dominance.

Get Started Today

How can you harness the power of VR Command Centers?

The best way to realize XR’s potential for your command center is to partner with those who are pioneering this application of the technology. Alberta, Canada-based VizworX, a software development company affiliated with the University of Calgary, draws on decades of experience developing industrial solutions, including next-generation control room interfaces for utilities and teams with expertise in virtual and augmented reality applications. Iowa-based Mechdyne is a large-scale technology integration company with deep capabilities in integrated virtual reality systems, network connectivity, hardware, support, and maintenance.

“We’ve joined forces not just to imagine and design the future of VR control rooms, but to bring our solutions to life,” says Kickbush. “With the right partners, we’ll create an entirely new paradigm for command and control.”

Who’s the right candidate for such a venture? “We’re looking for partners who understand just how powerful VR and XR technology are – and will become,” explains LaFrenz. “We want to join forces with a company or government entity that wants to be both a leader and a co-creator of a customized solution that’s tailorable to their unique needs.”

We live in a world rife with uncertainty and complexity. From extreme weather events to complex geopolitical challenges, knowledge will be the decisive factor in our ability to overcome obstacles and prevail in conflict. “XR technologies will enable us to process the critical information necessary to keep chaos at bay,” says Kickbush. “And the good news is, we have all the tools we need to make extended reality part of our shared reality.”

Interested in Learning More About Our Control Room Solutions? Contact Us Today!

Please fill out the form below. You can also contact Mechdyne by email (info@mechdyne.com) or by telephone (+1.641.754.4649)

Related Resources

5W’s of Mechdyne Mission Critical Control Rooms

Fortune 500 E & P Company

The company's goal was to utilize visualization technology to locate and drill the best oil wells while avoiding subsurface hazards, and reducing the risk of drilling dry holes. Mechdyne's solution was a 3D visualization system and collaboration room to allow high-definition viewing of underground formations with possible drilling sites and on-site space for improving research and drilling techniques.

Mission Critical Control

Control Room Design & Integration

VR Control Room Environments