For decades aviation training research has received significant attention and investment from both the U.S. Department of Defense and the commercial aviation industry, with the recognition that safety and performance excellence are key components within this domain. Recent advances in an emerging field of science and technology–augmented cognition–have inspired researchers to investigate its potential role in enhancing training and operational mission performance through better understanding of real-time brain activity.
As part of human factors research and technology development, augmented cognition was aggressively advanced under the Improving Warfighter Information Intake Under Stress program of the Defense Advanced Research Projects Agency (DARPA). The goal was to combine neurophysiological sensors and cognitive state gauges with an operational platform in order to produce a system that can use an individual’s neurophysiological data to dynamically modify system behavior, mitigating the cognitive bottlenecks responsible for limiting human performance. This advanced human-computer interaction (HCI) approach optimizes total system performance by assessing the operator’s cognitive state and modifying some aspect of the operational platform to dynamically compensate for limitations in human performance.
The Office of Naval Research (ONR), along with the Navy’s aviation training community, is beginning to explore how augmented cognition could support all aspects of flight training. Leveraging accomplishments and lessons learned from the DARPA program, researchers would select the best sensor technologies and cognitive state classification algorithms for integration within a training system, such as a flight simulator. This approach could yield a closed-loop, instructorless, adaptive training system. It could, for example, accelerate the transition from novice to expert, as instruction would be uniquely tailored to the trainee’s cognitive state at any point in time during the training process.
Imagine an aviation recruit experiencing a simulator that is tailored to the trainee at the most fundamental neurophysiological level. Imagine that this simulator’s integrated helmet and sensor suite are hooked up to a "black box" that modifies the simulated flight exercise based on a real-time assessment of the student pilot’s cognitive state, using information collected by the sensor suite. Some may perceive this as a radical idea; however, DARPA has demonstrated that such a scenario is possible.
DARPA’s progam has demonstrated how to modify the user interface of an operational system, based on integrated sensor data, cognitive state classification techniques, and mitigation strategies. It is not much of a stretch to conclude that the same is possible in a training system. When the recruit dons the flight helmet lined with neurophysiological sensors connected to cognitive state gauges that assess the trainee’s cognitive function, the output from those sensors/gauges may be delivered to a flight instructor to aid real-time observation and after-action review. Such a system could even assess and adapt to the student’s level–without the aid or intervention of a human instructor.
Fast forward 10 to 15 years from now, as naval flight schools apply augmented cognition technology to the selection and training of aviators. A new recruit eager to be accepted into a flight school is instrumented with neurophysiological sensors in order to generate an image of that recruit’s brain structure and function. Using what we know about the brain structure and function of highly successful pilots, perhaps we could compare the two images and predict our "combat ace of the future." Such a system conceivably could be used to predict who will succeed in flight school and, more importantly, who will succeed with flying colors.
Next, imagine that the recruit is shown to have such potential and is sent to flight school. Initial training would closely resemble the near-term application described above. The assessment system that was used to determine selection for flight school could also be used to help determine when the trainee is ready for live flight training. Additionally, during the live flight training exercise, the cognitive state assessment system can proactively alert the instructor of imminent trainee overload, allowing the instructor to adapt the scenario prior to any mishaps.
Eventually, the focus should be on how the systems we build can adapt to the humans who are operating or controlling them. It is no longer a question of if adaptive systems can be developed, but of how we can best cultivate and exploit this emergent capability. ONR’s Human Performance Training and Survivability (HPT&S) program is leading the effort to answer this question.
Dylan Schmorrow is a program officer at the Office of Naval Research (ONR).