Business & GA, Military

Helmet Mounted Displays: Adding Night Vision

By John Croft | September 1, 2006
Send Feedback

Day/night helmet mounted displays promise breakthrough abilities for night fighting in the areas of head comfort and safety, cockpit organization and situational awareness. Two manufacturers and two projects are leading the way for tactical fighter aircraft: VSI on the F-35 Joint Strike Fighter (JSF) and BAE Systems on the Eurofighter Typhoon. But night vision capabilities for these aircraft are still under development. Neither system is operational yet, and each has challenges to overcome.

Ultimately, the military wants to move away from night vision goggles (NVGs) that mount externally to the helmet to lighter-weight head gear that has integrated targeting and flight control symbology, in addition to night vision information, displayed on the visor. Bringing night vision inside the helmet will reduce the effects of g-forces on the pilot’s head and neck and will eliminate hardware configuration changes and stowage issues for day/night transitions. Optimizing HMDs also will mean moving from analog to digital technologies and reducing the weight of the lens assemblies needed to present the information on the visor. Though flight control and targeting symbology are relatively mature, thanks to the highly successful Joint Helmet Mounted Cueing System (JHMCS) built by VSI and used in F-15s, F-16s and F/A-18s, integrated night vision technology for HMDs is still in its infancy.

The JSF system, known as the helmet mounted display system (HMDS), is the most technologically progressive helmet to date, largely because it is destined to take on a new and critical role–the head-up display (HUD). "JSF was going down a path of trying to take the next step forward for HUD and JHMCS and night vision devices," says Eric Branyan, vice president of F-35 mission systems for Lockheed Martin.

"The best thing to do to keep them from being separate systems and have the pilot distracted was to integrate them into one device." By manipulating avionics and sensor data, the HMDS becomes a virtual HUD with imagery when the pilot looks forward, or in the boresight direction. Off-boresight, the HMDS will provide HUD-like aircraft performance details, threat information and targeting cues in addition to mid-wave and near-infrared (IR) imagery from a suite of six aircraft mounted IR sensors and a helmet mounted night vision sensor. Branyan says the aircraft mounted sensors will provide spherical coverage around the aircraft, even allowing pilots to look "sideways and down through the floor."

For the Typhoon, BAE Systems is designing a head equipment assembly (HEA) that will present night vision and off-axis targeting and cueing information via the helmet. The helmet is designed to use image intensification technology to amplify light from external sources, as well as an advanced laser head tracking system. The HEA does not, however, replace the HUD.

In both designs visual information provided by sensors either embedded in the helmet or located elsewhere on the aircraft will be projected onto a see-through visor patch located on the inside of the visor. This allows for a more compact design with an improved center of gravity relative to a traditional helmet/NVG system. But in order for the pilot to see the imagery correctly, the head gear–an inner lining and a hard outer shell–will require a more precise fit than ever before, hence the introduction of laser scans to measure the exact dimensions of each user’s head.

Typhoon Helmet

BAE Systems head equipment assembly for the Eurofighter Typhoon, when completed in the mid-2008 time frame, will bring the military closer to the notion of a single-helmet configuration for fighter aircraft. The system will include a laser head tracking system for target cueing. It incorporates twin image intensifier tubes on the sides of the helmet, providing a binocular 40-degree horizontal by 30-degree vertical field of view (FOV) with fully overlapped images. The helmet employs a dual-visor configuration–a clear blast/display visor for night and a glare/laser eye protective visor that swivels down for day operations. Analog symbology information is delivered from aircraft avionics to cathode ray tubes (CRTs) on the helmet via wires. The information is then transferred by a series of lenses and/or mirrors to the visor. The HEA’s optical path, however, includes a "brow" mirror that BAE Systems says allows it to minimize center-of-gravity (CG) offsets, and hence g-effects on the pilot’s head.

One issue that is still under investigation with this approach involves the location of the image intensifiers on the sides of the helmet near the ears. Because the distance between the two sensors is wider than the distance between the operator’s eyes, pilots can experience an effect called hyperstereopsis. "When viewed at close distances, objects such as the ground appear closer than actual, and false motion cues can occur–particularly when moving the head while viewing relative motion outside the aircraft," explains Dr. Chuck Antonio, a former Navy pilot and flight surgeon with extensive NVG and HMD testing experience, working for the Naval Air Warfare Center, Patuxent River, Md. Additionally, he says, due to the displaced eye position in combination with the see-through visor patch, parallax (misregistration) can occur between the virtual image on the visor and the real world, leading to, for example, seeing the intensified version of a light source offset from the real light source.

Flight test results have demonstrated that these effects are mitigated as the distance from the ground or objects is increased, and by approximately 1,000 feet (305 m) are no longer perceptible, Antonio says. Though putting the sensors in front of the eyes would avoid hyperstereopsis, it moves the CG forward and would cause excess g-forces, one of the main problems the helmet was supposed to eliminate. One potential solution being evaluated, according to George Lim, director of helmet systems for BAE Systems, is to switch off one of the image intensifiers during phases of flight where hypersteropsis may cause problems. The downside, however, is that going from two sensors to one results in a 40 percent reduction in image quality, he says.

Though the production helmet for Typhoon is slated to have image intensifier tubes, BAE Systems continues to experiment with using solid state cameras that work in the 340-nanometer to 1-micron range for night vision, making up for the lower-quality images (compared to analog image intensifier tubes) by using digital signal processing, says Lim. That processing includes symbology overlays and provides advantages such as reduced halo effects, increased contrast and reduced visual noise. The cleaned-up signal is then transferred to the visor via the lens assemblies on each side of the helmet. The technology, slated to be part of the next-generation Eurofighter helmet, remains too immature to incorporate in a flight system yet, however. Another big push within the company is to reduce the weight of the optics needed to transfer images to the visor. Lim says the company is investigating technologies that will trim 20 percent from the weight of the current optics.

To date, BAE Systems has built 30 functional and mechanically representative HEAs, with no night vision capabilities, as part of the development program. The Royal Air Force, as of July, had carried out 70 hours of flight testing in the UK Center of Aviation Medicine’s Hawk aircraft. Deliveries of night vision-capable systems are expected as part of the production deliveries in 2008, 10 years after the program was launched. Lim says the company expects to deliver 459 HEAs in two phases for Eurofighter Typhoon. BAE Systems is also under contract to build a similar cueing system next year for Gripen fighter jets bound for the South African Air Force.

HMDS Update

The F-35 HMDS is slated to be flying on Block 3 aircraft, beginning in 2009, says Lockheed’s Branyan. Made by VSI, the helmet displays the same symbology as the current Joint Helmet Mounted Cueing System, plus imagery from a single forward-looking night vision sensor at the top of the helmet and six 3-to-5-micron infrared sensors surrounding the aircraft, as part of the Northrop Grumman-built distributed aperture system, or DAS. (DAS was originally placed on the aircraft to detect plumes from surface-to-air missiles.) Since the helmet will provide head position, Lockheed can derive and present the appropriate view from the sensor array to the pilot. (VSI is a joint venture between Elbit subsidiary, EFW, and Rockwell Collins.)

Instead of CRTs, the helmet mounted display system has two 0.7-inch-diagonal, 1280-by-1024 active matrix liquid crystal display (LCD) projectors, one on each side of the helmet, that display imagery and symbology overlays. The night vision images are relayed to the visor via optics for a 40-degree horizontal by 30-degree vertical bi-ocular view (binocular vision from a single source). Images sent from the DAS to the helmet will be generated by the aircraft’s avionics systems, based on input from VSI’s electromagnetic head tracking system.

Branyan says Lockheed has evaluated several sources for the night vision sensor but has not yet selected a unit. The first prototype will have a customized 16-mm image intensifier tube built by ITT interfaced with a charge-coupled device (CCD), says Louis Taddeo, VSI marketing director. Visual information on the CCD will then be transferred to the visor via the optical tube above each eye, on the outside of the helmet.

Though the first JSF will fly Oct. 31, 2006, according to Branyan, the full HMDS suite will not begin flying on the aircraft until the third quarter of 2008. One major change at the moment involves switching out the HMDS visor. The original design called for a bifurcated visor (picture the armor on a knight’s face), which simplified the optics path and generated a large FOV. Unfortunately, it also had the potential to distort the outside world due to the peak in the middle of the visor, a finding that emerged in pilot demonstrations. "We were aiming to give the widest field of view possible, but found out that people got distracted by the centerline," says Taddeo, adding that the new design should be flying by next summer. Another potential issue is that the positioning of the night vision camera on the helmet could lead to the outside scene being partially blocked by the canopy bow, causing pilots to have to squat down in their seats to get a good view. This potential blockage of the outside scene by an aircraft structure is relevant to any design where the sensors are located other than in front of the eyes.

Part of the risk reduction for Lockheed will be to get the HMDS flying as early as possible on all three variants of the JSF. Branyan says the HMDS will first fly in February 2007, and the company "will continue to mature the capabilities." Lockheed’s Cooperative Avionics Testbed will begin testing the DAS inputs to the JSF helmet in late 2007 and early 2008. The aircraft will have a simulated JSF cockpit that pilots will fly as part of a fully integrated mock JSF, including DAS sensors. Other aspects of the helmet design have been tested in ground simulations and airborne labs over the past four years with a host of U.S. and international pilots, including using the Pitts Special aerobatic biplane to check out how the g-forces feel to pilots wearing the helmets.

If for some reason the HMDS is not ready for prime time on the Block 3 aircraft in 2009, the biggest impact could be the lack of a HUD in an otherwise fully functional aircraft. The HMDS is designed to show HUD information when the pilot is looking within about 30 degrees of the boresight, outside of which the attitude indicator is removed. "If we did have issues with development–and I don’t see any now–there are options to put a HUD in front of the pilot that shows typical flight data for legacy aircraft," Branyan says.

Helmet Evolution

Though far from mature, the JSF helmet stands on the shoulders of more than a decade of HMD efforts. In the 1980s the U.S. military held a competition to develop helmets with integrated night vision systems. The goal was to reduce the center-of-gravity offsets associated with external, clip-on NVGs, like the ITT or Northrop Grumman ANVIS-9 NVGs, the staple of the military’s fixed- and rotary-wing fleets. Having one visor that would perform both functions–day and night–would also avoid the need to switch out and stow systems during day-night transition flights.

While the NVGs are relatively lightweight (about 1.3 pounds, or 0.6 kg), the offset between the goggles and the pilot’s head amplifies the forces experienced during high-g maneuvers and was thought problematic for ejections. (Ejections thus far have not produced injuries, due mostly to the breakaway feature designed in the mount.)

"It was clear the [early HMD] technology wasn’t there yet," says Antonio. "The helmets were too big and heavy, and the imagery was not as good as with NVGs." However, the advantages offered by an integrated head-mounted system make it an important area to continue research, he stresses. With NVGs a pilot looks directly into the objective lenses of binocular, Generation 3, image intensifier tubes that can provide as good as 20/25 vision on a clear night with only starlight illumination. HMDs typically use a CRT or digital camera to capture the output of an image intensifier, transferring the image to the visor through a series of lenses or mirrors. Each extra step degrades the quality of the image, however.

The HMD concept caught a second wind around 1990, as the military wanted to find a way for a pilot to launch an AIM-9X Sidewinder missile by looking at the enemy aircraft through the helmet rather than the HUD. Pilots would not have to point the nose of the aircraft at their target to designate it, but could simply look at it, day or night. By default that also meant the system would have to support night vision. VSI ultimately won the contract for what became known as the Joint Helmet Mounted Cueing System, now in use on F-15s, F-16s and F/A-18s, though the requirement for night vision was later deferred.

Along with an in-helmet display of aircraft and weapons symbology (including heading, airspeed, altitude, angle of attack, target designation and steering cues), JHMCS also used an electromagnetic tracking system that determined the movement of the helmet in the cockpit, allowing the aircraft’s computers to georeference the pilot’s visual line of sight. Operationally, that meant pilots or weapons officers could visually mark targets for Sidewinders. They also could look at an object on the ground and automatically have the aircraft’s guidance system command a targeting pod forward looking infrared (FLIR) camera to the spot. Video from the high-fidelity, zoomable FLIR could then be displayed on the head-down display, allowing the crew to analyze the potential target and determine the course of action. VSI also included a symbol that points to a designated target so the operator can take a second look before firing.

To bring the symbology into the helmet, VSI sends analog data via a wire to a miniature CRT on the upper right side of the external helmet shell. The monochrome green picture in the "mini TV" on the right side of the helmet is then relayed optically to a portion of the visor (the screen) in front of the right eye, painting the symbology in a 20-degree conical field of view on the visor. (The visor has a coating on the inside that blocks light from the outside world in the same frequency band as the symbology information.) Since the information is presented to the right eye only, the system is classified as monocular. "Most people are right eye-dominant anyway," says Taddeo. "It’s also an easier problem to solve. With binocular vision, you usually have 100 percent overlap, and it’s hard to get complete overlap of the information."

To date, VSI has delivered 1,400 JHMCS helmets for 500 aircraft in the U.S. Navy fleet and in the air forces of 17 nations, says Taddeo, who asserts that it’s the only HMD flying today. At night, however, operators remove the visor and snap on traditional night vision goggles.

Efforts to combine JHMCS symbology and night vision are under way with a U.S. Navy competition between VSI and InSight, with an anticipated award in 2007. The idea is to inject symbology into the optical train of one of image intensifiers worn as traditional NVGs. Though the competition calls for a 40-degree field of view, a typical value for a binocular NVG system, the Navy is "giving points" for a modular wide-FOV system, also known as a panoramic NVG. This could increase FOV width to as much as 100 degrees by using four image intensifier tubes, all of which are slightly shorter and lighter than previous ANVIS-9 tubes, reducing head strain under increased g-forces.

VSI’s system features a modular design with up to four image intensifier tubes. The two center image intensifiers are overlapped 100 percent. The two outer intensifiers can be used to increase the FOV width to 100 degrees, or can be removed to revert to a standard binocular NVG configuration with a circular 40-degree FOV. In tests of the VSI binocular system, starting last September, both crew members of an F/A-18F Super Hornet wore the company’s night vision cueing and display (NVCD) system, and could independently survey air or ground targets at night, triggering the aircraft’s targeting pod to capture high-accuracy FLIR imagery for the head-down display.

Receive the latest avionics news right to your inbox