Stealthy, supersonic and sensor-rich, the Lockheed Martin F-35 Lightning II, or Joint Strike Fighter (JSF), will engage air and ground targets day or night. Key to using the F-35 in combat is a Helmet-Mounted Display System (HMDS) that superimposes flight and target symbology on the view seen through the pilot’s helmet visor. This virtual Head-Up Display slews sensors and weapons to pilot head movements, and it enables the wearer to de-clutter the picture and zoom into targets with hands on sidestick and throttle.
A second-generation HMDS flew for the first time on a Marine Corps F-35B in March as part of program’s System Development and Demonstration (SDD) phase. Manufacturer Vision Systems International (VSI), meanwhile, is building Low Rate Initial Production (LRIP) displays for the F-35 fleet.
By April, the LRIP display had been integrated with other pieces of the F-35 mission system on the Lockheed Martin Co-operative Avionics Test Bed (CATBird), a modified Boeing 737-330 designed to validate F-35 avionics.
“That’s what makes this a very challenging program,” acknowledged VSI President Drew Brugal. “SDD is going concurrent with the LRIP. The pilots are constantly evaluating the system, making recommendations.”
The F-35 blends radar, electro-optical and datalinked intelligence with digital maps, threat warnings and systems data on a 20-by-8 inch L-3 Display Systems panoramic cockpit display.
“It’s a fifth-generation airplane with fifth-generation sensors,” noted Brugal. “The whole idea of this helmet is multi-sensor fusion…. You can bring several different images together in the pilot’s field of view.”
The HMDS shows imagery from the Northrop Grumman Electro-optical Distributed Aperture System (EO DAS) annotated with composite symbology from other sensors. The head-slaved display affords all-round situational awareness and enables the pilot to see “through” aircraft structure via remote apertures. “If you can show it on a screen in the cockpit, you can bring it up on the visor,” Brugal said.
As currently integrated, the VSI helmet display does not now show the F-35 pilot magnified imagery from the aircraft’s Lockheed Martin Electro-Optical Targeting System (EOTS) or Synthetic Aperture Radar pictures from the Northrop Grumman active electronically scanned array (AESA) radar. Target analysis and designation are done largely on the cockpit display.
HMDS symbology nevertheless promises to reduce target location errors with the combined input of multiple sensors. Don Bolling, senior manager for business development with Lockheed Martin Missiles and Fire Control noted, “The thing that makes the F-35 very unique and powerful is that it has a fused architecture where all the sensors are working together. You don’t necessarily know whether that information comes from DAS, the EOTS or the radar.”
Clear View
VSI, based in San Jose, Calif., was formed by Elbit Systems Ltd., and Rockwell Collins to manufacture and market helmet displays for fixed-wing aircraft. The joint venture received an HMDS LRIP contract in 2009.
“The best way to think of it is anything that stays in the airplane is made by Elbit. Anything that is on the pilot is made by Rockwell Collins,” Brugal explained. Cockpit transmitters for the electromagnetic head tracker, for example, come from Elbit while helmet receivers are supplied by Rockwell Collins.
The HMDS shell is made by Helmet Integrated Systems Ltd., in Stranraer, U.K., and personalized with a laser map of the pilot’s head.
An Elbit display management computer interfaces the aircraft databus with the HMDS. A coaxial cable runs from the upper right side of the helmet down behind the pilot’s neck through a plug by the ejection seat to carry power, video feed, data feed and head position data.
DAS imagery or the unenhanced view through the polycarbonate visor is annotated with green monochrome symbology at all times to provide airspeed, altitude, heading, velocity vector and other spatial references. VSI Chief Technology Officer Bob Foote noted that color symbology would be technically difficult and is not part of HMDS requirements. “Our requirement from the JSF community is resolution, resolution and resolution and brightness,” Foote said.
Elbit provided HMDS software for the F-35 flight sciences test aircraft. In production fighters, the pilot will see symbology created by Lockheed Martin. “Those same symbols that he sees in the cockpit are replicated in front of his face,” said Brugal.
Without the color cues available on head-down displays, Lockheed Martin engineers flag items with flashing, cross-hatching and other man-machine interface tricks. “We make the machine so powerful, they can do just about anything they want. Our job is to provide good resolution and good tracking to implement it.” Foote said.
HMDS tracking accuracy is classified, but the head tracker from Elbit in Haifa, Israel, uses magnetic and optical references continuously to maintain confidence in the solution. “We have a requirement that there be no sort of alignment on-aircraft by the pilot,” explained Foote. “The optical portion of the tracker system provides an accurate auto-boresight to achieve this capability.”
VSI supplies the Joint Helmet Mounted Cuing System (JHMCS) on jet fighters around the world. The JHMCS uses a stroke display to put targeting symbology on a 20-degree circle over the pilot’s right eye. The head-tracking display steers aircraft sensors and missile seekers in high off-boresight engagements. It does not show the pilot sensor video or flight symbology, and it needs a QuadEye replacement assembly at night.
In contrast to the monocular JHMCS, the binocular HMDS covers a 30 by 40 degree field of view and supplements projected day and night video with raster-like symbology. The field of view is determined by packaging tradeoffs. “You have a limited amount of space available on the pilot’s head,” Brugal said. “You can only displace those projectors so many degrees. Displace them farther, and the helmet gets very wide and causes interference.”
The first-generation HMDS used a bifurcated visor to provide a 50-degree horizontal field of view. However, test pilots who flew the first helmet in U.K. Hawk trainers and the first F-35, AA-1, noted the witness line between the display segments. Raising the early visor also required the user to raise the entire optical unit. “You typically wouldn’t be doing that in flight because there are a lot of moving parts.” Brugal observed. The second-generation HMDS has a continuous-curve visor that slides up and down easily.
F-35 requirements say the ejection-safe HMDS can weigh no more than 4.5 pounds, slightly less than the JHMCS. VSI engineers leveraged flat-panel display technology to replace cathode ray tubes.
“That saved a considerable amount of weight,” Foote said.
A partially mirrored coating on the see-through visor, meanwhile, eliminated the heavy optical combiner and supporting structures of past helmet displays. “We’ve made a fairly sophisticated train of optics that allows us to bring the display from the back of the guy’s head to the visor,” Foote said. The HMDS also benefitted from modern computer-aided design technology. According to Foote, “The CAD tools have gotten much, much better. We can design to closer tolerances.”
Big Picture
The HMDS night imaging capability comes from sensors on the helmet and on the aircraft. On the helmet, an Electron Bombarded Active Pixel Sensor (EBAPS) from Intevac, Santa Clara, Calif., sends imagery directly to the visor. The camera works in the same wavelength as night vision goggles (NVG). “The sensitivity range is in the same range as image intensifier tubes,” said Foote.
Though the 1280-by-1024 pixel camera does not have the acuity of the latest NVGs, it does provide the 700-line video required for F-35 pilots to land at night. According to Foote, “It’s not really meant to be the main night sensor.”
By day and night, the F-35 AN/AAQ-37 EO DAS ( Avionics, August 2008, page 10) provides spherical coverage with near 20-20 visual acuity. Imagery from the half-dozen infrared sensors around the aircraft gives the HMDS a seamless picture anywhere in the EO DAS field of regard. The EO DAS also plays missile warning receiver to cue the F-35 pilot and weapons to threats on the ground. The pilot can turn EO DAS helmet imagery on or off and stabilize the image at a given point to look away, study targets on the head-down display and return to the head-up scene. The HMDS also cues the pilot to air and ground threats outside the immediate field of view.
In contrast to the fish-eyed EO DAS, the AN/AAQ-40 EOTS affords high magnification in a narrow field of view. EOTS packages a mid-wave forward looking infrared (FLIR) imager with a laser spot tracker and target marker. As an air-to-ground targeting sensor, the FLIR covers a 360-degree field of regard below the aircraft horizon. The F-35 pilot who spots an air defense or artillery site in the EO DAS helmet picture can slew and zoom the EOTS to the spot for closer study on the cockpit display. Air-to-air, the EOTS provides infrared search and track (IRST) symbology with target identification, azimuth, range and kinematics.
Like the EOTS, the Northrop Grumman AN/APG-81 radar generates HMDS cues for the pilot in combat. The high-resolution radar with Ground Moving Target Indicator functions zooms into ground targets or tracks and prioritizes targets in the air. “It’s really a target designator, whether it be an air-to-air target or a designated point on the ground. Any of that stuff can be put on that symbology on the HUD. You have an arrow pointing to that target in space,” said Peter Bartos, Northrop Grumman fifth-generation improvements and derivatives manager.
With all the sensors and symbology going to the helmet, HMDS engineers worked to traffic high-volume data without latency. Post-JHMCS processors gave them faster hardware, and some built-in prediction algorithms enhanced HMDS software. “Also, we very closely couple the head tracker and the image processing,” said Foote.
The same dedicated processor that tracks head orientation and position manages graphics. F-35 test pilots report the latency of early displays solved, but Foote acknowledges, “There’s going to be some lag; you can’t make it zero.”
The F-35 is due to achieve initial operational capability in 2012 with the U.S. Marine Corps, 2013 with the Air Force, and 2014 with the Navy. International operators will follow soon after, all using the HMDS. VSI has already received inquiries about HMDS for other platforms.
“There are helicopters who are considering this level of HMD. We are actively talking with the different services about applications on transports, gunships and other platforms,” Brugal said. “Obviously, we’re exploring those because the larger installed base, the lower the price point. The quality of the HMDS and its capabilities are very attractive on platforms you wouldn’t think would consider such a capable helmet.”