Although the Transportation Security Administration (TSA) had planned to begin testing a program next month to match passengers names to terrorist watch lists, an advisory committee to TSA cautions against the move, arguing that basic questions about the program have not been answered.

The Sept. 16 report by the nine-member Secure Flight Working Group (SFWG) was presented at the Sept. 22 meeting of the Aviation Security Advisory Committee (ASAC) and marks a considerable setback for the program intended to replace CAPPS II, the passenger prescreening system now in use. At the urging of TSA officials, the ASAC voted to forward the SFWG report to TSA "without recommendation."

The report, by an independent panel of security and privacy experts, documents considerable problems with Secure Flight and the policies surrounding it. The problem of identity matching is one of those problems, about which the report said:

"The challenge of Secure Flight is to reliably match passenger records to watch list records. In the context of 1.8 million passengers a day and a TSA watch list of 70,000- 160,000 names, merely comparing the two sets of names is almost worthless in two directions: because names on one list or the other may be misspelled, or because a single name can be written in various ways (Robert Smith, R. Smith, Bob Smith). A direct match misses valid matches (produces too many false negatives), while, at the same time, because there are many common names, it produces an intolerable number of false positives.

"To avoid false negatives, a name search must use fuzzy matches, comparing multiple variations of names to compensate for variations like Robert and Bob. ...

"The costs associated with mismatched data in Secure Flight are ... dramatic. A false negative is a risk to aviation safety, while a false positive might result in an individual losing the freedom to travel.

"The TSA has acknowledged that PNR [passenger name record] data flows from the airlines are inadequate as a basis for watch list matching, in particular because the PNR does not include the same categories of information that are on the watch list - especially date of birth. ...

"The TSA has discussed using logical systems that sound out a name and match with the spelling of other names associated with that name. This would likely reduce false negatives, but not false positives. ...

"TSA has not discussed with us the other side of the matching equation: use of commercial data to augment the watch lists. One of the greatest limitations of Secure Flight ... seems to be that not much identifying information is available on most suspected terrorists."

Based on the limited test results thus far, the SFWG concluded, "We cannot assess whether even the general goal of evaluating passengers for the risk they present to aviation security is a realistic or feasible one or how TSA proposes to achieve it."

Therefore, the SFWG posed six questions about Secure Flight that warrant answers before the program proceeds further. These critical issues must be addressed before the security and privacy impact of Secure Flight can be assessed:

1. What is the goal of Secure Flight? TSA is under a congressional mandate to match domestic airline passenger lists against the consolidated terrorist watch list. TSA has failed to specify with consistency whether watch list matching is the only goal of Secure Flight at this stage. On June 29, Justin Oberman, assistant administrator, Secure Flight/Registered Traveler, testified to a congressional committee that, "Another goal proposed for Secure Flight is its use to establish mechanism for ... violent criminal data vetting."

2. What is the architecture of the Secure Flight system? The Working Group received limited information about the technical architecture of Secure Flight and none about how software and hardware choices were made.

3. Will Secure Flight be linked to other TSA applications? Linkage with other screening programs (such as Registered Traveler, Transportation Worker Identification and Credentialing [TWIC], and Customs and Border Patrol systems like US-VISIT) that may operate on the same platform as Secure Flight is another unanswered question.

4. How will commercial data sources be used? One of the most controversial elements of Secure Flight has been the possible uses of commercial data. TSA has never clearly defined two threshold issues: what it means by "commercial data," and how it might use commercial data sources in the implementation of Secure Flight.

5. Which matching algorithms work best? TSA never presented the SFWG with test results showing the effectiveness of algorithms used to match passenger names to a watch list.

6. What is the oversight structure and policy for Secure Flight? TSA has not produced a comprehensive policy document for Secure Flight that defines oversight or governance responsibilities.

For the full report, go to http://www.tsa.gov/public/display?theme=73 and under the Sept. 22, meeting, scroll to Secure Flight.