Verocel, a software verification company, and Certification Services, Inc. (CSI), a consultancy specializing in airborne and ground-based aviation-related digital systems, have won a research contract from the Federal Aviation Administration (FAA) to study the use of reverse engineering techniques, which are prevalent in the development of safety- critical software for avionics and digital systems applications.

The study is expected to help the FAA formulate policies that will affect how reverse engineering processes can be used in the avionics software industry.

The research promises to impact accepted development practices in the avionics industry and also will apply to reverse engineering of commercial off the shelf (COTS) software. Reverse engineering processes are those where the development of requirements, design and code are not performed in a strict sequence.

The two-year project calls for Verocel and CSI to review current industry practices in reverse engineering and potential safety concerns, and will result in a proposed framework to help reduce potential risks. The ensuing guidance criteria that would implement such a framework are intended to be published as a report to help the FAA formulate future policies.

"Reverse engineering is widespread in the software avionics development industry, but guidance in this area is misunderstood and not applied uniformly, leading to confusion," says Mike DeWalt, chief scientist of CSI. Adds George Romanski, president of Verocel, "With the separation and globalization of the development and verification processes for high-integrity software, it is important to establish well-defined and coordinated process plans and procedures that provide confidence in the safety critical product."

Examples of reverse engineering include the development of source code before requirements are developed, or formalizing the design after the code is complete. However, concerns about using reverse engineering for software-critical avionics applications have been raised by the Certification Authority Software Team (CAST) in their position paper (CAST-18).

"These concerns will be addressed in this study," says Romanski, "which will result in a proposed framework of processes and procedures for the FAA that does not compromise safety expectations regarding the use of reverse engineering."

There are two phases to the research. Phase 1 will gather information across a wide range of sources using literature searches, direct solicitation from certification and industry authorities, information extracted from available data, and information gathered from regulatory materials. These activities are expected to lead to the formulation of a recommended reverse engineering framework.

Phase 2 activities will validate this framework through review of the results, performance of completeness checks, and the execution of a case study to demonstrate the applicability and efficacy of the proposed framework.

Software development that starts from some design artifact, such as the source code or low level requirements, and is followed by design and requirements development, is called reverse engineering. This approach has become popular, especially where the development of certification evidence is outsourced to offshore developers.

"Software development for safety-critical systems with taxing real-time constraints and robustness requirements is particularly difficult," says DeWalt. "In these systems, the required behavior is not always understood before the system is constructed. Reverse engineering has been used to develop prototype systems to help understand the system. In effect, the program is a specification of the intended behavior. Because the software development process is expensive, if the prototype proves successful, it is often used as the basis for the actual implementation. This is why reverse engineering has become so ingrained in the development of avionics applications."

However, the reverse engineering approach raises a number of potential problems in system development that may not satisfy requirements at the system level, or may contain additional behavior in the software that is not required. Reverse engineering traceability between software and system requirements that have been reverse engineered themselves may add vulnerabilities due to the process itself. These must be addressed to ensure confidence in the resultant system.

"The many issues raised in CAST-18 summarize the problems associated with reverse engineering. Among these are poor methodologies, inexperienced practitioners, and poor quality," DeWalt says. "However, other problems are much more profound. These include the potentially large differences in levels of abstraction, the extraction of intended design data from actual implementation data, and so on."

"Our research will explore the errors that can potentially be introduced by reverse engineering and provide techniques for mitigating these errors," Romanski says. "The research will also identify those areas and practices of reverse engineering that could produce results that cannot be shown to be compliant with current guidelines, or that represent potential safety problems. Once these have been determined, the research will propose guidance that can be used to provide assurance that DO-178B objectives can be fulfilled. If there is a need to provide alternate approaches, these will also be identified."