New tool qualification criteria

The need for tool qualification and the guidance to qualify the tools were provided in section 12.2 of DO-178C/ED-12C. This section, to avoid confusion, does not use anymore the terms “development tools” and “verification tools”, but identifies instead some “tool criteria”.  Three tool qualification criteria determining the applicable tool qualification level (TQL) depending on the software level are defined.

“Criterion 1” addresses the former “development tools”, while the two other criteria split the former “verification tools” depending of the certification credit claimed by the qualification of the tool.

Criterion 3 is the “classic” use of a verification tool: The purpose of the tool is to produce or verify an artifact, and the certification credit claim is only on objectives applicable to this artifact.

Examples:

  • A tool that produces the tool procedures from the test cases, the certification credit is limited to the correctness of the test procedures (Objectives A7-1).
  • The certification credit for a code checker, that verifies the compliance of source code to the coding standard, is limited to the objectives A5-4 Source code is compliant to standard)

The certification credit claimed is extended in case of application of criterion 2 to objectives that are beyond the data directly verified by the tool. In an appendix of the Tool Qualification Document, a Discussion Paper (DP#5) provides additional rationale about the need for these 3 criteria. It also includes some examples of distinguishing between criteria 2 and 3, using a “proof tool” and a “static code analyzer”

The idea is that the software verification process relies on multiple filters to improve the error detection. The certification credit claimed in application of criterion 2 is equivalent to removing one filter, since it has been replaced by the higher level of reliability of the tool. That’s why, for these tools, the Tool Qualification Level (TQL) is higher than for a “classic” verification tool: TQLThe TQL applicable for criterion 1 is the replacement for the development tool for each software level, while the TQL-5 for criterion 3 is the replacement for the verification tool in DO-178B/DO-278.

The TQL applicable for Criterion 2 basically requires an increased level of rigor for tools used on software level A and B in order to increase the confidence in the use of the tool (that is, TQL-4 instead of TQL-5). TQL-4 requires that the Tool Requirements data describe all functionality implemented in the tool and provide additional detail about the tool architecture. TQL-4 also requires verification of the compliance of the tool with Tool Requirements. TQL-4 objectives are considered as a minimum to claim confidence in the use of the tool.