Robustness

In DO-178B/ED-12B, robustness testing was sometimes misinterpreted as additional tests that supplemented requirements-based tests. This is clarified now and definitely states that all tests, normal and robustness, should be requirement based.

A note is added in the section §6.4.2 on requirements-based test selection: « Robustness test cases are requirements-based. The robustness testing criteria cannot be fully satisfied if the software requirements do not specify the correct software response to abnormal conditions and inputs. The test cases may reveal inadequacies in the software requirements, in which case the software requirements should be modified. Conversely, if a complete set of requirements exists that covers all abnormal conditions and inputs, the robustness test cases will follow from those software requirements »

To be more flexible, it is also recognized that some mechanisms as described in the standards may also be used to improve robustness. So, implicitly, some robustness tests should be developed to assess the correctness of the implementation of these mechanisms.

Additional text is also provided in FAQ#32 in DO-248C/ED-94C (What are defensive programming practices?). This FAQ makes a connection between programming practices and robustness but explains that programming practices don’t supersede the need for requirements specifying the correct software response to abnormal conditions and inputs.