The use of programmable electronics that includes the application software is spreading more and more, even in applications critica for human safety. For this reason, the latest generation of standards relating to the functional safety of electronic systems, almost always include sections dedicated to the application software (think, for example, to IEC 61508-3 or to CEI EN 50128). The requirements contained in these standards usually are very demanding, even for applications that require a not high (1 or 2) SIL.
A request that is often made by those standards, even for SIL lower levels, is that the translator of the software (compiler), used for the generation of the executable code, is certified or experienced with the use (that is, it is documented its use without problems in similar applications and equal SIL), or validated. However, it happens very often, based on the importance of the project and on the HW target platform used, not to be able to have a certified tool-chain, or proven with the use, therefore, the only viable solution is to proceed on one's own with the validation of the compiler.
The validation of a compiler can be an extremely challenging task that requires exhaustive knowledge and that involves the design of a large number of tests; the test design, in addition, in order to provide adequate coverage, requires a large specific experience in the compilation process relating to a specific programming language and a thorough knowledge of the language standard.
These difficulties often would get you to think that it is not possible to validate "on one's own" a compiler. However, in many cases, this is a a false impression, because the market offers, at least for the most common languages (eg C), the tools specially made to validate a compiler in a fast and exhaustive manner. The user is only required to have the skills and the experience necessary to be able to properly use these tools, that, however, always have some grade of complexity.
In recent years HINTSW - T & T Systems had the task of validating three different compilers related to the programming language C:
To carry out these validation activities, HINTSW - T & T Systems selected the validation suite "The Plum Hall Validation Suite for C" created by Plum Hall Inc., an American company, world leader in the production of validation suite for programming languages C, C ++, and more recently, Java. In particular, the Plum Hall suite used by HINTSW has two objectives:
Details related to the three validation activities follow here below.
The validation of the GCC compiler was performed in a "hosted" Linux, for two different hardware platforms: Itanium server with SUSE Linux Enterprise Server 10 operating system (ia64); Workstation Intel x86 with SUSE Linux Enterprise Server 10 operating system (x86_64). Both platforms were used in a SIL 2 certified project, compliant with standard CEI EN 50128.
The main objective of the test was to check the compliance of the GCC compiler to the level of the ISO / IEC 9899 standard chosen for the implementation of application software; the level of the standard chosen for the project was the C90 level that refers to the original ANSI C published in 1990, plus the subsequent corrections made in 1994 and 1996 and included in the amendment AMD1 and COR2.
The suite Plum Hall divides all the tests into two broad categories Conformity and Testing: Conformity contains the test strictly necessary for checking the conformity of the compiler to the standard chosen; Testing contains the tests that go beyond mere compliance and they are used to test the quality of the compiler, assigning a score based on the results obtained. In project done by T & T Systems you have performed both categories of tests, although not strictly necessary as the requirements of the Standard are already met in full by the first test category, because it is the one that ensures the compliance of the compiler to the reference standard.
As the suite Plum Hall is distributed as a directory tree that contains all internal tools needed to test the source of the test programs and all the tools (particularly the Makefile) needed to compile the target (in our case the two Linux platforms), it was necessary to make a customization of the suite modifying some prototype files, and creating the target part, that is, the branch where the tests are conducted.
T & T Systems, to allow easy and fast replay of all tests, has also engineered a dedicated validation environment, which, thanks to some script (developed in Perl language), allows fully automatic, install / reinstall suite of software, its customization, implementation, capturing the on-screen log in files, and separation of the output from the test programs. This allowed us to overcome some inconvenience of use of the suite, such as: the installation phase of the suite, which is fully manual, the execution phase, which requires manual activation of the variables used in the test environment, the great fragmentation and distribution of test results as they are distributed on the various branches of the tests, and the "log" of the test, because natively is directed only on video.
The validation of the Altera Nios II compiler was performed in environment "freestanding" (embedded) for a hardware custom platform based on the microprocessor Altera Cyclone II. The platform were used in a project to be certified SIL 4 in railway signalling.
As before, the main objective of the test was to check the compliance of the Altera Nios II compiler to the level of the ISO / IEC 9899 standard chosen for the implementation of application software. In this case the level of the standard chosen was the C99 level for free-standing platform that is related to the full compliance of the language as described in ISO / IEC 9899: 1999.
The validation of the compiler has been performed in two successive stages. The first phase consisted in the execution of all tests in the simulation environment (hence, not on the target) made available by the development environment Nios II; the second phase consisted of the replay of all compiler validation tests on the final target . Note that for the mere validation of the compiler the first step would be sufficient, but in view of the global target validation, the certification body requested the execution of this second phase. The execution of the compiler validation tests on the target implied the need for a greater customization of the Plum Hall validation suite and implied in a more complex engineering of the test environment . All of the suites customization were agreed and carried out in close collaboration with Dr. Thomas Plum Plum Hall Inc. Inc.
As mentioned the test target environment is quite complicated and it required, in order to perform the tests in a reasonable time, a non-trivial engineering. The architecture of the target system is composed of two cards, one called "vital" as related to the safety (SIL 4), a second, not vital, which performs non-related support operations with safety (SIL 0). The two cards communicate via a fast serial HDLC channel; the adapter board has a LAN connection and it mounts a MicroC Linux operating system and an FTP server.
The engineering of the validation environmental made it possible to automate the following steps:
The validation of the Texas Instruments CL2000 compiler was performed in a "freestanding" (embedded) for a hardware platform custom microprocessor-based Texas Instruments TMS320 DSP. The platform has been used in a project to be certified SIL 4 in railway signaling.
The main objective of the tests was to verify compliance of the compiler Texas Instruments CL2000 to the level of the ISO / IEC 9899 standard chosen for the implementation of application software. In this case the level of the standard chosen was the C90 level which refers to the original ANSI C published in 1990, plus subsequent corrections made in 1994 and 1996, and included in the amendment AMD1 and COR2.
Similarly to the previous case, the validation of the compiler has been performed in a simulated environment and on the final target.
In this case the test environment engineering on the real target was a bit 'easier since the programmable card is connected to the host computer via a USB connection that allows the programming of the card, the execution of the test and the capture the output of the same. For this purpose, the Plum Hall suite has been interfaced to the Debug Server provided by Texas Instruments in order to automate both the upload and the running on the device of the compiled code; Finally, through a suitable driver supplied by Texas Instruments, it was possible to manage the communication with the target, bringing the output of the tests on the host PC console, so we could save to file, as provided by the suite.