Test Farm
In order to ensure efficient development and dissemination of quality software tools, a farm of platforms is made available for debugging and testing purposes.
This facility is open to all developers within ETSF.
Aim of the facility
ETSF software offer consists of several large-scale products. Their development involves frequent updates by teams of programmers. In order to ensure that high levels of quality are attained, the portability and reliability of the applications are regularly checked on a diversity of environments by means of standardized procedures. They are performed on a farm of platforms running software builders detailed hereunder.
Users interested by the concepts behind a test farm may download this document presented during the CECAM tutorial in Zaragoza
Implementation details
The setup consists of several computers selected for their diverse characteristics controlled by a client/server software environment named Buildbot.
Software environment
Buildbot is a system designed to automate the configure - compile - test routine commonly exercised by software development projects in order to validate code changes as described on this utility homepage. It relies on a client/server paradigm:
- the server or build master is the central point of control. It timely receives software modification notices, distributes the build tasks and gathers the status information from the slaves to inform the user about the whole process success or failure.
- the client or build slave performs the actual compilation and test tasks. It proceeds through a series of prepared build steps consisting of shell scripts, compiler invocations, test commands ...
Hardware offer
Buildbot operations rely on several computers hosted at ETSF central node (Louvain-la-Neuve, Belgium). The build slaves offer a wide variety of configurations - a mixture of little endian and big endian (multi)processors systems representative of both desktop computers and large scale computing facilities.
A diversity of open source and proprietary operating systems, compilers and parallel processing libraries is provided. The following table summarizes the available hardware and its characteristics - number of CPU's, on board memory, software environment and build slave capacity for parallel processing. This facility is properly scaled in order to sustain a daily testing round from the different applications developed within ETSF.
| Builder | Brand | CPU | Procs | RAM | OS | bots |
|---|---|---|---|---|---|---|
| testf | Bull R423-E2 | Intel Xeon | 2x Quad | 12GB | CentOS 5.3 | 2 |
| chum | Sun X4200M2 | AMD Opteron | 2x Dual | 32GB | CentOS 5.3 | 1 |
| green-bb | Dell PowerEdge | Intel Xeon | 2x Quad | 16GB | SLinux 5.3 | 2 |
| bigmac | Apple Mac Pro | Intel Xeon | 2x Quad | 6GB | OSX 10.5 | 2 |
| chpit | HP rx4640 | Intel Itanium2 | 4 | 8GB | Debian 5.0.1 | 1 | buda | Transtec | Intel Xeon + GPUs | 2x Quad | 12GB | CentOS 5.5 | 2 |
| inca | HP DC7900 | Intel Core2 | 1x Quad | 4GB | CentOS 5.5 | 1 |
| shiva | HP Z400 | Intel Xeon | 1x Hexa | 12GB | CentOS 5.5 | 1 |
| fock | IBM OpenPower 720 | IBM Power5 | 2x Dual | 32GB | Suse 9.1 | 1 |
| ibm6 | IBM OpenPower 510 | IBM Power6 | 2x Dual | 8GB | AIX 6.1 | 1 | max | Apple XServe | IBM PowerPC G5 | 18x Dual | 18x 4GB | OSX 10.4 | 1 |
| coba2 | HP Z400 | Intel Xeon | 1x Quad | 3GB | CentOS 5.3 | 1 |
| littlebuda | Asus P55 | Intel Core2 + GPU | 1x Quad | 4GB | CentOS 5.5 | 1 |
| woopy | HP DC8100 | Intel Corei7 | 1x Quad | 4GB | Windows XP64 | 1 |
| ktulu | HP DC8000 | Intel Corei7 | 1x Quad | 8GB | Ubuntu 10.04 | 1 |
| toum | HP Z400 | Intel Core2 | 1x Quad | 8GB | Scientific Linux 6.0 | 1 |
Development tools
The following table displays the configured compilers and libraries on each slave. Considering that the ETSF software distribution mainly consists of applications written in Fortran 90/95 language, this family of compiler is reported hereunder.
The philosophy behind the diversity of compiler environments is
- to promote software availability by insuring that the code readily compiles and runs on platforms representative of end users equipment.
- to promote software reliability by making good use of the peculiar behavior of the different environments.
| Builder | g95 | Gfortran | ifort | Open64 | PGI | PathScale | SunStudio | XLF | MPI |
|---|---|---|---|---|---|---|---|---|---|
| testf | 4.4 | 11.1 | openMPI 1.3, 1.4 | ||||||
| chum | 0.9 | 4.2, 4.3 | 9.1, 10.1 | 7.3 | 3.2 | 12 | openMPI 1.2, 1.3 | ||
| green-bb | 0.9 | 4.2 | 10.1 | openMPI 1.3 | |||||
| bigmac | 4.3, 4.4 | openMPI 1.3, 1.4 | |||||||
| chpit | 4.4 | 11.1 | openMPI 1.3 | ||||||
| buda | 4.3, 4.4 | 11.1 | openMPI 1.3 | ||||||
| inca | 4.4 | mpich2 1.2.1 | |||||||
| shiva | 4.4, 4.5 | openMPI 1.4 | |||||||
| fock | 9.1 | mpich2 1.0.8 | |||||||
| ibm6 | 12.1 | POE | |||||||
| max | 0.9 | openMPI 1.3 | |||||||
| coba2 | 4.4 | 11.1 | openMPI 1.4 | ||||||
| littlebuda | 4.5 | mpich2 1.3.1 | |||||||
| woopy | x86_64_mingw32 | mpich2 1.3.1 | |||||||
| ktulu | 4.4 | openMPI 1.4 | |||||||
| toum | 4.4, 4.6 | openMPI 1.4 |
Target requirements
In order to run efficiently on this automatic build facility, candidate applications are expected to follow several design rules
- the process is triggered by modifications to the code. Therefore the source trees and revisions are accessible by the build master and slaves through a code management server as for example Subversion, CVS or Bazaar
- the build process is unattended. The code configuration process is assisted by utilities like Autotools to attain portability among the different slaves environment.
- the code comes with a selection of tests cases appropriate to the slaves workload in order to check the accuracy.
User interface
During the build cycles, relevant status information is saved by the slaves and forwarded to the build master. The latter dynamically feeds a webpage.
The results are sorted through build slaves, revision numbers and contributor names. Color codes allows to pinpoint offending build steps and to locate the corresponding code revision and slave. Links from the status grid provide statistics and further information for debugging
Access to the facility
The test farm facility and its management software are accessible to contributors to all ETSF funded applications. Interested parties are invited to use this form to request details and assistance. A support is provided to adapt the system to specific requirements.
Login accounts for users who wish to interactively use the development tools can be arranged on most of the platforms.