Skip to Main Content

Test Farm

In order to ensure efficient development and dissemination of quality software tools, a farm of platforms is made available for debugging and testing purposes.

This facility is open to all developers within ETSF.

Aim of the facility

ETSF software offer consists of several large-scale products. Their development involves frequent updates by teams of programmers. In order to ensure that high levels of quality are attained, the portability and reliability of the applications are regularly checked on a diversity of environments by means of standardized procedures. They are performed on a farm of platforms running software builders detailed hereunder.

Users interested by the concepts behind a test farm may download this document presented during the CECAM tutorial in Zaragoza

Implementation details

The setup consists of several computers selected for their diverse characteristics controlled by a client/server software environment named Buildbot.

Software environment

Buildbot is a system designed to automate the configure - compile - test routine commonly exercised by software development projects in order to validate code changes as described on this utility homepage. It relies on a client/server paradigm:

  • the server or build master is the central point of control. It timely receives software modification notices, distributes the build tasks and gathers the status information from the slaves to inform the user about the whole process success or failure.
  • the client or build slave performs the actual compilation and test tasks. It proceeds through a series of prepared build steps consisting of shell scripts, compiler invocations, test commands ...

Buildbot environment, client and server operations

Hardware offer

Buildbot operations rely on several computers hosted at ETSF central node (Louvain-la-Neuve, Belgium). The build slaves offer a wide variety of configurations - a mixture of little endian and big endian (multi)processors systems representative of both desktop computers and large scale computing facilities.

A diversity of open source and proprietary operating systems, compilers and parallel processing libraries is provided. This table summarizes the available hardware and its characteristics - number of CPU's, on board memory, software environment and build slave capacity for parallel processing. This facility is properly scaled in order to sustain a daily testing round from the different applications developed within ETSF.

 

Development tools

The table displays also the configured compilers and libraries on each slave. Considering that the ETSF software distribution mainly consists of applications written in Fortran 90/95 language, this family of compiler is reported hereunder.

The philosophy behind the diversity of compiler environments is

  • to promote software availability by insuring that the code readily compiles and runs on platforms representative of end users equipment.
  • to promote software reliability by making good use of the peculiar behavior of the different environments.

 

Target requirements

In order to run efficiently on this automatic build facility, candidate applications are expected to follow several design rules

  • the process is triggered by modifications to the code. Therefore the source trees and revisions are accessible by the build master and slaves through a code management server as for example Subversion, CVS or Bazaar
  • the build process is unattended. The code configuration process is assisted by utilities like Autotools to attain portability among the different slaves environment.
  • the code comes with a selection of tests cases appropriate to the slaves workload in order to check the accuracy.

 

User interface

During the build cycles, relevant status information is saved by the slaves and forwarded to the build master. The latter dynamically feeds a webpage.

The results are sorted through build slaves, revision numbers and contributor names. Color codes allows to pinpoint offending build steps and to locate the corresponding code revision and slave. Links from the status grid provide statistics and further information for debugging

 

Build slaves work steps reporting webpage

 

Access to the facility

The test farm facility and its management software are accessible to contributors to all ETSF funded applications. Interested parties are invited to use this form to request details and assistance. A support is provided to adapt the system to specific requirements.

Login accounts for users who wish to interactively use the development tools can be arranged on most of the platforms.

Request an access to the test farm