The size and complexity of software projects have increased considerably in the last years. Huge projects may involve hundreds or thousands of people working simultaneously on the same code base. This has increased the importance of Revision Control Systems. Revision Control Systems keep track of all changes done in a source code. More sophisticated systems may also be able to keep track of branches in the code. Without such tools, the development of huge softwares, e.g. operating systems would not be possible. The Revision Control Systems help, among other things, minimizing the cost of source code maintenance and management. This can be very critical for large projects. Also, these systems help finding out bugs introduced in the code base while adding new features or fixing other bugs. For example, consider a scenario of adding a new feature in a program. This may involve changing the behavior of a previously used function. This may impact other parts of the code base which use it.
The Kratos project  at CIMNE is an open source Multi-physics software based on the Finite Element Method. Kratos due to its multi-disciplinary nature contains a large collection of numerical algorithms, Finite Elements, etc. It is being actively used and developed in CIMNE and also worldwide.
Kratos currently uses CVS (Concurrent Versions System)  as its Revision Control System. Users can fetch the most current version of Kratos via CVS, and use / modify / improve it. Also, users can commit modifications / improvements back to the code base, without much effort.
While this open architecture of Kratos has great advantages, it may cause some problems. For a simple example, consider a user commits a change which affects a wide range of users. As the users regularly update their local code base to get the newest updates, the committed change propagates, and as a result, compilation of a previously working program may fail or results of a program may change. In former case, the existence of a conflict can easy be detected, but in the latter, it may be hidden for a long time, and lead to confusing results.
Although there is no reliable way to prevent such conflicts, and also it is very hard to distinguish them from other improvements done in the code base which may also lead to change of the results, a practical technique is needed to minimize the impact of changes by a user on the others. There are a number of test examples in the Kratos, which can be regarded as a representative of the code base functionality. A major change in the behavior of these test examples may indicate a possible conflict, and needs to be reviewed carefully.
The steady and careful checking process of the test examples is almost impossible for developers and users of Kratos, and needs to be automated. This has been the inspiration of development of an Automated Benchmarking module. During the nightly builds of Kratos, which will be done on the upcoming Kratos server, this module automatically runs all test examples and verifies the results against a reference data set. It will send an email containing the status of each test example to a list of recipients. This can be useful in several ways. First, it will be clear when a conflict has occurred. This will help greatly to find the user committed the changes, ask him / her to correct / undo it. Also, it will let other users to be aware of a conflict in the latest code base, and avoid updating their own local code base, before it has been corrected. This can save them a lot of time and effort.
 Kratos, open source multi-physics software, http://www.cimne.com/kratos
 CVS, Concurrent Versions System, http://www.nongnu.org/cvs
 Kratos wiki, http://kratos.cimne.upc.es/kratoswiki