Stager database scaling test with CASTOR 2.1.9-6

This test is about scalability of the stager database in terms of numbers of files on disk.

As CMS is already having > 20M files on disk, the target is to test 100M files on disk.

The only target being the scalability database, we will mostly deal with fake files that will only exist in the stager DB linked to fake machines and filesystems and will no have any physical counterpart on disk nor any entry in the namespace.

foreseen steps

1. prepare a database with 30M fakes files distributed randomly on 500 fakes machines and 2000 fake filesystems
2. let the system run idle and check the behavior of all automatic jobs (monitoring, synchronization, GC, dumps, ....)
3. delete and recreate ~40M files in order to create holes in the database tables
4. let the system run idle and check the behavior of automatic jobs
5. launch a short stress test (few hours) of the system (with real files this time) and watch the DB
6. add another 30M fake files to go to 60M and replay steps 4 and 5
7. add another 40M fake files to go to 100M and replay steps 4 and 5
8. rerun steps 4 and 5 against the new database hardware

-- SebastienPonce - 27-May-2010