Filesystem and Filesize profiling

The objectif of the questionnaire is to get an idea about :
  • the file size profiling (what is the most frequent filesize stored in a DPM, the maximum size of a file)
  • the filesystem used and its parameters (what type, tuned parameters, etc.)
Thus, the collected statistics will be used to run realistic performance test and to optimize DPM for the reality of the sites.

Questions

1) File size. We need to get some information about the file size. Please run the attached application.

2) What is the maximum file size?

3) What type(s) of filesystem do you have on your DPM disk server? (the command "mount" allows you to know the type)

4) What the current parameters of your filesystem(s)? If you don't use the default ones, we 'd appreciate an explanation of your choices. ( the parameters of a filesystem can be viewed using dumpe2fs -h /dev/XXX if ext3, xfs_info /mountpoint if xfs)

5) Are your DPM head node and DPM disk servers installed on the same machine?

6) What is the physical memory size of your DPM disk server?

To answer question 1) and 2) you must use the program attached. (it runs the SQL queries and report them on a file, so you can send it). Thanks for answering these questions as soon as possible and report any problems/comments.

Results

The result of the questionnaire is attached. It seems that the average file size is around a few MB. There are a lot fo small files with a size equal to 0.

-- LanaAbadie - 15 Jun 2007

Topic attachments
I Attachment History Action Size Date Who Comment
Compressed Zip archivetar GetFileSize_byVO.tar r1 manage 850.0 K 2007-07-13 - 17:38 LanaAbadie Get filesize profiling according to the VO
Compressed Zip archivetar GetFileSize_mysql.tar r1 manage 1860.0 K 2007-06-15 - 10:24 LanaAbadie Answer to questions 1) and 2) of the questionnaire. For DPM based on a MySQL DB only
Compressed Zip archivetar GetFileSize_ora.tar r1 manage 70.0 K 2007-06-15 - 16:19 LanaAbadie Answer to questions 1) and 2) of the questionnaire. For DPM based on an Oracle DB only
Microsoft Excel Spreadsheetxls filesystem_questionnaire.xls r1 manage 33.0 K 2007-07-31 - 12:08 LanaAbadie Result of the questionnaire : questions 1) to 6)
Microsoft Excel Spreadsheetxls site1_cppm1.xls r1 manage 134.5 K 2007-07-31 - 12:07 LanaAbadie Result of the post-questionnaire : Nb of files group by VO and by Size
Edit | Attach | Watch | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r7 - 2007-07-31 - LanaAbadie
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback