Extracting logs from a run to a text file

You would like to follow this procedure if you want to inject a particular run to a replay

  • Go to the TransferSystem folder:
     ssh vocms001
     cd /data/TransferSystem 
  • Execute the following command to filter the logs of a particular run, replacing the fields <RunNumber> for the desired run. Be careful with the quotes in the template:
     cat Logs/General.log* | grep "'Tier0Inject' => '1'" | grep "'RUNNUMBER' => '<RunNumber>'" > Run<RunNumber>.txt 
    An example command would be:
     cat Logs/General.log* | grep "'Tier0Inject' => '1'" | grep "'RUNNUMBER' => '270271'" > Run270271.txt 

The file created (Run270271.txt in the example) will contain the desired logs.

  • You can check the file content to be sure that you get what you need. It should be contain several lines like this:
     Tue Apr 19 11:55:52 2016: { 'Tier0Inject' => '1', 'APP_NAME' => 'CMSSW', 'APP_VERSION' => 'CMSSW_8_0_3',
     'HLTKEY' => '/cdaq/special/QuietBeams/v1.0.2/HLT/V1', 'NEVENTS' => '2205', 'RUNNUMBER' => '270271', 'CHECKSUM' => '66e3fd7b', 'T0FirstKnownTime' => '1461059712', 
     'PFN' => '/eos/cms/store/t0streamer/Data/Calibration/000/270/271/run270271_ls0001_streamCalibration_StorageManager.dat', 'STREAM' => 'Calibration', 'LUMISECTION' => '1',
     'TYPE' => 'streamer', 'START_TIME' => '1461059709', 'FILESIZE' => '2880293', 'STOP_TIME' => '1461059711', 'LFN' =>   
     /store/t0streamer/Data/Calibration/000/270/271/run270271_ls0001_streamCalibration_StorageManager.dat' } 

Every line is a log transferred transferred to the Tier-0 and it contains the necessary information to configure the processing there.

Checking progress of couch replication

Sometimes replication from local to remote couch may take a while. This replication is typically fired when the agent is restarted. To check the status of the replication you can run the following command within the agent

curl http://admin:couch@localhost:5984/_active_tasks
It will produce an output like the following
[{"pid":"0.659.0>",
"checkpoint_interval":5000,"checkpointed_source_seq":0,
"continuous":true,
"doc_id":"e7d11e5fcfac12f56ece6c3a5ee9fb92",
"doc_write_failures":0,
"docs_read":0,
"docs_written":0,
"missing_revisions_found":0,
"progress":18,
"replication_id":"bea298d8e259d8d36f3c48733f99a30d+continuous",
"revisions_checked":86022,
"source":"http://admin:*****@localhost:5984/wmagent_summary/",
"source_seq":52445521,
"started_on":1461232055,
"target":"https://cmsweb.cern.ch/couchdb/tier0_wmstats/",
"type":"replication",
"updated_on":1461246250},
{"pid":"0.719.0>",
"checkpoint_interval":5000,
"checkpointed_source_seq":0,
"continuous":true,
"doc_id":"e7d11e5fcfac12f56ece6c3a5eec239c",
"doc_write_failures":0,
"docs_read":5449,
"docs_written":5449,
"missing_revisions_found":5449,
"progress":99,
"replication_id":"53340fc64dc8c5e23eece5443305a734+continuous",
"revisions_checked":13238,
"source":"http://admin:*****@localhost:5984/t0_request_local/",
"source_seq":1002338,
"started_on":1461232056,
"target":"https://cmsweb.cern.ch/couchdb/t0_request/",
"type":"replication",
"updated_on":1461246241}]

-- JohnHarveyCasallasLeon - 2016-04-19

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2016-04-21 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Sandbox All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback