SAM Prod. Service Client-side failures -- No new tests sent to a resource

TODO

  1. Check if the resource (+site) is correctly registered in the SAM DB as explained on General checklist for site/node problems in the SAM DB.
    • NOTE: Carefully check the SERVICEINSTANCE > =ISMONITORED flag, often we have the problem there!!!
  2. Try to see what's going on, using the SAM submission framework manually from the SAM UI(s). NOTE: Due to the submission & publishing cron jobs, these operations might not be allowed due to the locking mechanism on the ~/.same/ directory.
    1. Check the status of the last test(using the samops account):
                   [samops]$ /opt/lcg/same/client/bin/same-exec --status <sensor> nodename=<nodename>
                
      This might be a slow operation. In case if you don't specify nodename filter, think about redirecting the long output (2>&1)
      • If there are any finished jobs, in case of (g)CE, (g)RB, FTS sensors, try to publish the results
                        [samops]$ /opt/lcg/same/client/bin/same-exec --publish <sensor> nodename=<nodename>
                      
      • If there are no pending jobs for the resource, try to submit manually
                       [samops]$ /opt/lcg/same/client/bin/same-exec <sensor> nodename=<nodename>
                    
        • In case of CE sensor, you might get error message:
                            Job already submitted to <nodename>
                         
          If the status check showed that the test on the CE is already in Done state, the job was stuck for some reason.
          1. move the ~/.same/CE/nodes/ directory to somewhere for further analysis and open a Savannah bug with the details
          2. re-submit the job to the site

-- JuditNovak - 01 Dec 2007

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2008-12-15 - JuditNovak
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    LCG All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback