ProfileCorrections < ArdaGrid

ArdaGrid Web>SiteStatusBoard>ProfileCorrections (2020-04-21, MaartenLitmaath) (raw view)
---+ How to apply corrections to a profile

%TOC%

---++ Introduction

A profile defines how to combine several metrics using an algorithm. The result is a new metric that calculates the availability.

The different operations that can be used to combine metrics are depicted in Figure 1.

<img align="middle" alt="Slide1.jpg" height="270" src="%ATTACHURLPATH%/Slide1.jpg" width="360" />

The last step before calculating the availability is to overwrite  the results obtained so far with a new metric, usually called '<PROFILE> corrections'. This allows to change manually the availability of sites. This document describes how to do that.

---++ How to populate a 'corrections' metric

When a profile gets created, a metric with the profile name appended by ' corrections' is also created automatically. This metric could be fed like any other metric in SSB, for instance giving a source URL where to fetch the data

, or by using the web UI to modify the values. Since the changes in corrections are not that common, the easiest way is to use the http post method, to send the new values whenever there is a need to change the availability. The format of the file that has to be uploaded is a tab separated file with:

&lt;start time&gt; &lt;end time&gt; &lt;metric id&gt; &lt;instance&gt; &lt;status&gt; &lt;color&gt; &lt;url&gt; &lt;options&gt;

See the 'Example' section for more details.

Once the new metric has been populated, SAM3 will recalculate the availability. Note that the system will recalculate everything starting from the time specified in the correction. In other words, if a correction is done for a six months ago, the system will recalculate everything from six month ago until now.  
---++ Timeline

This is a rough estimate of the time needed for the system to calculate the new availability:

   * Let's assume that the corrections metric is posted at t0
   * At t1 =~ t0 + 2 minutes, those values would be inserted in the database, and it should be visible in the history of the corrections metric at the history of the corrections of the profile
   * At t2 =~ t1 + 2 minutes, the collector that calculates the availability will take those changes. The time needed by the collector depends on the time period that has to be recomputed. For periods smaller than a month, a couple of minutes should be enough.
   * At t3 =~ t2 + <time to recalculate> + 2 minutes, the history of the corrected metric should be available at the history of the corrected profile
   * At t4 =~ t3 + 2 minutes, the daily and hourly availabilities should have been calculated, and the new availability should be visible in the SAM3 UI

So, in total, from the moment that the corrections metric gets populated until the changes are visible on the UI, there might be a 10 minute delay

The SAM3 UI provides also a checkbox to display the availabilities before and after the corrections.

---++ Current profiles and correction metrics

This table contains the metric ids for the current profiles and the corrections. It is possible to find these ids also in the [[http://wlcg-mon.cern.ch/dashboard/request.py/columnsform][ metric list]]

| *Profile Name* | *VO* | *Profile status Id* | *Corrections id* | *Profile status corrected id* |
| ALICE_CRITICAL | ALICE |  396 | 718 | 736  |
| ALICE_MONALISA  | ALICE | 703 | 716 | 734 |
| ATLAS_CRITICAL | ATLAS | 413 | 727 | 751 |
| ATLAS_AnalysisAvailablity | ATLAS | 790 | 788 | 791  |
| CMS_CRITICAL | CMS | 418 | 732 | 749 |
| CMS_CRITICAL_FULL | CMS | 561 | 728 | 745  |
| LHCb_CRITICAL | LHCb | 400 | 720 | 738 |

---++ Possible values for the availability

The following table describes the values that are understood for availability calculation

| *Status* |  *Color* | *Numerical Value* |
| OK | green | 0 |
| WARNING | yellow | 10 |
| DOWNTIME | saddlebrown | 15 |
| CRITICAL | red | 20 |
| UNKNOWN  | grey | 30 |

---++ Example

Let's assume that it is needed to change the availability of CERN to OK for the time period between the 1/11/2014 14:00 UTC to 1/11/2014 16:30 UTC, for the ALICE_MONALISA profile.

The steps to do are the following:
   1 *Find out the metric of the corrections.* Looking at the table above, it is metric 716.
   1 *Create the file with the changes that have to be applied.*
      1 echo -e '2014-11-01 14:00:00\t2014-11-01 16:30:00\t716\tCERN\tOK\tgreen\tNone\tnvalue=0 ' &gt; /tmp/my_data
   1 Upload the file. Please, *note that the user has to be authorized!*
      1 curl -k -X POST -T /tmp/my_data --cert .globus/usercert.pem --key .globus/userkey.pem 'https://wlcg-mon.cern.ch/dashboard/request.py/postMetricValues'

*NOTE:* =curl= does not seem able to handle the certificate key being protected with a password... You can create yourself a copy of
the key without a passphrase and perhaps put it in a special location that is just used for this purpose.  For example:
<verbatim>
    (umask 077; openssl rsa -in ~/.globus/userkey.pem -out ~/private/sam-recalc.pem)
</verbatim>
Then use the latter key in the =curl= command.

Next, the system is supposed to calculate the availability, and there are no more actions required from the user.
To validate the the change has been applied, the intermediate steps that can be done are:
   1 Verify that the metric history of the corrections has been updated at [[http://wlcg-mon.cern.ch/dashboard/request.py/siteviewhistory?columnid=716][ALICE_MONALISA corrections history]]. This might take a couple of minutes
   1 Verify that the metric of the corrected history has been updated at [[http://wlcg-mon.cern.ch/dashboard/request.py/siteviewhistory?columnid=734][ALICE_MONALISA corrected history]]. This might take around 5 minutes.
   1 Verify that the availability has been recalculated at [[http://wlcg-sam-alice.cern.ch/templates/ember/#/historicalsmry/heatMap?end_time=2014%2F11%2F02%2000%3A00&group=All%20Sites&profile=ALICE_CRITICAL&site=CERN&start_time=2014%2F11%2F01%2000%3A00&time=Enter%20Date...][SAM3 UI]]. This might take around 10 minutes

---++Troubleshooting:

Here we have a list of possible errors and the way to solve them:
   * <PRE>-bash-4.1$ curl  -k -X POST -T /tmp/my_data --cert .globus/usercert.pem --key .globus/userkey.pem 'https://wlcg-mon.cern.ch/dashboard/request.py/postMetricValues'
* Unable to load client key -8178.
</PRE> . The key in the userkey.pem file is not in unencrypted RSA format. Please, check the content of the file, If it is in  'ENCRYPTED CERTIFICATE', then <i>openssl rsa -in userkey.pem -out unencr_key.pem </i> can be used to convert the key to the required format.


-- Main.PabloSaiz - 2014-11-11
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
jpg	Slide1.jpg	r1	manage	80.6 K	2014-11-11 - 11:47	PabloSaiz
Topic revision: r5 - 2020-04-21 - MaartenLitmaath
ArdaGrid
ArdaGrid Web
ArdaGrid Web Home
Changes
Index
Search
Public webs
Welcome Guest
- Cern Search
- TWiki Search
- Google Search
ArdaGrid All webs
Copyright &© 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback