Show Children Hide Children

Main FTS Pages
FtsRelease22
Install
Configuration
Administration
Procedures
Operations
Development
Previous FTSes
FtsRelease21
FtsRelease21
All FTS Pages
FtsWikiPages
Last Page Update
Main.unknown
2010-04-30

Checksums support in FTS

This page describes how FTS handles checksums.

Parameters at Job Submission

As a global option one can enable comparison of source and destination checksums with the --compare-checksums option.

For each file one can specify a third parameter with checksum algorithm:checksum value, which will be used as the basis of comparison with the destination.

glite-transfer-submit ... --compare-checksums source destination ADLER32:ba0bab

See the status of this feature and the similar feature of lcg_utils.

Behavior of transfers

Get destination checksum

How to execute the "get destination checksum" step in the use cases:

  • if destination is a SURL:
    • if srmLs returns a value, use it
    • otherwise:
      • issue an srmPrepareToGet on the destination
      • once the TURL is ready, get the checksum using gridftp
  • if destination is a gsiftp:
    • get the checksum using gridftp

Use case 0: Client does not give any extra options

glite-tarnsfer-submit source-SURL destination-SURL

No checksum operations/comparison.

Use case 1: Client provides a checksum

glite-transfer-submit source-SURL destination-SURL ADLER32:12345678

Rationale: the client knows the checksum and wants to make sure that this is what arrives to the destination. If the source SE does not support checksums that is not a problem.

  1. Get source checksum.
    • Source is a SURL: get source checksum during the preparation phase via srmLs, but do not fall back to gridftp.
    • Source is a gsiftp: do not get checksum.
    • if checksum is the same as the user supplied value: CONTINUE
    • if checksum is not available: CONTINUE
    • if checksum is different: ERROR
  2. transfer succeeds
  3. If the destination is a SURL: call srmPutDone on the destination
  4. get destination checksum
  5. compare the checksums
    • if checksum is the same as the user supplied value: OK
    • if checksum is not available: ERROR
    • if checksum is different: ERROR

Use case 2: Client does not provide checksum, but requires comparison

glite-transfer-submit --compare-checksums source-SURL destination-SURL

Rationale: the client does not know the checksum, but wants to make sure that the transfer had no errors.

  1. Get source checksum.
    • If the source is a SURL: get source checksum during the preparation phase
      • if srmLs returns a value, use it
      • if not available, get the checksum using gridftp
    • Source is gsiftp: get the checksum using gridftp
  2. if checksum is not available: ERROR
  3. transfer succeeds
  4. If the destination is a SURL: call srmPutDone on the destination
  5. get destination checksum
  6. compare the checksums
    • if checksum is the same as the one from the source: OK
    • if checksum is not available: ERROR
    • if checksum is different: ERROR

Use case 3: Client provides checksum and requires comparison

glite-transfer-submit --compare-checksums source-SURL destination-SURL ADLER32:12345678

Rationale: the client knows the checksum and wants to make sure that this is what arrives to the destination. The client wants to make sure that the transfer does not even start, if the source SE has a corrupted file, not to waste the transfer bandwidth.

  1. Get source checksum.
    • If the source is a SURL: get source checksum during the preparation phase
      • if srmLs returns a value, use it
      • if not available, get the checksum using gridftp
    • Source is gsiftp: get the checksum using gridftp
  2. compare the checksums
    • if checksum is the same as the user supplied value: CONTINUE
    • if checksum is not available: ERROR
    • if checksum is different: ERROR
  3. transfer succeeds
  4. if the destination is a SURL: call srmPutDone on the destination
  5. get destination checksum
  6. compare the checksums
    • if checksum is the same as the user supplied value: OK
    • if checksum is not available: ERROR
    • if checksum is different: ERROR


Last edit: UnknownUser on 2010-04-30 - 15:44
Number of topics: 1

Maintainer: AkosFrohner


Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r6 - 2010-04-30 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EGEE All webs login

This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright & by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Ask a support question or Send feedback