Flickr Data Challenges


The goal of this data challenge is to execute feature extraction on images. The Image Feature Extraction tool is composed of a Java application, some Perl scripts and a C application. The Java code implements a client that contacts the Flickr database (, downloads a set of users (limited to 5 for interaction) and the images that these users are sharing over the Web. The Perl script and the C application are the core of the Feature Extraction process, they extracts feature from the images, create thumbnails and (using a Java client) store the results on a cluster located at CNR.

Since the application runs on Java 1.5 (not supported on gLite nodes) we have to download Java binaries before launching the application. However, this overhead is not so penalising and the Feature Extraction process ratio has been significantly improved. The input files are stored on our servers and the PPS support is limited to the computing power required to process such collection.

The characteristics of the data challenge are:

  • 1000 jobs submitted per day (through 2 WMSs), although this can be increased/decreased as needed
  • Each job processes 1000 images and requires at most 50 Mb of disk space and at least 512 of RAM
  • Jobs consume between 20 minutes and 1 hour of CPU time (depending on CPU)
  • Sites do not need to install any particular libraries or other software


  • Start: Monday 16 July
  • End: Friday 31 August


-- PedroAndrade - 30 Jul 2007

Edit | Attach | Watch | Print version | History: r7 | r5 < r4 < r3 < r2 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r3 - 2007-08-28 - PedroAndrade
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    DILIGENT All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback