YLI-GEO: Placing Task Datasets

Sample image: Space shuttleThe Multimedia Commons feature corpora are providing the data for the MediaEval Benchmarking Initiative‘s Placing Task for 2014, 2015, and 2016, including computed audio and visual features commonly used for automatic location estimation. The datasets for the three years are called MP14, MP15, and MP16. Together they comprise the YLI-GEO dataset.

The images and videos for YLI-GEO are drawn from the geotagged data in the YFCC100M. In total, YLI-GEO comprises around 6 million photos and around 100,000 videos, divided into training and test sets for each year’s release. The training set for each year remains fairly constant, with minor adjustments to reflect changes in the nature of the task. (Some items were removed from the 2015/2016 training index that were not mappable to geographical administrative units — cities, countries, etc. — as the 2015 and 2016 Placing Tasks focused on such units.) Each year’s test set is new.

Check out the 2016 Placing Task guidelines

Currently Available: As of June 2016, the YLI-GEO data available via our Amazon Web Services (AWS) data portal includes:

  • A full set of computed audio and visual features for MP14, in the subsets/YLI-GEO/features directory
  • In process: computed audio and visual features for MP16; these are being uploaded to the subsets/YLI-GEO/mp16 directory.

A complete index will be available soon — try checking AWS in case this explanation is out of date!

Features: The features available for each Placing Task dataset with YLI-GEO vary by year, as new features get added.

MP14 and MP15 features include:

  • Gist, LIRE, and SIFT visual features for static images and video keyframes
  • MFCCs20, Kaldi pitch, and SAcC pitch audio features for videos

MP16 features include:

  • Gist, LIRE, VGG, and hybrid-CNN visual features for static images (all uploaded as of 8/5/2016)
  • Gist, LIRE, and SIFT visual features for video keyframes (not yet uploaded as of 8/5/2016)
  • MFCCs20, Kaldi pitch, and SAcC pitch audio features for videos (all uploaded as of 8/5/2016)

Information about specific features, about how the features were computed and how they are organized on the Amazon Web Services (AWS) Public Data Sets platform, and about missing or invalid features, may be found on the YLI Feature Corpus page (Gist, LIRE, SIFT, all audio features) and the Additional Feature Corpora page (hybrid-CNN, VGG). (The parameters and organization of features for the YLI-GEO subset mirrors that of the full feature corpora.)

Miscellaneous Documentation:

  • Training and test indexes for the 2014 Placing Task are in subsets/YLI-GEO/docs.
  • Details about organization and format of the MP14 features are in subsets/YLI-GEO/features/README_YLI-GEO_10-9-2015.txt.
  • Place hierarchy metadata for the 2016 Placing Task is in subsets/YLI-GEO/mp16/metadata.
  • However, you will find the most complete and reliable information about the subparts of YLI-GEO in the 2016 Placing Task description or on the Placing Task home page, or by contacting the Placing Task organizers.

Media and Metadata: Currently, the original media files for YLI-GEO are not available on AWS as a separate bundle; they may be obtained by pulling the appropriate files from the top-level data/ directory. The original metadata (including geotags) may be obtained by requesting the YFCC100M dataset from Yahoo’s Webscope portal.

Public-Domain License: Most of the YLI-GEO feature sets are licensed under Creative Commons 0, meaning they are in the public domain and free for any use. However, we do appreciate credit as indicated below. Exception: The VGG features are licensed by ITI-CERTH and CEA List under a Creative Commons Attribution license (CC-BY); credit should be given to the producers. (The YFCC100M metadata and the original media for the items in YLI-GEO are subject to the Creative Commons licenses chosen by the uploaders.)

Preferred Citation: Jaeyoung Choi, Bart Thomee, Gerald Friedland, Liangliang Cao, Karl Ni, Damian Borth, Benjamin Elizalde, Luke Gottlieb, Carmen Carrano, Roger Pearce, and Doug Poland. 2014. The Placing Task: A Large-Scale Geo-Estimation Challenge for Social-Media Videos and Images. In Proceedings of the 3rd ACM Multimedia Workshop on Geotagging and Its Applications in Multimedia (GeoMM ’14). New York: ACM, 27-31. PDF.

Cheers to:

  • MediaEval Placing Task Organizers (2014, 2015, & 2016): Liangliang Cao, Jaeyoung Choi, Gerald Friedland, Claudia Hauff, Bart Thomee, and Olivier Van Laere.
  • YLI-GEO Feature Computation: Damian Borth, Carmen Carrano, Jaeyoung Choi, Benjamin Elizalde, Fabrizio Falchi, and Adrian Popescu.

Funding: The YLI-GEO features from the YLI Corpus were funded by a collaborative LDRD project led by LLNL under the U.S. Dept. of Energy (contract DE-AC52-07NA27344), and by a National Science Foundation grant for the SMASH project (grant IIS-1251276). Features drawn from the ISTI Deep Feature Corpus were funded by the European Commission (grant 325122) and the Region of Tuscany (grant CUP CIPE D58C15000270008). Features drawn from the VLAD-VGG-YFCC Dataset were funded by the European Commission (grants 287975, 611596, and 610928. (Any opinions, findings, and conclusions expressed on this website are those of the individual researchers and do not necessarily reflect the views of the funders.)

Contact: Questions may be directed to Jaeyoung Choi at jaeyoung[chez]icsi[stop]berkeley[stop]edu.

Advertisements