YLI Multimedia Event Detection

The YLI-MED corpus is an index of videos from the YFCC100M specialized for Multimedia Event Detection (MED) research. The videos are categorized as depicting one of 10 (so far) target events, or no target event, and annotated for additional attributes of interest. In addition to these annotations, the subsets/YLI-MED/ directory in our AWS S3 data store also includes extracted features for subsets of YLI-MED.

Event Annotations: The index includes annotations for ~2000 videos split between the 10 target events and ~48,600 videos that do not depict those events, divided into standard training and test sets for researchers wishing to compare results. The events included are: Birthday Party, Flash Mob, Getting a Vehicle Unstuck, Parade, Person Attempting a Board Trick, Person Grooming an Animal, Person Hand-Feeding an Animal, Person Landing a Fish, Wedding Ceremony, and Working on a Woodworking Project.

Event names sound familiar? For our first ten event sets, we chose events that were included in the TRECVID MED 2011 evaluation run by the National Institute of Standards and Technology. We are beginning experiments that compare results for classifiers trained on the two datasets, and hope to inspire more research exploring how dataset variation affects classification.

Additional Annotations: The index also includes annotations such as language spoken and whether the video has a musical score. Additionally, information is included on inter-annotator agreement and confidence scores for the event category assigned to each video.

Features: Features currently available are Kaldi pitch, SAcC pitch, and MFCC20s (audio) and AlexNet and LIRE (image). They are computed on (most of) the positive-instance videos in YLI. Additional features may be obtained by accessing the YLI features for all of the videos in the YFCC100M dataset and using the unique identifiers to extract features for the videos you want. Information about specific features, about how the features were computed and how they are organized on the Amazon Web Services (AWS) Public Data Sets platform, and about missing or invalid features, may be found on the YLI Feature Corpus page. (The parameters and organization of features for the YLI-MED subcorpus mirrors that of the full YLI feature corpus.)

Public-Domain License: The YLI-MED annotations and features are licensed under Creative Commons 0, meaning they are in the public domain and free for any use. However, we do appreciate credit as indicated.

Full Documentation/Preferred Citation: Julia Bernd, Damian Borth, Benjamin Elizalde, Gerald Friedland, Heather Gallagher, Luke Gottlieb, Adam Janin, Sara Karabashlieva, Jocelyn Takahashi, and Jennifer Won. 2015. The YLI‐MED Corpus: Characteristics, Procedures, and Plans. ICSI Technical Report TR-15-001. Also available in Computing Research Repository, arXiv:1503.04250. PDF.

Cheers to: Contributors to the YLI-MED subcorpus effort include (but are not limited to) Julia Bernd, Damian Borth, Carmen Carrano, Jaeyoung Choi, Benjamin Elizalde, Gerald Friedland, Heather Gallagher, Luke Gottlieb, Adam Janin, Sara Karabashlieva, Florin Langer, Karl Ni, Jocelyn Takahashi, and Jennifer Won.

Funding: The YLI-MED annotation effort was funded by a grant from Cisco Systems, Inc. (Any opinions, findings, and conclusions expressed on this website are those of the individual researchers and do not necessarily reflect the views of the funders.)

Contacts: Questions about corpus composition or the corpus-gathering process may be directed to Julia Bernd at jbernd[chez]icsi[stop]berkeley[stop]edu. Questions about connecting the corpus index with computed features, original media, or the user-supplied metadata may be directed to Jaeyoung Choi at jaeyoung[chez]icsi[stop]berkeley[stop]edu.