BOSS Dataset

Purpose of the annotation

The BOSS dataset is a good and fairly realistic dataset for pose, action and interaction recognition. It was originally captured as part of the Eureka's Celtic Initiative "BOSS : On Board Wireless Secured Video Surveillance BOSS" project.

We have manually annotated the original dataset typically for researchers to use as a ground truth to train and test classifiers to recognise overall poses (e.g. "standing"), actions (e.g. "walking") and interactions (e.g. "fighting").

The data is provided only for research purposes.

Acquisition details: Celtic-Boss
(Celtic-Boss official website)

BOSS was a project funded by the EUREKA Celtic Initiative from Oct 2006-June 2009. In particular, it focused on improving train passenger security through video and audio analysis. It generated a data corpus consisting of video/audio recordings with multiple calibrated cameras of acted scenarios inside a commuter train. As the train is moving, the data has challenging lighting conditions. As the cameras are mounted on-board, the data has challenging viewing conditions (including occlusions). The pictures below illustrate the captured environment, actions and interactions.


Two different kind of actions : laying down and walking.

Multiple people and Interactions

Multiple people running and two different kinds of interactions : handshaking and helping

Simple actions are :

  1. Walking
  2. Sitting
  3. Standing
  4. Laying Down
            Interactions include:

  1. Fighting
  2. Handshaking
  3. Kissing on the cheeks
  4. Hugging
  5. Helping

Video recordings

Different camera views of the same scene.

The original recordings used 10 calibrated cameras with audio installed in various places inside the moving train. The original data corpus only includes global descriptions, typically of security alarms, of each video/audio sequences e.g. "Harass". This is what we call here the "Original Dataset", which is an unaltered copy of the dataset provided by the BOSS project, and provided here for convenience.

We have augmented the Original Dataset with temporal localisation of actions and some interactions. That is what we call here the "Annotations File".

We are grateful to Mr Gaetan Crevin, a Masters student from Telecom Physique Strasbourg (Université de Strasbourg, France), who ground-truthed this dataset as part of his internship at the Applied Artificial Intelligence Research Group, Universidad Carlos III de Madrid, Spain. We are also grateful to Mr. Diego Gomez-Lira, then a student at the Universidad de Santiago de Chile, who also ground-truthed part of this dataset.