Clemson Cafeteria Dataset

Description

This website contains data for 276 people eating a meal at the Harcombe Dining Hall at Clemson University, a large cafeteria setting.

Data includes wrist motion (accelerometer x, y, z; gyroscope yaw, pitch, roll) and a table embedded scale (tray weight), all synchronized to video cameras positioned in the ceiling that recorded the head and torso of each subject as they ate.

For more background see the Bite Counter Project or this paper.

Example

An example of some data is shown in the following figure (click to enlarge). The 7 axes of data from top to bottom are accelerometer x, y, z, gyroscope yaw, pitch, roll, and scale. The green vertical line indicates the time displayed in the video panel.

Wrist motion and scale data

All data was recorded at 15 Hz. The following table shows a sample. Click here to see the full text data file from which the sample is an excerpt.

1.856 1.631 2.391 1.339 1.216 1.294 828.90
1.856 1.631 2.391 1.339 1.216 1.294 828.10
... ... ... ... ... ... ...

Columns 1-3 are from a ST Microelectronics LIS344alh accelerometer. Units are volts. The values can be converted to gravities using G=(V-1.65)x(5.0/3.3) where V is the voltage and G is the gravities. Columns 4-6 are from a ST Microelectronics LPY410al gyroscope. Units are volts. The values can be converted to deg/sec using D=(V-zero)x400 where V is the voltage and D is the degrees per second. The value for zero should be calculated independently for each file as the average value for each axis in the entire file. It will typically be around 1.25. This allows for small voltage drifts for the 0 deg/sec reference point. Column 7 is from a OHAUS Scout Pro SP4001 scale. Units are grams. The scale values will very occasionally read 0.0. These are an artifact of synchronization and should be ignored or replaced with the average of surrounding non-zero values.

Ground truth

Two types of ground truth are available, bites and gestures, each depicted in the following figures (click either to see an animation):

An excerpt of bite ground truth is shown in the following table (click here to see the full text file).

8 1537 right fork plate vegetable_stir_fry
9 1769 left hand glass water
... ... ... ... ... ...

Column 1 is the bite index. Column 2 is the data index at which the bite occurred. Column 3 is the hand used (left, right or both). Column 4 is the utensil used (fork, spoon, chopsticks or hand). Column 5 is the container used (plate, bowl, glass or cup). Column 6 is the food or beverage consumed (click here for a complete list of unique food items).

An excerpt of gesture ground truth is shown in the following table (click here to see the full text file).

utensiling 438 501
bite 503 528
... ... ...

Column 1 is the gesture type (bite, drink, rest, utensiling or other). Column 2 is the data index at which the gesture starts. Column 3 is the data index at which the gesture ends. Note that this ground truth only describes gestures taken with the dominant (instrumented) hand.

An enhanced gestures ground truth is available that describes bite and drink gestures taken with either hand. We used it for gesture recognition from video. It uses the same format as above.

Video

The full-size videos are 480 x 640 and were recorded at 30 Hz. These instructions describe how to crop each video to a smaller resolution and lower frame-rate for video-based gesture recognition. This list provides cropping details for each video file.

Demographics

The demographics for all subjects including handedness can be downloaded here.

Downloads

Data is stored by participant (p###) then course (c#) then filename. Each participant may have eaten their meal in more than 1 course if they left the table while eating their meal (for example went to obtain a second serving, or to get a dessert).

Click here to download all the wrist motion and scale data files (55 MB).

Click here to download all the ground truth bite files.

Click here to download all the ground truth gesture files (dominant/instrumented hand only).

Click here to download all the ground truth enhanced gesture files (either hand, useful for video-based gesture detection).

Click here to download all the full-size video files for each recording (18 GB).

Cafeteria Dataset / Clemson / ahoover@clemson.edu