Dishes: a restaurant-oriented food dataset

Dishes: a restaurant-oriented food dataset

Ruihan Xu, Shuqiang Jiang, Luis Herranz
Institute of Computing Technology, Chinese Academy of Sciences

Overview

Typical food recognition datasets only include food images and food categories. In contrast, Dishes is a restaurant-oriented dataset suitable to study both visual and context-based food recognition.

The dataset consists of dish (i.e. food category in a restaurant menu) images augmented with restaurant information. This information includes the geographic location of the restaurant and the menu (i.e. dish categories in that restaurant).

Some characteristics of the dataset:

Hierarchical organization: restaurant - (menu, location) - images (see Fig. 1).
All the images are collected in 6 cities (see Table I).

Figure 1: Example of data in the Beijing subset

Table I: Overall statistics (6 cities).

Data collection

Data was collected from a online restaurant review site. For each restaurant, we included the corresponding geographical coordinates and a menu with at least 3 dish categories. For each dish category, at least 15 images were included. Restaurants not meeting these requirements were discarded.

In the experiments we use 10 images to train classifiers, and the rest to test them.

Resources

Datasets:

Dish Images (.ZIP): Beijing Shanghai Tianjin Nanjing Hangzhou Guangzhou

Geographical Coordinates (.CSV): Beijing Shanghai Tianjin Nanjing Hangzhou Guangzhou

References:

Ruihan Xu, Luis Herranz, Shuqiang Jiang, Shuang Wang, Xinhang Song and Ramesh Jain,"Geolocalized Modeling for Dish Recognition, " IEEE Transactions on Multimedia (TMM), vol. 17, no. 8, August 2015,pp. 1187-1199 [pdf]
Luis Herranz, Ruihan Xu and Shuqiang Jiang," A probabilistic model for food image recognition in restaurants, "International Conference on Multimedia and Expo (ICME), June 2015, pp. 1-6 [pdf]
Luis Herranz, Shuqiang Jiang, Ruihan Xu," Modeling Restaurant Context for Food Recognition," IEEE Transactions on Multimedia, October 2016,PP19(2): 430-440 [pdf]
Weiqing Min, Shuqiang Jiang, Jitao Sang, Huayang Wang, Xinda Liu, Luis Herranz,"Being a Super Cook: Joint Food Attributes and Multi-Modal Content Modeling for Recipe Retrieval and Exploration," IEEE Transactions on Multimedia, vol. 19, no. 5 ,MAY 2017,PP:1100-1113 [pdf][dataset]

More details and examples (Beijing subset)

Location of the restaurants (See Fig.2).

Figure 2: Geographic distribution of the restaurants (Beijing city).

The number of dishes per restaurant and images per dish varies significantly (See Fig.3).

(a)

(b)

Figure 3: (a) number of dish categories per restaurant, (b) number of images per dish category (Beijing data)

Diversity of the data (see Fig. 4). Since the data was shared by users according to their interests (who tend to share unfamiliar dishes, nicely presented dishes, exotic dishes, etc.), and the strong filtering process (min 3 dishes/restaurant, 15 images/dish), the remaining dishes tend to be very special dishes, rather than everyday dishes eaten in more common restaurants. This makes this dataset different from other more generic food datasets, containing mostly everyday food.

Figure 4: Examples of dish names and photos available in Dishes.

Contact: sqjiang [at] ict.ac.cn

Key Laboratory of Intelligent Information Processing
Institute of Computing Technology, CAS