Fine-Grained Visual Recognition of Retail Products

Fine-Grained Visual Recognition of Retail Products

Grocery recognition in supermarkets comprises several challenges as groceries embed small inter-class and intra-class variance. Small inter-class variance is given because different products share substantial visual similarities. Datasets typically contain real-world images and reference images, which induces intra-class variance. The visual appearances of products change over time, and their number continuously grows because designs are reworked or new products are published. Standard object classification methods are inapplicable at scale because models need to be fine-tuned continuously to relax these changing conditions.

In this project, we leverage the burden of requiring all classes to be known at training time using methods derived from face recognition techniques and meta-knowledge derived from additional sensor information. The setting is based on recognizing groceries in unknown supermarkets, e.g., without substantial infrastructural changes. The core idea is to extend face-recognition methods and fine-tune known architectures to distinguish the fine-grained visual differences of grocery products. The required training images are semi-automatically generated using sensor data acquired with modern smart glasses, e.g., the user’s trajectory and a model of the environment. Product candidates in real-world images are found using a sliding window approach, which uses the observation that products are arranged on shelves.

This topic is under ongoing research. For questions refer to Marco Filax.

Publications

2021

Filax, Marco; Gonschorek, Tim; Ortmeier, Frank

Grocery Recognition in the Wild: A New Mining Strategy for Metric Learning Artikel

Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 2021.

Abstract | Links | BibTeX

2019

Filax, Marco; Gonschorek, Tim; Ortmeier, Frank

Data for Image Recognition Tasks: An Efficient Tool for Fine-Grained Annotations Inproceedings

Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods, 2019.

Abstract | Links | BibTeX

2018

Filax, Marco; Ortmeier, Frank

VIOL: Viewpoint Invariant Object Localizator - Viewpoint Invariant Planar Features in Man-Made Environments Inproceedings

Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP), S. 581-588, 2018, ISBN: 978-989-758-290-5.

Abstract | Links | BibTeX

2017

Filax, Marco; Gonschorek, Tim; Ortmeier, Frank

QuadSIFT: Unwrapping Planar Quadrilaterals to Enhance Feature Matching Inproceedings

Proceedings of the 25rd International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, WSCG 2017 - Short Papers Proceedings, 2017.

Abstract | Links | BibTeX

Fine-Grained Visual Recognition of Retail Products