Product recognition

I worked on visual product recognition from 2017-2019 at GrokStyle and Facebook AI.

Product recognition systems must generalize to novel products that are not seen during training. This can be achieved by approaching product recognition as a visual search task, searching images against a database of product images. Robust product recognition systems may additionally involve object detection or semantic segmentation to localize products within an image.

My work focused on visual search descriptor models, trained using deep metric learning and image retrieval techniques.

GrokNet KDD 2020

I worked on a visual product search model for Facebook and Instagram, which launched in 2019. This model was supervised with a combination of metric learning product recognition objectives and auxiliary classification objectives.

This system was expanded to become GrokNet, a comprehensive multi-task computer vision model for commerce.

GrokNet architecture. GrokNet trains on a large number of tasks over several datasets.

GrokNet trains on multiple objectives, including product recognition, category classification, and attribute prediction.