Product recognition

Prior to my work on copy detection, I worked on visual product recognition, first at GrokStyle and then at Facebook AI.

Typically, product recognition systems must generalize to novel products that are not seen during training. This can be achieved by approaching product recognition as a visual search task, searching images against a database of product images. Robust product recognition systems may additionally involve object detection or semantic segmentation models to localize products within an image.

My work focused on visual search descriptor models, trained using deep metric learning and image retrieval techniques.

GrokNet KDD 2020

I worked on a visual product search model for Facebook and Instagram, which launched in 2019. This model was supervised with a combination of metric learning product recognition objectives and auxiliary classification objectives.

This system was expanded to become GrokNet, a comprehensive multi-task computer vision model for commerce.

GrokNet architecture. GrokNet trains on a large number of tasks over several datasets.

GrokNet trains on multiple objectives, including product recognition, category classification, and attribute prediction.