Skip to content
John Hodge

← Projects

Multimodal sensing for Just Walk Out

Proprietary industry work. Described qualitatively, with no internal metrics or code.

Problem

Cashierless retail asks a hard sensing question: when a shopper takes or returns an item, which item was it, and how many? Thin shelf sensors and RFID give you noisy, multi-channel signals, cross-talk between lanes, and environments that drift over a day. The model has to commit to a decision anyway.

Approach

I led capacitive sensing science for AWS Just Walk Out and worked as a senior scientist on the RFID-enabled version, designing multimodal event-detection algorithms across both modalities.

Capacitive shelf sensing. A thin shelf-sensor pipeline for pick, return, and no-action detection: channel-to-lane fusion, CUSUM change detection to find the moment and location of an event, and Bayesian product-capacitance learning to turn a signal pattern into a per-lane product-and-quantity hypothesis.

RFID sensing. Antenna and algorithm work to improve event detection, including an overlapping-area figure of merit (OA-FOM) that quantifies how well an antenna separates target tags from stray tags, with a Python tool that hardware engineers used to evaluate antenna configurations against in-field data. I also applied attention-based transformers to model RFID tag transitions and built automated root-cause analysis to cut algorithm debugging effort. This work powered RFID-enabled Just Walk Out for NFL and MLB stadium retail.

Wireless power. Alongside the sensing work I served as Amazon’s research liaison to Professor Joshua R. Smith’s Sensor Systems Lab at the University of Washington (with PhD student Kedi Yan) on wireless power transfer for retail and robotic applications.

Result

The pipelines reached reliable action and quantity classification on lab data and shipped into retail and stadium deployments, and I aligned a science roadmap across research, hardware, firmware, and ML partners on lane mapping, confidence scoring, environmental compensation, and cross-talk. The throughline with my other work is the same: messy physical signals, real failure modes, and a model that has to survive contact with operational constraints.

Proprietary industry work, described qualitatively, with no internal metrics.