How Shoppermotion, an IoT platform for retailers, uses DigitalOcean for reliable data streaming

Shoppermotion is an IoT (Internet of Things) startup that anonymously collects paths customers take through a store automatically in real-time without any interaction. The technology processes millions of visits and serves custom analytics for merchandising, category, and marketing teams by tracking the movements of shoppers through a retail location. The innovation from Shoppermotion is cutting-edge technology that allows brick-and-mortar stores to leverage customer journey insights previously only available to e-commerce sites.

Often, retailers have ample data about the behaviors of online shoppers. Customer journeys for online segments are analyzed, mapped, and optimized every step of the way. Historically, what a shopper does upon entering a retail location has been unknown. Shoppermotion steps in and fills that gap, providing data on customer behavior inside the store through technology that allows them to track the movement of the shopper from entry to checkout.

Support for tech startups

The founding team at Shoppermotion participated in Techstars Accelerator in 2014. As the winners of that competition, they were introduced to DigitalOcean’s Hatch Startup Program. For a short time, Shoppermotion was multi-cloud, using DigitalOcean alongside two hyperscalers, but as they became familiar with the platform, they found DigitalOcean to be more developer-friendly and cost-effective without sacrificing performance, so they migrated their existing architecture to DigitalOcean.

“Hyperscalers are more difficult to use, which is fine if you have a huge DevOps team. But when you are starting out, it’s all about building and not wasting time having to tweak instances and do DevOps. We were amazed at how easy DigitalOcean is to use and it let us spend more time focused on our product.” - Marco Doncel, CTO, Shoppermotion

Reliable data streaming and reporting with DigitalOcean

In order to provide industry-leading analytics, Shoppermotion has to solve a complex set of problems with a distributed system that’s an ideal fit for DigitalOcean Kubernetes.

Shoppermotion places a device inside shopping carts that sends a continuous feed of data to their platform, tracking movements as shoppers use the carts. The device can be mapped to receipt numbers allowing retailers to align shopper movements to what was or wasn’t purchased without revealing the identity of individual shoppers. With these insights, brick-and-mortar retailers can make data-informed decisions about advertising, store layouts, and more. Most of the information generated by Shoppermotion is leveraged in Retail Media Networks to boost the advertising strategies and measure media execution in-store.

Shoppermotion deploys its own hardware to the stores and then maintains the software, data transfers, processing, and reporting platforms for its users. The organization has a machine learning algorithm that helps identify outliers in the data—for example, if a shopping cart has been moved by an employee or if an individual abandons their cart in the middle of a journey.

The tracking system will collect data in real time and transmit it over to hardware installed in the store that looks like a raspberry pi or microcomputer affixed with custom sensors. When anonymous data is captured, it’s sent to Shoppermotion’s hardware that holds lightweight containers running Python code and using MongoDB as a database store. They use MQTT (MQ Telemetry Transport) as a messaging system to send data to their data analysis cluster.

Within their data analysis cluster on DigitalOcean Kubernetes, Kafka plays an important role in building their data pipelines. These pipelines integrate with Apache HBase, which keeps huge volumes of data, and Apache Spark, which handles the data batch and streaming processes into MongoDB. Cloudera is used as a management plane for their pipelines. A data team can then consume this information through a Rails frontend or through 3rd party apps and APIs.The Shoppermotion platform can be customized for instant decision-making by personnel at a store or from several stores to find more insights.

Plans for expansion

The Shoppermotion team is just getting started. They’re expanding their regions, and more stores are adopting their technology every day. With half of the TOP 10 global grocery retailers leveraging their data daily, they provide their services in 25 countries, creating a new ecosystem of in-store analytics for the largest food retail players.

They’re pioneering a new way of delivering retail analytics, and the desire for their technology is bound to grow.