Motivation — The Importance of Analytics in Retail

Retail industry faces a difficult situations as retailers would like to understand the products that are very similar to each other in order to evaluate which product is better off not promoting from the similar products in the same week to increase sales activity and profit margins.

Retailers need to consider various aspects of individual store management in geographically distributed areas to allow continuous flow of inventory at a reduced cost and minimum wastage. In addition, huge volume of online sales have also promoted them to design automated Machine Learning (ML) retail platforms. The problems addressed by the ML platform includes:

• Inventory and supply chain management with assortment planning.
• Analysis of customer buying patterns with invalid/fraud requests.
• Analyzing customer interactions with virtual assistants and chatbots.
• Generating retail analytics for large scale distributed retail systems to understand yearly/monthly growth and aid in forecasting.
• Personalized recommendations using Collaborative filtering, Content-based filtering, Hybrid filtering and others.
• Identifying item displacement/removal in stores.
• Interpreting text and images from bills, packing lists, invoices.

This blog is structured as follows:

• Understanding the life-cycle of machine learning algorithms in retail so as to continuously learn and improve the model accuracy.
• Studying some of the applications of ensembling learning, clustering, PCA and time series forecasting in the space of retail.
• Investigate the retail areas, which can be leveraged with the advanced use of deep learning. In this context, I explain few of the deep CNN models like RCNN, Fast-RCNN, Faster-RCNN and Mask RCNN.

Life Cycle of Machine Learning Algorithms in Retail

The typical lifecycle of retail machine learning algorithms involve steps from gathering the training data, training the model, to continuously work in improving the model performance in a feedback loop.

Dynamic Pricing

Machine learning collect data from customers and help retailers to check and continuously monitor the prices of competitors. This enables retailers to automatically match them or even offer a lower price, so the customers get the best possible deal and stay with the same brand. 

Fashion Trend Analysis

Retailers have been successful in collecting social media data such as likes and comments, analyze customer sentiments, identify customer interests and the hottest trends and finally make relevant offers for specific customers as well as identify what’s selling.

Inventory Management

Machine learning has helped retailers to cope up with demand prediction to prevent additional expenses. It has facilitated easier inventory management, by ordering items to inventory based on the level of demand and replenished stocks.

Ensemble Modeling in Retail

Ensemble modeling in retail is mostly used for Uplift modeling for up-selling, cross-selling, churn prevention and retention activities. The objective is not to predict the likelihood of a customer buying, but what can be done to increase the likelihood of customers’ making a purchase. The models aims at predicting the outcome of a marketing campaign by selecting individuals for whom the action will be most profitable.

Ensemble methods like Bagging, Random Forests and Stacking can be used in uplift modeling. Other ensemble methods exist, such as Extremely Randomized Trees or Random Decision Trees. The difference in the methods lies only in the way by which randomness is injected into the tree learning algorithm to ensure that models in the ensemble are diverse.

Uplift modeling is a popular technique for retail industry by taking into consideration the incremental impacts of marketing campaigns. With limited access to collection of marketing datasets in real-time, success of uplift modeling lies in high model performance of ensembling techniques. This is due to the ensemble diversity arising from the predictor variables that are very weakly correlated with customer behavior.

Clustering in Retail

Store Clustering often used in retail industry, is employed to build relevant store segments which are homogenous in certain behavioral aspects and can be targeted using the same marketing strategy.

Retail store segmentation aids in segmenting stores which have a similar customer-base and location (for e.g. a store in the heart of the city may cater to customer-base similarly). Targeted advertising serves as an outcome of store segmentation and aids in differentiated marketing strategies for each store segment.

Category-based clustering strategy zooms in on specifics and helps to group items and products of a certain category in a specific group.

Customer Segmentation also uses unsupervised clustering techniques like k-means, latent class analysis, hierarchical clustering, etc.

Clustering techniques used in retail sales are: K-means, K-mediods, Density based Clustering, Hierarchical Clustering, Farthest First Clustering , Filtered Clustering.

Principal Component Analysis in Retail

Principal Component Analysis is an important tool for reducing large multivariate information sets to discover unsuspected and unknown relationships.

Principal component analysis is a kind of dimensionality reduction technique for factor analysis, aimed at reducing the initial variable set into a smaller sized set of linear mixes.

The eigenvalues contributes to the overall variation discussed by each contributing market factor governed by customer behaviour. Thats why retail industry uses principal component analysis to isolate the patterns in the information and to channelize the important information to enhance the business. Factor analysis is useful in : condensing variables , uncovering clusters of responses, automatically weighing each of the variables.

A customer items business wishes to evaluate client reactions to numerous qualities of a new moisturizer cream. Their marketing agency carries out a principal elements analysis to figure out whether they can form a smaller set of uncorrelated variables that are much easier to evaluate and analyze. The outcomes recognize the following patterns.

• Using smell, color and texture to isolate the principal components of a “Body Shampoo/”.
• Cleanliness, anti-aging, glowing factor that has an “Effect on skin”.
• Amount had to be applied and cost form a “Value” component.

The evaluation metrics are as follows:

1. Level of customer satisfaction obtained from reviews and feedbacks.
2. Product recommendation to friends and family members.
3. Probability of increase in sales and product enhancement in future for existing and returning customers.

In order to reveal the most striking features among lots of associated variables, principal component analysis can be used to reveal the most popular metrics of the high-dimensional information.

Time series Modeling in Retail

In this section, let us see what are the solutions available to handle missing and incomplete data that frequently occurs in retail analytics. In addition, become familiar how ARIMA/SARIMA can be used for forecasting different stocked/replenished items weekly, monthly, over any periods of time.

Sources of missing data and imputation techniques

Crowdsourced datasets are incomplete due to missing observations/values for important attributes. Data imputation methods can fill in missing values in crowdsourced, regular, discrete, retail price time series datasets. Retail Price Time Series Imputation (RPTSI) has been known to ensemble of three constituent methods in a univariate time series : Price Change Lookup, Central Moving Average, and Polynomial Interpolation. Other techniques of handling missing/sparse data are:

• Model Based Imputation (MBI): is a popular and effective collaborative approach for imputing missing data. In case a data set contains k features, then MBI builds k different classifiers, each trained to predict a different feature as target. Each classifier treats the observed values of the target feature as training set and the missing data as test set.
• Matrix Factorization (MF) exploits hidden features of a matrix by learning latent matrices whose dot product represent the original matrix. Recommender systems use various MF strategies to collaborative filtering of sparse user-item ratings with the aim of recommending unrated pairs.
• Supervised Matrix factorization is known to approximate the original data set by learning latent features. The dot product can reconstruct the original matrix. An important aspect of matrix factorization, is that it creates a latent feature space where all time-series instances jointly share information and all features are treated equally in the objective function, via a stochastic learning technique which is specifically designed for sparse data sets. It improves separability of latent data by enforcing a classification accuracy loss function (e.g. logistic regression loss function).
• Spline Interpolation: Splines are effective intra-series alternatives for reconstructing missing segments, by preserving the continuity and smoothness of the curve. Cubic Spline Interpolation (CSP), interpolate a missing segment by fitting a local cubic polynomial (spline), assuring that the first derivative (tangential) and the second derivative (speed of curvature change) . The process ensures the connection points to observed neighboring segments are preserved.

Forecasting with Time Series

SARIMA models (along with regression-SARIMA) assists in forecasting monthly retail commodity prices. Weekdays and holidays act as external variables in regression-SARIMA model, to identify the importance of predictor variables.

SARIMA model with the external factors (SARIMAX) serves the retail industry in forecasting the daily sales of any retail items such as grocery.

ARIMA and Holt-Winters (HW) models are used in forecasting time series of a group of perishable dairy products. To consider seasonality and holiday effects at different store locations, both ARIMAX and SARIMAX models are better.

Apart from time-series used in demand forecasting, Markov Chains also plays an important role in sales prediction

Word2Vec to Prod2Vec

Similar to word2vec model (in Continuous Bag of Words (CBoW)), where each sentence is composed of words and similar words to a given word are identified by considering contextual words, a similar algorithmic can be used as Prod2Vec. Prod2Vec assumes an individual basket is composed of products, and similar products are identified by considering other products purchased in the same basket.

A visualization of the resulting product vectors of each product when represented using a dimensionality reduction technique calledt-SNE (t-Distributed Stochastic Neighbor Embedding) is depicted in the below figure. Each color in the figure represents a certain set of products and closer the products are, more similar they are. This helps us to translate a product into an n-dimensional vector and solve multiple use cases in retail industry.

Deep learning in Retail Environments

There are several use-cases of item and video identification in retail business where deep learning has helped to solve such problems.

Detecting Item Removal/Displacement

Convolution Neural Networks plays an important role in detecting items removal in a store. The deep neural network takes people’s video who frequently places items on shelves on a vending machine. The camera is mounted at the top of the machine and triggered to record video only when the door is open. The below Figure illustrates a raw video frame.

Several different deep neural network architectures can be used to classify whether items have been removed or added to the shelf within the time frame of the video. Since the video dataset is small, use of pre-trained models available online along with transfer learning (to avoid overfitting) makes it easier to detect the item removed from shelves.

• Use image classification model, SqueezeNet. It helps to identify items through video classification, thereby easing search time.
• Use video classification models like C3D (3D ConvNets) with batch normalization and transfer learning to identify customer actions in the videos.

C3D based Neural Network composed of different Convolution Layers is used to identify objects and preserve scenes as illustrated in the diagram below.

Other Object Tracking methods are the observation model: generative method and discriminative method. These models helps to reduce the reconstruction error to search the object, such as PCA.

The discriminative method being one of the most robust mechanisms can be used to distinguish between the object and the background. To achieve the tracking by detection, candidate objects for all frames are detected and deep learning method used to recognize the wanted object from the candidates.

Product Classification

With the growth of online shopping it becomes essential for a retailer to know popular design styles of clothing products to increase the production of selected styles to achieve more profit. Thus it serves as an important characteristics to classify garments and provide target recommendations to end-users.

More knowledge on image’s textures can help in achieving this. Some Hand-Engineered feature extraction methods such as “Census Transform histogram (CENTRIST), Local Binary Pattern (LBP), Histogram of Oriented Gradient (HOG)” etc do exist and some of them have been able to achieve popularity because of their computational simplicities and better accuracies.

However a specific deep CNN model as stated in the figure below, has been able to achieve better accuracy results in identifying garments design class over Hand-Engineered feature extraction methods. FC layers and Convolutional layers used in a Deep CNN represent the features more elaborately, which are stronger than any of Hand-Engineered feature extraction techniques.

The Convolutional Neural Networks were also evaluated in different classification tasks like differentiating persons from products as well as classifying product images, gender and age of persons.

Object Detection with Region-based Convolutional Neural Networks (RCNN)

The first step in identifying objects is to output bounding boxes and labels for individual objects. There lies 2 classes of object classification, one with object bounding boxes and the other with non-object bounding boxes. For example, in a physical retail grocery basket, all items in the given basket needs to be detected from a given image with their respective bounding boxes.

One approach is to use Sliding Window technique to classify and localize images, where CNN could be applied to many different crops of the image. Because CNN classifies each crop as object or background, application of CNN scales to huge numbers of locations and scales becomes computationally expensive.

Neural network researchers have come up to define regions, where NN models are able to find “blobby” image regions that are likely to contain objects. This helps to scale and run with better latency.

The initial model designed was R-CNN (Region-based Convolutional Neural Network). In a R-CNN, a scan operation on the image looks for possible objects. This Selective Search technique generates~2,000 region proposals. Following this, CNN model is run on top of each of these region proposals. Finally, the output from each CNN is fed it into an SVM to classify the region and a linear regression to tighten the bounding box of the object. From object detection, the problem resolves to an image classification problem.

An advanced version of R-CNN is Fast R-CNN. It improves the detection speed through 2 augmentations:

1. Applying feature extraction techniques before proposing regions. This is achieved through applying CNN over the entire image, and
2. Using softmax layer, in neural network for predictions.

Fast R-CNN shown in the figure above, has tremendous processing speed. The main disadvantage lies in the selective search algorithm still exhibits lot of time to generate region proposals. The lack in improvement speed is because of the fact the region proposals are generated separately by another model.

The next invention in the field of R-CNN is Faster R-CNN, which is a canonical model for deep learning-based object detection. It replaces the slow selective search algorithm with another fast neural network. Here it inserts a Region Proposal Network (RPN) to predict proposals from features. The Faster R-CNN works principally for feature extractions and ROI proposals (ROI pooling) with 2 more convolution layers.

The RPN is acts as a chief decision-maker to infer areas of investigation within the image to reduce the computational requirements of the overall inference process. Its responsible for quick and efficient scan at every location, which helps to evaluate and continue further processing in a given region. The output from the scan results in k bounding box proposals each with 2 scores representing the probability of object or not at each location. The region proposals are then fed into straight into what is essentially a Fast R-CNN. A pooling layer, some fully-connected layers, and finally a softmax classification layer and bounding box regressor are added to the Fast R-CNN to convert it to Faster R-CNN.

Faster R-CNN achieves better speeds and higher accuracy. Future deep CNN models did a lot to increase detection speeds, few models managed to outperform Faster R-CNN by a significant margin. The drawback of Faster R-CNN lies in the feature map selected by RoIPool (Region of Interests Pool) were slightly misaligned from the regions of the original image.

MaskRCNN

MaskRCNN, a model built to very successfully conduct object detection and image segmentation, by adding a branch (Fully Convolutional Network on top of a CNN-based feature map) to Faster R-CNN that outputs a binary mask to detect whether or not a given pixel is part of an object. The Mask R-CNN is a multi-task network that works on a single input (image), to predict multiple kinds of outputs. MaskRCNN constitutes one more variant (on the right at the bottom) in building such mask.

ROI Align in Mask R-CNN

One of the unique features of Mask R-CNN is the refinement of the ROI pooling. In ROI, the warping is digitalized : the cell boundaries of the target feature map are forced to realign with the boundary of the input feature maps.

Mask R-CNN uses ROI Align(Region of Interests Align) which does not digitalize the boundary of the cells (top right) and make every target cell to have the same size (bottom right). The bilinear interpolation applied to calculate the feature map values within the cell better, avoids any error in rounding, preventing inaccuracies in detection and segmentation.

Mask R-CNN have gained popularity due to it’s major advantages-good inference speed, good accuracy, easier implementation process and extension capability. Mask R-CNN combines them with the classifications and bounding boxes from Faster R-CNN to generate precise segmentations. However the major drawbacks are false alerts and missing labels.

With a couple of retail items arranged on a table, is fed into the model, MaskRCNN is able to successfully identify a large variety of different scene objects including juice, glass, cold drinks bottle, detergent bottles, cups, mugs, snacks and other. Further, inside each identified bounding box, the colorful shaded region identifies exactly which pixels in an image correspond to the object. This is referred to as pixel segmentation, and each pixel in the image receives a predicted classification label about the kind of object that pixel belongs to (or background).

The model executes three things: object detection (green boxes), object classification, and segmentation (colorful shaded regions). The green bounding boxes on the image below are the outputs of the model and above each box is a prediction of what kind of object is contained within.

Detecting Text Objects

Object detection models like MaskRCNN are well known in retail space. It provides a flexible mechanism for identifying regions of interest inside images. Non-traditional OCR related to object detection primarily have two classes of objects: text objects and then everything else. A model very similar to MaskRCNN can be trained to identify regions of interest (RoI) in an image that are highly likely to contain text, a process known as textlocalization.

Some of the challenges for text extraction are: document of interest occurs alongside some background objects and the text within the document is highly unstructured. This necessities identification of all possible existing text blocks.

The output of the model is overlaid on the image above — regions of text are identified with dotted-line bounding boxes with the estimated pixel mask for the text. Above each box is the predicted class and confidence score, which, is “Text” in all detected cases, due to the presence of only one object class of interest.

The bounding boxes are tight and encapsulate the text regions fairly accurately. Model MaskRCNN needs modification and training with OCR-relevant datasets to design an effective approach for text localization. The inclusion of pixel masks in the multi-task learning forces the localization (bounding box regression) to be even more accurate.

Hence the primary task is to identify RoIs of an image that correspond to text blocks, then this is obviously of limited utility for OCR. The next job is to read the text contained in each image region; a process known as textrecognition.

OCR in Retail Industry

OCR systems have been a big help to the retail industry allowing companies to scan and extract data from a Bill of Lading, Packing List, Invoices, Purchase Orders and more. OCR’s advanced capture software is a dynamic system that looks for key words on each page to create a template for each invoice based on the defined keywords.

Conclusion

In this article, we have seen how traditional learning and deep learning serves the retail industry in different problem spaces with structured as well as unstructured input data. However there remains much work to be done in the following areas to increase model predictability:

• Nature of volume of discounts, inventory policy, safety stock maintained, assumptions regarding shoppers and use them in decision variables.
• Using transfer learning in similar retail stores to price the products.
• Use of Meta-learning to automate and expedite the learning efficiency.
• Enhance usage of Deep learning based Object Detection learning methods for intelligent store monitoring, maintenance and customer validation.

In the next blog let’s look at how IoT together with Machine Learning, Reinforcement Based Learning and BlockChain is set to create a revolution in the retail industry.

References

1. https://thesai.org/Downloads/Volume9No2/Paper_38-Machine_Learning_Techniques_for_Customer_Retention.pdf
2. https://pdfs.semanticscholar.org/ce4a/d1ad4134d9131af21d4213e598f03475cfd3.pdf