Using Machine Learning to Predict Amazon Search Rankings

Yet another report, this one from Jumpshot, a data intelligence firm, found that more consumer product searches occur on Amazon than Google. Moreover, 90 percent of Amazon’s product views come from the company’s organic site search and not from advertising or external channels, according to Jumpshot.

Thus given the importance to merchants of optimizing for Amazon’s search engine, A9, it’s worth understanding the ranking factors.

It’s widely reported that the goal of Amazon’s search engine is to rank products according to their sales potential. Many factors could influence sales, such as pricing, reviews, and product page copy. Presumably the products that excel in those areas are rewarded with better rankings.

…given the importance to merchants of optimizing for Amazon’s search engine, A9, it’s worth understanding the ranking factors.

It’s tricky to identify the relative importance of those factors, especially since Amazon doesn’t disclose them. So I attempted to find out.

I’ll explain my process in this article.

Predicting Sales Potential

While browsing Amazon’s “Best sellers” sections for various products, I noticed that in many key categories, such as “Electronics” and “Automotive,” the top sellers typically have the most reviews, or nearly the most.

Could the number of product reviews be a proxy for the sales of a product and thus for rankings? Presumably, reviewers purchase the product before writing about their experience.

To test, I used machine learning. Machine learning can do more than generate predictions. A little-known use of machine learning is to create a model and then learn (in some cases) which features are the most important in making the prediction. I’ll use that approach here, with these steps.

Prepare a machine learning source file with Amazon bestseller information, including reviews.

Augment this source file with review sentiment analysis using Google’s Natural Language API.

Upload this file to BigML, an easy-to-use machine learning tool.

Generate a deep neural network model (i.e., simulate the human brain to recognize patterns) to predict the number of reviews in the dataset.

Review the features that most influence the model’s predictions. These are the factors that are the most important in terms of getting more reviews and, by proxy, sales.

Source File

I found a list of best sellers from Q4 2017 at a JungleScout, an Amazon intelligence tool. The list includes around 10,000 unique products per category, across different categories. I focused on “Automotive.”

JungleScout’s site contained a list a Q4 2017 best sellers on Amazon.

The dataset contains 15 columns, such as the Amazon Standard Identification Number (ASIN), product subcategory, and product name. Here is the full list of columns.

gl_product_group_desc
Subcategory
asin
upc1
item_name
merchant_brand_name
customer_average_review_rating
customer_review_count
has_fba_offer
has_retail_offer
total_offers
min_price
max_price
min_3p_price
max_3p_price

I also wanted to extract the product review text and use it to calculate the sentiment of the reviews in case they are predictive. An assistant professor of computer science at the University of California at San Diego, Julian McAuley, has assembled Amazon reviews text. I downloaded automotive reviews from his site for my test.

That dataset has nine columns. Here is the list.

asin
helpful
overall
reviewText
reviewTime
reviewerID
reviewerName
summary
unixReviewTime

I combined both datasets, which provided many potential predictive factors, as follows.

reviewerID
asin
reviewerName
helpful
reviewText
overall
summary
unixReviewTime
reviewTime
gl_product_group_desc
Subcategory
upc1
item_name
merchant_brand_name
customer_average_review_rating
customer_review_count
has_fba_offer
has_retail_offer
total_offers
min_price
max_price
min_3p_price
max_3p_price

Next, I wanted to capture the sentiment of the reviews.

Sentiment of Reviews

Google’s Natural Language Processing API can help. I processed the review texts in that tool and captured four additional fields: Clearly Positive, Clearly Negative, Neutral, and Mixed. Each of those fields contained a “document score,” “magnitude per document,” and the “highest-scoring sentence.”

Google Natural Language Processing API can identify emotions and sentiments behind text — reviews on Amazon in this case.

To be sure, reviewers on Amazon also provide a rating (one to five stars) and I have that in the dataset. But I wanted to see if a more granular analysis would provide additional predictive factors.

Here are example document and sentence sentiments for product B00GG9FB8U.

{'asin': 'B00GG9FB8U',
'best_sentence_magnitude': 0.8,
'best_sentence_score': 0.8,
'document_magnitude': 7.3,
'document_score': 0.1}

After adding the sentiments to our dataset, I am ready for the most exciting part: learning which factors are the most predictive.

Machine Learning with BigML

I uploaded our source file to BigML, the aforementioned machine-learning tool.

I selected the customer_reviews_count as the predictive objective and a deep neural network as the type of machine learning model to build because it is typically the most powerful.

BigML searched 128 combinations of models to find the best performing. Here are the results in order — the top predictors of sales.

Subcategory 86.73%
Field1 (product number) 9.6%
Item_name 3.49%
Total_offers 0.06%
Upc1 0.04%
Customer_average_review_rating 0.03%
Max_price 0.02%
Min_price 0.01%

I was surprised that the review sentiment had no impact at all and that ratings (“6. Customer_average_review_rating”) and price (“7. Max-price” and “8. Min_price”) had very little predictive impact.

A product’s category on Amazon is the best predictor of sales, according to a machine-learning analysis using BigML.

But I can now see how the choice of product category and product name could have a significant impact because some products and categories are inherently popular with strong demand. Likewise, the number of product offers predicted overall sales, too.