CompareGo

The Problem

When I started shopping online more seriously, the friction of opening five tabs to compare the same product across Amazon, Flipkart, and a couple of other e-commerce sites got old fast. I wanted a single page that told me where the price was lowest — and ideally, whether the user reviews for that product were positive or negative, so I could tell a "cheap but junk" listing apart from a "cheap and solid" one.

This was around the time I was teaching myself Python and picking up the basics of web scraping and machine learning, so a price-comparison-plus-sentiment-analysis project was a natural way to combine all three into something I would actually use.

What I Built

CompareGo is a small Flask web app that does exactly that. You run python main.py, open http://localhost:5000, type in a product keyword, and the app scrapes a handful of e-commerce sites, extracts the prices, and renders them side by side. There is also a sentiment-analysis route where you can paste in a user review (or a batch of them) and the app rates each one as positive, negative, or neutral with a 1–5 star score.

Under the hood it is built from:

• Python + Flask for the web server, which listens on :5000
• BeautifulSoup + Requests to pull product listings from e-commerce sites
• A Support Vector Machine (SVM) classifier trained on the Amazon Reviews dataset from Kaggle to score the sentiment of user reviews

How It Works

The app has two mostly independent pipelines glued together by a thin Flask front end.

Price comparison pipeline

User types a keyword
  -> Flask route receives the request
  -> For each target e-commerce site:
        -> Requests fetches the search-results page
        -> BeautifulSoup parses out product names + prices
  -> Results are normalised (currency, decimals, sort order)
  -> Template renders a comparison table

Each target site has its own small scraper module, because every site has a different DOM, a different URL structure, and a different way of encoding prices in the HTML. The keyword is the only thing shared across them.

Sentiment analysis pipeline

User submits a review
  -> Flask route receives the text
  -> Text is tokenised + vectorised with the same TF-IDF vocabulary
     the model was trained on
  -> SVM classifier predicts a label (positive / negative / neutral)
  -> Predicted label + a 1-5 star rating are returned to the UI

The model itself is a straightforward SVM trained offline on the Amazon Reviews Kaggle dataset. The dataset already has labelled reviews and ratings, so the training step is just data prep, fitting, and serialising the model and its vectoriser to disk. At inference time the Flask app loads the pickled artefacts and runs the same vectorisation before predicting.

Key Design Decisions

Why SVM for sentiment?

SVM is a classic baseline for text classification and trains in seconds on a corpus the size of Amazon Reviews. It also serialises to a tiny .pkl, which makes it trivial to ship alongside a Flask app — no separate model server, no GPU, no ONNX conversion. For a hobby project I wanted to actually deploy on a laptop, that was the right tradeoff.

One Flask app, two features

I kept scraping and sentiment analysis in the same process instead of splitting them into two services. The traffic is hobby-scale, the deployment story is python main.py, and there is a single requirements.txt to install. Price comparison is read-only HTTP scraping, sentiment is pure CPU inference, so the two do not compete for resources in any meaningful way.

The Amazon Reviews dataset

I used the Amazon Reviews dataset from Kaggle directly rather than trying to scrape and label my own reviews. For a learning project, having a clean, labelled, large corpus meant I could spend my time on the model and the UI instead of the data pipeline. It also made the model reusable — even if the scraper breaks tomorrow, the sentiment model can be lifted out and dropped behind any other source of text.

A Note on Status

I should be upfront: this project is unmaintained. The e-commerce sites I was scraping have changed their DOMs (and in some cases their terms) since I last touched the code, and the price-comparison pipeline is no longer functional. The sentiment-analysis part may still work, since it does not depend on any external site.

I am not actively maintaining CompareGo, but I have left the repo up. It is a clean, small example of combining a scraping pipeline with a trained ML model behind a single Flask app, and it is a fine starting point if you want to fork it and point it at different sites or a fresh dataset.

Lessons Learned

Building CompareGo taught me a few things that have stuck with me since:

• Web scrapers are inherently fragile. Any change in the target site's markup silently breaks the pipeline. This project is the cleanest example I have made of that lesson — the sentiment model still works, the scraper does not, and the only thing that changed in the world is the DOM of the sites I was hitting.
• Public datasets are an underrated shortcut. The Amazon Reviews corpus on Kaggle let me skip the data-collection grind and focus on the model and the product. I default to "is there a labelled dataset for this?" before I think about scraping or labelling my own data now.
• SVM is a perfectly good baseline. For binary sentiment on bag-of-words text, a linear SVM is fast, small, and surprisingly competitive. I have used it as a sanity-check baseline in other projects since.
• Hobby projects do not have to be finished to be useful. CompareGo shipped, taught me a lot, and is now a small public artefact. I am happy with that even though the scraper is dead.

Links

• GitHub

Timeline

Built as a hobby project while I was learning Python, web scraping, and basic ML. The Flask app came together quickly — the price-comparison pipeline and the SVM sentiment classifier were both working by the end of the initial build. The scraper later broke when target e-commerce sites changed their DOM structure, and a final cleanup commit was pushed in April 2025 before the project was effectively archived.

• Jun 2023 — Initial development: price scraping pipeline with BeautifulSoup/Requests + SVM sentiment model trained on Amazon Reviews dataset
• Jun 2023 — Flask web server running on :5000, price comparison + sentiment analysis working
• Apr 2025 — Final cleanup commit; project archived as target e-commerce sites changed their DOM structure