Wine Review Intelligence Platform
Collects expert critic reviews from four paywalled sources, matches each to the industry wine ID and runs from a chat command.
Overview
A wine intelligence business needed a large, clean database of professional critic reviews. We built a scraping and enrichment platform that pulls reviews from four leading critic sources, matches every review to the LWIN11 wine identifier, checks quality automatically and is controlled through a dashboard and a plain-language AI agent.
The Challenge
The best wine reviews sit behind paywalls, logins and bot protection across several sites with different score scales. Collecting them by hand for tens of thousands of wines was impossible.
The Solution
Four-source scraping engine
For Jancis Robinson, Robert Parker, James Suckling and Decanter, each as its own plug-in module.
Wine matching and extraction
Score, critic, review text, drinking window, vintage and source URL, matched to LWIN11.
Automatic quality control
Catches paywalled rows, duplicates and stub entries.
Dashboard and AI control agent
Monitor jobs, download files and type commands like "start Decanter from row 3000".
How It Works
Wine list (CSV / batch)
Source modules
Authenticated scraping
Match to LWIN11
Quality control
Dashboard + export
Architecture
Backend
Flask API with blueprints and dynamic source dispatch.
Database
SQLite in WAL mode with resume snapshots for long batches.
Source modules
One module per critic site, each handling its own login and scale.
De-duplication
Note fingerprinting so repeated reviews are stored once.
Live updates
Socket.IO pushes job progress to the dashboard in real time.
Dashboard
React and Vite app with job control, wine list and file export.
Engineering Highlights
- Batches of 5,000 wines with resume support, so a long run never restarts from zero.
- Handles four different score scales (/20 and /100) in one consistent schema.
- Codebase audit and redesign: fixed CORS, removed hardcoded secrets and committed cookies, added a modern dashboard.
- Docker deployment on a VPS with a custom domain.
Outcome
- 35,000+ wines processed across four critic sources.
- 12,000+ verified reviews delivered in clean, structured files.
- Delivered in paid milestones.