DataLens AI
Full-stack data quality platform that ingests CSV datasets, profiles row- and column-level statistics, and computes an overall quality score. The ML pipeline uses Isolation Forest for anomaly detection, KNN imputation for missing values, and TF-IDF + cosine similarity for near-duplicate detection. LLM-driven root cause analysis built with LangGraph and Google Gemini explains why each quality issue occurred. JWT auth, Dockerized backend on AWS EC2 behind Gunicorn, Postgres on Neon, frontend on Vercel.
See more
- Problem
- "Is this dataset any good?" is normally answered by hand.
- Approach
- Automated profiling plus three ML passes, with an LLM layer explaining causes in structured output.
- Result
- Upload a CSV, get a scored, diagnosed, explained quality verdict.
Under the hood
Profiling
Row- and column-level statistics — type inference, null density, cardinality, distribution shape — rolled into a single quality score trackable across uploads.
The scikit-learn pass
| Model | Question it answers |
|---|---|
| Isolation Forest | Which rows don't belong? |
| KNN imputation | What should the missing values have been? |
| TF-IDF + cosine similarity | Which rows are near-duplicates of each other? |
Root cause
A LangGraph + Gemini layer converts raw findings into plain-language explanations plus structured fix recommendations under an enforced schema, so the frontend renders them instead of parsing prose.
Shipping it
Docker + Gunicorn on EC2, managed Postgres, frontend on Vercel. ML runs inside the request cycle, so latency was a design constraint rather than an afterthought.
