add datasets for space

.gitattributes

@@ -41,3 +41,4 @@ Web/ph_n_t_ch_th_tr_ng/screen.png filter=lfs diff=lfs merge=lfs -text
 Web/ph_ng_th_nghi_m_quy_t_nh/screen.png filter=lfs diff=lfs merge=lfs -text
 Web/t_ng_quan_quy_t_nh_chi_n_l_c/screen.png filter=lfs diff=lfs merge=lfs -text
 Web/tr_l_ai/screen.png filter=lfs diff=lfs merge=lfs -text
+clean_dataset.csv filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,321 +1,184 @@
 ---
-title: PropertyVision BI x RAG
-emoji: 🏢
-colorFrom: blue
-colorTo: indigo
-sdk: docker
-app_port: 7860
+language:
+- vi
+license: other
+task_categories:
+- tabular-classification
+- tabular-regression
+tags:
+- real-estate
+- vietnam
+- hanoi
+- ho-chi-minh-city
+- business-intelligence
+- property-analytics
+- tabular
+- pandas
+pretty_name: Hanoi & Ho Chi Minh City Real Estate Dataset
+size_categories:
+- 100K<n<1M
 ---
 
-# PropertyVision BI x RAG
-
-<p align="center">
-  <img src="docs/assets/readme-banner.svg" alt="PropertyVision BI x RAG banner" width="100%" />
-</p>
-
-> Executive-grade real-estate decision intelligence for **Ho Chi Minh City** and **Hanoi**.
-> BI dashboards, price prediction, what-if simulation, GIS/planning views, and a retrieval-first AI assistant for leadership reporting.
-
-<p align="center">
-  <a href="https://github.com/QuangVoAI/PropertyVision/releases/tag/v1.0.0"><img src="https://img.shields.io/badge/release-v1.0.0-0f766e?style=for-the-badge" alt="release v1.0.0" /></a>
-  <a href="https://github.com/QuangVoAI/PropertyVision"><img src="https://img.shields.io/badge/stack-FastAPI_%7C_React_%7C_RAG-1d4ed8?style=for-the-badge" alt="FastAPI React RAG" /></a>
-  <a href="https://huggingface.co/datasets/SpringWang08/hanoi-hcmc-real-estate"><img src="https://img.shields.io/badge/dataset-Hugging%20Face-f59e0b?style=for-the-badge" alt="Hugging Face dataset" /></a>
-  <a href="https://huggingface.co"><img src="https://img.shields.io/badge/AI-Hosted%20Qwen%20%2B%20RAG-7c3aed?style=for-the-badge" alt="Hosted Qwen plus RAG" /></a>
-  <a href="https://github.com/QuangVoAI/PropertyVision"><img src="https://img.shields.io/badge/NLP-Retrieval--First%20Assistant-0891b2?style=for-the-badge" alt="NLP assistant" /></a>
-</p>
-
-<p align="center">
-  <a href="https://github.com/QuangVoAI/PropertyVision"><img src="https://img.shields.io/badge/FastAPI-0ea5e9?style=flat-square&logo=fastapi&logoColor=white" alt="FastAPI" /></a>
-  <a href="https://github.com/QuangVoAI/PropertyVision"><img src="https://img.shields.io/badge/React-0f172a?style=flat-square&logo=react&logoColor=61dafb" alt="React" /></a>
-  <a href="https://github.com/QuangVoAI/PropertyVision"><img src="https://img.shields.io/badge/RAG-0f766e?style=flat-square" alt="RAG" /></a>
-  <a href="https://github.com/QuangVoAI/PropertyVision"><img src="https://img.shields.io/badge/NLP-7c3aed?style=flat-square" alt="NLP" /></a>
-  <a href="https://github.com/QuangVoAI/PropertyVision"><img src="https://img.shields.io/badge/Metro-2563eb?style=flat-square" alt="Metro impact" /></a>
-  <a href="https://github.com/QuangVoAI/PropertyVision"><img src="https://img.shields.io/badge/v1.0.0-111827?style=flat-square" alt="Version 1.0.0" /></a>
-</p>
-
-## Quick Links
-
-- [What You Get](#what-you-get)
-- [Quick Start](#quick-start)
-- [Environment Variables](#environment-variables)
-- [Hugging Face Space](#hugging-face-space)
-- [Metro Impact Data](#metro-impact-data)
-- [Useful Backend Endpoints](#useful-backend-endpoints)
-- [Documentation](#documentation)
-
-## What You Get
-
-- 📊 Executive dashboard with market KPIs and trend views
-- 🧩 Multi-dimensional slice-dice analysis
-- 📈 Price prediction and ROI simulation
-- 🗺️ Planning/GIS map with opportunity and risk views
-- 🤖 RAG-based assistant grounded in market, planning, legal, and metro context
-- 📝 Export-friendly periodic report view for leadership updates
-
-## At a Glance
-
-| Item | Value |
-|---|---|
-| Release | `v1.0.0` |
-| Main stack | `FastAPI + React + Vite` |
-| AI layer | `Hosted Qwen + retrieval-first RAG` |
-| Markets covered | `Ho Chi Minh City`, `Hanoi` |
-| Metro scope | `Bến Thành`, `Tham Lương`, `HCMC TOD`, `Hanoi TOD` |
-| Primary dataset | `datasets/clean_dataset.csv` |
-
-## Architecture
-
-```mermaid
-flowchart LR
-    U[User] --> F[React + Vite Frontend]
-    F --> B[FastAPI Backend]
-
-    HF[Hugging Face Dataset<br/>SpringWang08/hanoi-hcmc-real-estate] --> D[datasets/clean_dataset.csv]
-    D --> M[Runtime Data Mart<br/>SQLite + Pandas]
-    M --> B
-
-    B --> A[Analytics / Prediction / Simulation]
-    B --> G[GIS / Planning / Metro impact]
-    B --> R[RAG Retriever]
-    R --> Q[Hosted Qwen]
-    Q --> O[Executive response]
-
-    A --> F
-    G --> F
-    O --> F
-```
-
-## Repository Layout
-
-```text
-PropertyVision/
-├── backend/             FastAPI app, analytics, RAG, metro/planning data
-├── frontend/            React + Vite UI
-├── datasets/            Processed dataset, dataset notes, cached reference data
-├── docs/                Diagrams, baseline notes, demo scripts, UI spec
-├── data/                SQLite runtime artifacts
-├── README.md           Project overview and setup
-└── requirements.txt    Python dependencies
-```
-
-## Data Model
-
-The application works with a processed dataset and runtime-generated analytical layers:
-
-- `datasets/clean_dataset.csv` is the main processed dataset
-- `data/*.db` is created at runtime for facts, planning zones, legal notes, and metro impact profiles
-- the backend also builds a cached street-level reference for richer RAG answers
-
-### Automatic dataset behavior
-
-On first backend start, the app will try to:
-
-1. download the processed dataset from Hugging Face
-2. store it locally as `datasets/clean_dataset.csv`
-3. fall back to the local file if it already exists
-4. fall back to raw reference data in `datasets/raw/` if needed
-
-This means a fresh clone can usually start without manual data copying.
-
-Dataset links:
-
-- https://huggingface.co/datasets/SpringWang08/hanoi-hcmc-real-estate
-- https://huggingface.co/datasets/tinixai/vietnam-real-estates
-
-## Quick Start
-
-### 1. Clone
-
-```bash
-git clone https://github.com/QuangVoAI/PropertyVision.git
-cd PropertyVision
-```
-
-### 2. Set up the backend
+# Hanoi & Ho Chi Minh City Real Estate Dataset
 
-macOS / Linux:
+## Dataset Summary
 
-```bash
-python -m venv .venv
-source .venv/bin/activate
-pip install -r requirements.txt
-uvicorn backend.main:app --reload
-```
+This dataset is a cleaned, consolidated, and analysis-ready real estate dataset covering two major Vietnamese urban markets:
 
-Windows PowerShell:
+- **Hanoi**
+- **Ho Chi Minh City**
 
-```powershell
-python -m venv .venv
-.venv\Scripts\Activate.ps1
-pip install -r requirements.txt
-uvicorn backend.main:app --reload
-```
+It was prepared for **PropertyVision BI**, a business intelligence and decision-support project focused on property analytics, price prediction, ROI analysis, planning-risk exploration, and dashboard-based market monitoring.
 
-Backend URL:
+The dataset is designed for:
 
-```text
-http://localhost:8000
-```
+- exploratory data analysis
+- dashboarding and BI use cases
+- tabular machine learning
+- market segmentation
+- investment scenario analysis
 
-### 3. Set up the frontend
+## Why This Dataset Matters
 
-```bash
-cd frontend
-npm install
-npm run dev
-```
+This dataset was built to demonstrate more than simple data collection. It reflects a practical end-to-end analytics workflow that is highly relevant for portfolio and CV use cases:
 
-Frontend URL:
+- multi-source tabular data consolidation
+- schema harmonization across two major cities
+- feature standardization for business intelligence and machine learning
+- rule-based enrichment for incomplete records
+- validation-oriented preprocessing for downstream product stability
 
-```text
-http://localhost:5173
-```
+For recruiters or reviewers, this dataset highlights work across **data cleaning, dataset design, feature engineering, data validation, and analytics product thinking** rather than only model training.
 
-## Hugging Face Space
+## Quick Stats
 
-This repository already includes a `Dockerfile`, so you can upload it to **Hugging Face Spaces** as a Docker Space with minimal extra work.
+| Metric | Value |
+|---|---:|
+| Total rows | 106,205 |
+| Cities covered | 2 |
+| Ho Chi Minh City rows | 23,722 |
+| Hanoi rows | 82,483 |
+| Main data file | `clean_dataset.csv` |
+| Format | CSV |
+| Primary use | BI, analytics, ML experimentation |
 
-- The backend serves the built frontend from `frontend/dist`
-- The app runs on port `7860` in Spaces
-- Use the root `README.md` as the Space landing page
+## Dataset Structure
 
-If you want a shorter Vietnamese guide for the same project, see [README.vi.md](README.vi.md).
+The main file in this repository is:
 
-## Environment Variables
+- `clean_dataset.csv`
 
-The app works in retrieval-only mode without a hosted LLM token, but you can enable hosted generation for richer analysis.
+Each row represents a processed property listing or synthesized listing record prepared for downstream analytics.
 
-Recommended variables:
+### Core Columns
 
-```bash
-HF_TOKEN=your_hugging_face_token
-PROPERTYVISION_HF_QWEN_MODEL=Qwen/Qwen2.5-1.5B-Instruct
-PROPERTYVISION_HF_INFERENCE_PROVIDER=auto
-PROPERTYVISION_USE_HOSTED_QWEN=true
-```
-
-Optional `.env` file at the project root is supported.
+- `Location`: normalized location label
+- `Price`: human-readable property price
+- `Type of House`: property category
+- `Land Area`: human-readable area label
+- `Bedrooms`: bedroom count label
+- `Toilets`: toilet count label
+- `Total Floors`: number of floors
+- `Main Door Direction`: main door orientation
+- `Balcony Direction`: balcony orientation
+- `Legal Documents`: legal status label
+- `price_vnd`: numeric property price in VND
+- `area`: numeric land/building area
+- `price_per_m2`: numeric price per square meter
+- `district`: district label
+- `purchase_price`: estimated purchase price proxy
+- `current_price`: current price proxy
+- `ROI`: return-on-investment proxy
+- `date`: normalized date field
+- `city`: city label
+- `source_dataset`: original source file identifier
 
-### Notes
+## Data Sources
 
-- If no hosted model is available, the app still runs with retrieval-backed analysis.
-- If you want faster local debugging with less AI overhead, keep the hosted model disabled.
+This dataset is a curated derivative dataset built from cleaned source data used in the PropertyVision project.
 
-## Core Features
+The underlying information was assembled from public and educational data sources referenced in the project, including public planning/GIS references and real estate market proxy data.
 
-### 1. 📌 Tổng quan điều hành
+This repository publishes the **processed analytical dataset**, not a claim of ownership over all raw source records.
 
-- KPI trọng yếu
-- xu hướng điều hành dài hạn
-- kiểm tra giả định tăng trưởng
-- khuyến nghị dành cho ban điều hành
+Because this is a derivative analytical release, users should review upstream source terms before using the dataset for redistribution or commercial applications.
 
-### 2. 🏙️ Thông tin thị trường
+## Preprocessing and Curation
 
-- so sánh khu vực
-- mặt bằng giá
-- phân tích phân khúc
-- insight theo thành phố / quận / loại tài sản
+The published dataset was prepared through several curation steps:
 
-### 3. 🔎 Phân tích đa chiều
+1. schema standardization across city-level datasets
+2. column normalization for price, area, ROI, district, and date fields
+3. consolidation of Hanoi and Ho Chi Minh City records into one tabular dataset
+4. generation of business-ready derived fields such as `purchase_price`, `current_price`, and `price_per_m2`
+5. consistency validation for property-type-specific rules
 
-- slice-dice theo khu vực và phân khúc
-- bảng phân đoạn tiềm năng cao
-- xem danh sách địa chỉ theo từng record
+## Important Note on Synthetic Enrichment
 
-### 4. 📊 Mô phỏng đầu tư
+Some Hanoi records originally had fewer attributes than the Ho Chi Minh City dataset.
 
-- giá trị tương lai
-- lợi nhuận vốn
-- ROI tích lũy
-- thời gian hoàn vốn
-- khuyến nghị mua thêm / giữ / bán bớt
+To create a unified schema for analytics and demo applications, selected fields for Hanoi were enriched using **deterministic rule-based preprocessing**, including:
 
-### 5. 🗺️ Bản đồ quy hoạch
+- property type inference
+- bedroom and toilet count estimation
+- floor count estimation
+- legal document label approximation
+- directional field completion
+- normalized location labeling
 
-- opportunity score
-- risk level
-- bộ lọc theo ROI, score và rủi ro
-- dữ liệu quy hoạch, legal, và metro impact
+These enriched values were created to support **schema consistency, dashboard stability, and machine learning experimentation**.
 
-### 6. 🤖 Trợ lý phân tích
+They should be treated as **synthetic analytical enrichments**, not guaranteed ground-truth metadata.
 
-- hỏi đáp theo ngữ cảnh RAG
-- nguồn trích dẫn rõ ràng
-- khuyến nghị ngắn gọn theo giọng điều hành
+## Intended Use
 
-### 7. 📝 Báo cáo định kỳ
+Recommended use cases:
 
-- bản tóm tắt kiểu executive report
-- hỗ trợ in ra PDF từ trình duyệt
+- BI dashboards
+- market comparison between Hanoi and Ho Chi Minh City
+- regression experiments for property price estimation
+- ROI analysis
+- feature engineering practice
+- educational data projects
+- portfolio projects in analytics, data science, and data engineering
 
-## Metro Impact Data
+## Skills Demonstrated
 
-The backend now includes a dedicated metro-impact layer for real estate analysis:
+This dataset release demonstrates the following practical skills:
 
-- Ho Chi Minh City metro line 1
-- Ben Thanh central station
-- Tham Luong station / metro line 2 gateway
-- Hanoi TOD and urban rail corridor references
+- data cleaning and tabular preprocessing
+- schema design and dataset consolidation
+- feature engineering for property analytics
+- rule-based synthetic enrichment for incomplete records
+- validation rule design for data consistency
+- business intelligence thinking for dashboard-ready data products
+- preparation of shareable ML and analytics assets for public repositories
 
-This layer is available through the RAG pipeline and the data-ops view so the assistant can answer questions like:
+## Author
 
-- “Metro ảnh hưởng giá nhà như thế nào?”
-- “Bến Thành và Tham Lương tác động ra sao?”
-- “Hà Nội và TP.HCM khác nhau thế nào quanh ga metro?”
+Curated and released by **Xuan Quang Vo** as part of the **PropertyVision BI** project.
 
-There is also an API endpoint:
+## Limitations
 
-```text
-GET /api/metro/impact
-```
+- This dataset is intended primarily for **analytics and educational use**.
+- Some fields are cleaned proxies rather than directly verified transaction records.
+- Some Hanoi attributes are **synthetically enriched** for consistency.
+- The dataset should not be used as the sole basis for legal, financial, or investment decisions.
 
-## Refreshing Data
+## Citation
 
-If you change planning, legal, metro, or market sources, refresh the runtime layers:
+If you use this dataset, please cite it as:
 
-```text
-POST /api/etl/run
-POST /api/rag/reindex
+```bibtex
+@dataset{springwang08_hanoi_hcmc_real_estate,
+  author = {Xuan Quang Vo},
+  title = {Hanoi \& Ho Chi Minh City Real Estate Dataset},
+  year = {2026},
+  publisher = {Hugging Face},
+  url = {https://huggingface.co/datasets/SpringWang08/hanoi-hcmc-real-estate}
+}
 ```
 
-You can also use the **Theo dõi dữ liệu** page in the UI to do this.
-
-## Useful Backend Endpoints
-
-- `GET /api/metadata`
-- `POST /api/analytics`
-- `POST /api/slice-dice`
-- `POST /api/predict`
-- `POST /api/what-if`
-- `GET /api/map/districts`
-- `GET /api/planning/zones`
-- `GET /api/metro/impact`
-- `POST /api/rag/reindex`
-- `GET /api/etl/status`
-- `GET /api/ai/status`
-
-## Notes For Contributors
-
-- The repository is designed so that a new clone can run end-to-end with minimal manual setup.
-- Avoid committing generated runtime files from `data/` and downloaded dataset artifacts unless intentional.
-- If you update the dataset or knowledge base, reindex RAG so the assistant reflects the latest state.
-
-## 📚 Documentation
-
-- [Project Diagrams](docs/PROJECT_DIAGRAMS.md)
-- [Technical Baseline](docs/BASELINE.md)
-- [Demo Script](docs/DEMO_SCRIPT.md)
-- [UI Design Spec](docs/UI_DESIGN_SPEC.md)
-
-## 🎁 Release Notes
-
-- Official `v1.0.0` release for executive-grade real-estate intelligence.
-- Adds a cleaner onboarding README so new contributors can clone and run faster.
-- Includes metro-impact data and RAG coverage for Ho Chi Minh City and Hanoi.
-- Keeps the AI experience retrieval-first, with hosted Qwen available when configured.
-
-## 🪪 License / Data Use
+## Acknowledgement
 
-This project aggregates public, processed, and derivative analytical data for BI and demonstration purposes.
-Please review the source terms of any upstream data before redistribution or commercial use.
+This dataset card and processed release were prepared as part of the **PropertyVision BI** project for academic, portfolio, and demonstration purposes.

clean_dataset.csv

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5d523121a35db3987fd8b5eff64758bcd7c11a5c085878119211eef213fa600
+size 28136118