A comprehensive pipeline for building a content-based recommendation system for hotel-related businesses using user reviews, tips, and business attributes.
This project implements a multi-stage data processing and analysis pipeline to create a recommendation engine based on hotel characteristics, user feedback, and sentiment analysis. The system filters, processes, and enriches hotel data to generate meaningful content representations for recommendation tasks.
- Data Cleaning & Filtering: Remove redundant information and focus on relevant hotel-business interactions
- Data Reduction: Consolidate multiple data sources into a unified, optimized format
- Content Creation: Generate rich text representations from reviews, tips, and business attributes
- Sentiment Analysis: Distinguish positive feedback for building high-quality content profiles
- Bad Business Identification: Flag hotels with consistently poor reviews for exclusion
- Input: Raw Yelp dataset (reviews, users, businesses, tips)
- Process:
- Analyze rating distributions across 5,857 unique hotel businesses
- Identify user-business interaction patterns
- Remove duplicate user-business pairs, keeping the latest review
- Filter and structure data into clean CSVs
- Output:
filtered/directory containing:review_filtered.csv- De-duplicated reviews (320,925 ratings)user_filtered.csv- User profilesbusiness_filtered.csv- Hotel business detailstips_filtered.csv- User tips
Key Statistics:
- 245,334 unique users
- 5,857 unique hotel businesses
- 320,925 ratings (after deduplication)
- Average rating: 3.38 stars
- Rating distribution highly skewed toward 4-5 stars
-
Input: Filtered CSVs from stage 1
-
Process:
Users:
- Aggregate 11 compliment types into a single
mean_complimentsmetric - Remove irrelevant metrics (useful, funny, cool, elite status)
- Retain key trustworthiness indicators (fans, average_stars, friends_count, review_count)
Businesses/Hotels:
- Remove location-specific data (coordinates, postal codes) - not essential for recommendations
- Remove operational status (
is_open) - recommendations apply regardless of current status - Remove temporal data (
hours) - users show no time-based preferences - Retain: business name, location (city/state), star rating, review count, attributes, categories
- Aggregate 11 compliment types into a single
-
Output:
reduced/directory containing:users.csv- 9 core user featuresbusiness.csv- 8 core business features
-
Input: Reduced dataframes + original reviews/tips
-
Process:
Positive Sentiment Filtering:
- Extract reviews with ≥ 3 stars as reliable descriptors
- Aggregate positive reviews + tips by business
- Creates content profiles capturing what users appreciate
Attribute Enrichment:
- Parse structured business attributes (WiFi, parking, credit cards, etc.)
- Convert boolean/nested attributes into human-readable text
- Include only meaningful positive attributes
Content Aggregation:
- Combine: positive review text + tips + business attributes
- Results in rich, multi-source content representations for each hotel
Bad Business Flagging:
- Identify 151 hotels with exclusively negative reviews (< 3 stars)
- Saved to
bad_business/bad_business.csvfor exclusion from recommendations
-
Output:
bad_business/bad_business.csv- 151 flagged hotels- Content representations ready for vectorization and similarity matching
Raw Yelp Data
↓
[1_data.ipynb] - Filtering & Deduplication
↓
filtered/ ← CSVs (cached, .gitignored)
↓
[2_data_reduction.ipynb] - Feature Engineering
↓
reduced/ ← Optimized CSVs
↓
[3_content_creation.ipynb] - Content & Attributes
↓
Recommendation-Ready Content Profiles + Bad Business List
project/
├── filtered/ # Cached filtered data (stage 1)
│ ├── review_filtered.csv
│ ├── user_filtered.csv
│ ├── business_filtered.csv
│ └── tips_filtered.csv
├── reduced/ # Cached reduced data (stage 2)
│ ├── users.csv
│ └── business.csv
├── bad_business/ # Hotels to exclude
│ └── bad_business.csv
└── [notebooks]
Note: Intermediate CSVs are .gitignored to keep repository size manageable. They're recreated by running notebooks in sequence.
- 5,742 / 5,857 hotels (98%) have positive review content
- 3,578 hotels have tips; 5,706 hotels have positive reviews
- 151 hotels (2.6%) flagged as consistently poor-performing
- Average user gives ~1.3 ratings
- Distribution is highly skewed (few very active users)
- Top 50 hotels account for disproportionate review volume
- Median: 4 stars (50th percentile)
- Mean: 3.38 stars
- Quartiles: 25% at 2★, 75% at 5★
- Strong positive skew in user ratings
-
Ensure you have the raw Yelp dataset in the working directory (or adjust paths in notebooks)
-
Run notebooks in sequence:
jupyter notebook 1_data.ipynb # Generates filtered/ jupyter notebook 2_data_reduction.ipynb # Generates reduced/ jupyter notebook 3_content_creation.ipynb # Creates content profiles -
Access outputs:
- Cleaned user/business data in
reduced/ - Bad business list in
bad_business/ - Content profiles ready for vectorization (TF-IDF, embeddings, etc.)
- Cleaned user/business data in
- Vectorize content using TF-IDF or embeddings (Word2Vec, BERT)
- Compute business similarity matrices
- Implement collaborative filtering or hybrid approaches
- Evaluate on ranking/recommendation metrics
- pandas - Data manipulation and aggregation
- Python 3 - Core language
- Jupyter - Interactive notebook environment
- All intermediate CSV files are cached to avoid reprocessing large datasets
.gitignoreprevents bloat from large CSV files- Positive reviews (≥ 3 stars) are prioritized to avoid negative bias in content profiles
- Business attributes are normalized from nested JSON structures into readable text
Developed for the CAC Assignment 1 project.