Recommender Systems

7.

Evaluation of Recommender Systems

7.1.

Importance of Evaluation

7.1.1.

Measuring System Effectiveness

7.1.2.

Aligning with Business Goals

7.1.3.

Comparing Different Approaches

7.2.

Offline Evaluation

7.2.1.

Data Splitting Strategies

7.2.1.1.

7.2.1.2.

Cross-Validation

7.2.1.3.

Temporal Splitting

7.2.1.4.

7.2.2.

Prediction Accuracy Metrics

7.2.2.1.

Mean Absolute Error

7.2.2.2.

Root Mean Square Error

7.2.2.3.

Mean Squared Error

7.2.3.

Ranking and Classification Metrics

7.2.3.1.

7.2.3.2.

7.2.3.3.

7.2.3.4.

Mean Average Precision

7.2.3.5.

Normalized Discounted Cumulative Gain

7.2.3.6.

7.2.3.7.

Area Under Curve

7.2.4.

Beyond Accuracy Metrics

7.2.4.1.

7.2.4.1.1.

Catalog Coverage

7.2.4.1.2.

7.2.4.2.

7.2.4.2.1.

Intra-List Diversity

7.2.4.2.2.

Inter-List Diversity

7.2.4.3.

7.2.4.3.1.

7.2.4.3.2.

7.2.4.4.

7.2.4.4.1.

Unexpected Relevance

7.3.

Online Evaluation

7.3.1.

7.3.1.1.

Experimental Design

7.3.1.2.

Statistical Significance

7.3.1.3.

Sample Size Determination

7.3.2.

7.3.2.1.

Team Draft Interleaving

7.3.2.2.

Probabilistic Interleaving

7.3.3.

Multi-Armed Bandit Testing

7.3.4.

Key Business Metrics

7.3.4.1.

Click-Through Rate

7.3.4.2.

Conversion Rate

7.3.4.3.

User Engagement

7.3.4.4.

7.3.4.5.

7.3.4.6.

User Satisfaction

7.4.

Evaluation Challenges

7.4.1.

Evaluation Bias

7.4.2.

Temporal Effects

7.4.3.

User Feedback Quality

Previous

6. Advanced Recommender Models

Go to top

Next

8. Practical Challenges and System Design