Regression Model Performance Evaluation

[thehanggit]

The regression models are trained every 3 months using data from the 3rd day of the beginning of each season. Since imputed data lacks ground truth, we cannot directly evaluate the performance by comparing predictions on imputed data. Instead, the evaluation focuses on assessing how well the trained model generalizes to missing data.

Evaluation Steps:

1. EDA analysis on the training data to examine whether distribution, correlation, and normality assumption are satisfied.
2. Choose one month of data within a season and randomly drop a subset of ground truth data (observed data) to create a test set.
3. Use the fixed coefficients from trained model to make predictions for the test set.
4. Compare predictions against the actual values in the test set using MAE, RMSE, and R^2.
5. Repeat the random sampling and evaluation through k-fold cross-validation to avoid bias.
6. Compare the results of different regression models.

[ian-r-rose]

Love this!

[thehanggit]

Love this!

Thank you! @ian-r-rose. Will see what happens