Why is my model accuracy high in train-test split but actually worse than...

drywall_dan

I have trained a XGboost model to predict survival for the Kaggle Titanic ML competition . As with all Kaggle competitions there is a train dataset with the target variable included and a test dataset without the target variable which is used by Kaggle to compute the final accuracy score that determines your leaderboard ranking. My problem: I have build a fairly simple ensemble classifier (based on XGboost) and evaluated it via standard...

Originally asked by Fnguyen on Stack Exchange (ai) (CC BY-SA 4.0).

uphilltech

Great question — answering as the UphillTech team:

Looking at your code, one set of data transformations were applied to the train data and a different set of transformations were applied to the test data. Different data transformations could account for different evaluation metric performance. It is best practices to put all data transformations in a function so they can be applied to all data in a similar way. Since you are using scikit-learn, sklearn.compose.ColumnTransformer is designed for this purpose. Example code for the Titanic dataset is here .

Full answer by Brian Spiering: https://ai.stackexchange.com/a/17814 (CC BY-SA 4.0).

Need it done for your business? UphillTech.