What is model tuning?
Model tuning is the cycle of adjusting how a model is built and trained until its performance on validation data stops improving. It covers choosing hyperparameters, trying different feature engineering, changing the loss function, and comparing algorithms. Each change is a new experiment, scored against the same validation set, and the best configuration is kept.
Model tuning is different from fine-tuning. Fine-tuning changes a pre-trained model’s weights by training it on new data. Tuning changes the settings around the training process, and the weights that result are a consequence. Vendors often use the two terms loosely, so ask which one they mean.
Model tuning in practice
A collections agency has a model that predicts which accounts are likely to pay after a first contact. The first version ranks accounts only slightly better than the existing rules. Over three weeks, the analyst tunes it: a search over tree depth and learning rate, a new feature for days since the last payment, and a loss function that penalizes missed high-balance accounts more heavily. Each run is logged in an experiment tracking tool. The final model improves recovery enough to change the daily call list.
What to watch for
- Tuning against one validation set for too long will overfit to that set, so confirm the final result on held-out test data.
- Diminishing returns arrive quickly, and better data usually beats another week of tuning.
- Keep every experiment’s settings and scores, or nobody will know why the production model looks the way it does.