A tech knowledge scientist’s stack to enhance cussed ML fashions
This text is one in every of a two-part piece documenting my learnings from my Machine Studying Thesis at Spotify. Be sure you additionally take a look at the second article on how I carried out Function Significance on this analysis.
In 2021, I spent 8 months constructing a predictive mannequin to measure person satisfaction as a part of my Thesis at Spotify.
My objective was to know what made customers glad with their music expertise. To take action, I constructed a LightGBM classifier whose output was a binary response:
y = 1 → the person is seemingly glad
y = 0 → not a lot
Predicting human satisfaction is a problem as a result of people are by definition unhappy. Even a machine isn’t so match to decipher the mysteries of the human psyche. So naturally my mannequin was as confused as one might be.
From Human Predictor to Fortune Teller
My accuracy rating was round 0.5, which is the worst attainable final result you will get on a classifier. It means the algorithm has a 50% likelihood of predicting sure or no, and that’s as random as a human guess.
So I spent 2 months making an attempt and mixing completely different methods to enhance the prediction of my mannequin. In the long run, I used to be lastly in a position to enhance my ROC rating from 0.5 to 0.73, which was an enormous success!
On this submit, I’ll share with you the methods I used to considerably improve the accuracy of my mannequin. This text would possibly come in useful everytime you’re coping with fashions that simply received’t cooperate.
Because of the confidentiality of this analysis, I can not share delicate data, however I’ll do my perfect for it to not sound complicated.
