Designing feedback loops in machine learning system design
In machine learning systems, we often receive some kind of feedback from the model’s environment, that is then fed back into the system. This can take many forms, such as using the model’s output to train newer versions of the model, or using user feedback on the model’s decisions to improve the model.
While many feedback loops are useful and will improve the performance of your model over time, some feedback loops will actively degrade the performance of the machine learning system over time. Designing helpful feedback loops is an important component of machine learning system design, and needs to be done thoughtfully in order to ensure that your system is sustainable.
In this post I will go over examples of both helpful and harmful feedback loops, and explain why each type has the result that it does.
A beneficial feedback loop typically involves bringing in unbiased, external information into your machine learning system. This often occurs in the form of obtaining labels through a source that is not strongly correlated to the outputs of the machine learning models.
