What building an AI system taught me.

Building an AI system is very different from simply training a model. The model is one component inside a much larger system.

My work on an AI-powered disinformation detector made that particularly clear. Data preparation, labels, evaluation and model selection all affected the final behaviour of the system.

The model is only part of the problem

It is easy to focus on the model because it is the most visible technical component. In practice, the surrounding data pipeline and evaluation process can have just as much influence on the result.

Evaluation changes how you think

Looking at accuracy alone can hide important behaviour. Class imbalance, confusion between labels and the difference between macro and weighted metrics all provide additional information about how a system is performing.