AI as a target

The AI itself is worth attacking

Earlier lessons treated AI as a weapon and as a shield. There is a third angle: the AI system is a target in its own right. As organisations put more and more AI into decisions that matter, the model itself becomes something worth attacking, and something that has to be defended.

This is a shift from the threats you face as an individual toward how the systems around you are protected. You will not be defending a model yourself, but it helps to understand why the organisations you rely on, your bank, your employer, public services, have to guard the AI behind their services.

Poisoned training data

As training vs using a model explained, an AI model learns from data, and whatever it is trained on shapes how it behaves. That dependence is a weakness an attacker can aim at.

If someone can slip bad examples into the data a model learns from, they can quietly bend its behaviour. This is called data poisoning. Done carefully, it can teach a model to make a specific mistake, to ignore a certain kind of threat, or to carry a hidden weakness that the attacker can trigger later. The model looks fine in everyday use, which is what makes poisoning hard to spot, and the harm can be baked in long before anyone notices.

This is why where training data comes from matters. A model trained on whatever can be scraped from the open internet inherits whatever anyone chose to put there.

Stolen and copied models

A well trained model is valuable. It can represent a great deal of money, data, and effort, so it becomes a target for theft.

That theft takes more than one form:

• Outright theft, where an attacker who breaks into a system simply copies the model, taking the finished work without paying for it.
• Copying by interrogation, where an attacker cannot reach the model directly but questions it many times and uses the answers to build a close imitation of their own.
• Extracting the training data, where an attacker probes a model to pull back out some of the sensitive information it was trained on, which can expose the private data of real people.

A stolen or copied model can leak confidential information, hand a competitor years of work, or give an attacker a private copy to study for weaknesses at their leisure.

Why organisations must protect their AI

Put these together and the message is clear: an AI system is critical infrastructure, and it needs the same care as any other. That means controlling and checking the data it learns from, limiting who can access the model and how much they can query it, watching for signs of tampering or theft, and keeping humans able to review important decisions.

For you, the practical point is expectation, not action. When an organisation asks you to trust an AI-driven service, part of trusting it well is expecting that they treat the AI itself as something to protect, because attackers certainly treat it as something worth attacking.