How machines learn

Learning from examples instead of rules

So far you have told the computer exactly what to do, step by step. Move the turtle forward, check a condition, repeat a loop. That works when you can write the rule down. But some problems are almost impossible to spell out in rules.

Think about telling apart a photo of a cat from a photo of a dog. You know the answer in a fraction of a second, yet you would struggle to write the exact rules that separate them. This is where AI and machine learning come in. Instead of writing the rule yourself, you show the machine many examples and let it work out the pattern.

Machine learning is programming a computer to get better at a task by using example data or past experience. It is useful exactly when a plain rule is hard to write: recognising speech, reading handwriting, spotting fraud, or steering a robot across ground no human has walked before.

And you do not have to build all of this from nothing. As one of our workshop guests put it: you do not need to write a book when you already have it in a library. In Python you just import the right library and use it. We will show a tiny taste of that at the end.

Three kinds of learning

Almost every machine learning method falls into one of three families.

Supervised learning learns from labelled examples. You give it data where the answer is already attached, many photos each marked "cat" or "dog", and it learns the pattern that connects the input to the label. Afterwards it can put a label on a new photo it has never seen. Most everyday predictions, from spam filters to price estimates, work this way.

Unsupervised learning gets data with no labels at all. Its job is to find structure on its own, for example grouping customers who behave alike into clusters. Nobody tells it the "right" groups; it discovers them from the data.

Reinforcement learning learns by trying, then getting a reward or a penalty. Imagine teaching a self-driving car through a series of moves like "left, straight, right, straight". A safe, smooth run earns a good score, say +18.5. A run that swerves and brakes hard earns a bad one, say -3. Over many attempts the machine learns which actions lead to the higher reward, the same way you learn a game by playing it.

A worked example: predicting from features

The pieces of information you feed a model are called its features. Choosing good features is often what decides whether a model works.

Here is an example from one of our data science workshops. Suppose you want to predict whether a person is a man or a woman, and the only feature you have is their height.

You read the heights from a file, plot them, and look for a boundary. Say you decide that anyone taller than 170 cm is predicted to be a man, and shorter is predicted to be a woman. This rule is right a lot of the time. But when you count the mistakes, you see plenty: short men and tall women get labelled wrong. The accuracy, the share of predictions the model gets right, is only so-so.

So you add a second feature: weight. Now each person is a pair, height and weight together, and the two features give the model much more to separate people by. The same kind of model, fed both features, predicts far more accurately than height alone.

That is the everyday rhythm of machine learning: pick features, measure accuracy, and add or change features until the predictions are good enough for the job.

Finally, a short taste of the "just import it and use it" point. This tiny program uses a ready-made library to learn the link between years of experience and salary, then predicts a new value. You do not need to understand every line; notice how few lines it takes.

from sklearn.linear_model import LinearRegression

# years of experience and the matching salaries
X = [[1], [2], [3], [4], [5]]
y = [110000, 120000, 130000, 140000, 150000]

model = LinearRegression()
model.fit(X, y)

# predict the salary for 6 years of experience
print(model.predict([[6]]))

Check yourself

• A spam filter is trained on thousands of emails already marked "spam" or "not spam". Which of the three kinds of learning is that, and why?
• In the height example, why did adding weight as a second feature make the predictions more accurate?
• In the police-sketch picture of a GAN, which network is the sketch artist and which is the eyewitness, and what does each one try to do?