Refresher: Describing Data with Numbers
Summarise a list of numbers: the mean, median and mode, why outliers pull the mean, spread (range, variance and standard deviation), percentiles and quartiles, and reading a histogram. These are the first things you compute on any dataset.
Summarising a list of numbers
Before building any model you look at the data, and the first step is to summarise each column with a few numbers: where is the middle, and how spread out are the values? These descriptive statistics are what df.describe() prints in pandas.
The middle: mean, median and mode
Take the waiting times (minutes) :
- Mean (average): add them up and divide by how many there are: . In symbols .
- Median: the middle value after sorting: 7. With an even count, average the two middle values.
- Mode: the most common value: 7.
An outlier (an unusual value) pulls the mean but barely moves the median. Replace 13 with 130 and the mean jumps to 31.4, while the median stays 7. That is why house prices and incomes are usually reported as medians.
Spread: range, variance and standard deviation
Two classes can both average 70% while one has everyone between 65 and 75 and the other ranges from 20 to 100. Spread measures that difference.
- Range: largest minus smallest: . Simple, but decided by just two values.
- Variance: the average squared distance from the mean. For our data the distances are , their squares , and their average .
- Standard deviation : the square root of the variance, minutes, back in the original units. Roughly, "a typical distance from the mean".
Squaring makes every distance positive and weighs big distances more. Note: statistics software often divides by instead of for a sample (the "sample standard deviation", 3.32 here); pandas does this by default, NumPy does not. The statistics lesson explains why.
Percentiles, quartiles and the box plot
The p-th percentile is the value below which p% of the data falls. The median is the 50th percentile; the quartiles Q1 and Q3 are the 25th and 75th. The interquartile range (IQR) is the spread of the middle half, and it ignores outliers. A box plot draws exactly these: a box from Q1 to Q3, a line at the median, and whiskers out to the rest. "Your baby is in the 90th percentile for height" means taller than 90% of babies that age.
Histograms: the shape of the data
A histogram splits the range of values into bins and draws a bar for how many values fall in each. It shows the shape: symmetric like a bell, skewed with a long tail to one side (incomes, waiting times), or with two peaks (two different groups mixed together). Always look at the shape before trusting a single average.
Try it: change the data
Edit the numbers and watch the mean, median and standard deviation update. Add one huge value and see which summaries move.
Interactive: this part runs in your browser.
Practice: check yourself
Work each question out (on paper, in your head, or in Python), type the answer and press ✔ Check. Fractions like 1/4 are fine. Hints and a worked answer are there when you need them.
Summary
Key words
- mean
- the sum divided by the count
- median
- the middle value when sorted
- mode
- the most common value
- outlier
- an unusual value far from the rest
- variance
- the average squared distance from the mean
- standard deviation
- the square root of the variance; a typical distance from the mean
- percentile
- the value below which a given % of the data lies
- IQR
- Q3 − Q1: the spread of the middle half
- histogram
- bars showing how many values fall in each range
Common mistakes
- Reporting only the mean for skewed data like prices: report the median too.
- Forgetting to sort before finding the median.
- Using the variance when you meant the standard deviation (it is in squared units).
- Comparing a sample standard deviation (n − 1) with a population one (n) without noticing.
Quiz
1. What is the mean of 2, 4, 6, 8?
2. What is the median of 9, 1, 5, 3, 7?
3. One huge value is added to a dataset. Which changes most?
4. The standard deviation is…
5. Q1 = 20 and Q3 = 50. What is the IQR?