← All work Public policy · English Indices of Deprivation 2019

One in five English neighbourhoods is under-ranked by the deprivation index

Data IMD 2019, 32,844 areasMethod Spearman correlation, k-meansTools R, tidyverse, sf
TL;DR

Segmenting England's neighbourhoods on the seven deprivation domains found 6,360 areas (19.4%) whose real problem is access to housing and services. The overall index ranks them as less deprived than a moderately deprived group, so funding allocated by rank alone would pass them over.

Open the full report (PDF, 20 pages) ↗3,260 words · includes the full R script

The decision at stake

Councils and funders use the Index of Multiple Deprivation rank to decide where money goes. The single score assumes its seven domains move together. If one domain behaves differently, areas deprived mainly in that domain look better off than they are.

  • How are the seven domain scores related?
  • Can neighbourhoods be grouped into distinct deprivation profiles?
  • Where are those profiles in England?

Data and approach

  • CDRC shapefile of IMD 2019 (32,844 areas, 317 districts) joined to the ONS district-to-region lookup; every area matched, no missing values or duplicates.
  • Three domains are strongly right-skewed (1.20 to 1.34), so I used Spearman correlation and z-scored every domain before clustering.
  • K-means with 25 random starts and a fixed seed, fitted for both three and four clusters, then cross-tabulated.
Why four clusters, not three

Findings

0.95correlation between income and employment deprivation
0.04between barriers to housing & services and income
61%of London neighbourhoods are in the high-barriers group
The high-barriers group is better than average on income, work, education and healthBar chart of average standardised scores per domain for each cluster
Figure 3 from the report. Positive means more deprived than the national average. Cluster 2 is deprived on barriers (+1.27) and living environment (+0.93) only.
Multiple deprivation sits in the North; high barriers dominate London and recur in the rural SouthMap of England coloured by the four clusters
Figure 4 from the report. In Knowsley (55.1%) and Liverpool (54.4%) more than half of neighbourhoods are multiply deprived.

Inside the high-barriers group, the cause differs by place. London areas score +4.29 on wider barriers (affordability, overcrowding) and −0.63 on geographical barriers. Outside London it reverses: +1.04 geographical (distance to GP, school, shops) and −0.47 wider. The published barriers score averages two parts that move in opposite directions, so it partly cancels itself out.

All figures

Select a figure to enlarge it.

Recommendation

For anyone allocating funding by IMD rank: add a second screen on the barriers domain and read its two sub-domains separately.

  • London high-barriers areas: affordability and overcrowding programmes.
  • Rural high-barriers areas: transport and service-access programmes.
  • Tradeoff: a second screen adds complexity to a ranking people trust because it's simple.

Limitations

  • Results describe areas, not people (ecological fallacy).
  • One year of data, so no view of change over time or cause.
  • K-means puts each area in exactly one group, which simplifies mixed areas.

Under the hood

domains_z <- scale(select(imd_flat, all_of(domain_cols)))
set.seed(1234)

km3 <- kmeans(domains_z, centers = 3, nstart = 25, iter.max = 50)
km  <- kmeans(domains_z, centers = 4, nstart = 25, iter.max = 50)

# Does the high-barriers group survive with k = 3?
table(km3$cluster, km$cluster)   # it splits 4,110 / 2,249

# Join check: every area must get a region
imd <- left_join(imd, lookup, by = "lad_code")
sum(is.na(st_drop_geometry(imd)$region))   # 0

The full script is in the appendix of the report and runs end to end.

Check it yourself