◂ CONTENTS SKILL · METRICS AND LEARNING ANALYTICS FOUND IN 2 OF 4 PROJECTS

SKILL 7 OF 9 · METRICS AND LEARNING ANALYTICS

A/B testing

Changing one thing for some people and comparing what they do. I read the result with Alex Edmans’ Ladder of Misinference?LADDER OF MISINFERENCEAlex Edmans’ four checks before trusting a finding: is it accurate, is the sample representative, could something else explain it, and does it hold beyond this case? before anything is claimed.

WHERE IT SHOWS UP · PICK A PROJECT

PROJECT 03JUMP Math Lab
PROJECT 04The Artificial Leaf
BEFORE A RESULT IS CLAIMED

Four questions on the ladder

  1. RUNG 1Is the number accurate?
  2. RUNG 2Is the sample representative?
  3. RUNG 3Could something else explain it?
  4. RUNG 4Does it hold beyond this case?
PROJECT 02 · LEARNING METERFROM P. 9–14

Redesigning the gamified loops

I redesigned Animal Planet’s gamified loops from engagement data and A/B tested the redesign.

PARTICIPATION+40%learner participation, A/B tested.
ACTIVITIES A WEEK3–4 → 7–8per learner.
OPEN PROJECT 02 ▸
PROJECT 01 · BEELINEFROM P. 3–8

Randomising the gateway cards

Sessions comparing two or more gateways?GATEWAYSThe seven broad sectors in the MBacc, like Digital or Financial, that young people explore careers through., before and after the order of the gateway cards was randomised. Part of last year’s figure was people clicking the next card down, so the lower number is the truer reading.

LAST YEAR20.9%
RANDOMISED17.2%
z = −6.75 · p < 0.001 · LM-03 DECISION QUALITY OPEN PROJECT 01 ▸
TOOLS
Google AnalyticsHotjarMicrosoft ClarityBigQuery
RELATED SKILLS
WORK WITH MEMetrics and learning analyticsMetric frameworks, dashboards and A/B tests
ASK ME FOR A DAY RATE ▸