ScribbleScope

The problem

Students are taught that AI "can be biased" and that "confidence isn't accuracy," as slides. Actually showing either one happening normally needs a Python environment, a GPU, and an hour of setup, which rules out most classrooms, and every student without a capable laptop.

Approach

Solo entry for the ML Empowerment Build Challenge 2.0. A 4-block convolutional network (~850K parameters) is trained from scratch on Quick, Draw! doodles across 195 countries; three independently-seeded copies form a deep ensemble. Everything runs client-side: the models are quantized to ONNX and served through onnxruntime-web, so nothing a student draws ever leaves their device, and it works offline on a school Chromebook after the first load. Four labs turn one abstract concept each into something measured live on the student's own drawing: watching 50 class probabilities update while you draw, deleting a stroke and re-running the network to see what it cost, drawing something outside the model's 50 known classes to watch it guess confidently anyway, and a per-country accuracy breakdown across 44 countries with bootstrap confidence intervals, comparing a baseline model against one trained on a rebalanced dataset.

Results

Each model runs client-side at roughly 20ms per prediction. The fairness audit (44 countries, each with at least 2,000 held-out drawings) is reported with 95% bootstrap confidence intervals rather than a single number, since a per-country gap this small needs the interval to mean anything.

What broke

Training silently ran on CPU for a while because the device check only tested for CUDA, never Apple's MPS backend. It was about 7x slower than it needed to be. Once fixed, epoch time went from 22 to 89 minutes; it looked like thermal throttling but was actually PyTorch's MPS memory allocator growing unbounded until the machine was swapping. And the fairness experiment was inverted once: the per-country sampling cap had been applied to the baseline model instead of the mitigated one, which would have shown the comparison backwards. Caught it by checking the data before training on it, not after.