An educational project that demonstrates how to extract and visualize the knowledge learned by a simple neural network trained on handwritten digit recognition.
No frameworks, no dependencies — just plain JavaScript running in the browser.
A feed-forward neural network trained on the MNIST dataset can recognize handwritten digits with ~98% accuracy. But what has it actually learned? This project explores two complementary techniques for answering that question:
Chapter 1: Causal Index Analysis — Computes the influence of each input pixel on each digit class by tracing weighted paths through the network. The result is a set of heat maps that look remarkably like the digits themselves. Live demo
Chapter 2: Iterative Input Adaptation — Starts from a blank image and iteratively modifies pixels to maximize the network’s confidence for a target digit. A simple hill-climbing approach that lets the network “dream” its ideal input. Live demo
.
├── README.md # This file
├── chapter1.md # Article: Causal Index method
├── chapter2.md # Article: Iterative Adaptation method
├── demos/
│ ├── causal-index.html # Interactive demo for Chapter 1
│ ├── iterative-adaptation.html # Interactive demo for Chapter 2
│ ├── net.js # Pre-trained network weights
│ └── logo.png # Site logo
├── images/
│ ├── char0.png # Sample heat map (digit 0)
│ └── char3.png # Sample heat map (digit 3)
└── LICENSE
No build step required. Clone the repo and open the HTML files in your browser:
git clone https://github.com/cleardatalabs/knowledge-extract-ffnn-mnist.git
cd knowledge-extract-ffnn-mnist
open demos/causal-index.html
Based on the neural network demo by Hubert Eichner. The network was trained in MATLAB on the MNIST dataset and exported to JavaScript.
Created by ClearDataLabs & Kostiantyn Chumychkin (2018-2026). Distributed under the MIT License.