Karpathy's char-rnn essay turned RNN theory into demos anyone could train

In 2015 the common wisdom said recurrent neural networks were difficult to train, and the papers that used them were heavy going, with equations and benchmark tables that gave a reader little sense of what the network had actually learned. Andrej Karpathy had found the opposite in his own work on image captioning, where a small, arbitrarily configured model produced surprisingly good captions within minutes, and he set out to show why the models deserved attention.

His essay taught the RNN as one stateful step function, then built on it with a character-level language model that predicts the next character of a text. The article included a minimal 100-line implementation in numpy and an MIT-licensed Torch/Lua codebase, char-rnn, so the reader could reproduce every experiment by feeding in one large text file.

The demonstrations made the learning legible. Sampling after training showed Shakespearean dialogue, Wikipedia markdown with correctly nested XML and brackets, LaTeX that almost compiled into fake algebraic geometry, and Linux kernel C code with balanced braces and plausible comments. A War and Peace model improved from random jumbles at iteration 100 to properly spelled sentences by iteration 2000, and neuron visualizations found cells that had learned to track whether the text was inside a URL, quote or wiki link, rather than memorizing strings.

The edit list appended to the post documents the response: Reddit discussions, a talk at the London Deep Learning meetup, a comparison against n-gram baselines by Yoav Goldberg, and community replications training char-rnn on cooking recipes, Eminem lyrics, Obama speeches, Irish folk music, ABC-notation music and the Bible. The essay resurfaced on Hacker News on February 3, 2019, drawing 161 points and 51 comments.

Show learning as it happens: one character model per corpus, with code and neuron maps

A 100-line numpy implementation plus a released repository made the claim falsifiable: any reader could train the same models instead of trusting the author's screenshots.

Familiar corpora did the explanation for free, Shakespeare, Wikipedia, LaTeX and Linux source, because readers could judge whether the sampled output looked right without needing any ML expertise.

Showing samples at iterations 100, 300, 700 and 2000 replaced the abstract statement that networks learn with the visible process of structure emerging, from spacing to spelling to grammar.

Neuron firing maps answered the skepticism about black boxes, displaying cells that learned quote detection, URL tracking and bracket counting from raw characters alone.

MIT licensing let derivative projects spread the technique into recipes, music and political speeches, which is how a single blog post seeded a visible ecosystem.

When a technique is distrusted because its internals are invisible, ship the smallest working version and show learning as it happens; evidence readers can rerun beats claims that it is too hard.

The post's appended edit list records immediate follow-ups: discussions on r/machinelearning and r/programming, a London Deep Learning meetup talk, Yoav Goldberg's n-gram baseline comparison, and community models trained on cooking recipes, Eminem lyrics, Obama speeches, Irish folk music and the Bible, plus a Terminal.com snapshot with char-rnn preinstalled. The essay resurfaced on Hacker News on February 3, 2019, drawing 161 points and 51 comments, years after the underlying LSTM work it popularized had been published.