hackertube.io

The Simple Essence of Automatic Differentiation - Conal Elliott

Microsoft Research · talk · Watch on YouTube

Machine learning & deep learningFunctional programmingPure mathematics

What it is

Conal Elliott presents his paper on automatic differentiation, deriving a simple, generalized AD algorithm from a natural specification. Varying the representation of derivatives yields forward and reverse mode algorithms that avoid the complexity and statefulness of backpropagation.

Why Hacker News recommends it

Cited as the more accessible companion to the paper, and as the most interesting treatment of AD via compiling to categories.

  • 3 people recommended it
  • 4 different months
  • 1 strong recommendations
  • first 2018, last 2023

What HN commenters said

  1. “The most interesting thing I've seen on AD is”

    Compiling to categories makes forward and reverse mode AD composable

    infogulch, 2022-04 · Strong recommendation · in “Differentiable Programming – A Simple Introduction”

  2. “He also gave nice presentation on AD”

    Elliott's nice presentation on automatic differentiation

    noblethrasher, 2019-11 · Recommendation · in “Differentiable Programming Manifesto”

  3. “There's a talk that Conal gives that makes the paper a bit more accessible”

    Talk that makes the AD paper somewhat more accessible

    sriku, 2022-09 · Recommendation · in “Automatic Differentiation in 38 lines of Haskell”

Excerpts from Hacker News comments, linked to the originals. Summaries and labels were written by a language model from those comments.

Topics: automatic differentiation, functional programming, category theory, backpropagation, haskell

Also recommended by the same people