hackertube.io

Lexical Scanning in Go - Rob Pike

GoogleFOSSSydney · talk · Watch on YouTube

Compilers, interpreters & program analysisC, C++, Rust, Go & systems languages

What it is

Rob Pike shows how the Go text/template lexer is built, using state functions, goroutines and channels. The talk was given at a Google Technology User Group in Sydney in 2011.

Why Hacker News recommends it

HN returns to it for years as an elegant introduction to writing a lexer that is easy to follow even without a parsing background.

  • 13 people recommended it
  • 12 different months
  • 4 strong recommendations
  • first 2012, last 2025

What HN commenters said

  1. “Rob Pike has an excellent video about this”

    Pike on parsers returning functions that consume the next token.

    willvarfar, 2012-06 · Strong recommendation · in “Parsing huge XML files with Go”

  2. “It's far more interesting than it sounds, and even if you don't know much about lexing/parsing it's easy to follow.”

    Explains lexing in text/template; easy to follow even without parsing background.

    TomNomNom, 2014-05 · Strong recommendation · in “Templating in Go”

  3. “I love everything about this talk.”

    Builds a lexer from scratch simply and elegantly using coroutines, with humor.

    grose, 2016-10 · Strong recommendation · in “Ask HN: What's your favorite tech talk?”

  4. “I love Rob Pike's talk about lexical Scanning in Go (2011). Educational and elegant approach.”

    Educational and elegant approach to lexical scanning.

    hu3, 2024-11 · Strong recommendation · in “Rust for tokenising and parsing”

  5. “watching Pike's talk on lexing and parsing in Go really hits this point home.”

    Shows how goroutines and channels change how you design algorithms.

    drbawb, 2012-03 · Recommendation · in “Go is amazing, period.”

  6. “There's a video by Pike describing how he built the parser at”

    Pike describes building the text/template lexer using recursive state functions.

    jbarham, 2013-02 · Recommendation · in “Function types in Go”

  7. “A nice example for this is shown in his talk about lexical scanning in Go”

    Example of concurrency as a way to structure software, without parallelism.

    mseepgood, 2013-02 · Recommendation · in “Concurrency is not Parallelism. Rob Pike at Waza 2012 [video]”

  8. “There is a better way, as demonstrated in Robe Pike's video[1] detailing how the Go template language is implemented.”

    Shows a better alternative to regex for parsing templates.

    thisishugo, 2013-04 · Recommendation · in “Building a toy template engine in Python”

Excerpts from Hacker News comments, linked to the originals. Summaries and labels were written by a language model from those comments.

Topics: go, lexer, rob pike, state machines, concurrency, parsing

Also recommended by the same people