Shopify/liquid: Performance: 53% faster parse+render, 61% fewer allocations (https://github.com/Shopify/liquid/pull/2056)

PR from Shopify CEO Tobias Lütke against Liquid, Shopify’s open source Ruby template engine that was somewhat inspired by Django when Tobi first created it back in 2005 (https://simonwillison.net/2005/Nov/6/liquid/).

Tobi found dozens of new performance micro-optimizations using a variant of autoresearch (https://github.com/karpathy/autoresearch), Andrej Karpathy’s new system for having a coding agent running hundreds of semi-autonomous experiments to find new effective techniques for training nanochat (https://github.com/karpathy/nanochat).

Tobi’s implementation started two days ago with this autoresearch.md (https://github.com/Shopify/liquid/blob/2543fdc1a101f555db208fb0deeb2e3bf1ae9e36/auto/autoresearch.md) prompt file and an autoresearch.sh (https://github.com/Shopify/liquid/blob/2543fdc1a101f555db208fb0deeb2e3bf1ae9e36/auto/autoresearch.sh) script for the agent to run to execute the test suite and report on benchmark scores.

The PR now lists 93 commits (https://github.com/Shopify/liquid/pull/2056/commits) from around 120 automated experiments. The PR description lists what worked in detail - some examples:

• Replaced StringScanner tokenizer with String#byteindex. Single-byte byteindex searching is ~40% faster than regex-based skip_until. This alone reduced parse time by ~12%.

• Pure-byte parse_tag_token. Eliminated the costly StringScanner#string= reset that was called for every {% %} token (878 times). Manual byte scanning for tag name + markup extraction is faster than resetting and re-scanning via StringScanner. […]

• Cached small integer to_s. Pre-computed frozen strings for 0-999 avoid 267 Integer#to_s allocations per render.

This all added up to a 53% improvement on benchmarks - truly impressive for a codebase that’s been tweaked by hundreds of contributors over 20 years.

I think this illustrates a number of interesting ideas:

• Having a robust test suite - in this case 974 unit tests - is a massive unlock for working with coding agents. This kind of research effort would not be possible without first having a tried and tested suite of tests.

• The autoresearch pattern - where an agent brainstorms a multitude of potential improvements and then experiments with them one at a time - is really effective.

• If you provide an agent with a benchmarking script “make it faster” becomes an actionable goal.

• CEOs can code again! Tobi has always been more hands-on than most, but this is a much more significant contribution than anyone would expect from the leader of a company with 7,500+ employees. I’ve seen this pattern play out a lot over the past few months: coding agents make it feasible for people in high-interruption roles to productively work with code again.

Here’s Tobi’s GitHub contribution graph (https://github.com/tobi) for the past year, showing a significant uptick following that November 2025 inflection point (https://simonwillison.net/tags/november-2025-inflection/) when coding agents got really good.

He used Pi (https://github.com/badlogic/pi-mono) as the coding agent and released a new pi-autoresearch (https://github.com/davebcn87/pi-autoresearch) plugin in collaboration with David Cortés, which maintains state in an autoresearch.jsonl file like this one (https://github.com/Shopify/liquid/blob/3182b7c1b3758b0f5fe2d0fcc71a48bbcb11c946/autoresearch.jsonl).

Via @tobi (https://x.com/tobi/status/2032212531846971413)

Tags: django (https://simonwillison.net/tags/django), performance (https://simonwillison.net/tags/performance), rails (https://simonwillison.net/tags/rails), ruby (https://simonwillison.net/tags/ruby), ai (https://simonwillison.net/tags/ai), andrej-karpathy (https://simonwillison.net/tags/andrej-karpathy), generative-ai (https://simonwillison.net/tags/generative-ai), llms (https://simonwillison.net/tags/llms), ai-assisted-programming (https://simonwillison.net/tags/ai-assisted-programming), coding-agents (https://simonwillison.net/tags/coding-agents), agentic-engineering (https://simonwillison.net/tags/agentic-engineering), november-2025-inflection (https://simonwillison.net/tags/november-2025-inflection), tobias-lutke (https://simonwillison.net/tags/tobias-lutke)