Build pipeline (build/build_blog.rb)

Architecture — part 9 of 13

The script is the pass itself; the domains it calls live beside it, one file each -- build/blocks.rb (a content block becomes HTML), build/output.rb (writing, pruning, and getting a file into public.nosync), build/feeds.rb (RSS, Atom, JSON Feed, sitemap), build/discovery.rb (the search index and what feeds it), build/cards.rb (link and social cards), build/languages.rb (which language a run renders, where its reader is sent, and the checks a language configuration has to pass), build/tags.rb (the tag pages and the index of them), build/archive.rb (the archive map and a page per year) and lib/series.rb (what a series is and what order its parts go in, which the CLI needs too). The phases in tags.rb and archive.rb are functions with their inputs spelled out -- the posts, the tag map -- called from the point in the pass where they always ran, so the pages are written in the order they always were.

⚠️ Moving a method out of build_blog.rb is not a free refactor: the templates in templates/ are rendered against the script's own binding, so every helper they call has to stay reachable from there. A helper that moves into a module keeps a one-line wrapper at the top level, or the build dies on the first page that uses it.

A single linear pass, no framework:

  1. Load & guard. Read every post JSON; abort on a year+slug collision (two posts would silently overwrite each other's output and media).
  2. Order. Sort by date with slug as tiebreaker -- sort_by isn't stable and imported posts can share timestamps; without the tiebreaker, page contents would shuffle between builds.
  3. Partition. Drafts leave the main flow: each gets only its own page at /draft/<token>/<slug>/ with noindex, and appears in no listing, feed or index.
  4. Consult the record (lib/build_cache.rb). The last build wrote down a key describing the inputs of each page it produced -- the post, the template, the locale, the configuration, the engine itself -- and a digest of every file emit wrote. A page whose key still matches is not rendered; a file whose digest still matches costs one stat instead of a read-back. The whole record is keyed to a fingerprint of templates/, lib/, build/, locales/ and config/, plus the facts the build works out for itself (the menu, the types present, the page size), so a template edit or a new menu item throws it away. assets/ is deliberately not among them: a page links a stylesheet, a script and a favicon rather than carrying their bytes, and there is no ?v= on those links, so nothing under assets/ can change a rendered page. Assets have their own path -- emit_copy compares them byte for byte on every build, before the cache is opened -- and colors.css, the one asset a page's own colours come from, is generated out of config/, which IS in the fingerprint. The cache is an optimisation and never an authority: anything it cannot vouch for is rendered and compared the old way, and --full believes none of it while still recording what it wrote.
  5. Render. Per-post pages, then listings: homepage, per-tag, per-content-type -- the last only for types with at least one published post (PRESENT_TYPES); the nav menu and the sitemap follow the same set, so an empty type has no pages, no menu item and no sitemap entry. Pagination is anchored to the oldest post -- page 1 is the oldest ten, the newest posts live on the landing page, which splits only after holding 2x the page size. Page boundaries therefore never shift, so adding a post rewrites a handful of files instead of the whole archive.
  6. Indexes & feeds. Client-side search index split into recent (search-index.json, newest 500, loaded eagerly) and archive (search-index-archive.json, loaded on first query). Each entry carries one folded blob (title, text and tags together), which is what assets/js/search.js matches on; it scores the hits afterwards -- a word in the title above a word in the text, a whole word above the same letters inside a longer one, date as the tiebreak -- and draws at most RESULT_LIMIT of them while reporting the true count; RSS (last-build date = newest post, not "now", to keep the file byte-stable); sitemap; robots.txt.
  7. Assets, colors and the root favicon. Before assets/ is copied into the build, any live name missing under assets/images/ is seeded from the tracked assets/images/defaults/ -- the live banner and favicon are per-install files outside git (see decisions.md), so a fresh clone renders immediately while an owner's own artwork is never overwritten; defaults/ itself is not published. config/site.yml's 7-key palettes compile into assets/css/colors.css, together with the header's font stacks and sizes and any @font-face a site declared for its own files in assets/fonts/; site.css itself contains no palette values -- its only color literals are palette-independent neutrals (white, two fixed grays, black/white alpha overlays) -- and no site-specific typography. One generated stylesheet rather than two on purpose: it is already linked from the layout, so adding fonts to it changes two files on a deploy instead of every page in the archive. build_favicon_ico wraps assets/images/favicon.png in an ICO container (a 22-byte header, then the PNG verbatim) and emits /favicon.ico -- pages link the PNG, so this exists purely for clients that request the root path without reading the <link>. No image library involved, the same "smallest correct slice of a format" approach as lib/qr_code.rb; returns nil with no source PNG, so a site without a favicon simply doesn't get the file. ICO's dimension fields are one byte each and 0 means 256, so a larger source can't state its size -- browsers load it and report 256, immaterial at favicon sizes.
  8. Write & prune. emit writes a file only when its bytes actually changed and records every generated path; prune_public then deletes whatever the build didn't produce this run, deepest directories collapsing first. The sweep runs on every build regardless of the cache, because a record can only miss what it once held.

Renders are memoized per post (content HTML, parsed time, dominant type, list item) keyed by object identity -- a post appears on its own page plus every listing, and would otherwise be rendered 4-6x. ERB partials (nav, aside, footer) are cached by name+locals since they don't depend on page content.

One site, several languages

A site that names site.locales is built once per language, and every run writes into the same public.nosync/. The build itself knows one language at a time: BLOG_SH_LANG says which (lib/i18n.rb reads it and nothing else may), and Publishing#run_build runs the site's own language first and the others after it, all under one lock, so a scheduled publish cannot deploy a tree that is half one language. What a run needs to know about the others it asks: which roots exist (SITE_LOCALES) and what a post says in them.

  • Roots. The site's own language keeps the site root (LANG_ROOT is empty); every other writes under /<lang>/ (CONTENT_ROOT). loc() turns a site path into this language's, and SHARED_ROOTS -- /assets/ and /write/ -- are the two it leaves alone: one copy for every language.
  • The sweep. Each run removes what it did not write, but only in the part of the tree it owns (prune_root): another language's run owns its root and nothing else; the site's own run owns everything except those roots. Without that the second run would carry off the first one's site.
  • Written once. robots.txt, sitemap.xml (one document naming every language, with xhtml:link alternates only where an address exists in more than one), the sidebar's JSON and the writing app's site.js come from the site's own run alone. A crawler reads the robots.txt at the origin, the sidebar fetches its data by absolute address, and the writing app speaks the language its author writes in -- a copy per language would be files nothing reads, and for site.js, the last run's language.
  • What a post says here. Translations.for_lang gives the post as this language renders it: its own text when it has one, otherwise the nearest language on the fallback chain in config/site.<lang>.yml, ending at the post itself. A language gets a PAGE only for a post that has words in it; the listing shows the fallback text under a link to the one copy that exists, so the same words never stand at two addresses. PostAddress builds the address from the translation's own slug.
  • What the site says here. SiteConfig.localize!, called at the top of the build and nowhere else, lays the chrome of config/site.<lang>.yml over site.yml -- title, description, banner words, about, footer, menu, widget headings, tag labels -- before anything reads a single one of those keys. Only the build: every other program that reads the config also writes it back. lib/language_file.rb says which keys a language file may carry and what is wrong with one (an unknown key, a translation of something site.yml lacks, a menu or footer list that leads elsewhere); the build refuses those and check reports the same list.
  • The engine's own words. A heading the site does not write -- about, the footer's links and social columns, a widget's -- is chrome.* from the locale of this run (site_heading); a tag shows TAG_LABELS[slug] when the language file gives it a word (tag_label), which the tag map, the pills, the feed's categories and the JSON-LD all go through.
  • Telling the reader and the crawler. The switcher offers every language the site publishes; a post with nothing in one leads to that language's front page. The hreflang alternates are narrower on purpose and name only addresses that were written (language_links).
  • The cache. config/ is in the fingerprint, so an edit to a language file re-renders the pages it speaks on like any other config change.

A site without site.locales goes through none of this: one run, an empty LANG_ROOT, no alternates, no switcher -- the markup it has always had.