Content model
How the engine works inside -- one page, following a post's life from keystroke to visitor. The why behind these shapes lives in Decisions.
$EDITOR (markdown) → CLI parse → JSON blocks → build (ERB) → static HTML → deploy backend → host
↑ ↑
importers target the cron refreshes widget
same schema JSON + post statsOne post = one JSON file at content.nosync/posts/<year>/<slug>.json:
slug,title,date(ISO 8601),tags,state(published/draft),source(which platform it came from), optionallymastodon_url(its comment toot),draft_tokenandcreated_at(drafts),type(explicit content type override).contentis an array of typed blocks:text(withsubtypeheading1-6/quote),list,table,code,image,video,audio,file,link,chat,hr.- Inline formatting (bold/italic/strikethrough/code/link) is stored as codepoint offset ranges into plain text, not nested HTML -- the same NPF-style shape Tumblr's API uses, which is what the importers naturally produce.
- Media lives next to its post in
media.nosync/<year>/<slug>/, referenced by bare filename; nothing is ever hotlinked.
Everything that walks those directories walks them through lib/path_glob.rb. Dir.glob reads its whole argument as a pattern, and the installation's own absolute path used to be the first half of it -- so an install in ~/Sites/blog [1] had [1] read as a character class, every listing came back empty, and build reported an empty archive without naming a path. PathGlob.under(dir, '*', '*.json') passes the directory as base:, which Dir.glob never parses, and re-joins the results, so callers keep the absolute paths and the order they always had. The pattern half stays a pattern: a slug, a filename or a typed directory that lands there goes through PathGlob.literal first, which escapes the metacharacters so the name stands for itself.
The question one step before that -- may this value become a path at all? -- has one answer, in lib/path_safety.rb. A slug typed at the CLI, a media filename off an import, a deploy target read from config and a directory named in a redirect all used to decide it for themselves, in four places with four sets of rules, which is three places to forget. It answers whether a string may be one path segment (safe_segment?) or a relative path (safe_relative_path?), and whether a resolved path is inside a directory it must not leave (within?, contained!). Containment resolves the deepest directory that exists rather than comparing strings, so a symlink pointing out of the archive is caught where a prefix check would have waved it through.
Field reference
The authoritative schema for anyone writing an importer -- new importers should produce exactly this and write it through PostWriter.write (which handles slug uniqueness and re-import dedup by source).
Post object:
| Field | Type | Notes |
|---|---|---|
slug | string, required | URL segment; Slug.slugify output |
date | string, required | ISO 8601 with offset |
content | array, required | blocks, see below |
title | string | optional. A post without one borrows the title of its first link block -- that is where a link post keeps what the reader takes for its title, and it becomes the heading (an h1 on the post's page, an h2 in a listing, both pointing where the link points), the <title>, the feed item and the structured data. With neither, the post is named from its own opening words (see PostText.name_and_rest), and only a post with no words at all falls back to the slug |
tags | array of strings | rendered as pills, slugified for tag URLs |
state | "published" | "draft" | absent = published |
draft_token | string | drafts only -- the hidden preview URL segment |
created_at | string | drafts only -- publish-time "was the date edited?" check |
type | string | explicit dominant content type; absent = derived from blocks. type: page belongs in FRONT MATTER only: what a stored post carries is page: true, and nothing reads type when deciding where a post is served |
source | object | platform plus optionally account, original_id -- the re-import dedup key |
mastodon_url | string | the post's comment toot (Mastodon sites), set on publish/toot |
bluesky_url | string | the announcement's human link (Bluesky sites), set on publish/bluesky |
bluesky_uri | string | the announcement's at:// URI -- what the thread API takes; stored alongside the URL because converting between them needs a handle→DID resolution round-trip |
former_slugs | array of strings | every address the post used to have, as "year/slug" frozen at rename time; the build emits a redirect stub for each (see props → rename). Engine-side history like the announcement URLs: edits and re-imports carry it over untouched |
unpublished_from | string | drafts only -- the "year/slug" address the post vacated when it was unpublished. Publishing consumes it: back under a different slug it becomes a former_slugs redirect, back under the same one it just disappears |
series | string | the series this post belongs to; with two or more published parts the build gives it a listing at /series/<slug>/, a line in the index at /series/, and puts "part 2 of 5" navigation on the post |
series_part | integer | position within the series -- without it, parts are ordered by date, which is the usual case; the number is for the rare insert |
pinned | string/boolean | one post held at the top of the front page. Truth test is strict (true/yes/1), so a hand-edited false cannot pin by accident; more than one pinned post warns and the newest wins. Listings past page one, the archives and the feeds ignore it |
toc | string/boolean | table of contents on the post page. Absent = automatic (from 4 headings up); an explicit false suppresses it, an explicit true forces it below that threshold |
hero | string/boolean | whether the first non-degenerate image runs full width above the post. Absent = whatever layout.hero says for the whole site |
scheduled | boolean | set on a draft whose date is its future publish time; the publish-scheduled cron publishes it then and drops the key |
unlisted | string/boolean | a published post out of the listings, the archives, the feeds, the sitemap and the search index, noindex on its page -- but on its ordinary dated address, with its redirects intact. Not a password (see Decisions); the truth test is loose, so a typo hides rather than exposes |
page | boolean | a page rather than a post: a permanent address (/<slug>/), out of the listings, the archives and the feed, but in the sitemap and the search index. Written as type: page in front matter; page: true is the older spelling and is still read |
translations | object | the post's text in the OTHER languages the site publishes, keyed by language code: {"de": {"title": …, "content": [ … ]}}. An entry may carry title, content (blocks, the same shapes as below), excerpt, slug -- the address that language serves the post at, which is NOT the post's own slug (that names the file and the media directory; internally the translated one travels as address_slug) -- and former_slugs, the "year/slug" addresses THIS LANGUAGE'S text is owed a redirect at: the ones it had as a post of its own before it was merged in as a translation. The build serves each at the site's root and in that language's tree, leading to that language's page (Translations.former_slugs). Nothing else: the date, the tags, the series, the state and the source belong to the post and hold in every language (Translations::TEXT_KEYS is the whole list). A language counts as written once it has a body. Absent = a post in the site's own language, which is every post of an archive that publishes one. See Localization |
redirect_from | array of strings | site-root paths the post answered at on its PREVIOUS platform ("/old-post/", "/2009/05/old-post.html"), written by importers when the new site keeps the old domain. The build emits a redirect stub for each, after everything real -- a live page, listing or site file always wins over a stub, out loud. Deliberately separate from former_slugs: that is rename history inside this site, this is where the post lived before it arrived. Paths ending .html/.htm become literal files (Blogger-era URLs had no trailing slash); first segments the site itself owns (posts, page, tag, type, assets, search, markdown) are refused. Published posts only; edits and re-imports carry it over untouched |
Blocks (content array entries), by type:
type | Fields |
|---|---|
text | text; subtype (heading1-heading6, quote; absent = paragraph); formatting; a quote may carry cite (attribution, rendered as a right-aligned line) |
list | style ("ul"/"ol"); items -- each {text, formatting?, children?} where children is a nested list block |
table | align (array of left/center/right); header (array of cells, absent when the table has none -- written as a table opening with the separator row, rendered without a <thead>); rows (array of cell arrays); a cell is {text, formatting?} |
code | text (verbatim, blank lines preserved); lang (cosmetic) |
chat | lines -- array of {name, text}; name may be nil for a continuation line |
hr | no fields |
image | media (see below); alt_text; caption |
file | media (url, size in bytes); label -- an attachment offered for download; the post's type becomes document when its text is caption-short |
video | one of four shapes: local file -- media (+ optional imported poster, same shape) and caption (the authoring CLI requires it); YouTube -- provider: "youtube", url, youtube_id, caption; another platform the address alone identifies (Vimeo, PeerTube, archive.org) -- provider, url, embed_id (+ embed_hash for an unlisted Vimeo, embed_origin for the PeerTube instance), see lib/embed.rb; imported embed -- embed_html (+ provider, url). A url alone renders as a polite "video unavailable" notice |
audio | local file -- media (first entry's url, no dimensions needed) and caption; a platform the address identifies (Spotify, SoundCloud, Mixcloud) -- provider, url and, where there is one, embed_id/embed_kind, see lib/embed.rb; a platform that had to be asked (Funkwhale, Bandcamp) -- provider, url, embed_src resolved once at save time by lib/embed_lookup.rb (+ embed_origin for the Funkwhale instance); imported embed -- embed_html (+ provider, url). A url alone renders a polite "audio unavailable" notice |
link | url; title; description. On a post with no title of its own the FIRST such block lends its title to the post (see title above) and then renders without it, leaving the description; anywhere else -- a titled post quoting a link mid-article, or a second link block -- it renders in full as a link card |
An unrecognized type renders as its raw JSON in a <pre> -- loud, not silent.
media is an array whose first entry is used: {url, width, height}. url is a bare filename inside the post's media.nosync/<year>/<slug>/ directory; width/height reserve layout space and should be there for every image -- without them the browser has nothing to hold the space with and the page jumps as the picture arrives. Only a real 1x1 is treated as degenerate and dropped from rendering: that is the tracking pixel the rule exists for, and an image of unknown size is shown rather than thrown away, because a picture somebody put in a post is not the engine's to discard over a missing number. The CLI reads the dimensions from PNG/JPEG/MP4 headers, and an importer should supply them too.
formatting is a flat array of {type, start, end} spans over the block's plain text -- offsets are Unicode codepoints (Ruby string indexing), end exclusive; spans may nest and overlap, and shorter spans render innermost. Span types the authoring CLI produces: bold, italic, strikethrough, code, link (url + optional title). Additionally accepted from imports: small, mention (blog.url), color (hex).