Importing
An import splits along the line between what every platform needs and what only one does, so that adding a source means writing its mapping, not another copy of the plumbing every existing adapter shares:
Import::Mediacollects one post's media forPostWriter, from either a URL (download, following redirects, retrying transient failures) or a path an export already contains. Filenames are numbered per post in registration order, which is what makes a re-import land on the same names soPostWriter's "skip if it exists" copy is a no-op instead of a duplicate. In dry-run it allocates and counts without fetching -- so adapters must take image dimensions from platform metadata, never from the downloaded bytes.Import::Rundrives: walk the source, hand each item to the adapter, write it, count why items were skipped, report. Skips are Symbols, so the summary can answer the question anyone reads a summary for -- why fewer posts arrived than the source has. Two callbacks,on_scanandon_post, let a wizard show progress without this layer knowing what a terminal is.- An adapter implements only
label,each_item(paging the source) andmap(item, media), returning a post hash or a skip reason. Five optional hooks cover what sources differ on:preamble(a line to print before a slow read),total(the source's size, often knowable only once the first page arrives),postscript(a one-line note after a run or preview -- scheduled posts becoming drafts, posts with no usable original address),platform_tag(the tag posts are filed under when the platform's name makes a poor one -- "medium.com" says more than "feed") andkeep_permalinks=(a setter whose presence tells the wizard this adapter can record old addresses asredirect_from, so the question is asked per capability and defaults to no). Import::HtmlBlocksconverts a post body that arrives as markup into blocks, for the sources that hand over HTML rather than structured data. A tolerant tokenizer and a stack-based tree (an XML parser refuses real post HTML outright), then a conservative mapping: exactly what the schema can represent, unknown wrappers walked through, and anything with no representable shape dropped and counted rather than guessed at.Import::Cliis the non-interactive front end thescripts/migrate_*.rbwrappers share, so each is a handful of lines. Every source is therefore reachable both ways -- wizard or script -- over one implementation of the mapping.
Import::Feed covers WordPress and RSS/Atom in one adapter, because they are one format: a WXR export is RSS 2.0, with a wp: namespace layered on for what a feed has no room for (post_type to filter by -- in a stock export menu items, attachments and pages outnumber the posts -- status for drafts, post_id for dedup, post_name for the slug the site already published under). It takes the date from pubDate rather than wp:post_date, which carries no offset and would otherwise be read in site.timezone and shift by hours. Images referenced in the markup are downloaded and then measured: HTML rarely states dimensions, and a block without them renders with nothing reserving its place, so the page jumps as each picture lands.
An adapter judges items rather than pre-filtering them: map returns the reason as a Symbol -- :reply, :retweet, :quote, :empty and their kin; the wizard's TRANSLATED_REASONS in scripts/import.rb names two dozen across the adapters -- instead of quietly dropping the item, so the run's summary can account for every item the source held. That also means a written-post counter is not a progress fraction -- a source that skips half its items never writes as many posts as it has -- so both counters are reported and the caller picks: against a LIMIT the goal is posts written, against a whole source it's position scanned.
Import::Bluesky is the reference adapter. Two things it has to get right generalize to any API source: facet offsets are UTF-8 byte positions while the schema's formatting spans are codepoints, so every boundary is converted (skip that and spans land several characters off on any non-ASCII text), and video arrives as an HLS playlist rather than a file, so the poster frame is imported as an image -- a video block with no media would render as "video unavailable".
scripts/import.rb is a second wizard rather than a menu entry in blog.sh, and always previews in dry-run before writing. lib/launcher.rb is the one piece that is for a single platform: on a Mac setup.sh can put an icon in Applications that opens Terminal in the site's folder and runs ./blog.sh, built with tools macOS ships. lib/site_header.rb is shared by both wizards so the site-identity block can't drift between them.
Supporting casts: lib/slug.rb (one folding/slugification used for post slugs, tag slugs, heading anchors and the search index -- and mirrored client-side by fold() in search.js), lib/content_type.rb (dominant type: video > audio > image > chat > quote > link > text, where "quote" means the post's first block is one, and media win only while the post's text stays caption-sized -- past 500 characters it's an article with illustrations, i.e. text), lib/media_dimensions.rb (width/height straight from PNG/JPEG/MP4 headers -- EXIF orientation included, so a photo taken sideways reserves the space it is shown at), lib/video_probe.rb (the video track's codec from the same box walk, two levels further down at stsd, so a save can say that a clip is HEVC).
