Zimi 1.9.3 - #72
Merged
Merged
Zimi 1.9.3#72
Conversation
The 1.9.2 entry claimed this issue and fixed a different one. What was reported is a checkbox that does nothing on any ZIM, read in dark mode, and that was still true after 1.9.2 shipped. The decision had two answers, darken or leave, so it could only ever add darkening to a light page. Modern Wikipedia ZIMs carry their own dark mode and follow the OS through skin-theme-clientpref-os, so the article was already dark: ticking had nothing to add, unticking nothing to remove. Three answers now, and unticked asks the page for its light face through color-scheme and through MediaWiki's theme class, remembering what the page shipped with so ticking puts it back. Verified in the reader on a Wikipedia ZIM with the OS in dark mode: unticked 248,249,250 with dark text; ticked 32,33,34. A capture left on the default is still untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1.9.2 shipped the two pictures for every engine that goes through Zimi's own Creator, and quietly not for the two that do not. warc2zim writes those ZIMs, accepts no arbitrary metadata, and seals the file — so the engine most likely to produce a capture worth doubting was the one engine that could not show you the comparison. The pictures go beside the library instead, under <data>/shots/, and the same two routes serve them with a fallback. The live picture was already being taken by the recording pass and thrown away; it is kept now. The packaged one is taken from the finished file, by serving that file's own entries to the browser over the same synthetic origin the in-flight path uses. Deleting a ZIM forgets both. Verified end to end on draculatheme.com/contribute: both pictures stored, both served as image/jpeg, and the packaged one shows the replayed page in full including its 773 contributor avatars. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
warc2zim writes the ZIM for the alive and zimit engines and takes nine flags, so everything the capture learned was dropped at that door: which entries are pages, what URL each came from, what the live page looked like. The cost was not confined to the capture. The title index took all 876 entries of a two-page site, so the library's shared vocabulary filled with asset URLs and every ZIM started offering "com" as a correction for ordinary words. Random article answered "no articles found". The card said two entries for a whole site, one of them a third-party widget. The file is now rewritten once at creation, adding the front-article flags the capture actually knows, the source URL warc2zim parses and never writes, both pictures, and a capture record under X-Zimi-Capture. Measured on a real 15.7 MB alive capture: 1.1s, same size, indexes rebuilt, replay machinery byte-identical. Two rules, both tested: only standard fields used as the spec intends plus X- metadata, and nothing else moves. The front-article flag is the field every viewer already reads for whether an entry is a page, so Kiwix gets a better ZIM from the same change. On any failure the converter's original file is kept exactly as written. The read side now uses it: the title index indexes pages (816 rows to one), and random draws from them. This replaces the store beside the library that 1.9.3 briefly used; the pictures belong in the ZIM. Measured on a real capture, before and after: suggest an avatar URL as a title -> Contribute - Dracula Theme random no articles found -> the page article count 2, one a vendor widget -> 1 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Wave one put the facts in the file; this is the read side using them, and it is deliberately three small changes rather than a subsystem. The domain map learns a capture's real address from its main entry path, which is the discovery interlang already had and this copy was missing despite its docstring promising the two were in sync. A just-created capture registered under dracula.com, .org, .io and .net and nothing could link to it until a restart; now it registers under draculatheme.com, and /resolve finds it immediately. The panel and the card read the record: one page, not the converter's two, one of which was a vendor's feedback widget. Verified on the real capture: source, both pictures, suggest, random and resolve all correct, article count 1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nothing set a colour scheme on the capture browser, so every capture was taken in the default, which is light. A site with its own dark mode serves a different page for each, so a person reading in dark got a capture that looked nothing like the site in front of them. The session takes a color_scheme now. Left unset it stays the browser's own default rather than forcing light, which is the honest answer when nobody asked. Also #68: the Create page kept hiding the topbar's controls, which on a wide window turned a desktop toolbar into three icons and an overflow menu — it reads as the bar breaking rather than as focus. Create is an ordinary page under the same bar now; Almanac still hides them, because it paints its own full-screen scene. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Eric: every site a user shares should work faithfully and stay on a list we keep validating. A site can be broken in one way and fine in another, and the broken half was simply absent from the list. An entry may now carry a second check with a known_broken reason. The suite runs it and reports it, does not fail the build while it fails, and DOES fail the day it starts passing — because that is the day somebody is owed news. draculatheme.com's palette switch goes in as the first one, with what was measured: every chunk is in the ZIM, all inline scripts parse, no console, page or rejection errors, the RSC payload sits unconsumed where the live page drains it, and React never mounts. It fails identically with the ZIM served bare, no Zimi reader involved, so it is warc2zim/wabac replay and not ours. Neither the shipped patch nor the full upstream fix moves it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three of the reporter's issues, all the same shape: the app knew something and did not say it. #67 — the home screen sorted by article count, which rewards big files rather than the one you are looking for and disagreed with the list in Settings. A–Z is the default now, with Recently added, Recently updated and Most articles beside it. Order and view share one control, because both answer "how do I want to look at my library". #69 — the update frequency greys out when ZIMI_AUTO_UPDATE is set, and the reason lived in a hover tooltip. He saw "Daily" that would not change and no way to learn why. The reason is beside the control now. #70 — BitTorrent says "unavailable" when libtorrent has no wheel for this Python. The server already worked out the exact next step and only wrote it to the log, where the person looking at the greyed-out switch in a browser cannot see it. It is shown. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Eric: "Capture all sites in light and dark modes if they have them." A site whose dark mode is a media query serves a different page to a reader who prefers dark. A capture could only ever keep one, so reading a captured site in dark got you the light one, which is not what the site does. The other face is read from the page already loaded and settled, before serialization mutates it: flipping the media query is a repaint, where a second visit would be another download and could disagree about more than the theme. A cheap fingerprint of the painted colours decides whether the page actually changed, so a site with one face is still stored as one — and a site whose theme is a class some script sets from localStorage does not flip, which is the honest answer rather than a duplicate. It is stored as an ordinary entry beside the first with a metadata key saying which is which, so any other viewer can still open it, and it is rewritten through the same asset carrier: the second face is the same page repainted, so its images and fonts were already carried. MDN's CSS page holds both in 1.1 MB. The reader opens whichever matches the theme in front of the person. Verified end to end: reading in dark opens A/index~other at rgb(24,25,27); reading in light opens A/index at white. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The disk-cache guard did its job: a new key appeared and the test asked for a deliberate decision. It is written only when a capture actually kept two faces, and read with .get() everywhere, so an older Zimi reading the same cache ignores it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first cut of this had a real defect I found before shipping it: the other face was read from the same page as the capture, and _PREPARE_JS mutates the page it serializes. Asking for the second face first handed the real capture a DOM that had already been stripped and rewritten once — on exactly the sites this feature exists for. Reordering it fixed the corruption and broke the feature: by then the scripts that answer the theme are gone, so the flip changes nothing and MDN came out with one face. So it is two steps now. While the page is still live and before anything is serialized, a non-destructive check flips the media query, compares the painted colours, and flips back. Only if that says the site really paints differently does the other face get fetched, on a visit of its own, in the other scheme. Cost: nothing for the web that has one face (example.com, 6s, unchanged), one extra visit for the sites that have two (MDN, 12s, both faces in 1.2 MB). Verified through the reader: light opens A/index at white, dark opens A/index~other at rgb(24,25,27), same content in both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The red CI badge on the README was this test: it called Kiwix's real OPDS feed and accepted 200 or 502, so the whole four-runner matrix depended on the network. On a slow macOS Intel runner the CLIENT timed out before either status arrived, and the failure said nothing about Zimi. The upstream fetch is stubbed, so it tests our proxy — which is all it ever claimed to test — and a second case covers the unreachable-feed path returning 502 rather than a 500. Test count badge to 2711. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The reporter said draculatheme.com's theme switcher and colour tabs did
nothing in the ZIM. They were right, and it was not the block-scoped
globals bug: rebuilding with both the shipped patch and the full upstream
fix changed nothing.
warc2zim stores a page's references relative, so `src="/_next/app.js"`
becomes `src="_next/app.js"`. Right for the browser, which resolves it
against the page. Wrong for the page's own JavaScript when it reads the
attribute back — and Turbopack, which Next.js builds with, identifies a
chunk by script.getAttribute("src") and strips a leading `/_next/` to get
the key it filed that chunk under. With the slash gone the strip fails,
every chunk registers under a name nothing waits for, and the entry
module never runs: no error, no failed request, a page that renders and
does nothing. 191 module factories run live; none in the ZIM.
wombat already patches the .src PROPERTY to answer with the original URL.
The shim does the same for the attribute, for scripts only, deriving its
answer from that same patched property so the two agree.
Then our own reader turned out to be breaking what the shim fixed. The
consent-wall sweep and the chrome settle edit the article's DOM, which is
safe on a frozen capture and not on one that is about to hydrate: React
found the DOM changed under it, threw "removeChild: the node to be
removed is not a child", and re-rendered a stump. The page worked in a
bare iframe and broke in ours, which is how it was found. On a live
replay both passes now wait for the page to settle.
Verified through Zimi's reader on a fresh capture: Alucard moves the
palette from #282A36 to #FFFBEB, and Hex/RGB/HSL all switch format. The
reported-sites suite carries the interaction as an ordinary expectation
now rather than a known failure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three defects, none of which had a failing test, which is why each one has one now. The seed page's live picture was not the seed's. A crawl fetches every page in the frontier through the same capture, and each fetch overwrote last_shot — so the About panel showed whichever page the crawl happened to end on as "the live page", beside a packaged picture of the site's front page. Two unrelated pages presented as before and after, and a height comparison between them that could warn about a collapse that never happened. The first page wins now. Both readers of the faces record promised they never raise and did. A ZIM whose metadata parses to a list, a string or a number reached .get() on it and threw AttributeError, which aborted metadata extraction for the whole ZIM in one path and would have 500'd an info request in the other. The type is checked before the value is used. A recording pass was paying for a second face it never reads. The alive engine's archive is what becomes the ZIM and a second visit is not recorded into it, so every alive capture of a site with a dark mode — much of the modern web — was taking twice the time and bandwidth for something nothing would ever look at. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e is still a place Four of the sixteen things Eric found on the 1.9.3 walkthrough. The tracker for all sixteen is docs/plans/2026-09-11-v193-walkthrough-fixes.md. The library order was applied where the full list was built, which covers categories — they are partitioned out of the already-sorted list — but not Favorites or Collections, which are built from stored NAME lists in the order things were starred or added. Sorting the source is not sorting the views when two of the three views do not descend from it. "A–Z" becomes "Alphabetical" in all ten locales. The old label named the mechanism; the others in the menu name the intent. The world clock's 28 cards are a curated tour, not a list of zones. Matching fell back to "the first card sharing this offset", which covers a resolved zone like Europe/Berlin landing on Paris's +2 column — and covers nothing at all for a fractional zone, because no card is at +8:45. Clicking Eucla on the map left every card dark, which reads as a click that did nothing. The decision is now a named function, _almTzCardMatch, whose -1 is a real answer: the caller gives that zone a card of its own at the head of the row. Fourteen of the map's 173 dots take that path. Being off the tour is a property of the instant, not the zone — Lord Howe sits on Sydney's +11 in January and drifts to +10:30 in July — and the new test pins that. The Messages Across Time pill kept a permanent underline to advertise that a second tap opens the article. An affordance can wait for the pointer; it now appears on hover and focus. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five of Eric's walkthrough findings, all on draculatheme.com, and four of them turn out to be two defects wearing different clothes. THE PICTURE WAS OF A PAGE THAT NEVER SHIPS. finish_zim photographed warc2zim's output and then patched the file. But the patch is what inserts the loader shim — the fix for #64 — so the photograph was of a ZIM still carrying the defect: a Next.js bundle whose module loader cannot find its own chunks, so nothing runs and the page is a shell with an empty middle. Then shot_verdict compared that shell against the live page and said "the packaged page is much shorter, something did not survive capture" about a ZIM that renders perfectly the moment you open it. Eric: "it does look better than the screenshot said it would??" It did. The picture is now taken DURING the rewrite, from the bytes going into the file, through a callback zimpatch hands the finished document and an accessor over the assets it is carrying across untouched. AND IT WAS TAKEN TOO EARLY. shoot_packaged waited for `load` and nothing else. For a frozen capture that is right — the document IS the finished DOM. For a live one it is not: every script is served out of the ZIM at memory speed, so the network goes quiet while the framework has not yet run a line of render code. Hydration is CPU work. _settle_boot watches the page's own height instead and stops when it has held still. Measured on this capture: quiet at 900px, finished at 19,851. Both changes together take the stored dims from 1280x19858,1280x900 to 1280x19858,1280x19858, and the warning stops firing. TWO FACES, FOR SITES THAT KEEP THEIR THEMES THEMSELVES. The probe flipped prefers-color-scheme and looked for a repaint, which is most of the web and not this site: draculatheme.com stamps data-theme="dark" whatever the query says and flips only when you press its moon. So the probe was right that the query changed nothing and wrong that the site has one face, and the capture stored a page with a visible theme button that did nothing. It now falls back to looking for a control the page itself labels as a theme switch — an aria-label, a title, a data attribute, never a guess from a glyph — and the second visit presses it. Which face came back is then MEASURED from what it paints, because a site that ignored our query cannot be filed under it. AND THE SECOND FACE BRINGS ITS OWN PICTURES. The comfortable assumption was that a second face is the same page repainted and needs nothing new. Dracula's hero is a different file in light, images/hero/default-light.svg, so the stored light page kept an absolute remote URL — offline, a gap where the hero was. Eric: "in light mode the dracula image doesn't appear". The second visit now collects its own subresources, minus the set the first face already carried, and render_other carries them through the same sink. Verified in Zimi's own reader, not in the code: dark opens A/index at rgb(14,13,17), light opens A/index~other at rgb(233,231,226), 473 images and none broken in either, and the light hero is the Alucard art rather than a gap. Also stores X-Zimi-Screenshot-Dims for warc2zim ZIMs, which every other engine already wrote, so About can compare the pair for these too. 2,729 tests pass. tests/test_second_face.py drives the whole path end to end against a fixture site shaped like this one: no media query, a labelled theme button, an image that swaps file when it is pressed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Eric found 20 zombies on the NAS in 21 hours of uptime — 12 chrome-headless, 7 python3, 1 wget — all parented to the container's PID 1, which is `python3 -m zimi serve` and reaps nothing. Both streaming runners collected the child only when the command finished normally. _run_stream's reap sat on the line AFTER its try/finally, and the sink it streams through is the job's note(), which is the cancellation checkpoint and raises by design — so cancelling a capture, an ordinary thing to do, skipped it every time. _run_streaming had three holes: an uncaught TimeoutExpired, a KeyboardInterrupt branch that called terminate() without wait(), and the same raising-note path. The zombies are the cheap part. What precedes each one is a browser that should have stopped and did not, still reading from a saturated disk, for a capture whose result was already thrown away. zimi/subproc.py is the one answer to "how does a child die here". Every child starts in a process group of its own, because the things Zimi shells out to are TREES — the sidecar starts Chrome, which starts a renderer per tab, a GPU process and a zygote, and terminating the single PID Python holds orphans the rest. Stopping asks the group, kills the group, and always reaps, in a finally, never raising: it runs on the way out, often while another exception is already travelling, and a failure to kill must not replace the reason the caller was leaving. Windows has no process groups to signal, so it goes through taskkill /T. init: true on all three compose files is the backstop for anything the code still misses. A net, not the fix — tini collects zombies, it does not stop a runaway Chrome from burning CPU. The tests are written against the expensive half rather than the visible one: after a raising sink, is the child actually dead, and did its grandchild die with it. Verified they fail against the old shape. Reported-by: Eric Pheterson Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…after Three things Eric found on the Create page, plus the picture he asked for. THE PROBE ONLY RAN WHEN THE FIELD LOST FOCUS. 'change' fires on blur, so pasting an address and pressing Create started the job without ever asking the server what was there — and the probe is not decoration: it picks the engine, and it can move the mode, because a video address typed under Web page is a video. Eric: "I could've clicked create right away and missed that step." Now typing asks too, after a 600ms pause, and Create waits for an unanswered probe before starting, then re-reads the form because answering it may have changed both. A FINISHED RUN LEFT ITS ANSWER IN THE FORM. The address, the title, the preview and every per-mode stash survived into the next creation. They are cleared when a job completes — but only when the form still holds the job that just finished, so someone who started typing the next address while the last one ran never has it taken away mid-word. AND THE PICTURE OF THE PAGE IS ON SCREEN WHILE IT IS BEING CAPTURED. It is taken seconds into a job that can run for minutes, and was only ever visible once the ZIM existed. Eric: "if we're taking a screenshot of source why not splash that up at the top while creating." The engines announce it on the progress channel they already have, the web sink takes the bytes off the event and keeps them (the event tail is polled JSON — a JPEG cannot ride in it), and /manage/create/shot serves them under the job's own id so each run's picture is its own cacheable resource. Terminal sinks ignore events, as they always have. Not built, deliberately: periodic snapshots through the run. Eric floated it and called it overkill himself. One picture within seconds is what was missing; a stream of them is a screenshot per interval on the machine that is already the bottleneck, for a page that mostly does not change. 2,737 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… restart Eric: "Creator tab is still super slow to load in manage view.... then after it loads the made here is super slow all of that blows it should be snappy." Two costs, both paid once per process — which sounds fine until you notice a deploy restarts the process, so every deploy the first admin to open the pane paid them both. THE CAPABILITY ROWS. Finding out whether the rendered engine works means launching a browser: 2.5s measured here, and the pane waited for it, plus a subprocess for the sidecar's version. They are probed on a thread now and the pane answers with whatever is known — 21ms instead of ~4s. The three capability fields are null rather than false while that is outstanding, because "we have not looked" is not "you cannot do this", and showing the second for the first tells an admin their browser is missing when it is not; the client renders "Checking…" and asks again. The answer is kept but goes stale after 20s, so a refresh runs BEHIND the answer already on screen — without that, running `zimi import --setup` would never show up in the pane that told you to run it, which the existing tests caught me taking away. THE MADE-HERE LIST. "Which ZIMs did Zimi make" was answered by opening every archive in the library and reading three metadata fields, memoized per process — and the memo dies with the process. It is a property of the FILE, keyed by the same name-and-size signature the memo uses, so it now lives in the metadata cache beside every other fact about that file, written once per walk rather than once per ZIM. A fresh process with a warm cache does zero archive reads; on the 73-ZIM NAS that is 73 opens removed from every restart. None is stored as a real answer, because most ZIMs were published by somebody else and that is exactly the case worth remembering. 2,743 tests pass. Reported-by: Eric Pheterson Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… the pane kind_store makes "which ZIMs did Zimi make" survive a restart, but somebody still has to pay for the first walk, and left alone that somebody is whoever opens Manage -> Creator first after a deploy — which is exactly the wait the cache was meant to remove. It runs on the shape worker's existing thread, after its settle, so no new thread and nothing on the serving path waits for it. A library whose cache already holds the answer costs nothing here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Eric, 2026-09-11: "icons disappear and redownload when i toggle compact or full", "the whole site is held up", "pulsing on the top a lot". The icon at /w/<zim>/-/icon, and the two capture pictures beside it, were sent as `max-age=0, must-revalidate` with an ETag over the CONTENT. Correct and ruinous. The browser asked about every picture on every render, and answering the ask meant opening the archive and hashing the illustration — under the global libzim lock, the same one every search and every article read needs. Seventy-four sources is seventy-four locked reads on every sort, every view toggle, every return to the page. Two changes. The tag now names the FILE — mtime and size, read from the list cache — so a revalidation is answered BEFORE the lock is taken, by a branch that never touches the archive. And there is a real freshness window, so a re-render does not ask at all. Still not `immutable`, and the window is seconds: a ZIM's bytes can change at the same URL, and an earlier version of this was immutable for a week, which is how Eric once watched a ZIM he had just made show a missing icon in one browser and the right one in another. When the list cache cannot identify a file the content digest is still there to fall back on. Measured on a 74-source library, four page loads and six view toggles: 74 archive reads in total, all on the first paint, none after. Icons draw in ~85ms on a revisit against ~390ms cold. Also coalesces the home re-renders that follow late data. Three separate things land in the first second of a cold load — the library list, the collections, the provenance walk — and each one re-rendered the whole grid, which is what was visibly pulsing. Worth recording what did NOT turn out to be the lever: the service worker routes /w/ subresources network-first, and I changed it to serve these three from its own cache instead. Measured against the unchanged worker, that bought exactly nothing — the browser's HTTP cache was already absorbing them once the server stopped saying max-age=0 — so it is not in this commit. The bug was one header the whole time. 2,749 tests pass. Reported-by: Eric Pheterson Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Eric, 2026-09-11: "the whole site is held up while it's loading the discover stuff something is inefficient there." It was. /random took the global libzim lock ONCE and held it across every attempt — up to 50 for a Wiktionary card, 30 for Gutenberg and Wikiquote — and ran all of them regardless, because the judging happened after the loop. So a single Discover card read fifty random articles, extracted fifty previews, and held the lock that every search and every article read needs for all of it, even when the first pick was perfect. Discover fires one of those per card, in parallel, on page load. libzim needs the lock around each READ, not across a sequence of them. Holding it for the sequence was a throughput trade that cost fairness completely. Now it is one acquisition per attempt, and each pick is judged as it arrives, so the usual case reads once instead of fifty times. The judging moved into _random_pick_verdict, which is what made "judge it as it arrives" possible without taking the lock along with it. It is a faithful transcription of the chain it replaces, including the part that is easy to get wrong: a Gutenberg pick that IS the cover page still has to satisfy the thumbnail requirement, so it falls through rather than being accepted outright. Also, on the library cards: the date a list is ordered by now appears on the card when — and only when — it is the sort key. Eric asked whether it should always show in full view. It should not: the card already says what it is, how much of it there is and how big it is, and a fourth standing fact answers a question nobody asked. But an order whose key you cannot see is one you have to take on trust. And the Messages Across Time pill loses its underline entirely, hover included, as asked. The tooltip and the aria-label still say the tap does something. And an off-tour timezone card is inserted where its clock belongs rather than at the head of the row — the row reads west to east, and a +8:45 card to the left of Honolulu made the whole line nonsense. 2,749 tests pass. Reported-by: Eric Pheterson Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported by Joe (WB3IHY), with the cause and the fix, on a fresh 1.9.2 in two browsers: every PDF opened through the reader stayed permanently blank, while the same viewer URL opened directly rendered fine. Two reasonable behaviours collided. The reader loads Zimi's bundled pdf.js viewer into the same iframe it loads ZIM articles into, and that iframe's onload runs the passes that tidy up a CAPTURED page — stopping runaway animations, dropping empty ad slots, collapsing hollow blocks. At the instant onload fires, pdf.js has parsed the document but has not painted a canvas; that happens a moment later, off its own IntersectionObserver. So #viewerContainer is right then a tall, empty, transparent box containing no embedded media, which is exactly how the hollow-block sweep recognises an abandoned ad slot. It got height:0 !important, nothing was ever in view again for pdf.js's observer, and no page rendered. A hard inline write, so no resize or reload in that session undid it. The rest of the onload handler already knew about this distinction — three blocks below the sweep check the frame's path for /static/ before touching anything. These two did not. The check is now a named function, so there is one answer to "is this our own page" and the next pass added has something to ask. Reproduced first: a zimgit-shaped ZIM with a real PDF, opened through the deep link, #viewerContainer at height 0px with zero canvases. After: 812px, pages rendered, and the sweep's marker style absent from that document entirely. The regression test drives the guard directly and reads the call site to check both passes are still behind it; verified it fails with the guard removed. An end-to-end browser test would be better and is not here — the bare-ZimHandler fixture the other reader tests use does not boot the SPA far enough to open a deep link, and fixing that is its own piece of work. Reported-by: Joe (WB3IHY) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The section was written on the 7th and stopped there. Two of its claims had also stopped being true: the second face is a separate visit that can press the site's own theme control, not a repaint, and it is no longer the case that a site theming itself by script has one face; and the alive engine's two pictures go inside the ZIM now rather than beside the library, because the file is being rewritten at creation anyway. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five things, found by looking rather than by being told. THE MATRIX FOUND A CAPTURE THAT DIES ON A FONT. nerdfonts.com asks for /assets/fonts/Symbols-2048-em Nerd Font Complete v233.woff2 — spaces and all — and the builtin engine ended the whole run on it. Two faults: the URL was handed over as written instead of encoded, which every browser does, and _http_asset_reader caught only OSError while InvalidURL is an HTTPException. Its sibling sixty lines up in the same file already caught all three families, with a comment explaining why after a nonsense reference took down a fifteen-page sqlite.org crawl. This was the twin nobody had fixed. Confirmed pre-existing on main before changing anything. Both fonts are now carried and all three engines pass both sites. /LIST WAS CARRYING MY BOOKKEEPING. The icon ETag needs the file's identity, and that lives on the library entry — which is the dict /list returns. So every client was being handed an mtime and a full copy of the provenance record that /zim-info?kinds=1 already serves on purpose: 9 KB of a 47 KB payload, 19%, on the path the home screen waits on. Stripped at the response, kept in the cache. THE RENDERED ENGINE'S LIVE PICTURE WALKED. A crawl calls render() per page and the shot was reassigned each time, so once the engines started announcing it the running job's picture stepped from page to page instead of showing the site that was asked for. Same first-page-only rule the recording engine already had. THE CREATE PAGE SHOWED THE SAME PICTURE TWICE. The live shot's job is to fill the wait; once the run is done the card below carries the live page and the packaged one side by side. It goes away when the job does. And --engine on the site suite, so the matrix can be swept at all. 2,761 tests pass. alive 2/2, rendered 2/2, builtin 2/2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…n a card Five of Eric's findings from the walkthrough. MADE HERE NEVER FINISHED, AND THE NAS GROUND. Mine. The metadata cache is read-modify-written in four places — the shape worker, the provenance walk I added, and registering or unregistering a ZIM — on different threads, with no lock. Whichever saved second was building on a copy taken before the first, so the first's work vanished. Losing a shape is not cosmetic: the shape worker re-measures anything without one, and measuring walks every entry in the file, under the global libzim lock, every fifteen minutes for ever. That is the grinding, and it starves the request that would have drawn Made Here. One lock now covers all four. The test forces the interleaving rather than racing for it — racing passes with the lock removed, which is worth nothing. THE ENGINE RECOMMENDED FOR AN APP WAS THE ONE THAT BREAKS IT. looks_like_spa asks whether the document arrived empty. draculatheme.com is server-rendered, so it arrives full of text and read as an ordinary page — and its palette, theme switch and search are all JavaScript. looks_like_app asks the other question, whether the page ships a framework runtime, from markers frameworks put in their own output. And since a recommendation you cannot see is one you cannot disagree with, the preview now says which kind of page it found. THE AGE WAS YELLOW AND TOO LONG. "21 hours ago" pushed the detail line onto two rows on a phone. It reads "21h" now, tabular, in the same colour as the count and the size beside it — it is one more fact about the file, not a warning. THE FAST ENGINE SHOWED NOTHING WHILE IT WORKED. It took the live picture at packaging time, so the engine most likely to be picked for a quick capture was the one that never showed the page during the wait. Taken at fetch now, same single visit. And the live picture goes away when the run ends, because the card below it already carries that picture beside the packaged one. 2,765 tests pass, 31 JS suites, and the three-engine matrix is green. Reported-by: Eric Pheterson Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sorting changes only the sequence — the same cards, in a different order — and the whole library was being thrown away and rebuilt for it. That is the flash, and it re-requested every icon on the way. The cards are reordered in place now and travel from where they were to where they belong: first/last/invert/play, one movement for the whole shelf, released on the same frame so it reads as the shelf rearranging rather than as seventy-four separate animations. Measured during a re-sort: zero icon requests, no card left holding a transform. prefers-reduced-motion gets the new order without the travel. Anything the move does not recognise — a card whose ZIM is no longer in the library — falls back to the rebuild, and the test covers that path rather than assuming it. The age that comes with a date sort is rewritten in place for the same reason: one span per card, so nothing else on the card is touched. And the Discover strip loses its scrollbar. The thumb was a short line reaching about halfway across, sitting under the cards and reading as an artefact rather than a control. The next card is always cut off at the edge, which is what actually says there is more. 2,765 tests pass, 31 JS suites. Reported-by: Eric Pheterson Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…he dots THE AUDIT FOUND A FIFTH WRITER. The Q-ID pass opened cache.json itself, merged has_qids and wrote the whole file back — outside the lock the other four now share, so it could land on top of a shape or a provenance record saved a moment earlier and erase it, and be erased the same way. Reading the raw file also skipped the cache's version check, which means merging into a format this build no longer understands and writing it back out. It goes through the shared read-modify-write now, and a contract test walks zimi/*.py for anyone else calling _save_disk_cache or reaching for the cache path directly. Also audited and found sound, so it does not get changed: the shape walk takes the libzim lock per RUN of entries and releases between runs, which is why a 40 GB file does not starve the server while being measured — the grinding was the repetition caused by lost updates, not one walk. And the almanac's Q-ID title fallback holds the lock across a batch of up to 400 lookups, which its docstring justifies as microseconds each; that is true of a warm page cache and not of a cold NAS, so it is worth a second look, but not a change made the night before a release. THE DATE ON A CARD SAYS WHICH DATE IT IS. Two orders put a date there and they mean opposite things, so it reads "Added 12 Sep" or "Updated 12 Sep". Written out rather than counted down: the elapsed form was longer than the date it stood in for and said less. Both dates carry their own separator, so either can be taken out without leaving a dangling middot — which is what lets the in-place re-sort swap them without rebuilding the card, and stops a ZIM's own date sitting next to the sorted one saying the same thing twice. THE STEP RAIL RAN THROUGH THE DOTS. It spanned centre to centre, so the line went under each dot and out the other side, and the dot's fill is not the page's. It runs between them now. 2,766 tests pass, 31 JS suites. Reported-by: Eric Pheterson Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Eric: "when I said timestamp I meant the higher timestamp not age. Age I wanted small. One line it's too much info there right!?" Right. The line carries a count and a size already, and a third fact spelled out was more than a glance needs. So the visible half is two characters and a unit — 7h — and the whole answer moved to the timestamp behind it: "Added September 12, 2026 at 4:58 PM". Which of the two dates it is still has to be said, because they mean opposite things, and the tooltip is where there is room to say it. And the release notes are cut from 2,705 words to about 1,200. They were stories; the story belongs in the commit message, which is where these ones have been all along. Twenty-three entries, each a fact. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nothing was wrong with the server. The endpoint answers in 41ms and the provenance it needs was already on disk — 75 of 76 ZIMs had it. I spent the last two rounds fixing real bugs in the part that was working. _creatorHtml builds the pane as a string, and part way through building it it calls _creatorLoadInventory(). With the inventory already in hand that filled the slot SYNCHRONOUSLY — writing into the DOM the pane is about to replace — and then the string being built, which still contains the "Loading…" placeholder, was assigned over the top. Nothing ever filled it again. So it hung precisely when the answer arrived fastest, which is why warming the provenance cache and taking the lock off the walk made it worse rather than better. The fill is deferred past the caller's own innerHTML assignment now, the same way _msCreatorHtml already defers its first render. Reproduced in a browser first — 200 on the request, _creatorInventory set, _msSection 'creator', and the slot still showing a spinner — and the test is verified to fail with the deferral removed. 2,766 tests pass. Reported-by: Eric Pheterson Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Made Here's names are anchors and nothing styled them, so they got the user agent's blue and an underline — the only place in the app that happened. They read as ordinary text now and answer to the pointer in amber, the way the sortable headers beside them already do. The three "Checking…" rows in the same screenshot are not a bug: a cold process has not launched a browser yet, and the row resolves about a second and a half later. Verified against a freshly restarted server rather than a warm one, which is what hid it the first time I looked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Eric: "I don't like those release notes, bulleted facts simpler better." They were still paragraphs wearing a bullet. 1,207 words to 367, one line each, what changed rather than how. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Eric: they describe polish on things that did not exist in 1.9.2, so there is no slower version anybody upgrading has experienced. Calling them faster implies a regression that was never shipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thirty-two commits, closing every open report from the 1.9.2 walkthroughs.
What it fixes
script.getAttribute("src")and strips a leading/_next/, so with the slash gone nothing ran. Fixed upstream in python-scraperlib#337; Zimi ships a shim until that releases.Also: captures keep both of a site's faces, a capture's own pictures stopped libelling it, cancelled captures stop their browser, a filename with spaces stopped ending a capture, the engine suggested for a JavaScript app is one that can capture it, and five background writers stopped erasing each other's work on the metadata cache.
Verification
Release notes are the
[1.9.3]section of CHANGELOG.md.