Every answer, every search vector, every transcript and every description is made on this machine. One background job holds the device at a time, and it steps aside whenever a person is waiting for an answer.
NVIDIA L4: 16,583 of 23,034 MiB in useanswer model runningsearch vectors on the GPU0 hosted-model callsbackground slot: vision
The device
Driver 595.91.07 · 91% busy · 67 °C · 71 W · the rule is to keep 3,072 MiB free after any model is loaded.
On the device
MiB
pipeline worker: vision (process 520897)
13,014
the answer model (llama.cpp server) (process 138057)
1,788
the query-vector service (process 208593)
1,762
free
6,451
What is waiting
the pipeline is running
Work
Tier
Waiting
Batches queued / running
Per hour
text vectors
1
0
0 / 0
–
image vectors
2
0
0 / 0
–
scene vectors
2
0
0 / 0
–
faces
3
0
0 / 0
–
curate
5
–
0 / 0
86
film speech
3
0
0 / 0
–
film vision
3
0
0 / 0
–
photo vision
3
10490
3 / 1
2,960
extract
2
0
0 / 0
–
Workers: vision-79425-21 (busy, pl.photo_vision)
Photograph descriptions
what a model sees in each photograph, made findable by what it shows
29,760 of 40,154 photographs described (9,536 with words read in them); 10,394 left. Measured: 2,106 an hour while the pass runs (since it began 1,565 an hour, with the stops and the waits for other work counted), so about 4.9 hours of running are left.
A photograph takes 1.61 s of the device when it has the device to itself. A description is written by a model, kept apart from every printed caption, and says so wherever it is shown; the words read in a picture are shown as read, never as a name.
Who has the device, and how long work waits
foreground answers come first; background work takes turns
Held by vision for 1,527 s. Nothing else is waiting.
The rules: a kind keeps the device for at most 15 minutes while another kind of its own tier waits; a kind of a better tier takes it at the holder's next checkpoint; aging is off on the device, so a kind that has waited 30 minutes is given one turn of at least 3 minutes whatever its tier; a film runs at most 3 minutes at a time while other work of its priority waits. Work is done in units of a few photographs, one scene of a film or three pages, each committed as it is finished.
Queue (last 24 hours)
Batches
Mean wait (s)
95% within (s)
Longest (s)
curate
18
2151
3572.8
6317.3
extract
41
15.8
31.3
455.4
extract sweep
440
20.5
23.9
1172.9
faces
12
5.5
12.7
13.2
film speech
55
11
38.3
123.7
film vision
55
262.5
756.4
1847.7
photo vision
3,719
45.9
129.6
5019.3
previews
1
0.6
0.6
0.6
text vectors
13
16.9
43.5
47.7
verify
199
1.4
1.9
93.6
The last 24 hours
no batch failed
Kind
Batches
Items
Seconds
Given to answers (s)
Peak MiB
Failed
Stopped
embed
31
977
966.4
40
4,925
0
0
extract
485
1,426
5564.5
184
3,564
0
4
faces
15
220
40.9
0
4,621
0
0
speech
58
52
324.4
0
4,789
0
3
vision
3,784
31,618
58709.4
6978.5
18,339
0
27
Films and new text
55 films held · speech heard in 55 · 55 looked at (2,518 frames, 2,515 described, 829 with words read, 16,431 faces found in frames)
Local extraction of new text: 0 pages waiting · read so far: 2 applied, 14 carried, 14 nothing, 679 proposed
The models held
downloaded once, with the revision and a checksum of every file
Model
Revision
Size
Used for
mobiuslabsgmbh/faster-whisper-large-v3-turbo
mit
0a363e9161cb
1.5 GB
multilingual transcription with segment timestamps (English, Urdu, mixed)
Qwen/Qwen3-VL-4B-Instruct
apache-2.0
ebb281ec70b0
8.3 GB
scene and photograph descriptions, reading text visible in a frame, local structured extraction from text
BAAI/bge-m3
mit
5617a9f61b02
2.1 GB
passage and query vectors, 1024-d
google/siglip-base-patch16-224
apache-2.0
7fd15f0689c7
778 MB
photograph, scene and picture-query vectors, 768-d
insightface buffalo_l
325 MB
face detection (SCRFD-10G) and ArcFace R50 embeddings
google/gemma-4-E2B-it-qat-q4_0-gguf
apache-2.0
3.1 GB
answers to questions (the local generator)
The answer model is gemma-4-E2B_q4_0-it.gguf on llama.cpp (cuda), started with -c 8192 -np 2 -ngl 99 -t 2 at 9 Oct 2026 19:27 PKT.
Corrections and retractions
every change to a verification record, with the runs it concerns
#14 correction of selection:requoted_fact (run rule:newer_capture@1): Fact 3032 (SM Jaffar is remembered as an all-time hockey great ...; page Hockey) was made inactive by the newer-capture replacement because its quotation ended with a clause ('... can be read on this') that the newer capture's text does not have, although the page still states the claim (checked on the live site, 10 October 2026). It was re-quoted with the two sentences the page carries and moved to the capture in use (revision 4196). — made by one-off correction, 10 October 2026
#13 correction of selection:text_lost@1 (run review:text_lost@1): The first application of the newer-capture rule replaced the older capture of the Archives page (revision 94, a text export) by its newest capture (revision 3605), whose TEXT is 521 characters because the page keeps its content in slides hidden until clicked and the text extraction dropped hidden elements; the stored page of the newer capture and the live site hold the museum visiting hours, the telephone number and the history. The 18 claims, 6 passages, 17 links and 1 membership the replacement had retired were put back (selection.reinstate_captures), the rule now never replaces a capture by one whose text is poorer than its own page, and the text extraction keeps hidden panels that hold the page's own text (collect/htmlextract.py: is_hidden_content; over 2,632 stored current pages it changes only this one). — made by selection.reinstate_captures, 10 October 2026 09:2x UTC
#12 correction of selection:newer_capture@1 (run rule:newer_capture@1): A page, portal page or document captured again is now stood for by its newest complete capture: the 198 older live captures that a newer capture existed for were looked at; 196 were replaced by it (what the new text confirms word for word moved to it, everything else the older capture said was kept but made inactive, its passages are no longer searched), 1 was kept in use because the newer capture lost the content of its own page (Archives), 1 because the newer capture is a failed render (Past Principals). The full account is var/state/NEWER_CAPTURE_REPORT_20261010.md; the snapshots around the run are var/state/NEWER_CAPTURE_before_*.json and NEWER_CAPTURE_after_*.json. — made by python -m aitchison.process.selection (rule:newer_capture@1), applied 10 October 2026 09:08 UTC
#11 retraction of extraction:local_gemma4e2b_sweep1 (run local_gemma4e2b_sweep1), 679 records: Low measured precision; retain proposals for further evaluation
Paid inference is off
No answer, vector, transcript or description is made by a hosted model. The record of every attempt to reach one is kept: 0 were made and 0 were refused. Collection health has the rest.
— made by local_extract retract local_gemma4e2b_sweep1
#10 retraction of extraction:local_gemma4e2b_sweeptest (run local_gemma4e2b_sweeptest), 5 records: rehearsal of the supplement — made by local_extract retract local_gemma4e2b_sweeptest
#9 correction of notices:retired_by_a_rehearsal (run local_gemma4e2b_sweeptest): A rehearsal of the supplement on revisions 85, 109, 117, 125, 131 and 137 was undone with `local_extract retract`, which retired every active notice of those revisions, including the four that the manual readers had made (notices 82, 163, 165 and 196); the retraction retired what a label had not made. The four were set active again exactly as they were; `retract` now retires only the notices and the document dates that the label's own applied bundles wrote (tests/test_local_extract.py::test_retracting_a_supplement_leaves_the_notice_and_the_date_that_another_reading_gave_the_document). — made by the author of the supplement, after its rehearsal on 10 October 2026 (04:04 UTC)
#8 retraction of extraction:local_gemma4e2b_sweeptest (run local_gemma4e2b_sweeptest), 5 records: rehearsal of the supplement — made by local_extract retract local_gemma4e2b_sweeptest
#7 correction of facts (run captures@1), 249 records: claims of an older capture of a page that the current capture of the same page says again (the same sentence, the same or other figures) were retired, so that an old and a new value are never both eligible as current; kept, inactive, undone by captures.undo — made by extraction round 3 (corrections)
#6 correction of extraction evaluation: a duplicate decides before a false positive (run lx-p2 and lx-p3 readings of the 55 documents (tests/fixtures/extraction_reading_lxp2, extraction_reading_lxp3), measured 2026-10-09 21:50 UTC), 3 records: A false positive can be a fragment ("Saturday, 14th February") that any longer sentence contains, so a duplicate judged on the whole sentence was counted as a false positive. A record that matches a duplicate and a false positive is now counted as a duplicate. (The same minute two judgements were added: another form of the trek event of revision 46961, and one duplicate of revision 7180.) — made by session 7193cac8 (Claude Code, local extraction measurement, 2026-10-09): aitchison/ingest/extract_eval.py; written down afterwards
#5 correction of extraction evaluation: matching of facts whose value is a short number (run lx-p2 reading of the 55 documents (tests/fixtures/extraction_reading_lxp2), measured 2026-10-09 21:05-21:10 UTC), 6 records: A fact whose value is a bare number of one or two digits ("65", "96") was compared by words longer than two letters, so it matched no gold record, not even an identical one, and showed as unjudged or as a false positive. Found while comparing the readings of the 55 documents; the match now accepts the very same value when the two facts are about the same thing (the same predicate or subject). — made by session 7193cac8 (Claude Code, local extraction measurement, 2026-10-09): aitchison/ingest/extract_eval.py; written down afterwards
#4 correction of gpu_runs (run pl.photo_vision batches between 19:52 and 19:56 UTC on 2026-10-09): migration 037 (gpu_runs.queued_s, slot_wait_s) was applied about four minutes after the workers began to write the new columns, so the batches of that interval have no gpu_runs row (the insert failed and the ledger carries on without a row); their photographs are recorded in processing_outputs (stage photo_vision) and are counted from there. The device seconds per photograph are measured over the batches that have a row — made by session 7193cac8 (Claude Code, GPU migration, 2026-10-09): aitchison.migrate 037 applied after the worker restart; written down afterwards
#3 correction of gpu_runs (run batches handed back at a stop request, 2026-10-09), 1 record: a batch that was handed back at a stop request (jobs.Release) was recorded as failed; it is not a failure (what was done is committed and the next worker finishes the batch). The ledger now records status stopped with the note 'asked to stop; the batch was handed back', and the rows already written were changed from failed to stopped by SQL — made by session 7193cac8 (Claude Code, GPU migration, 2026-10-09): code fix in aitchison/gpu/ledger.py and a manual SQL update; written down afterwards
#2 deletion of external_inference_calls (run a unit test of GET /api/gpu (tests/test_api.py), before it monkeypatched inference._record), 3 records: three rows saying that hosted inference had been refused were written into the live ledger by a test that exercised the refusal guard against the production database; they did not record real attempts and were deleted, so that the count of hosted calls stayed an honest 0 made / 0 refused. The test now patches inference._record and tests/conftest.py makes the default database unreachable — made by session 7193cac8 (Claude Code, GPU migration, 2026-10-09): manual SQL delete; written down afterwards
#1 retraction of extraction:local_gemma4e2b_lxp1 (run local_gemma4e2b_lxp1, first live run (gpu_runs 140-149, 17:16-17:29 UTC)), 56 records: the first live run of the local extractor read page titles, calendar grids and lists of letters as events and people (junk); its output was undone with local_extract retract at 17:29:53 UTC and the rules were tightened; the label was then run again (gpu_runs 152 onwards, 17:41-17:49 UTC) — made by session 7193cac8 (Claude Code, GPU migration, 2026-10-09): local_extract retract local_gemma4e2b_lxp1; written down afterwards