U8 — Context, processing, native & Rust

120 Python modules / 40,103 LOC + 13.4k LOC Rust workspace / 726 churn commits (12m). Reviewer: parallel unit review, 2026-09-03.

Overview

The native/Python split is genuinely roofline-aware for CPU-bound work (tokenize, similarity, pattern-match, chunking) and the fallback discipline is real — but semantic parity has drifted in exactly the two functions that guard the LLM context budget (token counting, context fitting), and a 4^th "protocol" accelerator stack plus a dead victor/context/manager.py multiply maintenance cost. GIL handling and FFI data movement are good on similarity (batch, py.detach), absent everywhere else.

Findings

ID	Sev	Location	Finding	Why it matters	Fix sketch	Effort
F1	P1	victor/processing/native/context_fitter.py:144-152 vs rust/crates/python-bindings/src/context_fitter.rs:124,152-155	Strategy vocabularies diverged: Rust accepts fifo/priority/smart (default smart); Python fallback accepts recency/priority/balanced (default recency). Production passes strategy="smart" (victor/agent/conversation/assembler.py:388) which the fallback silently maps to balanced; "recency" passed to Rust silently maps to fit_smart	Which conversation history survives truncation depends on whether the wheel is installed — the exact fallback-drift correctness risk the architecture mandates against. Zero parity tests (only tests/benchmarks/test_hot_path_performance.py:154 exercises native)	One shared Strategy enum (contracts or processing/native/_base); reject unknown names loudly; native-vs-fallback equality matrix over all strategies	S
F2	P1	victor/native/rust/tokenizer.py:70; victor/processing/native/tokenizer.py:57-66,85; rust/.../tokenizer.rs:283-341	Three token-count semantics in production: "exact" native = Rust punct/CJK-weighted heuristic (~90%); fallback exact = tiktoken cl100k; fast fallback = word×1.3. A full BPE BpeTokenizer pyclass exists (tokenizer.rs:195) but is constructed only in a benchmark	Token budgets shift ~±20% (worse on code/punctuation-dense text) between native and fallback installs; fit_context budgets, providers/base.py:964 and store.py estimates inherit the skew	Ship cl100k ranks once (lazy) into BpeTokenizer; make exact paths (native and fallback) both BPE; keep heuristic only for count_tokens_fast	M
F3	P2	rust/Cargo.toml:35 (panic = "abort") vs context_fitter.py:93,198,265	Release profile builds the cdylib with panic="abort"; the pervasive except Exception: pass → Python fallback pattern is not a panic boundary — any Rust panic aborts the whole Python process	Silent-fallback wrappers create a false safety story for a long-lived agent/serve process; one slicing/indexing bug in 10k LOC of Rust becomes a hard crash	Override to panic="unwind" for the extension (or catch_unwind in bindings); or document panics as fatal	S
F4	P2	context_fitter.py:74,93; victor/native/rust/context_fitter.py:97; context_fitter.rs:53	msg.get("priority", 1.0) feeds a float to MessageSlot(priority: u8); PyO3 rejects floats → TypeError for every message lacking an int priority; the flat shim swallows it (line 93) and silently downgrades to the Python path	The Rust fitter is effectively dead for default-shaped inputs, hidden by the silent except; EWMA dispatch never sees honest rust samples	Default to 50 (int) at both call sites; log at debug on native-path exceptions	S
F5	P2	rust/.../tokenizer.rs:274; context_fitter.rs:125; chunking.rs	GIL released only on the similarity path (py.detach, similarity.rs:191). All other batch entry points — count_tokens_fast_batch (rayon), fit_context, chunk batches — hold the GIL for the whole call	Rayon parallelism buys nothing for multi-threaded victor serve: every other request thread stalls during batch counting/fitting	Add py: Python<'_> + py.detach around the pure-Rust impl bodies (the _impl/wrapper split already exists in similarity.rs)	M
F6	P2	victor/native/python/* + victor/native/rust/* (2,539 LOC, 14 files); context_fitter.py:100-174; context/manager.py:242-358	Triple accelerator stacks: flat shims (production path), protocol classes (only real consumer: victor-coding chunker's get_text_chunker), plus manager.py pruning. Three near-copies of the fitting algorithm with three strategy vocabularies	Every semantics fix must land 3-4 places — F1/F2 are the direct product; churn concentrates on the drifted, duplicated surfaces	Declare flat shims canonical; reduce protocol classes to thin delegates (or delete unused: token counter, context fitter, arg normalizer have zero production consumers); delete manager.py	M
F7	P2	victor/agent/response_sanitizer.py:415-417 vs rust/.../streaming_filter.rs:124,236	_NativeFilterWrapper.flush() returns "" with comment "native filter has no buffering", but the Rust StreamingFilter does buffer partial <think… tags — never emitted at stream end. Python fallback's flush emits its buffer (response_sanitizer.py:353)	Tail-of-stream content dropped only on the native path — backend-dependent content loss on the streaming hot path	Add a Rust flush() returning the residual buffer; wrapper delegates	S
F8	P3	victor/context/manager.py:273-283,244,255	_prune_fifo iterates reversed(other_messages) dropping the newest messages — FIFO inverted (drops newest, keeps oldest). Module is production-dead (one unit test) yet churn=13; uses print() with emoji	401 LOC of maintained dead code containing a logic bug invites cargo-cult reuse into the real path	Delete the module — assembler.py is the real pruning path	S
F9	P3	context_fitter.rs:145-147	kept_indices.iter().filter_map(|&idx| messages.iter().find(...)) — O(n²) in kept messages when computing totals	Negates part of the native win on long histories (200 msgs → 40k scans)	Index map or fold totals during selection	S
F10	P3	victor/classification/pattern_registry.py:181-184,1118,1134	compiled_pattern property docstring says "(cached)" but calls re.compile fresh on every access, inside match loops over 76 patterns	Per-query overhead; relies on re's internal 512-entry cache for the claim to hold	Precompile in __post_init__ (or functools.cached_property)	S
F11	P3	victor/processing/native/streaming.py:204	Infrastructure-layer fallback imports victor.agent.response_sanitizer (runtime layer); response_sanitizer itself imports victor.processing.native — circular-by-design cross-layer dependency	Streaming fallback impossible to extract/test independently; no guard covers this direction	Move StreamingContentFilter fallback into processing/native	S
F12	P3	victor/agent/conversation/assembler.py:383-384,395-396	Mixed budget currencies: assembler budgets in chars, converts via //4 chars-per-token to a token budget for fit_context, then re-sums results in chars	Two approximation errors compound with F2's estimator skew; no single unit of budget accounting	One currency end-to-end; convert once at the provider boundary	M

Co-design opportunities

1. One-crossing slot fitting: fit_context costs N FFI crossings to build MessageSlot pyobjects + 1 to fit. Accept parallel lists (roles, token_counts, priorities, recencies) or build slots inside Rust.
2. Zero-copy embeddings: similarity takes Vec<Vec<f32>> (full copy per call, similarity.rs:128) while the NumPy fallback wins benchmarks partly because it's zero-copy. Enable PyO3 buffer-protocol support; persistent EmbeddingIndex pyclass as the only batch path.
3. Content-hash memoized token counts: stored messages are immutable; hashing.rs already provides xxh3 signatures. Cache counts by signature in the conversation store/assembler instead of re-tokenizing the whole history every pass.
4. Release the GIL on batch entry points (tokenizer.rs:274, context_fitter.rs, chunking.rs) using the pattern proven in similarity.rs.
5. Parity harness as CI law: generated test matrix asserting native == fallback for every flat shim (strategies × token estimators × streaming flush). F1/F2/F7 are precisely the drift class it would have caught.
6. Ship the BPE ranks: one lazy BpeTokenizer("cl100k", ranks) at startup unifies exact counting across both backends.

Keep intact

• victor/processing/native/_base.py:30-65 — single _NATIVE_AVAILABLE flag, one-time install hint, _PYTHON_PREFERRED_OPERATIONS benchmark-aware dispatch (NumPy wins batch similarity); the honest single source of truth.
• pattern_match.rs:34-68 — bounded (512-entry) Arc-shared Aho-Corasick automaton cache keyed by flags+patterns; exactly the right compiled-pattern amortization.
• victor/native/observability.py adaptive EWMA dispatch (69-174, consumed by accelerator.py:460-498): per-backend EWMA with update_ewma=False for non-attributable observers — a genuinely closed measure→dispatch loop.
• API-reported usage written back via store.py:1208 update_message_token_count — self-corrects estimator drift after each real LLM call; F2 is tolerable only because this exists.
• project_context.py signature caching keyed by (path, mtime, size) with force_reload — correct incremental recompute for the I/O-bound side of context assembly.

Quick metrics

• LOC: processing 17,837 | security 9,307 | native 3,875 | context 3,698 | optimization 3,697 | classification 1,689 | rust/crates 13,371 (python-bindings 10,258).
• Rust: pyo3 0.29, rayon, aho-corasick, lru, no numpy interop; py.detach only in similarity.rs; release opt-level=3, lto=fat, panic=abort.
• 27 per-call hasattr(_native, …) checks in flat shims; 6 silent except→fallback sites in processing/native.
• Token estimators reachable in production: 4 distinct formulas (tiktoken exact; word×1.3; Rust punct/CJK heuristic; chars//4 in assembler).
• fit_context implementations: 4 (Rust, flat-shim fallback, PythonContextFitter, manager._prune_*); strategy vocabularies: 2 disjoint sets; parity unit tests: 0.
• Near-dead weight: ProjectContextLoader (401 LOC, 1 test consumer), project_manifest.py (892, fan-in 0), command_parser.py (630, fan-in 0) ≈ 1,923 LOC.