v0.7 — Free-Threaded Python and the Interpreter Executor¶

Status: AMENDED — Round 1 review findings folded in (see Review amendments). Ready to implement.

The two forward bets deferred from v0.6: prove pyarallel on free-threaded CPython, and add executor="interpreter" (PEP 734). Neither touches the engine — the windowed driver, retry, rate limiting, and checkpointing are executor-agnostic by construction, and the probes below confirm it.

Ground truth (measured 2026-07-06, macOS arm64, uv-managed CPython)¶

Every claim in this plan was verified with throwaway scripts before writing. No design decision below rests on documentation alone.

Probe	Result
Full suite (341 tests) on 3.14.6	green, full dev group installs
Full suite on 3.14.6 free-threaded	green, `sys._is_gil_enabled() == False`
Full suite on 3.13.14 free-threaded	green with pytest-only deps; dev group fails to install — `mkdocs-material[imaging]` → cairocffi → cffi fails to build on the free-threaded ABI
`parallel_map` thread executor, 4× CPU-bound spin	1.0× speedup under GIL → 2.4× on 3.14t (4 workers, 4 items)
`InterpreterPoolExecutor`, same task vs threads (GIL build)	3.4× — interpreters get true CPU parallelism on standard builds
Interpreter worker startup	~30 ms for pool-of-1 spin-up + roundtrip
Class shape	`InterpreterPoolExecutor` subclasses `ThreadPoolExecutor`: has `initializer`/`initargs`, no `max_tasks_per_child`
pyarallel's exact submit shape — `partial(_execute_outcome, module_fn, retry=Retry(...), limiter=None)`	crosses the boundary; `_Outcome` and `Retry` pickle across
Lambda / closure / `__main__`-defined task fn	`NotShareableError` — workers cannot see the parent's `__main__`; same constraint story as process
Task exceptions	arrive typed (`ValueError` stays `ValueError`, `__cause__` = `ExecutionFailed`)
`contextvars.Context.run` as submit wrapper	`NotShareableError` — no contextvars propagation, interpreter must take the process-style submit branch
Closure `worker_init`	`BrokenInterpreterPool` at first submit — must be rejected upfront like process
Raising future (unshareable args, broken pool)	already absorbed per-item by `_absorb`'s `except Exception` — no engine change
`import numpy` inside an interpreter worker (3.14.6, standard build)	`ImportError` on every item — numpy's C extension does not support subinterpreters (single-phase init); pandas fails transitively. Deterministic, recurs per item
`__main__`-defined function as target	process succeeds (spawn bootstraps the script as `__main__`), interpreter fails per-item with `AttributeError` — the sub-interpreter gets a fresh empty `__main__`. Verified with pyarallel's exact partial shape
`__main__`-defined `worker_init`	passes `pickle.dumps` (pickles by reference), then dies as `BrokenInterpreterPool` at first submit — the pickle pre-check alone cannot deliver "rejected upfront" here
`if sys.version_info >= (3, 14): from concurrent.futures import InterpreterPoolExecutor`	passes mypy strict on 3.12 — mypy narrows on the version check; a plain `if executor == "interpreter":` branch does not narrow

Part A — Free-threaded and 3.14 CI¶

The suite already passes with the GIL off on both t-builds. Part A makes that a kept promise instead of a one-off observation.

A1. Split a `test` dependency group¶

[dependency-groups] gains test = ["pytest>=8.3.4", "pytest-asyncio>=0.23"]; dev includes it via {include-group = "test"}. Forced by ground truth: the dev group cannot install on 3.13t (cffi), and free-threaded jobs only need pytest anyway. Lint/type/docs already run on every standard job — re-running them per t-build adds cost, not coverage.

A2. CI matrix¶

checks job: ["3.12", "3.13", "3.14"] — full gates (lint, mypy, pytest, mkdocs). 3.14 is currently untested in CI at all; that gap closes here, independent of free-threading.
New free-threaded job: ["3.13t", "3.14t"] (setup-uv accepts these version strings — verify in the PR run; fallback is the full cpython-3.1Xt spelling). Steps: uv sync --no-default-groups --group test, a GIL guard, uv run pytest -q.
The GIL guard is a CI step, not a test file: uv run python -c "import sys; assert not sys._is_gil_enabled()" — uv run, not bare python, so the guard hits the same interpreter pytest will use (a bare python resolving outside the synced venv is precisely the wrong-build scenario the guard exists to catch). It fails the job if a dependency silently re-enables the GIL or setup resolved a standard build. It is a start-of-job sanity check, not a continuous invariant — a dependency added later could re-enable the GIL mid-suite; acceptable while the test group is pytest-only.
No continue-on-error. Both builds are green today; if they flake, that is signal about pyarallel's timing tests under a new scheduler, not noise to suppress. (Known risk, accepted: timing-margin tests may flake more under free-threading. Respond to real flakes; don't pre-soften.)

A3. Metadata¶

Classifiers gain 3.13 and 3.14. requires-python stays >=3.12 — nothing in the suite or probes demands raising it.

A4. Docs (small, honest)¶

Best-practices, under Choosing the Right Executor: on free-threaded builds threads parallelize CPU-bound work too (measured 2.4× on 4 cores), CI runs the suite with the GIL off on 3.13t/3.14t. No performance promises beyond the measured number; free-threaded builds labeled as what they are.

Part B — `executor="interpreter"` (Python 3.14+)¶

The contract: process constraint semantics in one OS process, with cheap worker startup. Every process rule applies — importable module-level functions, picklable retry callables, no shared limiter, no contextvars — plus two rules process does not have: functions defined in __main__ are rejected (a sub-interpreter cannot see the parent's __main__; spawn can bootstrap it, interpreters can't), and max_tasks_per_worker is rejected (no worker recycling). What you buy: true CPU parallelism on standard GIL builds (measured 3.4×) with ~30 ms worker startup instead of fork/spawn, no OS-process management. What you do NOT save: per-worker import cost and module state — each interpreter re-imports the target module, re-running import-time side effects and duplicating module memory, exactly like a process worker. And the hard boundary: C extensions that don't support subinterpreters fail with ImportError inside workers — numpy and pandas among them today. The interpreter executor is for pure-Python CPU-bound work; C-extension workloads stay on executor="process".

B1. Type and validation¶

ExecutorType = Literal["thread", "process", "interpreter"] (core.py:58). The literal is unconditional — a version-gated literal would poison every signature with conditionals for zero runtime safety. (Verified: mypy strict accepts the widened literal on 3.12 with no fallout in decorators or TypedDicts.)
Value validity and runtime capability are separate gates. _validate_common accepts "interpreter" as a valid value on every Python — its "must be thread or process" message updates. The 3.14+ capability gate lives with the pool validation (_validate_worker_options, already called only on the non-sequential path): on sys.version_info < (3, 14), raise ValueError('executor="interpreter" requires Python 3.14+ ...'). Rationale — the existing debug-mode contract: sequential=True ignores the executor entirely (pinned by existing tests, stated in the code comment at the call site). A team running executor="interpreter" in 3.14 production must be able to debug the same call with sequential=True on a 3.12 laptop. Gating in _validate_common would break that; gating on the pool path keeps it.
__main__-defined targets are rejected upfront for interpreter. When _resolve_process_target reports module == "__main__" (and for a worker_init with __module__ == "__main__"), raise ValueError telling the user to move the function into an importable module. Probe-verified: process works for these (spawn bootstraps the script), interpreter fails — as a per-item AttributeError storm for the task fn, as BrokenInterpreterPool for worker_init, and the pickle pre-check alone catches neither (__main__ functions pickle fine by reference). An upfront error beats N identical failures.

B2. Touchpoints (all in `core.py`, mirroring the process branch)¶

Site	Change
`_make_pool`	`InterpreterPoolExecutor(max_workers=workers, initializer=worker_init)` — imported under `if sys.version_info >= (3, 14):`, which is what makes mypy strict pass on 3.12/3.13 (mypy narrows on the version check; an `executor == "interpreter"` branch does not narrow — probe-verified)
`_submit_task`	no change — interpreter already lands in the non-thread `else` (no `copy_context()`; probe: `Context` is not shareable)
`_effective_workers`	caution: interpreter groups with THREAD here, `min(32, cpu_count + 4)` — that is what the inherited `ThreadPoolExecutor` actually does, and window sizing must match pool reality. The "joins the process branch" mental model applied blindly at this site would under-window streams
`_build_task_fn`	interpreter joins the process branch: retry pickle pre-check (message names the actual executor), `_resolve_process_target` re-import shipping with the `__main__` rejection from B1, `limiter=None`
`_validate_worker_options`	`max_tasks_per_worker` + interpreter → ValueError, and the existing message "thread workers have no per-worker task budget" is reworded — as-is it would misdescribe an interpreter caller. `worker_init`: pickle pre-check as process (message names the actual executor) plus the `__main__` rejection; the 3.14+ capability gate from B1 also lives here
`parallel_starmap`	the `executor == "process"` resolution special-case becomes `in ("process", "interpreter")`
`_procexec.py`	docstring updates from "process workers" to "out-of-process-context workers (process and interpreter executors)"; the (module, qualname) re-import code is reused as-is. No rename: the module is 55 lines and findable

Not touched: the driver loop, aio.py (async has no executor parameter), checkpointing (its picklability requirement already matches), rate limiting at submission time (paces in the parent, executor-blind).

B3. Failure modes, stated¶

Unshareable task fn (lambda/closure): surfaces as that item's failure entry via _absorb's infrastructure-failure catch — run continues, other items unaffected. Probe-verified; a parity test pins it.
Task exceptions: typed, as with every executor. Test pins that a ValueError raised in an interpreter worker is isinstance-checkable in result.failures().
Retries inside interpreter workers pace by backoff only (no shared limiter) — same documented behavior as process.
Subinterpreter-incompatible C extension imported inside the task fn (numpy et al.): undetectable upfront — the import happens in the worker. Every item fails with the same ImportError, collected in result.failures(); with max_errors set, the run aborts cheaply. Docs carry the constraint (B5); no code can.

B4. Tests — `tests/test_interpreter.py`¶

Version-gated per-test (skipif < 3.14), except the gate test itself which runs only below 3.14. Each test names the production bug it prevents:

Test	Bug prevented
basic map parity (order, values, failures collected)	interpreter path silently diverging from the engine contract
typed exception across boundary	user's `except ValueError` handling breaks because errors arrive wrapped
lambda task fn → per-item failure, run continues	one unshareable item poisons the whole run
retry with module-level fn retries N times	retry state fails to cross, attempts silently become 1
decorated function via `.map(executor="interpreter")`	decorated targets unresolvable in workers (the `_procexec` case)
starmap	starmap resolution special-case missed for interpreter
`parallel_iter` streaming	streaming path builds pool differently and misses the gate
`max_tasks_per_worker` rejected	option silently ignored, user believes recycling happens
closure `worker_init` rejected at call time	`BrokenInterpreterPool` + stderr dump at first submit instead of an upfront ValueError
module-level `worker_init` runs per worker	initializer support regresses
module-global isolation: a global mutated in a worker is not visible to the parent (contrast test asserts thread executor does see it)	semantics documented wrong; also the honest replacement for a flaky timing-based "is it really parallel" assert
contextvars do not propagate	docs claim propagation that doesn't exist
`executor="interpreter"` on < 3.14 → clear ValueError	cryptic ImportError deep in `_make_pool` on older CPython (this is the test the 3.12/3.13 CI jobs run)
`sequential=True` + `executor="interpreter"` works on < 3.14, for both `parallel_map` and `parallel_iter`	version gate placed too early breaks the debug-mode contract — production config must stay one flag away from inline on any supported Python
`__main__`-defined task fn → upfront ValueError (not N per-item AttributeErrors)	script users get a failure storm instead of one actionable error
checkpoint + `timeout` under interpreter (cached admission respects the deadline)	the v0.6 cached-checkpoint deadline fix silently not holding for the new pool type
source-iterator error mid-run with slow in-flight interpreter tasks propagates promptly	poison-source hang regression (v0.5 contract) reappearing on the new executor
streaming: breaking out of `parallel_iter` stops admission, run ends cleanly	generator-close/cancel path never exercised against `InterpreterPoolExecutor` semantics

The last three are stop-path regressions, not happy-path parity: the engine's driver claims executor-blindness, and these are the paths where that claim has historically been earned rather than assumed.

No timing/speedup assertions in CI — the 3.4× number lives in this plan and the docs as a measured-once figure, not a regression gate.

B5. Docs¶

Best-practices Choosing the Right Executor: third entry — interpreter, when (pure-Python CPU-bound, 3.14+, want cheap workers / one process), constraints (process rules, no worker recycling, no __main__ functions), and the C-extension boundary named explicitly: numpy/pandas-style extensions fail with ImportError in subinterpreters — use executor="process" for those. This caveat is non-negotiable: the library's own executor pitch lists data crunching and scientific computation, which is exactly the workload that hits it.
API reference core.md: executor parameter gains the third value + version requirement + the C-extension caveat in one sentence.
README What You Get: one bullet, wording pinned here so no out-of-context number ships: "Interpreter executor (Python 3.14+) — executor=\"interpreter\": true CPU parallelism for pure-Python work without process overhead (PEP 734)". No benchmark figure in the README; the measured, scoped numbers live in best-practices.
Quickstart CPU-bound section: one sentence pointing at it.
CHANGELOG under a new Unreleased heading, including one sentence on the ExecutorType literal widening for downstream code that matches exhaustively on it.

Phases¶

Part A — pyproject test group, CI matrix, classifiers, docs note. Ships standalone value (3.14 was never in CI).
Part B code + tests — the executor, gated by the full local matrix run (3.12 gate error test, 3.14 full suite + new tests).
Part B docs + CHANGELOG.

Gates per phase: pytest -q, ruff check + format --check, mypy strict, mkdocs build --strict, plus for phase 2 the suite on 3.14.6 and both t-builds locally.

Non-goals¶

Async interpreter executor — async has no executor parameter today; inventing one is a separate design question.
max_tasks_per_worker emulation for interpreters — rejected loudly instead.
Cross-interpreter Limiter sharing via PEP 734 queues — revisit on a concrete user ask; submission-time pacing already covers the main rate-limit story.
Raising requires-python, or claiming free-threaded support beyond what CI proves per-commit.
Interpreter backport below 3.14 (InterpreterPoolExecutor does not exist there; _interpreters is private API in 3.13).

Review amendments¶

Round 1 — 2026-07-06. Four reviewers: Codex review, Codex adversarial, Opus, Sonnet. Verdicts: clean / needs-attention / AMEND / AMEND.¶

Every finding below was independently re-verified before amending.

Version gate moved out of _validate_common (Codex adversarial, high). The draft gated executor="interpreter" on 3.14 inside _validate_common, which runs before the sequential branch — breaking the pinned debug contract that sequential=True ignores the executor. Verified in core.py: _validate_worker_options is already inside if not sequential: with a comment stating exactly this contract. Gate lives there now; B1 rewritten as value-validity vs runtime-capability; two sequential tests added to B4.
sys.version_info guard for the import, not just "inside the branch" (Opus, P1). mypy strict on 3.12/3.13 rejects the InterpreterPoolExecutor import regardless of runtime reachability; only a version check narrows. Reproduced both the failure and the fix with mypy strict on 3.12. _make_pool row updated.
__main__-defined targets: process parity is false (Opus, P2). Process spawn bootstraps the script's __main__; a sub-interpreter cannot — per-item AttributeError storm for task fns, BrokenInterpreterPool for worker_init, and pickle.dumps catches neither. Upfront ValueError added to B1/B2; test added; contract sentence rewritten to name the divergence.
C-extension incompatibility documented as a named constraint (Sonnet P1; independently probe-verified here: numpy → deterministic per-item ImportError on 3.14.6, pandas transitively). Ground-truth row, B3 failure mode, and B5 docs requirements added. "Pure-Python CPU-bound" is now the stated scope of the feature.
"Process semantics at thread prices" rescoped (Sonnet, P2). N interpreters re-import the module N times — import-time side effects and module memory are process-like, only OS-process overhead is saved. Contract paragraph rewritten.
Stop-path tests added (Codex adversarial, medium): checkpoint + timeout deadline binding, poison-source prompt propagation with slow in-flight work, streaming break/close — the paths where executor-blindness is earned, not assumed.
Hardcoded validation messages (Sonnet P2 + Opus P3): the two _validate_worker_options strings ("thread workers have no per-worker task budget", "for executor='process'") misdescribe interpreter callers; rewording folded into the B2 table.
GIL guard runs via uv run python (Opus, P3) and is documented as a start-of-job sanity check, not a continuous invariant (Sonnet, P3).
README wording pinned; no benchmark number in the README (Sonnet, P2). CHANGELOG sentence on the ExecutorType literal widening (Sonnet, P3).

Confirmed by reviewers, no change needed: touchpoint inventory complete (no executor branches outside core.py); widened Literal has no mypy fallout; the free-threaded CI split is a real coverage story, not theatre; _submit_task needs no change at all.

Implementation correction (found at Phase 1, not by review): a reviewer claimed uv sync --only-group test installs the project — it does not (python -c from the repo root masked it by importing ./pyarallel off the CWD; pytest collection then failed with ModuleNotFoundError). The CI step is uv sync --no-default-groups --group test, verified by running the full suite against the synced env on 3.13t.

Implementation discovery (Phase 2): interpreter workers do not inherit runtime sys.path additions — multiprocessing spawn explicitly ships the parent's sys.path to children; InterpreterPoolExecutor does not. First seen as every parity test failing with ModuleNotFoundError under pytest (which adds tests/ to sys.path at runtime), but it is a real user-facing gap for any target importable only via a runtime path tweak. Fix, mirroring spawn exactly: the pool initializer sets the worker's sys.path to a snapshot of the parent's, then resolves the user's worker_init by (module, qualname) — resolution must happen after the path is set, because unpickling the init by reference would need the import to succeed first. Consequence: interpreter worker_init must be a named module-level function (validated upfront); the pickle check alone was never sufficient anyway (amendment 3).