Add Python repl_prepare runtime preparation#135
Draft
t-kalinowski wants to merge 20 commits into
Draft
Conversation
repl_prepare for Python session initializationExplain that Python repl is self-starting and that repl_prepare should not be called preemptively.
…tool-repl_prepare
Address Codex review findings for Python repl_prepare. Finding 1: <<<FINDING 1 START>>> [P1] Pass sandbox metadata into prepare restarts — /Users/tomasz/.codex/worktrees/92b2/mcp-repl/src/server.rs:948-951: When the server runs with `--sandbox inherit`, `repl_prepare` receives call metadata but drops it before replacing the Python worker. Prepares that need to spawn or replace the worker can fail with missing sandbox metadata or restart under stale policy; handle `meta` the same way as `repl`/`repl_reset` before replacing the worker. <<<FINDING 1 END>>> Finding 2: <<<FINDING 2 START>>> [P2] Check full requirement specifiers before preserving Python — /Users/tomasz/.codex/worktrees/92b2/mcp-repl/src/python_prepare.rs:360-363: The probe strips requirements to distribution names and only checks metadata exists. Requirements like `numpy>=2`, extras, or direct references can be treated as satisfied when they are not, allowing `repl_prepare` to commit an unsatisfied environment without invoking uv; validate full requirements with packaging/uv or only short-circuit bare names. <<<FINDING 2 END>>> Finding 3: <<<FINDING 3 START>>> [P2] Clear stale timeout bundles after prepare replacement — /Users/tomasz/.codex/worktrees/92b2/mcp-repl/src/server.rs:552-555: When `repl_prepare` replaces the worker while a request is timed out, worker state is cleared but `ResponseState` active/staged timeout output remains. The next normal `repl` call can be finalized as a timeout follow-up and reuse or append to the discarded request's bundle; retire or clear the timeout bundle when prepare discards pending work. <<<FINDING 3 END>>> Finding 4: <<<FINDING 4 START>>> [P2] Reset user-state tracking in files mode — /Users/tomasz/.codex/worktrees/92b2/mcp-repl/src/worker_process/output_state.rs:237: `user_state_may_exist` is cleared only on the pager reset path. Files-mode resets/replacements leave the flag set after the old Python process is discarded, so later `repl_prepare` with `restart: "no"` can be rejected as if it would discard user state even when the fresh session has not run user code; apply the same reset in files mode. <<<FINDING 4 END>>>
[P1] Stage sandbox metadata before probing executables — /Users/tomasz/.codex/worktrees/92b2/mcp-repl/src/server.rs:597-599. When the server runs with `--sandbox inherit`, it resolves the requested Python before `stage_prepare_sandbox_state` is called, so a `repl_prepare` call with missing or invalid sandbox metadata still executes the supplied executable on the host via `query_python_runtime_config` before being rejected. This bypasses the inherited-metadata gate used by the other tool paths; validate/stage the sandbox state before running the probe. [P2] Avoid treating supersets as satisfying removals — /Users/tomasz/.codex/worktrees/92b2/mcp-repl/src/python_prepare.rs:220. When the current/default interpreter already has an extra package that was just removed or omitted by `action: "set"`, this shortcut returns the current interpreter because it satisfies the remaining packages. `active_matches` then leaves the session unchanged while the result reports the package removed, even though it remains importable; narrowed manifests need to force a managed uv target instead of accepting any superset.
[P1] Validate sandbox metadata before invoking uv — /Users/tomasz/.codex/worktrees/92b2/mcp-repl/src/server.rs:519-522 Under `--sandbox inherit`, requirements prepares should fail closed before side effects when `_meta["codex/sandbox-state-meta"]` is missing or malformed. The patch resolves the manifest first, which can spawn `uv tool run --with ...` for user-supplied requirements before `sandbox_state_update()` is called, so a rejected tool call can still perform package resolution/downloads outside the accepted sandbox state. Stage or validate the sandbox update before calling the requirements resolver, as the explicit Python path already does. [P2] Check the active Python before falling back to uv — /Users/tomasz/.codex/worktrees/92b2/mcp-repl/src/python_prepare.rs:233-234 After `repl_prepare` has selected an explicit venv or executable, the active session can satisfy a later `requirements` add/no-op even when the original server Python does not. This helper probes `resolve_python_runtime_config()` from the server environment, not the worker launch, so `restart: "no"` can fail or `if_needed` can restart and discard state even though the current venv already has the manifest packages. Use the active worker executable for the shortcut, or skip the shortcut when it is unavailable.
[P1] Run explicit Python probes inside the sandbox — /Users/tomasz/.codex/worktrees/92b2/mcp-repl/src/python_prepare.rs:136-138 When `repl_prepare` receives `python.executable`, this path runs the caller-supplied absolute executable in the server process via `query_python_runtime_config` before the sandboxed worker is spawned. Under `read-only`, `workspace-write`, or `inherit` sandbox modes, a caller can point this at any executable that accepts or ignores the probe args and execute it with server permissions, bypassing the worker sandbox. Probe the interpreter under the effective worker sandbox or otherwise avoid executing arbitrary paths in the parent process. [P1] Run uv resolution under the sandbox policy — /Users/tomasz/.codex/worktrees/92b2/mcp-repl/src/python_prepare.rs:340-341 For `requirements` requests, this launches `uv` directly from the server process, so a package request can access the network, write uv caches, and potentially run package build code outside the configured worker sandbox and managed-network policy. This affects restricted `read-only`/`workspace-write`/`inherit` sessions where `repl` code would otherwise be contained. Resolve/install requirements through a sandboxed helper or explicitly enforce the same filesystem and network policy here.
Finding addressed: [P2] Track auto-selected Python executable for prepare matching — /Users/tomasz/.codex/worktrees/92b2/mcp-repl/src/worker_process.rs:306-309 When the Python server is started normally without `MCP_REPL_PYTHON_EXECUTABLE`, the worker still has an active auto-selected `sys.executable`, but this returns `None` for `WorkerLaunch::Builtin`. As a result, `repl_prepare` cannot detect that `python.executable` points at the current interpreter, so it restarts and discards user state even though the session already matches; the same missing hint also prevents requirements shortcuts for an already-satisfying auto-selected runtime. Response: Built-in Python workers now report their resolved runtime executable in worker_ready metadata, and WorkerManager stores that live hint for repl_prepare executable matching. A public repl_prepare regression covers the auto-selected Python launch path.
[P2] Preserve Python prefixes for prepared workers — /Users/tomasz/.codex/worktrees/92b2/mcp-repl/src/python_ffi.rs:433-433 When `repl_prepare` restarts into a prepared Python, `module_search_paths` is non-empty and this calls `Py_SetPath`, which bypasses CPython's normal path/prefix calculation; after a default prepare, `sys.prefix` and `sys.exec_prefix` are empty. Code and packages that use `sys.prefix`/`sysconfig` to locate the active environment will see a broken Python environment, so the prepared worker should initialize with config/home/prefix information instead of only overriding `sys.path`.
Finding 1: [P2] Persist prepared Python target on no-op prepares — /Users/tomasz/.codex/worktrees/92b2/mcp-repl/src/server.rs:598-604 When the resolved target already matches the live process, this commits the manifest but leaves `worker_launch` as the previous builtin auto-discovery. If a later Ctrl-D reset or sandbox-cwd change respawns the worker, it can discover a different Python and lose the prepared manifest even though `repl_prepare` reported success; the explicit-python `active_matches` branch has the same issue. Record the resolved `PythonExecutable` target for future spawns even when reusing the current session. Finding 2: [P2] Avoid concatenating bare Python version requests — /Users/tomasz/.codex/worktrees/92b2/mcp-repl/src/python_prepare.rs:162-164 When a user prepares one Python version and later asks for another with the default `action: "add"`, this builds a value like `3.11,3.12` and passes it to `uv --python`. `uv` treats that as a single executable name/invalid request, so the second prepare fails instead of switching versions while keeping the package manifest; store or require replacing the version request rather than appending bare versions.
# Conflicts: # .github/workflows/ci.yml
# Conflicts: # .github/workflows/ci.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Python-only
repl_preparetool for cases where a caller wants to change, select, or preflight the Python environment before laterreplcalls. The Python REPL itself still starts ready for evaluation;repl_prepareis for environment preparation, not for makingreplusable.Tool schema
repl_prepareis advertised only by Python servers whenuvis available. R servers continue to advertiserepl_reset; Python servers no longer advertiserepl_reset.Contract details:
repl_preparejust to makereplavailable; the REPL starts ready.requirementswhen the caller needs managed packages or a managed Python version before evaluation.pythonwhen the caller needs an existing interpreter or virtual environment.packages=["numpy"]andpython_version=null.requirementsorpython, not both.python, provide exactly one ofexecutableorvenv.requirements.actionupdates the persistent managed manifest withadd,remove, orset.requirements.restartcontrols whether preparation may restart the active Python session.requirements.python_versionis forwarded touvas supplied.pythonselections use the requested interpreter or venv as-is and do not mutate the managed manifest.timeout_msargument.Scenario examples
The snippets below use pseudo tool-call syntax. Each
repl(...)call is a later Python REPL evaluation afterrepl_prepare(...)returns.Add a package before importing it:
Build up a persistent managed manifest:
Replace the whole managed manifest and request a Python version constraint:
Preserve existing session state if preparation would require a restart:
Force a fresh session for the current managed manifest:
Switch to an existing project virtual environment:
Switch to an explicit Python executable:
Remove a package from the managed manifest before returning to that managed environment:
Reset Python without environment preparation:
R keeps the existing reset tool:
repl_reset() # new session startedInternal changes
uv.Diff composition
Measured against the merge base with
main, this PR is2079insertions and107deletions across22files.src/:+1058/-95(52.7%of churn)src/:+3/-0(0.1%of churn)tests/:+924/-8(42.6%of churn)+94/-4(4.5%of churn)Largest files:
tests/python_repl_prepare.rs:+904/-0src/server.rs:+387/-49src/python_prepare.rs:+399/-0src/worker_supervisor.rs:+88/-25src/python_runtime.rs:+71/-17