How much does tracing slow an LLM app? Arize Phoenix vs MLflow, run locally
Short answer: On 14 Nov 2026 we timed 200 calls to a local model per setup: no tracing, Arize Phoenix, and MLflow, both servers on the same machine. Whatever tracing added was smaller than the run-to-run noise: MLflow added 10.1 ms at the median and Phoenix came out 13.0 ms faster than its own untraced baseline, while two identical untraced runs differed by 21.2 ms. A burst of 2,000 traced no-op calls showed the real difference: Phoenix's default setup took 3.3 ms per span and kept every trace, and MLflow took 0.16 ms but stored only 845 of 2,050.
Limits: read these before quoting a number
- A shared, noisy machine. Other jobs were running on the same 8-vCPU box. Two untraced runs of the same code had medians of 104.5 ms and 125.7 ms. Differences of 10 to 30 ms per call are inside that noise.
- We ran it twice, and the first run is not the primary result. During run A we rendered a video on the same machine for about 10 seconds. We wrote that down, added a note to the pre-registration and repeated the identical script (run B). Run A's numbers are below, unchanged.
- Default settings only. Phoenix used
phoenix.otel.register()with its default (synchronous, one export per span); itsbatch=Trueoption was not tested. MLflow usedmlflow.openai.autolog()with its default async queue. Both behave differently once tuned. - Local servers, one small model. Hosted Arize, Phoenix Cloud and managed MLflow were not tested. Calls to a local 3B model are short (about 110 ms); with slower hosted models a few ms matter even less.
- Not a feature or UI comparison. See the tool pages for that.
Method (pre-registered)
The method was committed before the first measured call: PREREGISTRATION.md (with the run-B addendum).
| Workload | OpenAI Python client 3.51.0 calling qwen2.5:3b on Ollama 0.40.16 (local), prompt "Reply with the single word OK.", temperature 0, max 5 tokens. |
|---|---|
| Phoenix | arize-phoenix 21.0.0, phoenix serve with SQLite; app instrumented with openinference-instrumentation-openai 0.1.73. |
| MLflow | mlflow 3.18.0, mlflow server with SQLite; app uses mlflow.openai.autolog(). |
| Order | Each setup in its own process. Pass 1: baseline, Phoenix, baseline, MLflow. Pass 2 in reverse. 10 warm-up calls, then 100 timed calls per setup per pass: 200 timed calls each. Each tool is compared with an untraced baseline run in its own Python environment. |
| Burst test | 2,000 calls to a traced function that does nothing (plus 50 warm-up), per tool, then a count of traces actually stored in each server's database. |
Results
| Setup | Median per call | p95 per call | Added vs its baseline (95% CI) | Traces stored | Readable after last call |
|---|---|---|---|---|---|
| No tracing (Phoenix env) | 125.7 ms | 214.4 ms | baseline | n/a | n/a |
| Arize Phoenix | 112.6 ms | 225.1 ms | -13 ms (-31.8 to 1.1) | 220 of 220 | 0.35 s, 0.38 s |
| No tracing (MLflow env) | 104.5 ms | 172.5 ms | baseline | n/a | n/a |
| MLflow | 114.6 ms | 234.2 ms | +10.1 ms (4.5 to 14.2) | 220 of 220 | 0.31 s, 0.35 s |
Run A, for comparison: Phoenix +9 ms (5.2 to 12.2), MLflow +34.6 ms (22.1 to 48.7). The two runs disagree by more than their own intervals, which is why we read both as "smaller than the machine's noise" and not as a ranking. Both tools stored every trace from the real calls in both runs.
| Tool | Median per traced call | p95 | Traces stored | Server memory after runs | Database on disk |
|---|---|---|---|---|---|
| Arize Phoenix (default register) | 3.31 ms | 4.18 ms | 2,050 of 2,050 | 468 MB | 31 MB |
| MLflow (default async) | 0.16 ms | 0.39 ms | 845 of 2,050 | 755 MB | 7 MB |
Phoenix's default sends each span to the server before your code continues, so a burst costs about 3 ms per span and nothing is lost. MLflow queues traces in the background, so your code barely waits, but when the queue filled it logged "Trace export queue is full, trace will be discarded" and dropped 1205 traces. Run A logged the same warning; its stores were wiped before we could count them. If you trace many small steps per request, set Phoenix to batch mode or raise MLflow's queue size (MLFLOW_ASYNC_TRACE_LOGGING_MAX_QUEUE_SIZE), then measure again.
The run itself
Terminal recording of run B (asciinema), sped up, with pauses longer than 2 seconds shortened.
Data, scripts and licence
Download every timed call in run B (CSV), the run B summary, the run A summary and run A calls, bench.py, run_all.sh and the terminal recording. Licence: CC BY 4.0, attribution Evals Review editorial team, 14 Nov 2026, https://evalsreview.com/tests/tracing-overhead-phoenix-vs-mlflow.