How much does tracing slow an LLM app? Arize Phoenix vs MLflow, run locally

Short answer: On 14 Nov 2026 we timed 200 calls to a local model per setup: no tracing, Arize Phoenix, and MLflow, both servers on the same machine. Whatever tracing added was smaller than the run-to-run noise: MLflow added 10.1 ms at the median and Phoenix came out 13.0 ms faster than its own untraced baseline, while two identical untraced runs differed by 21.2 ms. A burst of 2,000 traced no-op calls showed the real difference: Phoenix's default setup took 3.3 ms per span and kept every trace, and MLflow took 0.16 ms but stored only 845 of 2,050.

By Evals Review editorial team · Run 14 Nov 2026, 3:45 PM to 3:48 PM IST (run B) · How we review. No affiliate links on this page. $0: open-source tools run locally, no accounts.

Limits: read these before quoting a number

  • A shared, noisy machine. Other jobs were running on the same 8-vCPU box. Two untraced runs of the same code had medians of 104.5 ms and 125.7 ms. Differences of 10 to 30 ms per call are inside that noise.
  • We ran it twice, and the first run is not the primary result. During run A we rendered a video on the same machine for about 10 seconds. We wrote that down, added a note to the pre-registration and repeated the identical script (run B). Run A's numbers are below, unchanged.
  • Default settings only. Phoenix used phoenix.otel.register() with its default (synchronous, one export per span); its batch=True option was not tested. MLflow used mlflow.openai.autolog() with its default async queue. Both behave differently once tuned.
  • Local servers, one small model. Hosted Arize, Phoenix Cloud and managed MLflow were not tested. Calls to a local 3B model are short (about 110 ms); with slower hosted models a few ms matter even less.
  • Not a feature or UI comparison. See the tool pages for that.

Method (pre-registered)

The method was committed before the first measured call: PREREGISTRATION.md (with the run-B addendum).

One machine, one model, three setups
WorkloadOpenAI Python client 3.51.0 calling qwen2.5:3b on Ollama 0.40.16 (local), prompt "Reply with the single word OK.", temperature 0, max 5 tokens.
Phoenixarize-phoenix 21.0.0, phoenix serve with SQLite; app instrumented with openinference-instrumentation-openai 0.1.73.
MLflowmlflow 3.18.0, mlflow server with SQLite; app uses mlflow.openai.autolog().
OrderEach setup in its own process. Pass 1: baseline, Phoenix, baseline, MLflow. Pass 2 in reverse. 10 warm-up calls, then 100 timed calls per setup per pass: 200 timed calls each. Each tool is compared with an untraced baseline run in its own Python environment.
Burst test2,000 calls to a traced function that does nothing (plus 50 warm-up), per tool, then a count of traces actually stored in each server's database.
Watch the 50-second video of this test (AI-voiced narration)

Results

Real LLM calls, 200 per setup (run B, primary)
SetupMedian per callp95 per callAdded vs its baseline (95% CI)Traces storedReadable after last call
No tracing (Phoenix env)125.7 ms214.4 msbaselinen/an/a
Arize Phoenix112.6 ms225.1 ms-13 ms (-31.8 to 1.1)220 of 2200.35 s, 0.38 s
No tracing (MLflow env)104.5 ms172.5 msbaselinen/an/a
MLflow114.6 ms234.2 ms+10.1 ms (4.5 to 14.2)220 of 2200.31 s, 0.35 s

Run A, for comparison: Phoenix +9 ms (5.2 to 12.2), MLflow +34.6 ms (22.1 to 48.7). The two runs disagree by more than their own intervals, which is why we read both as "smaller than the machine's noise" and not as a ranking. Both tools stored every trace from the real calls in both runs.

Burst: 2,000 traced no-op calls per tool (run B)
ToolMedian per traced callp95Traces storedServer memory after runsDatabase on disk
Arize Phoenix (default register)3.31 ms4.18 ms2,050 of 2,050468 MB31 MB
MLflow (default async)0.16 ms0.39 ms845 of 2,050755 MB7 MB

Phoenix's default sends each span to the server before your code continues, so a burst costs about 3 ms per span and nothing is lost. MLflow queues traces in the background, so your code barely waits, but when the queue filled it logged "Trace export queue is full, trace will be discarded" and dropped 1205 traces. Run A logged the same warning; its stores were wiped before we could count them. If you trace many small steps per request, set Phoenix to batch mode or raise MLflow's queue size (MLFLOW_ASYNC_TRACE_LOGGING_MAX_QUEUE_SIZE), then measure again.

The run itself

Terminal recording of run B (asciinema), sped up, with pauses longer than 2 seconds shortened.