HN practitioner mentions of LLM eval and observability tools
Pull date 2 Nov 2026 · dataset v1.1 · page published 2026-11-09
Across 7,985 Hacker News comments from 2 Nov 2024 to 2 Nov 2026 (UTC), 282 kept tool mentions remain after filters (dataset v1.1, pull 2026-11-02). Ranked by distinct threads, Langfuse leads with 60 threads (111 comments), then promptfoo 38, Braintrust 34 and LangSmith 30; TruLens and Galileo have 0 kept mentions.
Counts by tool
Sorted by distinct HN threads first (then raw comment mentions). Thread count is the primary measure because one busy thread can inflate comment counts. Stance columns are keyword labels on a short window around the tool name; most mentions are labelled neutral.
| Tool | Unique threads | Mentions (comments) | Top-thread share | Max in one thread | Distinct authors | Recommend | Neutral | Complaint |
|---|---|---|---|---|---|---|---|---|
| Langfuse | 60 | 111 | 22.5% | 25 | 100 | 14 | 94 | 3 |
| promptfoo | 38 | 45 | 8.9% | 4 | 33 | 6 | 38 | 1 |
| Braintrust | 34 | 41 | 9.8% | 4 | 39 | 4 | 36 | 1 |
| LangSmith | 30 | 38 | 10.5% | 4 | 36 | 1 | 36 | 1 |
| Arize Phoenix | 8 | 10 | 20% | 2 | 9 | 1 | 9 | 0 |
| Helicone | 8 | 9 | 22.2% | 2 | 9 | 3 | 6 | 0 |
| DeepEval | 6 | 12 | 58.3% | 7 | 9 | 1 | 11 | 0 |
| LM Evaluation Harness | 4 | 4 | 25% | 1 | 4 | 0 | 4 | 0 |
| Ragas | 3 | 4 | 50% | 2 | 4 | 1 | 3 | 0 |
| Comet Opik | 3 | 3 | 33.3% | 1 | 3 | 0 | 3 | 0 |
| Inspect AI | 2 | 2 | 50% | 1 | 2 | 0 | 2 | 0 |
| OpenAI Evals | 2 | 2 | 50% | 1 | 2 | 0 | 2 | 0 |
| W&B Weave | 1 | 1 | 100% | 1 | 1 | 0 | 1 | 0 |
| Galileo | 0 | 0 | 0% | 0 | 0 | 0 | 0 | 0 |
| TruLens | 0 | 0 | 0% | 0 | 0 | 0 | 0 | 0 |
What the totals mean
- 7,985 comments scanned from the Algolia queries (the scanned set in METHODOLOGY.md).
- 505 raw tool-mention rows before self-promo and launch-thread exclusions; 282 kept in v1.1 after removing 3 non-tool Braintrust matches from v1.0's 285.
- 199 tool-thread pairs in
thread-counts.jsonl(one row per tool + HN story). - Langfuse concentration: 25 of 111 comments (22.5%) sit in one thread (ClickHouse acquires Langfuse). Without that thread: 86 comments across 59 threads.
- Zero rows: TruLens and Galileo were searched and kept as explicit 0s.
Limits
- HN only. Reddit blocked unauthenticated access on 2 Nov 2026; the audience leans startup/infra and open-source.
- Mentions ≠ use or quality. Passing lists and news spikes count the same as first-hand reports until stance is read with care.
- Stance is crude. On a hand-checked sample of 40 kept rows, tool identity was 40/40 and stance 37/40; recommend is a floor.
- Self-promo filter catches disclosed affiliations only. Undisclosed vendor replies can remain.
- Algolia caps each query at 1,000 hits; common terms may be truncated.
FAQ
Which LLM eval tool is mentioned in the most HN threads?
In this v1.1 pull, Langfuse appears in 60 distinct HN threads, ahead of promptfoo (38), Braintrust (34) and LangSmith (30). The table is sorted by distinct threads, not raw comment count.
Do mention counts mean market share?
No. A count is how often a tool's name appeared in HN comments after filters. One news thread can move the numbers: 25 of Langfuse's 111 comments come from the ClickHouse acquisition thread alone.
Why are TruLens and Galileo listed as zero?
Both were in the search terms. After filters, each has 0 kept mentions in the window. Zero is kept as an explicit row, not dropped.
What is excluded from the counts?
Job and hiring threads, off-topic name matches, disclosed self-promotion and vendor launch threads. Of 505 raw tool-mention rows, 282 remain in v1.1. Comment text and usernames are not published.
Files and sources
- Summary table: summary as CSV (this page's download; same columns as
data/practitioner-mentions/summary.csv). - Method: METHODOLOGY.md.
- Source: Hacker News via Algolia HN API · pull 2026-11-02 · v1.1.