IndustrialKnowEval (IndKnow)
by Haiteng Wang, Haoze Li, Jingwei Guo, Jingbo Zhou, Si Liu and Lei Ren
What is IndustrialKnowEval (IndKnow)? IndustrialKnowEval (IndKnow) is a reasoning & knowledge eval made by Haiteng Wang, Haoze Li, Jingwei Guo, Jingbo Zhou, Si Liu and Lei Ren. Industrial and engineering question answering for LLMs: 43,282 samples across 21 task types in seven industrial sectors, including aerospace, electric power and metallurgy. Questions are sorted into four levels of cognitive demand, from recalling facts through comprehension and reasoning to engineering application, and the authors say enterprise domain specialists took part in building and checking the data. The paper (arXiv:2611.20695) evaluates 18 LLMs (the README says 16) and proposes an Industrial Cognitive Capability Gap metric, which measures how far a model's reasoning and application results trail its basic industrial knowledge.
Editor note
Industrial and engineering question answering for LLMs: 43,282 samples across 21 task types in seven industrial sectors, including aerospace, electric power and metallurgy. Questions are sorted into four levels of cognitive demand, from recalling facts through comprehension and reasoning to engineering application, and the authors say enterprise domain specialists took part in building and checking the data. The paper (arXiv:2611.20695) evaluates 18 LLMs (the README says 16) and proposes an Industrial Cognitive Capability Gap metric, which measures how far a model's reasoning and application results trail its basic industrial knowledge.
- What it tests
- Industrial domain knowledge, from memory to engineering application, across seven sectors.
- Original source
- github.com/J4cksonLee/IndKnow
- Last updated
- arXiv:2611.20695 (16 Nov 2026); repo last push 2026-11-16 (GitHub API, read 19 Nov 2026)
- License
- Apache-2.0 (GitHub API, read 19 Nov 2026)
Related reasoning & knowledge evals
- GPQA — Expert-level science reasoning.
- Humanity's Last Exam — Frontier academic knowledge and reasoning.
- LiveBench — Hard, contamination-limited tasks with verifiable ground-truth answers (no LLM judge) in v1; LiveBench 2.0 (14 Oct 2026) adds long-horizon real-work tasks.
- MMLU — World knowledge and problem solving across many school and professional subjects.
Discussion
No one has discussed IndustrialKnowEval (IndKnow) yet. Have you run it? Share what it caught, what it missed or how you set it up.