Reasoning & knowledge

IndustrialKnowEval (IndKnow)

by Haiteng Wang, Haoze Li, Jingwei Guo, Jingbo Zhou, Si Liu and Lei Ren

What is IndustrialKnowEval (IndKnow)? IndustrialKnowEval (IndKnow) is a reasoning & knowledge eval made by Haiteng Wang, Haoze Li, Jingwei Guo, Jingbo Zhou, Si Liu and Lei Ren. Industrial and engineering question answering for LLMs: 43,282 samples across 21 task types in seven industrial sectors, including aerospace, electric power and metallurgy. Questions are sorted into four levels of cognitive demand, from recalling facts through comprehension and reasoning to engineering application, and the authors say enterprise domain specialists took part in building and checking the data. The paper (arXiv:2611.20695) evaluates 18 LLMs (the README says 16) and proposes an Industrial Cognitive Capability Gap metric, which measures how far a model's reasoning and application results trail its basic industrial knowledge.

Page last updated 2026-10-18 · Evals Review

Editor note

Industrial and engineering question answering for LLMs: 43,282 samples across 21 task types in seven industrial sectors, including aerospace, electric power and metallurgy. Questions are sorted into four levels of cognitive demand, from recalling facts through comprehension and reasoning to engineering application, and the authors say enterprise domain specialists took part in building and checking the data. The paper (arXiv:2611.20695) evaluates 18 LLMs (the README says 16) and proposes an Industrial Cognitive Capability Gap metric, which measures how far a model's reasoning and application results trail its basic industrial knowledge.

What it tests
Industrial domain knowledge, from memory to engineering application, across seven sectors.
Original source
github.com/J4cksonLee/IndKnow
Last updated
arXiv:2611.20695 (16 Nov 2026); repo last push 2026-11-16 (GitHub API, read 19 Nov 2026)
License
Apache-2.0 (GitHub API, read 19 Nov 2026)

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Sign in with GitHub to review

Related reasoning & knowledge evals

  • GPQA — Expert-level science reasoning.
  • Humanity's Last Exam — Frontier academic knowledge and reasoning.
  • LiveBench — Hard, contamination-limited tasks with verifiable ground-truth answers (no LLM judge) in v1; LiveBench 2.0 (14 Oct 2026) adds long-horizon real-work tasks.
  • MMLU — World knowledge and problem solving across many school and professional subjects.

All Reasoning & knowledge evals · Comparison table

Discussion

No one has discussed IndustrialKnowEval (IndKnow) yet. Have you run it? Share what it caught, what it missed or how you set it up.

Sign in with GitHub to comment

Be specific and kind. No promotion or affiliate links. Community rules · All discussions