Trustworthy validation of industrial energy assessment reports: a neuro-symbolic validator and seeded-error benchmark
96.2% mutant-level exact-match
Research summary
96.2% exact-match with rules and a unit-aware calculator.
Research record
This record preserves the title and author order of the supplied manuscript. Its research stage remains separate from the existence of a PDF or conference-formatted layout.
Evaluation results, cohort definitions, limitations, and source locators are maintained in the related project. This record does not duplicate the project metrics.
Selected finding
The headline result from the related case study, with its sample and source.
Held-out seeded-error evaluation
1,620 mutated reports. 81 source reports; 3,249 seeded errors; excludes 27 calibration reports.
Mutant-level exact-match
Reported in sourceHighest reported value: Rules plus calculator, 96.2%.
Data table
| Condition | Value |
|---|---|
| Rules plus calculator | 96.2% |
| Hybrid 8B | 43.0% |
| Direct LLM 8B | 0.1% |
Denominator: 1,620 mutated reports in the full held-out evaluation.
Adapted from Trustworthy validation of industrial energy assessment reports: a neuro-symbolic validator and seeded-error benchmark, Abstract; Tables 4–6; Figure 1.
Evaluation Scope & Methodology
Samples, denominators, uncertainty, sources and limits
Approach
Compare exact expected finding sets on a held-out source-report slice; preserve separate calibration and model replay cohorts.
Results and scope belong to the related case study, AuditGuard: verifying industrial energy reports.
Cohorts and samples
| Cohort | Sample | Definition |
|---|---|---|
| Held-out seeded-error evaluation | 1,620 mutated reports | 81 source reports; 3,249 seeded errors; excludes 27 calibration reports. |
Reported measures
| Measure | Unit | Cohort | Denominator | Uncertainty | Evidence |
|---|---|---|---|---|---|
| Mutant-level exact-match | Percent | Held-out seeded-error evaluation | 1,620 mutated reports in the full held-out evaluation | None reported | Reported in source |
| Minimum per-class recall | Score from 0 to 1 | Held-out seeded-error evaluation | Seven seeded-error classes on the full held-out slice | None reported | Reported in source |
| False-positive findings | Count | Held-out seeded-error evaluation | Full held-out slice: 81 reports and 1,620 mutants | None reported | Reported in source |
Limitations
- Seeded errors are controlled mutations, not an observed production defect rate.
- Confidential client reports cannot be released as demonstration fixtures.
- Larger-model results use separate subsets and cannot be pooled with the full evaluation.
Sources
- Trustworthy validation of industrial energy assessment reports: a neuro-symbolic validator and seeded-error benchmark, Abstract; Tables 4–6; Figure 1 (Supplied manuscript reviewed November 2026). Public release pending
Cite this record
Asif Faisal Chowdhury, Muhammad Tahir Khan, Chao Wang. Trustworthy validation of industrial energy assessment reports: a neuro-symbolic validator and seeded-error benchmark. Manuscript.
Formatted from the supplied manuscript. Publisher metadata has not been verified, so no BibTeX is offered yet.