ConstructionQA: Evaluating Multimodal Large Language Models on Industrial Construction Drawing Set
Research summary
A task-specific benchmark for multimodal answers about construction drawings.
Research record
This record preserves the title and author order of the supplied manuscript. Its research stage remains separate from the existence of a PDF or conference-formatted layout.
Evaluation results, cohort definitions, limitations, and source locators are maintained in the related project. This record does not duplicate the project metrics.
Evaluation Scope & Methodology
Samples, denominators, uncertainty, sources and limits
Approach
Compare models within each task and supplied-context condition; do not pool different task metrics.
Results and scope belong to the related case study, ConstructionQA: reading supplied drawing context.
Cohorts and samples
| Cohort | Sample | Definition |
|---|---|---|
| ConstructionQA supplied-context questions | 120 question–answer pairs | Approximately 150+ sheets across eight disciplines; task-specific subsets. |
Reported measures
| Measure | Unit | Cohort | Denominator | Uncertainty | Evidence |
|---|---|---|---|---|---|
| Information extraction F1-BoW, GPT-5.4 | Score from 0 to 1 | ConstructionQA supplied-context questions | Information-extraction task subset of the 120-pair benchmark | None reported | Reported in source |
| Information extraction F1-BoW, Gemini 2.5 Flash | Score from 0 to 1 | ConstructionQA supplied-context questions | Information-extraction task subset of the 120-pair benchmark | None reported | Reported in source |
| Cross-discipline coordination combined score, GPT-5.4 | Score from 0 to 1 | ConstructionQA supplied-context questions | Cross-discipline coordination task subset, Table 1 | None reported | Reported in source |
| Sheet-reference F1: GPT-5.4 | Score from 0 to 1 | ConstructionQA supplied-context questions | Same ConstructionQA sheet-reference task under distractor and guaranteed-context conditions | None reported | Reported in source |
| Sheet-reference F1: Gemini 2.5 Flash | Score from 0 to 1 | ConstructionQA supplied-context questions | Same ConstructionQA sheet-reference task under distractor and guaranteed-context conditions | None reported | Reported in source |
| Sheet-reference F1: GPT-4.1 Nano | Score from 0 to 1 | ConstructionQA supplied-context questions | Same ConstructionQA sheet-reference task under distractor and guaranteed-context conditions | None reported | Reported in source |
Limitations
- Supplied relevant sheets do not test full-corpus retrieval.
- Guaranteed context does not improve every task.
- Native-PDF conditions are separate from Table 1.
Sources
- ConstructionQA: Evaluating Multimodal Large Language Models on Industrial Construction Drawing Set, Benchmark description; Table 1; Figure 2 (Supplied manuscript reviewed November 2026). Open approved PDF
Cite this record
Obiora Odugu, Mohammad Hadi Foroozandeh Dehkordi, Asif Faisal Chowdhury, Muhammad Tahir Khan, Chao Wang. ConstructionQA: Evaluating Multimodal Large Language Models on Industrial Construction Drawing Set. International Conference on Industrialized Construction. Manuscript, 2026.
Formatted from the supplied manuscript. Publisher metadata has not been verified, so no BibTeX is offered yet.