ManuscriptCo-author and assessment researcher

Temporal assessment of excavator proficiency

Multi-agent assessment across 119 operators and three skill levels.

All projects

My contribution

Contributed to temporal behavior analysis and evidence-linked operator assessment reports.

Separate evidence lanes

The assessment framework combines simulator telemetry calculations, video analysis, simulator-report extraction, and report assembly. Pairwise Simultaneity Index and Bimanual Coordination Score quantify temporal control organization. Each report should preserve the difference between a computed measurement and a model-generated interpretation.

Reported group comparisons

The study covers 119 operators across three skill levels. Expert versus novice Arm–Boom simultaneity is 0.542 versus 0.125, and bimanual coordination is 0.333 versus 0.109. These are aggregate comparisons, not reconstructed participant distributions or universal thresholds.

The abstract describes three or more simultaneous controls, while other text uses three-control wording. That ambiguity blocks promoting the corresponding 35.7% versus 6.3% result as an unqualified chart. The supplied recordings and report recommendations also do not establish prospective learning gains.

System architecture

Select a step, or a block in the drawing, to see its input, computation, output and source.

Redrawn from manuscript

Three evidence lanes, one assessment report

Video, telemetry and the simulator report are analyzed in separate lanes. Telemetry calculations produce computed measures. Report assembly writes the interpretation.

  • Computed measurement
  • Analysis or extraction
  • Model-written interpretation
Three evidence lanes, one assessment reportAssessment framework from the 119-operator study. Three lanes run in parallel: session video to video analysis to behavior observations; simulator telemetry to PSI and BCS calculations to computed measures; simulator report to report extraction to extracted fields. All three feed report assembly, which produces the evidence report.SessionvideoVideoanalysisBehaviorobservationsSimulatortelemetryPSI and BCScalculationsComputedmeasuresSimulatorreportReportextractionExtractedfieldsReportassemblyEvidencereport
  1. Session videoInput
    Input
    Recorded simulator session video.
    Computation
    None.
    Output
    Video for analysis.
    Source
    Excavator assessment manuscript, Figures 3-1 and 3-2
  2. Video analysisProcessing step
    Input
    Session video.
    Computation
    Analyzes operator behavior in the recording.
    Output
    Behavior observations.
    Source
    Excavator assessment manuscript, Figures 3-1 and 3-2
  3. Behavior observationsProcessing step
    Input
    Video analysis output.
    Computation
    None.
    Output
    Observations for the report.
    Source
    Excavator assessment manuscript, Figures 3-1 and 3-2
  4. Simulator telemetryInput
    Input
    Control telemetry from the simulator.
    Computation
    None.
    Output
    Time-series control data.
    Source
    Excavator assessment manuscript, Figures 3-1 and 3-2
  5. PSI and BCS calculationsDeterministic
    Input
    Control telemetry.
    Computation
    Computes the Pairwise Simultaneity Index and Bimanual Coordination Score.
    Output
    Computed temporal coordination measures.
    Source
    Excavator assessment manuscript, Figures 3-1 and 3-2
  6. Computed measuresDeterministic
    Input
    Calculation output.
    Computation
    None.
    Output
    Measurements for the report.
    Source
    Excavator assessment manuscript, Figures 3-1 and 3-2
  7. Simulator reportInput
    Input
    The simulator’s own session report.
    Computation
    None.
    Output
    Report document.
    Source
    Excavator assessment manuscript, Figures 3-1 and 3-2
  8. Report extractionProcessing step
    Input
    Simulator report.
    Computation
    Extracts recorded fields from the report.
    Output
    Extracted fields.
    Source
    Excavator assessment manuscript, Figures 3-1 and 3-2
  9. Extracted fieldsProcessing step
    Input
    Extraction output.
    Computation
    None.
    Output
    Fields for the report.
    Source
    Excavator assessment manuscript, Figures 3-1 and 3-2
  10. Report assemblyModel
    Input
    Observations, computed measures and extracted fields.
    Computation
    Assembles a written assessment that interprets the evidence.
    Output
    Draft assessment narrative.
    Source
    Excavator assessment manuscript, Figures 3-1 and 3-2
  11. Evidence reportOutput
    Input
    Assembled narrative and its evidence.
    Computation
    Keeps computed measurements distinct from model-written interpretation.
    Output
    Assessment report. Coaching efficacy is not established.
    Source
    Excavator assessment manuscript, Figures 3-1 and 3-2

Adapted from Multi-Agent AI Framework for Objective Assessment of Excavator Operator Proficiency Through Temporal Behavior Analysis, Abstract; Tables 4.1–4.2; Figures 3-1/3-2.

Study of 119 operators across three skill levels, separate from the wearable-sensor samples. Video evidence remains an approved-media dependency.

Animated pulses show conceptual signal flow only. They do not show measured execution time or a live inference run.

Findings

Reported results, grouped by sample. Each figure keeps its own scale and links to its source and method.

Temporal assessment sample

119 operators. Three skill levels; separate assessment/reporting study.

Arm–Boom Pairwise Simultaneity Index

Reported in source

Highest reported value: Expert, 0.542.

Data table
Arm–Boom Pairwise Simultaneity Index. Expert and novice group aggregates in the 119-operator study.
ConditionValue
Expert0.542
Novice0.125

Denominator: Expert and novice group aggregates in the 119-operator study.

Adapted from Multi-Agent AI Framework for Objective Assessment of Excavator Operator Proficiency Through Temporal Behavior Analysis, Abstract; Tables 4.1–4.2; Figures 3-1/3-2.

Source and methodOpen approved PDF

Bimanual Coordination Score

Reported in source

Highest reported value: Expert, 0.333.

Data table
Bimanual Coordination Score. Expert and novice group aggregates in the 119-operator study.
ConditionValue
Expert0.333
Novice0.109

Denominator: Expert and novice group aggregates in the 119-operator study.

Adapted from Multi-Agent AI Framework for Objective Assessment of Excavator Operator Proficiency Through Temporal Behavior Analysis, Abstract; Tables 4.1–4.2; Figures 3-1/3-2.

Source and methodOpen approved PDF

Evaluation Scope & Methodology

Samples, denominators, uncertainty, sources and limits

Approach

Compare reported control-coordination aggregates across skill groups, preserving the telemetry/video/report branches.

Cohorts and samples

CohortSampleDefinition
Temporal assessment sample119 operatorsThree skill levels; separate assessment/reporting study.

Reported measures

MeasureUnitCohortDenominatorUncertaintyEvidence
Arm–Boom Pairwise Simultaneity IndexScore from 0 to 1Temporal assessment sampleExpert and novice group aggregates in the 119-operator studyNone reportedReported in source
Bimanual Coordination ScoreScore from 0 to 1Temporal assessment sampleExpert and novice group aggregates in the 119-operator studyNone reportedReported in source

Limitations

  • This 119-operator study is separate from the N=36 classifier and N=40 gaze analysis.
  • Generated assessment reports do not establish coaching efficacy.
  • Three-control versus three-or-more wording requires reconciliation.
  • Control-usage categories must not be stacked without resolving overlap definitions.

Sources

  • Multi-Agent AI Framework for Objective Assessment of Excavator Operator Proficiency Through Temporal Behavior Analysis, Abstract; Tables 4.1–4.2; Figures 3-1/3-2 (Supplied manuscript reviewed November 2026). Open approved PDF

Research records

LSU Industrial Training & Assessment Center · September 2024 to present

Energy Assessment Team Leader & Researcher

I lead industrial energy assessments: utility-bill analysis, on-site measurement, energy-balance calculations, and client technical reports. Totals are team assessment outcomes.

Recommended annual electricity savings

8,208,899 kWh/year

16 assessments with recommendations

Implemented annual electricity savings

5,950,754 kWh/year

11 assessments with implementation reports