Korean Journal of Construction Engineering and Management, 26(1), 74-82
Safety|구충완
Synthetic video generation process model for enhancing the activity recognition performance of heavy construction equipment - Utilizing 3D simulations in unreal engine environment
2025Domestic Journals
Research AreaSafety
Professor구충완
AuthorsShin, Y., Seo, S., and Koo, C. (Corresponding Author)
Publication Year2025
Journal (Volume, Issue, Pages)Korean Journal of Construction Engineering and Management, 26(1), 74-82
AbstractThere has been a growing interest in AI (Artificial Intelligence)-based smart management for heavy
construction equipment, aiming at real-time monitoring of safety, productivity, and environmental impact. In addition,
deep learning-based computer vision technologies have advanced to identify the activities of construction equipment
through visual information from CCTV (Closed-Circuit Television) at construction sites. Ensuring the performance
of such vision technologies requires a substantial amount of training video datasets collected from construction
sites; however, there are limitations in gathering datasets across diverse scenarios due to the nature of construction
environments. To address this challenge, this study aimed to develop a synthetic video generation process model to
enhance the activity recognition performance of heavy construction equipment. The proposed process model can
closely simulate real videos of construction equipment using 3D simulation in game engine. This study validated the
stepwise performance improvement of the proposed process model using the 3D ResNet-18 model for excavator
activity recognition. The performance of the final stage, measured by the weighted F1-score, showed a 90.89%
performance, marking an approximate 25% improvement compared to the first stage (66.02%). This performance is
very similar to the activity recognition performance for real videos (90.12%). The confusion matrix demonstrated that
the recognition performance and patterns for both real and synthetic videos were considerably similar. The synthetic
videos produced through the proposed process model can be utilized as training datasets and serve as a foundational
model for simulating excavator operations.
construction equipment, aiming at real-time monitoring of safety, productivity, and environmental impact. In addition,
deep learning-based computer vision technologies have advanced to identify the activities of construction equipment
through visual information from CCTV (Closed-Circuit Television) at construction sites. Ensuring the performance
of such vision technologies requires a substantial amount of training video datasets collected from construction
sites; however, there are limitations in gathering datasets across diverse scenarios due to the nature of construction
environments. To address this challenge, this study aimed to develop a synthetic video generation process model to
enhance the activity recognition performance of heavy construction equipment. The proposed process model can
closely simulate real videos of construction equipment using 3D simulation in game engine. This study validated the
stepwise performance improvement of the proposed process model using the 3D ResNet-18 model for excavator
activity recognition. The performance of the final stage, measured by the weighted F1-score, showed a 90.89%
performance, marking an approximate 25% improvement compared to the first stage (66.02%). This performance is
very similar to the activity recognition performance for real videos (90.12%). The confusion matrix demonstrated that
the recognition performance and patterns for both real and synthetic videos were considerably similar. The synthetic
videos produced through the proposed process model can be utilized as training datasets and serve as a foundational
model for simulating excavator operations.
Synthetic Video Generation ProcessHeavy Construction EquipmentActivity RecognitionF1-ScoreGame Engine3D Simulations
PrevA strategic approach to enhancing the practical applicability of vision-based detection and classification models for construction tools - Sensitivity analysis of model performance depending on confidence thresholdNextInvestigation of individual differences in thermal comfort of elderly residential spaces: Utilizing IoT-based physiological model