Activity identification in modular construction using audio signals and machine learning.

Modular construction is an attractive building method due to its advantages over traditional stick-built methods in terms of reduced waste and construction time, more control over resources and environment, and easier implementation of novel techniques and technologies in a controlled factory setting. However, efficient and timely decision-making in modular factories requires spatiotemporal information about the resources regarding their locations and activities which motivates the necessity for an automated activity identification framework. Thus, this paper utilizes sound, a ubiquitous data source present in every modular construction factory, for the automatic identification of commonly performed manual activities such as hammering, nailing, sawing, etc. To develop a robust activity identification model, it is imperative to engineer the appropriate features of the data source (i.e., traits of the signal) that provides a compact yet descriptive representation of the parameterized audio signal based on the nature of the sound, which is very dependent on the application domain. In-depth analysis regarding appropriate features selection and engineering for audio-based activity identification in construction is missing from current research. Thus, this research extensively investigates the effects of various features extracted from four different domains related to audio signals (time-, time-frequency-, cepstral-, and wavelet-domains), in the overall performance of the activity identification model. The effect of these features on activity identification performance was tested by collecting and analyzing audio data generated from manual activities at a modular construction factory. The collected audio signals were first balanced using time-series data augmentation techniques and then used to extract a 318-dimensional feature vector containing 18 different feature sets from the abovementioned four domains. Several sensitivity analyses were performed to optimize the feature space using a feature ranking technique (i.e., Relief algorithm), and the contribution of features in the top feature sets using a support vector machine (SVM). Eventually, a final feature space was designed containing a 130-dimensional feature vector and 0.5-second window size yielding about 97% F-1 score for identifying different activities. The contributions of this study are two-fold: 1. A novel means of automated manual construction activity identification using audio signal is presented; and 2. Foundational knowledge on the selection and optimization of the feature space from four domains is provided for future work in this research field. The result of this study demonstrates the potential of the proposed system to be applied for automated monitoring and data collection in modular construction factory in conjunction with other activity recognition frameworks based on computer vision (CV) and/or inertial measurement units (IMU). • An audio-based activity identification framework for manual activities in modular construction is presented. • Methods for feature extraction, optimization, and synthesis for classifying audio-signals is provided. • A multi-class SVM model with 5-fold cross-validation is used to validate the designed feature space. • Designed feature space and window-size are used to evaluate the SVM classification model to identify different activities. [ABSTRACT FROM AUTHOR]