Huffman Deep Compression of Edge Node Data for Reducing IoT Network Traffic

Data compression at the Internet of Things (IoT) edge node aims to minimize data traffic in smart cities. The traditional Huffman Coding Algorithm (HCA) is shown as the most effective compression algorithm for sensor data. However, implementing the algorithm at IoT edge nodes is hindered due to memory limitations; HCA requires a large amount of memory to construct a Huffman tree to compress data. To address this issue, this paper proposes a new lossless Huffman Deep Compression (HDC) algorithm that incorporates the sliding window technique to fit in memory, reduces the complexity of the Huffman tree using deep learning pruning and pooling techniques, and uses pattern matching with pattern weights instead of using symbol matching and symbol frequencies in HCA. This paper introduces a sliding window approach to minimize memory usage, leveraging pattern matching and weights for higher compression and employing deep learning techniques to reduce the Huffman tree size through pruning and pooling. Experiments were performed using the Esp8266 MCU IoT node on eight numerical attributes from sensors of six of Malaysia’s air pollution station datasets. The findings demonstrate that the HDC algorithm has substantially reduced data size (p-value