This study introduces a comprehensive approach for classifying individual malting barley kernels, involving dual-sided kernel imaging, a specifically designed image processing algorithm, an optimized deep neural network architecture, and a mechanical sorting system. The proposed method achieves precise classification into multiple classes, aligning with quality standards for malting material assessment. Throughout the study, various image analysis techniques were assessed, including traditional feature engineering, established transfer learning deep neural network architectures, and our custom-designed convolutional neural network tailored for barley kernel image analysis. Comparative analysis underscores the superior performance of our network model. The study reveals that our proposed deep learning network achieves a 94% accuracy in classifying barley kernel defects and varieties, outperforming well-established transfer learning models to complex architectures that attain 93% accuracy. Additionally, it surpasses the traditional machine learning approach involving feature extraction and support vector machine classifiers, which achieve accuracy below 90% in detecting defective kernels and below 70% in varietal classification. However, we also noted the traditional approach's advantage in morphological feature recognition. This observation guides new research toward integrating morphological feature extraction techniques with modern convolutional networks. This paper presents a deep neural network designed specifically for the analysis of cereal kernel images in two applications: defect and variety classification. It emphasizes the importance of standardizing kernel orientation and merging images from both sides of the kernel, and introduces a device for image acquisition that fulfills this need., (© 2024. The Author(s).)