Drowsiness has become one of the major causes of road accidents now-a-days. In order to alleviate this issue, a system has been developed, which uses electroencephalogram (EEG) signals to detect drowsiness with sufficient reliability. This experiment was conducted on a small population and the EEG signals were acquired using a 14-channel wireless headset, while they were in a virtual driving environment. To extract the eye closures, the EEG signal was segmented, and pre-processed. Further the scalograms which describes the time-frequency characteristics of these segments were taken. Pretrained Convolutional Neural Network based architectures viz. ResNet-152, ResNet101, VGG16, VGG19, AlexNet were used to distinguish three states of the driver namely “Sleepy or Drowsy”, “Asleep” and “Awake”.