COVID-19 is an infectious disease that has spread globally, and the best way to slow down transmission is to maintain a safe distance. Due to the COVID-19 spread, social distancing has become very vital. Furthermore, the formation of groups and crowds cannot be left unseen. Even when the necessary regulations have been implemented by governments worldwide, people tend not to follow the rules. We wanted to make it possible for authorities in areas like schools, universities, industries, hospitals, restaurants, etc., to monitor people breaking social distancing rules and take appropriate measures to control the virus from spreading. To monitor and control the crowd, society requires a system that does not put other people's lives at risk. Therefore, it is critical that we stop it from spreading further. Initially, the government imposed a lockdown to control the spread of the virus. Due to the lockdowns, the economy had experienced some negative effects. Due to the economic slowdown, people were allowed to go out and carry on with their regular tasks, leading to crowding in many places, intentionally or unintentionally. The research work aims to make a crowd detection and alert system in public places like hospitals, schools, universities, and other public gathering events. The proposed idea has two modules; a deep CNN CrowdNet people counting algorithm to detect the distance between humans in highly dense crowds and an IoT platform for sending information to the authorities whenever there is a violation. Image processing is carried out in two parts: extraction of frames from real-time videos using YOLO CV, and the second is processing the frame to detect the number of people in the crowd. The crowd counting algorithm, along with the vaccination, will enforce safety rules in people-gathering places and minimize health risks and spread. The image processing YOLO model mainly targets people not following social distancing norms and standing very close by. The data for the violations are sent online to the IoT platform, where the value is compared to a threshold. The platform aids in sending alerts to the concerned authorities in case of significant violations. Warnings are sent through e-mail or personal messages to the concerned authorities and the location. This model prevents the presence of an official to check whom all are violating the rules. There is no need for human intervention and risking their lives; direct messages can be sent through the IoT platform to authorities if there is a crowd formation. Data analytics can help find out the peak hours of crowding and help control the crowd much more efficiently. CrowdNet, a deep CNN algorithm, will estimate the number of humans in a given frame to classify the locations where most people communicate and check whether the safe distance is not reached and the number of times it is not reached. Our system sends the number of people available in the frame at that moment and whether they are maintaining social distancing or not. The Deep CNN algorithm will filter the objects by capturing high-level semantics required to count only the humans and calculate the distance between the humans alone. The base neural network is Alexnet to estimate whether it is safe or not and then send it to the respective authority. This proposed idea using CrowdNet CNN and IoT combination will help find out peak hours of crowding and help control the spread of the disease during social distance violations without human intervention. Thus, social distancing in public places is automated using the real-time deep learning-based framework via object detection, tracking, and controlled disease.