Background Driving restrictions in epilepsy are intended to safeguard public and personal safety; however, these limitations inhibit socialization, restrict employment, and reduce self-esteem in patients with seizures. A large proportion of patients with seizures continue to drive, and factors leading to noncompliance with driving regulations are poorly understood. Thus, the patients' perspective on driving safety is not incorporated into the existing counseling tools on driving safety in epilepsy. The present study assessed social, economic, and psychological perceptions related to driving restrictions in patients with refractory and pharmacotherapy-controlled seizures at the single epilepsy center and identified impediments for safe driving. Methods Data were obtained from an anonymous survey completed by 25 adult patients in the presurgical group (PG) with refractory epilepsy and 46 patients in the ambulatory group (AG) with confirmed epilepsy which did not meet criteria for refractoriness. The questionnaire (administered via Research Electronic Data Capture (REDCap)) addressed seizure and driving history, knowledge of driving restrictions, and social consequences of losing driving privileges. Results Eighty-seven percent of all responders experienced seizures with alteration of awareness; however, 34% of patients continued to drive during the time when they were legally restricted, and 6% had accidents related to seizures. All responders reported their seizure status accurately to the treating physician, and 93% understood state-based driving restrictions. The median time from the last seizure was shorter, and the duration of last driving restriction was longer in the PG compared with the AG (1 vs. 20 weeks, and 12 vs. 24 weeks, respectively). Despite that, the proportions of patients driving at the time of survey were not significantly different between the two groups. Nearly 80% of all patients stated that driving restrictions reduced their quality of life, and 70% believed that these restrictions carry a social stigma. Employment was chosen to be the most affected by driving restrictions from a list of four social domains by the majority of patients in both groups. Notably, the employment rate was 26% higher in the AG compared with the PG. The lack of public transportation was regarded as a hurdle by more than 60% of patients in each group with greater than two-thirds of patients relying on other drivers for transportation. Conclusions These findings suggest that patients with refractory and pharmacotherapy-controlled seizures are similarly likely to drive a vehicle, disregarding a practitioner's advice and state restrictions. The lack of public transportation is a shared constraint and likely leads to reduced compliance with driving regulations. Driving restrictions carry social stigma and limit the employment of patients with epilepsy, regardless of the refractory seizure status.