Background: Simulation-enhanced education is being used in many health care curricula as an answer to an increasing demand for allied health professionals and a paucity of clinical sites crucial to providing requisite clinical education. The effective use of simulation may reduce and provide better use of the time required in clinical environments. If the use of simulation is to become a valuable addition to traditional clinical experiential learning, a reliable and valid measurement tool designed to measure a radiological technologist's performance in a practice environment is vital., Objective: This study assessed the reliability of a clinical competency evaluation tool used in the radiological technology setting. The objective was to determine the inter-rater reliability of the evaluation tool as applied to six clinical scenarios., Methods: In order to test the clinical competency evaluation tool, standardized patients portrayed radiological technologists and patients in six videotaped scenarios depicting one-on-one interactions between these two groups. Nine trained and qualified participants watched the videos and used the study tool to evaluate each radiological technologist's clinical communication competency., Results: Inter-rater and intraclass correlations were generated to examine the internal consistency of rater scores on the videotaped scenarios. Face and content validity were established through an authenticity exercise involving content educator experts., Conclusion: Overall, this tool was shown to be a reliable, valid, feasible, and usable methodology to assess communication skills with clinical students. Further investigation of the tool is required to examine the tool's ability to reliably identify borderline or mixed skill level students under cultural and non-culture-based scenarios.