Published on May 2nd, 2017 | from CAMH Education
Evaluation in Education: A year in review (Part One)
By Megha Bhavsar, Evaluation Co-ordinator, CAMH Education
Happy New Fiscal Year, everyone! A new year brings an opportunity not only to look forward to the exciting things that are ahead, but also to reflect on the things that we are grateful for, that we have accomplished and that we have learned over the past year.
This is the first in a four-part series looking at CAMH Education’s “small but mighty” Evaluation team consisting of Erica McDiarmid (Manager), Megha Bhavsar (Evaluation Co-ordinator) and Alyssa Kelly (Research Analyst). Through the posts, you will gain a good understanding of the variety of projects that we have been involved with over the past year, our capacity-building efforts and how we have disseminated knowledge. You may also come away with an idea on how you can engage the Evaluation team for your project. So, let’s get started!
The Student Experience
Our department has a strong emphasis on the student experience. Working with the Evaluation team, our colleagues at the Student Centre administer the Student Experience Survey, a tool used TAHSN wide, to students completing placements. On a regular basis (quarterly, semi-annually and annually), the Evaluation team analyzes and reports data in various formats to internal and external stakeholders. These stakeholders include the Toronto Applied Health Science Network (TAHSN), our partners in Performance Improvement for the Balanced Scorecard, the Education leadership team, the Clinical Practice Advisory Council, Education Co-ordinators, and CAMH Legal. Our annual results show that CAMH continues to excel in providing a positive student experience with top ratings on many measures, including the percentage of students who would recommend a placement here. Our response rate is consistently higher than the average rate of TAHSN hospitals. This would not be possible without our colleagues at the Student Centre who diligently organize the students and the surveys. We will continue enhancing the Student Experience through efforts to align data collection with other departments at CAMH.
Notable this year is our continued partnership with the CAMH Legal Team. Committed to providing an excellent student experience, the Legal and Evaluation teams have worked to include legal students on the Student Experience Survey and conduct exit interviews with legal students to evaluate their learning experience. As a result of these efforts, the Legal team has incorporated students’ suggestions to improve the experience they provide the next cohort, which has been evident in the results.
We also had a student of our own – Riley Saikaly, who was completing his Master of Public Health practicum. Riley worked on a number of projects including evaluating the Opioid Dependency Treatment course. The objective of the evaluation was to understand, from the perspective of program graduates, the importance/benefit of workshop-based training to deliver material on interprofessional collaboration and reducing stigma regarding MMT and patients with opioid addiction (e.g., stigma, client-centred care). Riley also conducted a needs assessment for a customer service and de-escalation training with shopping mall security.
Going into the new fiscal year, we will be hosting another practicum student beginning this month. We are also working with the Student Centre to explore ways to integrate evaluation activities into the Student Registration System. We are also looking to measure how many students eventually become CAMH staff and how the student experience leads to the development of mental health advocates.
Simulation and Evaluation
This year, the Simulation team held Simulation Workshops at the Mental-Physical Interface (SWAMPI). We conducted an evaluation of SWAMPI. For more on the exciting intersection of simulation and evaluation, click on our previous blog post.
The Department of Psychiatry, University of Toronto has created a partnership with the Centre for Faculty Development, Allan Waters Family Simulation Centre at St. Michael’s Hospital and CAMH, to create a faculty development program to provide clinical education within the Department of Psychiatry with knowledge of simulation-based learning. A series of workshops were developed based on key informant interviews and were evaluated to understand quality and relevancy of course content and delivery methods, and improve the program.
We also collaborated with the Simulation team and our partners at the Temerty Centre for Therapeutic Brain Intervention to evaluate their electroconvulsive therapy training delivered to psychiatry residents annually. We proposed a new evaluation methodology to use simulation as an evaluation method and implemented this to understand the program’s areas of success and improvement.
Read the rest of this series: