|
|
 
 |
| ORIGINAL ARTICLE |
|
| Year : 2021 | Volume
: 11
| Issue : 1 | Page : 27-31 |
|
|
Mini-clinical evaluation exercise as a tool for formative assessment of postgraduates in psychiatry
Sujata Sethi, Vivek Srivastava, Piyush Verma
Department of Psychiatry, Pt. B.D. Sharma PGIMS, Rohtak, Haryana, India
| Date of Submission | 29-May-2020 |
| Date of Decision | 12-Jun-2020 |
| Date of Acceptance | 10-Sep-2020 |
| Date of Web Publication | 26-Jan-2021 |
Correspondence Address:
Sujata Sethi
122/8 Shivaji Colony, Rohtak - 124 001, Haryana
India
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/ijabmr.IJABMR_305_20
 Abstract | | |
Context: Introduction of competency-based medical education in India has necessitated a diversion from traditional methods of teaching and assessment. Aim: The aim of this study was to assess the feasibility and acceptability of mini-clinical evaluation exercise (mini-CEX) as a tool for formative assessment of postgraduates in psychiatry at our institute. Settings and Design: It was a prospective, quasi-experimental study carried out in the Department of Psychiatry, PGIMS, Rohtak (India). Materials and Methods: After obtaining written informed consent, trainees (18) were sensitized and faculty members (5) were trained to use mini-CEX and regarding the feedback process with the help of workshops. Statistical Analysis Used: Data collected were analyzed using descriptive statistics. Mean differences between the sessions were assessed using a paired sample t-test. P < 0.05 was considered as level of significance. Results: Statistically significant difference was found between session 1 and session 6 for counseling skills but not for three sessions of diagnostic skills. Mini-CEX was accepted as a better learning method by 61% of the postgraduates, 36% felt that it improved their clinical skills, 64% reported a boost in their confidence, and 89% looked forward to these encounters. Majority of the postgraduates and faculty members suggested that the mini-CEX should be regularly used in academic settings and should be introduced early in the training. Conclusions: Mini-CEX is a feasible formative assessment tool for postgraduates in psychiatry and an effective method of learning by the postgraduates, inducing a significant improvement in the counseling skills of students. Teaching faculty also accepted mini-CEX as an effective method of formative assessment.
Keywords: Formative assessment, mini-clinical evaluation exercise, postgraduates, psychiatry
How to cite this article:
Sethi S, Srivastava V, Verma P. Mini-clinical evaluation exercise as a tool for formative assessment of postgraduates in psychiatry. Int J App Basic Med Res 2021;11:27-31 |
How to cite this URL:
Sethi S, Srivastava V, Verma P. Mini-clinical evaluation exercise as a tool for formative assessment of postgraduates in psychiatry. Int J App Basic Med Res [serial online] 2021 [cited 2021 Mar 8];11:27-31. Available from: https://www.ijabmr.org/text.asp?2021/11/1/27/307987 |
| Introduction | |  |
Assessing the postgraduates for various clinical skills at workplace is important though it needs a lot of resources and time. With a shift from traditional model to competency-based medical education, the role of formative assessment is paramount. This becomes all the more important in a specialty like psychiatry where interviewing and counseling skills are critical for successful clinical practice. However, medical students report that they are rarely being observed during patient encounters and one main reason for this is lack of faculty time.[1] Direct observation is mandatory for the reliable and valid assessment of interviewing and counseling skills.[2] It also requires multiple assessments over time.[3] In-training assessments done at the end of a term introduce a “halo effect.”[4] Mini-clinical evaluation exercise (mini-CEX) developed by the American Board of Internal Medicine helps to overcome these issues.[5] The mini-CEX involves direct observation of trainees in a focused clinical encounter followed by immediate feedback. Mini-CEX has not been used that often as a feedback tool in psychiatry, probably because of lack of awareness and limited resources. To best of our efforts, we could identify only three studies in this regard.[6],[7],[8] Hence, we planned to take up this study.
| Materials and Methods | | |
After obtaining the ethical approval from the Institutional Ethics Committee and written informed consent from the participants, the study was conducted from April 2019 to September 2019 in the Department of Psychiatry, Pt. BD Sharma PGIMS, Rohtak. The department has an annual intake of ten postgraduate students for M. D Psychiatry course of 3 years.
For the purpose of this study, all the 2nd- and 3rd-year trainees (19) and faculty members (6) were invited to participate in this study. Consenting postgraduates (18) were sensitized and teaching faculty members (4) were trained to use mini-CEX and regarding the feedback process with the help of workshops conducted by the investigator (SS). Feedback from the participants was used to address any anticipated difficulties. A mock drill was conducted to pilot test the introduction of mini-CEX with 3 first year postgraduates. These residents were not involved in the main study nor was this data used in the analysis. A tentative roster of mini-CEX sessions was prepared, ensuring that the trainee rotates through all the 5 assessors (including investigator) and that no two consecutive sessions are with the same assessor. The minimum time interval between two consecutive mini-CEX encounters was 2 weeks. One of the postgraduates faced only 3 mini-CEX encounters of diagnostic skills and left the institute. After the completion of the study, feedback from faculty members and postgraduate students was gathered using the feedback forms specially designed for the purpose and validated by the senior faculty members who did not participate in the study and medical education unit members. Postgraduates were asked to complete another scale also, Learning Self-Efficacy Scale for clinical skills.[9]
Permission to use these scale has been granted by the authors.
Data collected from mini-CEX encounters and feedback from faculty members and students were analyzed using SPSS. Descriptive statistics were generated for each of the domains on the mini-CEX. Mean differences between the sessions were assessed using a paired sample t-test.
| Results | | |
Every postgraduate (except one student as mentioned earlier) faced 9 mini-CEX encounters: 3 encounters for diagnostic skills and 6 encounters for counseling skills. A total of 156 mini-CEX encounters were carried out by 5 faculty members for 17 (18) postgraduates. Level of case complexity was based on the clinical expertise and judgment of the assessing clinician [Table 1].
[Table 2] shows the statistical comparison between various sessions of mini-CEX on two different competencies, i.e., diagnostic skills (interviewing and mental state examination) and counseling skills. A comparison of sessions 1 and 3 of mini-CEX for diagnostic skills shows that there was no statistically significant difference between 2 sessions except in the domain of evaluator satisfaction. | Table 2: Comparison of mean scores for diagnostic skills assessment sessions and for counseling skills assessment sessions
Click here to view |
However, a comparison of session 1 with session 6 of counseling skills shows a statistically significant difference among all the domains.
[Figure 1] and [Figure 2] depict the residents and faculty feedback on mini-CEX. In response to open-ended questions, 61% of the postgraduates mentioned that the mini-CEX was a better method of teaching as compared to other methods, 36% felt that it improved their clinical skills, 64% reported a boost in their confidence, and 89% looked forward to these encounters. Majority of the postgraduates did not find any drawback in the exercise though around a third (28%) of them reported experiencing anxiety and stress in the initial parts of assessment. A significant proportion (88%) of residents suggested that the mini-CEX should be regularly used in academic settings. On a similar note, most of the consultants who were already trained in various assessment exercises found that mini-CEX was better than the traditional assessment methods such as outpatient department and bedside case discussion or case conference as it provided immediate feedback (80%), remains focused on the task assigned for the encounter (60%), and has a structured pattern of evaluation (60%). However, subjective bias and time constraints for assessment were considered to be significant drawbacks by 40% of the faculty members. Despite these shortcomings, most of the faculty members (70%) recommended mini-CEX as a good formative assessment tool and that it should be introduced early in the training. [Table 3] shows the positive perception of residents in all three domains on the Learning Self-Efficacy Scale. [Table 3]. | Figure 1: Faculty feedback on mini-clinical evaluation exercise . *Questions that were negative and scores were reversed
Click here to view |
 | Figure 2: Student feedback on mini.clinical evaluation exercise (n = 17). *Questions that were negative and scores were reversed
Click here to view |
| Discussion | | |
The direct observation-cum-evaluation approach is an effective way of learning for residents, particularly in the specialty of psychiatry, where diagnostic as well as communication skills are of paramount importance. Direct observation and timely feedback lead to a significant improvement of the clinical skills of the trainee residents, and findings of the index study support this assertion.
Because of the time constraints, we could focus only on two competencies, i.e., diagnosis and counseling skills. It was noticed that the difference between the mean score for each of the items on mini-CEX for session 1 and session 6 of counseling skill assessment was statistically significant. Findings of the index study corroborate that of Gupta et al.[10] on pediatric residents and Saeed et al.[11] on undergraduates. Khalil et al.[12] in their study with pediatric residents drew similar inference and concluded that it was feasible to use mini-CEX as a formative assessment tool with high acceptability among both faculty and residents.
However, the difference of mean scores between session 1 and session 3 of diagnostic skill assessment was found to be statistically significant only in the area of evaluator satisfaction. The absence of significant difference in mean score for most of the items on diagnostic skill evaluation can be attributed to insufficient number of sessions for this competency. It can be possible that a few more sessions could have led to better diagnostic skills and hence significant difference in most of the items on mini-CEX. Alves de Lima et al.[13] used mini-CEX for 17? cardiology residents and suggested that although the tool was valid and reliable, it was not feasible. It was evident in their study that it was not easy to achieve the number of encounters required and a greater number of sessions are required to mark a difference as has been suggested by the developers of the tool.[5]
The feedback from the participants reflects that both the students and faculty accepted the mini-CEX very well. It is further corroborated by the feedback? of students wherein they looked forward to mini-CEX encounters as well as desired for more of these. Although some of the residents (28%) reported initial anxiety and stress, they became comfortable as the encounter progressed. These students wanted to continue with further encounters, and most of the students agreed that this exercise helped them to learn better and improved their clinical skills. A significant proportion (88.8%) of residents suggested that the mini-CEX should be regularly used in academic settings.
Most of the faculty members who were already trained in various assessment exercises found that mini-CEX was better as it provided immediate feedback (80%), has focused (60%) and structured pattern of evaluation (60%), and should be introduced early in the training. However, subjective bias and time constraints for assessment were perceived to be significant drawbacks by both faculty and students, which was in line with the results of previous studies.[10],[13],[14] However, the completion of all the planned encounters proves its feasibility. Regular use, constant reinforcement, and positive attitude can overcome these constraints.
This is further corroborated by the perceptions of the residents on the Learning Self-efficacy Scale, which strongly suggested improvement in clinical skills in all the domains – cognitive, affective, and psychomotor. It underlines the perception of improvement in self-skills with mini-CEX among residents, which justifies its utility as suggested by other studies also.[10],[15] It can be, therefore, implied that mini-CEX as a tool has good acceptability and feasibility, as suggested by the feedback responses, and this substantiates the findings of earlier studies.[6],[7],[8],[10],[16],[17]
| Conclusions | | |
The study concludes that it is feasible to use mini-CEX for formative assessment of postgraduates in psychiatry. Furthermore, it was considered to be an effective method of learning by the postgraduates and was found to induce a significant improvement in the counseling skills of students. Teaching faculty also accepted mini-CEX as an effective method of formative assessment.
The strength of the index study lies in the fact that it is for the first time in this part of India, and the faculty and residents were exposed to workplace-based method of formative assessment using a wide diversity of psychiatric diagnoses. However, due to time constraint, the number of mini-CEX encounters for each competency was restricted and assessment could be made only in two competencies.
We continue to use mini-CEX for formative assessment as a routine practice in the department and plan to introduce mini-CEX as a method of formative assessment as early as possible in the training.
Acknowledgment
We acknowledge the guidance and assistance provided by the CMCL FAIMER Team onsite and online during the Progress Report, all the 2017 and 2018 Fellows, and my fellow colleagues. We extend special thanks to Dr. Dinesh Badyal and Dr. Anil Kapoor for constant support and guidance. We thank Dr. R. Gupta, Head, Department of Psychiatry, for the administrative support. Enthusiastic and unconditional participation by the participating faculty and year 2 and year 3 postgraduates is highly appreciated.
Ethical clearance
Study was conducted after approval from the Institutional Ethics Committee.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | Fromme HB, Karani R, Downing SM. Direct observation in medical education: A review of the literature and evidence for validity. Mt Sinai J Med 2009;76:365-71.  |
| 2. | Holmboe ES, Hawkins RE, Huot SJ. Effects of training in direct observation of medical residents' clinical competence: A randomized trial. Ann Intern Med 2004;140:874-81. |
| 3. | Williams RG, Dunnington GL, Klamen DL. Forecasting residents' performance-Partly cloudy. Acad Med 2005;80:415-22. |
| 4. | Wilkinson TJ, Wade WB. Problems with using a supervisor's report as a form of summative assessment. Postgrad Med J 2007;83:504-6. |
| 5. | Norcini JJ, Blank LL, Arnold GK, Kimball HR. The mini-CEX (clinical evaluation exercise): A preliminary investigation. Ann Intern Med 1995;123:795-9. |
| 6. | Kim S, Willett LR, Noveck H, Patel MS, Walker JA, Terregino CA. Implementation of a mini-CEX requirement across all third-year clerkships. Teach Learn Med 2016;28:424-31. |
| 7. | Wang PW, Cheng CC, Chou FH, Tsang HY, Chang YS, Huang MF, et al. Using multiple assessments to evaluate medical students' clinical ability in psychiatric clerkships. Acad Psychiatry 2011;35:307-11. |
| 8. | Meresh E, Daniels D, Sharma A, Rao M, Mehta K, Schilling D. Review of mini-clinical evaluation exercise (mini-CEX) in a psychiatry clerkship. Adv Med Educ Pract 2018;9:279-83. |
| 9. | Kang YN, Chang CH, Kao CC, Chen CY, Wu CC. Development of a short and universal learning self-efficacy scale for clinical skills. PLoS One 2019;14:e0209155. |
| 10. | Gupta S, Sharma M, Singh T. The acceptability and feasibility of mini-clinical evaluation exercise as a learning tool for pediatric postgraduate students. Int J Appl Basic Med Res 2017;7:S19-22. |
| 11. | Saeed NS, Tariq N, Jaffery T. Mini-CEX (clinical evaluation exercise) as an assessment tool at Shifa College of Medicine, Islamabad, Pakistan. Rawal Med J. 2015;40:220-4. |
| 12. | Khalil S, Aggarwal A, Mishra D. Implementation of a mini-clinical evaluation exercise (Mini-CEX) program to assess the clinical competence of postgraduate trainees in pediatrics. Indian Pediatr 2017;54:284-7. |
| 13. | Alves de Lima A, Barrero C, Baratta S, Castillo Costa Y, B?ortman G, Carabajales J, et al. Validity, reliability, feasibility and satisfaction of the mini-clinical evaluation exercise (mini-CEX) for cardiology residency training. Med Teach 2007;29:785-90. |
| 14. | Singh T, Sharma M. Mini-clinical examination (CEX) as a tool for formative assessment. Natl Med J India 2010;23:100-2. |
| 15. | Miller A, Archer J. Impact of workplace based assessment on doctors' education and performance: A systematic review. BMJ 2010;341:c5064. |
| 16. | Sidhu RS, Hatala R, Barron S, Broudo M, Pachev G, Page G. Reliability and acceptance of the mini-clinical evaluation exercise as a performance assessment of practicing physicians. Acad Med 2009;84:S113-5. |
| 17. | Kogan JR, Bellini LM, Shea JA. Feasibility, reliability, and validity of the mini-clinical evaluation exercise (mCEX) in a medicine core clerkship. Acad Med 2003;78:S33-5. |
[Figure 1], [Figure 2]
[Table 1], [Table 2], [Table 3]
|
