Home About us Editorial board Search Ahead of print Current issue Archives Submit article Instructions Subscribe Contacts Login 
  Users Online: 487 Home Print this page Email this page Small font sizeDefault font sizeIncrease font size  

 Table of Contents  
ORIGINAL ARTICLE
Year : 2021  |  Volume : 11  |  Issue : 1  |  Page : 27-31  

Mini-clinical evaluation exercise as a tool for formative assessment of postgraduates in psychiatry


Department of Psychiatry, Pt. B.D. Sharma PGIMS, Rohtak, Haryana, India

Date of Submission29-May-2020
Date of Decision12-Jun-2020
Date of Acceptance10-Sep-2020
Date of Web Publication26-Jan-2021

Correspondence Address:
Sujata Sethi
122/8 Shivaji Colony, Rohtak - 124 001, Haryana
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/ijabmr.IJABMR_305_20

Rights and Permissions
   Abstract 


Context: Introduction of competency-based medical education in India has necessitated a diversion from traditional methods of teaching and assessment. Aim: The aim of this study was to assess the feasibility and acceptability of mini-clinical evaluation exercise (mini-CEX) as a tool for formative assessment of postgraduates in psychiatry at our institute. Settings and Design: It was a prospective, quasi-experimental study carried out in the Department of Psychiatry, PGIMS, Rohtak (India). Materials and Methods: After obtaining written informed consent, trainees (18) were sensitized and faculty members (5) were trained to use mini-CEX and regarding the feedback process with the help of workshops. Statistical Analysis Used: Data collected were analyzed using descriptive statistics. Mean differences between the sessions were assessed using a paired sample t-test. P < 0.05 was considered as level of significance. Results: Statistically significant difference was found between session 1 and session 6 for counseling skills but not for three sessions of diagnostic skills. Mini-CEX was accepted as a better learning method by 61% of the postgraduates, 36% felt that it improved their clinical skills, 64% reported a boost in their confidence, and 89% looked forward to these encounters. Majority of the postgraduates and faculty members suggested that the mini-CEX should be regularly used in academic settings and should be introduced early in the training. Conclusions: Mini-CEX is a feasible formative assessment tool for postgraduates in psychiatry and an effective method of learning by the postgraduates, inducing a significant improvement in the counseling skills of students. Teaching faculty also accepted mini-CEX as an effective method of formative assessment.

Keywords: Formative assessment, mini-clinical evaluation exercise, postgraduates, psychiatry


How to cite this article:
Sethi S, Srivastava V, Verma P. Mini-clinical evaluation exercise as a tool for formative assessment of postgraduates in psychiatry. Int J App Basic Med Res 2021;11:27-31

How to cite this URL:
Sethi S, Srivastava V, Verma P. Mini-clinical evaluation exercise as a tool for formative assessment of postgraduates in psychiatry. Int J App Basic Med Res [serial online] 2021 [cited 2021 Jun 24];11:27-31. Available from: https://www.ijabmr.org/text.asp?2021/11/1/27/307987




   Introduction Top


Assessing the postgraduates for various clinical skills at workplace is important though it needs a lot of resources and time. With a shift from traditional model to competency-based medical education, the role of formative assessment is paramount. This becomes all the more important in a specialty like psychiatry where interviewing and counseling skills are critical for successful clinical practice. However, medical students report that they are rarely being observed during patient encounters and one main reason for this is lack of faculty time.[1] Direct observation is mandatory for the reliable and valid assessment of interviewing and counseling skills.[2] It also requires multiple assessments over time.[3] In-training assessments done at the end of a term introduce a “halo effect.”[4] Mini-clinical evaluation exercise (mini-CEX) developed by the American Board of Internal Medicine helps to overcome these issues.[5] The mini-CEX involves direct observation of trainees in a focused clinical encounter followed by immediate feedback. Mini-CEX has not been used that often as a feedback tool in psychiatry, probably because of lack of awareness and limited resources. To best of our efforts, we could identify only three studies in this regard.[6],[7],[8] Hence, we planned to take up this study.


   Materials and Methods Top


After obtaining the ethical approval from the Institutional Ethics Committee and written informed consent from the participants, the study was conducted from April 2019 to September 2019 in the Department of Psychiatry, Pt. BD Sharma PGIMS, Rohtak. The department has an annual intake of ten postgraduate students for M. D Psychiatry course of 3 years.

For the purpose of this study, all the 2nd- and 3rd-year trainees (19) and faculty members (6) were invited to participate in this study. Consenting postgraduates (18) were sensitized and teaching faculty members (4) were trained to use mini-CEX and regarding the feedback process with the help of workshops conducted by the investigator (SS). Feedback from the participants was used to address any anticipated difficulties. A mock drill was conducted to pilot test the introduction of mini-CEX with 3 first year postgraduates. These residents were not involved in the main study nor was this data used in the analysis. A tentative roster of mini-CEX sessions was prepared, ensuring that the trainee rotates through all the 5 assessors (including investigator) and that no two consecutive sessions are with the same assessor. The minimum time interval between two consecutive mini-CEX encounters was 2 weeks. One of the postgraduates faced only 3 mini-CEX encounters of diagnostic skills and left the institute. After the completion of the study, feedback from faculty members and postgraduate students was gathered using the feedback forms specially designed for the purpose and validated by the senior faculty members who did not participate in the study and medical education unit members. Postgraduates were asked to complete another scale also, Learning Self-Efficacy Scale for clinical skills.[9]

Permission to use these scale has been granted by the authors.

Data collected from mini-CEX encounters and feedback from faculty members and students were analyzed using SPSS. Descriptive statistics were generated for each of the domains on the mini-CEX. Mean differences between the sessions were assessed using a paired sample t-test.


   Results Top


Every postgraduate (except one student as mentioned earlier) faced 9 mini-CEX encounters: 3 encounters for diagnostic skills and 6 encounters for counseling skills. A total of 156 mini-CEX encounters were carried out by 5 faculty members for 17 (18) postgraduates. Level of case complexity was based on the clinical expertise and judgment of the assessing clinician [Table 1].
Table 1: Basic details of mini-clinical evaluation exercise encounters

Click here to view


[Table 2] shows the statistical comparison between various sessions of mini-CEX on two different competencies, i.e., diagnostic skills (interviewing and mental state examination) and counseling skills. A comparison of sessions 1 and 3 of mini-CEX for diagnostic skills shows that there was no statistically significant difference between 2 sessions except in the domain of evaluator satisfaction.
Table 2: Comparison of mean scores for diagnostic skills assessment sessions and for counseling skills assessment sessions

Click here to view


However, a comparison of session 1 with session 6 of counseling skills shows a statistically significant difference among all the domains.

[Figure 1] and [Figure 2] depict the residents and faculty feedback on mini-CEX. In response to open-ended questions, 61% of the postgraduates mentioned that the mini-CEX was a better method of teaching as compared to other methods, 36% felt that it improved their clinical skills, 64% reported a boost in their confidence, and 89% looked forward to these encounters. Majority of the postgraduates did not find any drawback in the exercise though around a third (28%) of them reported experiencing anxiety and stress in the initial parts of assessment. A significant proportion (88%) of residents suggested that the mini-CEX should be regularly used in academic settings. On a similar note, most of the consultants who were already trained in various assessment exercises found that mini-CEX was better than the traditional assessment methods such as outpatient department and bedside case discussion or case conference as it provided immediate feedback (80%), remains focused on the task assigned for the encounter (60%), and has a structured pattern of evaluation (60%). However, subjective bias and time constraints for assessment were considered to be significant drawbacks by 40% of the faculty members. Despite these shortcomings, most of the faculty members (70%) recommended mini-CEX as a good formative assessment tool and that it should be introduced early in the training. [Table 3] shows the positive perception of residents in all three domains on the Learning Self-Efficacy Scale. [Table 3].
Figure 1: Faculty feedback on mini-clinical evaluation exercise . *Questions that were negative and scores were reversed

Click here to view
Figure 2: Student feedback on mini.clinical evaluation exercise (n = 17). *Questions that were negative and scores were reversed

Click here to view
Table 3: Learning Self.Efficacy Scale for clinical skills

Click here to view



   Discussion Top


The direct observation-cum-evaluation approach is an effective way of learning for residents, particularly in the specialty of psychiatry, where diagnostic as well as communication skills are of paramount importance. Direct observation and timely feedback lead to a significant improvement of the clinical skills of the trainee residents, and findings of the index study support this assertion.

Because of the time constraints, we could focus only on two competencies, i.e., diagnosis and counseling skills. It was noticed that the difference between the mean score for each of the items on mini-CEX for session 1 and session 6 of counseling skill assessment was statistically significant. Findings of the index study corroborate that of Gupta et al.[10] on pediatric residents and Saeed et al.[11] on undergraduates. Khalil et al.[12] in their study with pediatric residents drew similar inference and concluded that it was feasible to use mini-CEX as a formative assessment tool with high acceptability among both faculty and residents.

However, the difference of mean scores between session 1 and session 3 of diagnostic skill assessment was found to be statistically significant only in the area of evaluator satisfaction. The absence of significant difference in mean score for most of the items on diagnostic skill evaluation can be attributed to insufficient number of sessions for this competency. It can be possible that a few more sessions could have led to better diagnostic skills and hence significant difference in most of the items on mini-CEX. Alves de Lima et al.[13] used mini-CEX for 17? cardiology residents and suggested that although the tool was valid and reliable, it was not feasible. It was evident in their study that it was not easy to achieve the number of encounters required and a greater number of sessions are required to mark a difference as has been suggested by the developers of the tool.[5]

The feedback from the participants reflects that both the students and faculty accepted the mini-CEX very well. It is further corroborated by the feedback? of students wherein they looked forward to mini-CEX encounters as well as desired for more of these. Although some of the residents (28%) reported initial anxiety and stress, they became comfortable as the encounter progressed. These students wanted to continue with further encounters, and most of the students agreed that this exercise helped them to learn better and improved their clinical skills. A significant proportion (88.8%) of residents suggested that the mini-CEX should be regularly used in academic settings.

Most of the faculty members who were already trained in various assessment exercises found that mini-CEX was better as it provided immediate feedback (80%), has focused (60%) and structured pattern of evaluation (60%), and should be introduced early in the training. However, subjective bias and time constraints for assessment were perceived to be significant drawbacks by both faculty and students, which was in line with the results of previous studies.[10],[13],[14] However, the completion of all the planned encounters proves its feasibility. Regular use, constant reinforcement, and positive attitude can overcome these constraints.

This is further corroborated by the perceptions of the residents on the Learning Self-efficacy Scale, which strongly suggested improvement in clinical skills in all the domains – cognitive, affective, and psychomotor. It underlines the perception of improvement in self-skills with mini-CEX among residents, which justifies its utility as suggested by other studies also.[10],[15] It can be, therefore, implied that mini-CEX as a tool has good acceptability and feasibility, as suggested by the feedback responses, and this substantiates the findings of earlier studies.[6],[7],[8],[10],[16],[17]


   Conclusions Top


The study concludes that it is feasible to use mini-CEX for formative assessment of postgraduates in psychiatry. Furthermore, it was considered to be an effective method of learning by the postgraduates and was found to induce a significant improvement in the counseling skills of students. Teaching faculty also accepted mini-CEX as an effective method of formative assessment.

The strength of the index study lies in the fact that it is for the first time in this part of India, and the faculty and residents were exposed to workplace-based method of formative assessment using a wide diversity of psychiatric diagnoses. However, due to time constraint, the number of mini-CEX encounters for each competency was restricted and assessment could be made only in two competencies.

We continue to use mini-CEX for formative assessment as a routine practice in the department and plan to introduce mini-CEX as a method of formative assessment as early as possible in the training.

Acknowledgment

We acknowledge the guidance and assistance provided by the CMCL FAIMER Team onsite and online during the Progress Report, all the 2017 and 2018 Fellows, and my fellow colleagues. We extend special thanks to Dr. Dinesh Badyal and Dr. Anil Kapoor for constant support and guidance. We thank Dr. R. Gupta, Head, Department of Psychiatry, for the administrative support. Enthusiastic and unconditional participation by the participating faculty and year 2 and year 3 postgraduates is highly appreciated.

Ethical clearance

Study was conducted after approval from the Institutional Ethics Committee.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.



 
   References Top

1.
Fromme HB, Karani R, Downing SM. Direct observation in medical education: A review of the literature and evidence for validity. Mt Sinai J Med 2009;76:365-71.  Back to cited text no. 1
    
2.
Holmboe ES, Hawkins RE, Huot SJ. Effects of training in direct observation of medical residents' clinical competence: A randomized trial. Ann Intern Med 2004;140:874-81.  Back to cited text no. 2
    
3.
Williams RG, Dunnington GL, Klamen DL. Forecasting residents' performance-Partly cloudy. Acad Med 2005;80:415-22.  Back to cited text no. 3
    
4.
Wilkinson TJ, Wade WB. Problems with using a supervisor's report as a form of summative assessment. Postgrad Med J 2007;83:504-6.  Back to cited text no. 4
    
5.
Norcini JJ, Blank LL, Arnold GK, Kimball HR. The mini-CEX (clinical evaluation exercise): A preliminary investigation. Ann Intern Med 1995;123:795-9.  Back to cited text no. 5
    
6.
Kim S, Willett LR, Noveck H, Patel MS, Walker JA, Terregino CA. Implementation of a mini-CEX requirement across all third-year clerkships. Teach Learn Med 2016;28:424-31.  Back to cited text no. 6
    
7.
Wang PW, Cheng CC, Chou FH, Tsang HY, Chang YS, Huang MF, et al. Using multiple assessments to evaluate medical students' clinical ability in psychiatric clerkships. Acad Psychiatry 2011;35:307-11.  Back to cited text no. 7
    
8.
Meresh E, Daniels D, Sharma A, Rao M, Mehta K, Schilling D. Review of mini-clinical evaluation exercise (mini-CEX) in a psychiatry clerkship. Adv Med Educ Pract 2018;9:279-83.  Back to cited text no. 8
    
9.
Kang YN, Chang CH, Kao CC, Chen CY, Wu CC. Development of a short and universal learning self-efficacy scale for clinical skills. PLoS One 2019;14:e0209155.  Back to cited text no. 9
    
10.
Gupta S, Sharma M, Singh T. The acceptability and feasibility of mini-clinical evaluation exercise as a learning tool for pediatric postgraduate students. Int J Appl Basic Med Res 2017;7:S19-22.  Back to cited text no. 10
    
11.
Saeed NS, Tariq N, Jaffery T. Mini-CEX (clinical evaluation exercise) as an assessment tool at Shifa College of Medicine, Islamabad, Pakistan. Rawal Med J. 2015;40:220-4.  Back to cited text no. 11
    
12.
Khalil S, Aggarwal A, Mishra D. Implementation of a mini-clinical evaluation exercise (Mini-CEX) program to assess the clinical competence of postgraduate trainees in pediatrics. Indian Pediatr 2017;54:284-7.  Back to cited text no. 12
    
13.
Alves de Lima A, Barrero C, Baratta S, Castillo Costa Y, B?ortman G, Carabajales J, et al. Validity, reliability, feasibility and satisfaction of the mini-clinical evaluation exercise (mini-CEX) for cardiology residency training. Med Teach 2007;29:785-90.  Back to cited text no. 13
    
14.
Singh T, Sharma M. Mini-clinical examination (CEX) as a tool for formative assessment. Natl Med J India 2010;23:100-2.  Back to cited text no. 14
    
15.
Miller A, Archer J. Impact of workplace based assessment on doctors' education and performance: A systematic review. BMJ 2010;341:c5064.  Back to cited text no. 15
    
16.
Sidhu RS, Hatala R, Barron S, Broudo M, Pachev G, Page G. Reliability and acceptance of the mini-clinical evaluation exercise as a performance assessment of practicing physicians. Acad Med 2009;84:S113-5.  Back to cited text no. 16
    
17.
Kogan JR, Bellini LM, Shea JA. Feasibility, reliability, and validity of the mini-clinical evaluation exercise (mCEX) in a medicine core clerkship. Acad Med 2003;78:S33-5.  Back to cited text no. 17
    


    Figures

  [Figure 1], [Figure 2]
 
 
    Tables

  [Table 1], [Table 2], [Table 3]



 

Top
 
 
  Search
 
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
    Abstract
   Introduction
    Materials and Me...
   Results
   Discussion
   Conclusions
    References
    Article Figures
    Article Tables

 Article Access Statistics
    Viewed427    
    Printed6    
    Emailed0    
    PDF Downloaded89    
    Comments [Add]    

Recommend this journal