Home About us Editorial board Search Ahead of print Current issue Archives Submit article Instructions Subscribe Contacts Login 
  Users Online: 245 Home Print this page Email this page Small font sizeDefault font sizeIncrease font size  

 Table of Contents  
ORIGINAL ARTICLE
Year : 2015  |  Volume : 5  |  Issue : 4  |  Page : 66-70  

Formative assessment in postgraduate medical education - Perceptions of students and teachers


1 Department of Pediatrics, Dr. Rajendra Prasad Government Medical College and Hospital, Tanda, Kangra, Himachal Prades, India
2 Department of Orthopedics, Dr. Rajendra Prasad Government Medical College and Hospital, Tanda, Kangra, Himachal Pradesh, India

Date of Submission16-Mar-2015
Date of Acceptance26-Jun-2015
Date of Web Publication5-Aug-2015

Correspondence Address:
Dr. Seema Sharma
H. No. 23, Type 5, Block B, Dr. RPGMCH, Tanda, Kangra, Himachal Pradesh
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/2229-516X.162282

Rights and Permissions
   Abstract 

Context: One of the most important factors of medical education that can revolutionize the learning process in postgraduate students (PGs) is assessment for learning by means of formative assessment (FA). FA is directed at steering and fostering learning of the students by providing feedback to the learner. However, though theoretically well suited to postgraduate training, evidence are emerging that engaging stakeholders in FA in daily clinical practice is quite complex. Aims: To explore perceptions of PGs and teachers (Ts) about factors that determines active engagement in FA. Subjects and Methods: It was a descriptive qualitative study involving focus group discussions (FGDs) with PGs and Ts from Departments of Pediatrics and Orthopedics. FGDs data were processed through points/remarks, data reduction, data display, coding followed by theme generation for content analysis. Results: Four higher order themes emerged: Harsh reality of present summative assessment structure, individual perspectives on feedback, supportiveness of the learning environment, and the credibility of feedback and/or feedback giver. Conclusions: Engaging in FA with a genuine impact on learning is complex and quite a challenge to both students and Ts. Increased acceptability along with the effective implementation of FA structure, individual perspectives on feedback, a supportive learning environment and credibility of feedback are all important in this process. Every one of these should be taken into account when the utility of FA in postgraduate medical training is evaluated.

Keywords: Focus group discussion, formative assessment, students, summative assessment, teachers


How to cite this article:
Sharma S, Sharma V, Sharma M, Awasthi B, Chaudhary S. Formative assessment in postgraduate medical education - Perceptions of students and teachers. Int J App Basic Med Res 2015;5, Suppl S1:66-70

How to cite this URL:
Sharma S, Sharma V, Sharma M, Awasthi B, Chaudhary S. Formative assessment in postgraduate medical education - Perceptions of students and teachers. Int J App Basic Med Res [serial online] 2015 [cited 2019 Dec 6];5, Suppl S1:66-70. Available from: http://www.ijabmr.org/text.asp?2015/5/4/66/162282


   Introduction Top


It promotes learning by providing feedback to the learner [1] resulting in an efficient postgraduate training along with transparent and credible assessment. Consequentially, the role of purely summative assessment (SA), at the end of the training period is waning and FA, is gaining ground, thereby facilitating its implementation in many postgraduate training curricula worldwide. [2] However, whereas in SA, validity and reliability are seen as dominant determinants of utility, in FA, utility, defined as learning that results from the assessment process, is much more dependent on how stakeholders that is, PGs and Ts employ the instrument in practice. [3]

So far, few studies have addressed the issue of the effect of FA on doctors learning and performance. [4] Moreover, it is becoming increasingly clear that, even though FA is theoretically well suited to postgraduate medical training, engaging both PGs and Ts in meaningful FA is quite complex. [5] We set out to qualitatively explore PGs and Ts perceptions on what factors determine active engagement in FA in postgraduate medical education in India. Both PGs and Ts views were sought because engaging in meaningful FA requires efforts from both feedback giver and feedback recipient. [6] Our objective has been to formulate the themes generated by focus group discussion (FGD) regarding perceptions of FA.


   Subjects and Methods Top


This was a descriptive qualitative study conducted over 6 months in a rural medical college in a hilly terrain, where postgraduate training comprises a full-time training program lasting 3 years. Training consists of both in- and out-patients services supervised by faculty. Although this study was exempted from ethical approval, considerable effort was taken to protect the interest of participants; participants were informed about the voluntary nature of participating in the study, about the aim of the study and that data would be analyzed anonymously. Twenty-three PGs of all 3 academic years and 14 Ts were enrolled from Departments of Pediatrics and Orthopedics of our institute. To create an optimally safe environment, FGDs with PGs and Ts were held separately. Ten focus group sessions were held, four with Ts and six with PGs. FGDs for PGs of both the departments were held separately. Each group of FGD consisted of PGs of the single academic session from the same department. The group sizes of the FGDs with PGs and Ts ranged from 3 to 5 participants.

A FGD approach rather than in-depth interviews or questionnaires was preferred, as this would provide more information, make interconnections visible and perhaps even trigger the formulation of new ideas or theories on the subject. At the start of each FGD session, all participants were informed on the purpose of the study and guaranteed full confidentiality. For FGD in Department of Pediatrics, the moderator was from the same department, and the same moderator moderated the FGD in Department of Orthopedics too. The moderator was not involved directly or indirectly in the assessment process. Moderator initiated the discussion using a predefined list of nine questions for guidance [Table 1]. The first questions were meant to elucidate the current assessment structure, followed by questions on expectations and needs regarding assessment for learning. FGDs data were analyzed using QDA Minor 4 Lite and was processed through points/remarks, data reduction, data display, coding followed by theme generation for content analysis.
Table 1: Questions-focus group PGs/focus group Ts


Click here to view



   Results Top


After content analysis, four higher order themes emerged, explaining the level of active engagement in FA by PGs and Ts-harsh reality of present SA structure, individual perspectives on feedback, and supportiveness of the learning environment and the credibility of feedback and/or feedback giver [Table 2]. We have presented the results, interlaced with distinctive quotes, organized according to these themes. Quotes have been coded as follows: PG denotes postgraduate student, T denotes teacher, 1-3 refer to training year, and P and O refer to Pediatric and Orthopedic Departments.
Table 2: Key themes of FGDs


Click here to view


Harsh reality of present summative assessment structure

In spite of guidelines, still there is thinking of old era that being in good books of someone is the only key to pass the examination. Fear of failure and untoward consequences instead of zeal for knowledge and constant improvement is the driving force for a graduate or PG. The ground reality is that actively engaging in seeking and obtaining feedback does not prevail in the system. The likely causes are lack of acceptability, poor implementation, and poor sensitization of faculty toward FA. Some of the comments read:

  • Negative impression is always fatal for students (PG2-O)
  • I do not think that my supervisor is well informed on how I'm progressing in my training (PG3-P).


Lack of acceptability

Lack of acceptability of FA is prevailing in the system due to the centralized decision-making structure in the departments.

Poor implementation

FA is difficult to implement due to time constrains, scarcity of faculty and inbuilt inertia to a possible change by coming out of their comfort zone of SA structure.

Lack of faculty sensitization

Lack of faculty sensitization is one of the reasons for inadequate assessment as there are shortcomings in providing hands-on training workshops on FA.

When Ts were asked regarding present assessment structure mostly referred to specific guidelines and little was revealed on their individual perspective on it.

Individual perspectives on feedback

During the FGDs, it became clear that the individual perspectives on feedback were largely determined by ownership and goal orientation.

Ownership

Ownership can best be described as the belief that making the most of one's training period is a personal responsibility.

I strongly believe that it is my training, and thus my responsibility to ask for feedback (PG3-O).

Typically, feelings of ownership grew with years of experience; PGs in the beginning of specialist training were initially more focused on learning how to cope with the new working environment and were less actively involved in their own training pathway.

In the beginning, you tend to have a more consuming attitude (PG1-P).

However, once training progressed, their competence grew, and transition to progressive independence was set in motion. At the same time, growing awareness of their personal responsibility to make the most of training resulted in more self-reflection and active search for learning opportunities and feedback.

I do feel that my ability to self-reflect has grown during my training. It has happened in my 2nd year and I'm more aware that it is important to select appropriate learning goals and that I'm actively engaged in getting there (PG3-P).

Ts were of the opinion that how important it is to them that PGs show ownership right from the start of training.

You would expect them to study spontaneously, just being involved in patient care should provide enough incentives to start looking things up (T-O).

Goal orientation

Both PGs and Ts expressed varied assessment and feedback preferences. These seemed to be determined by their achievement goal orientation (performance- or mastery-orientated). Performance orientated PGs and Ts preferred SA in which competence is assessed against a predefined standard, and in which failing has clear consequences.

I would prefer good old knowledge exams: Clear study materials, clear pass/fail standards and clear consequences. It helps me to start studying and in this way I know once I have mastered a subject (PG3-P).

I would prefer rigorous assessment system with clear consequences. In my experience PGs study most when they get targeted assignments followed by assessment. They like it when they notice that they have actually mastered a subject (T-O).


Mastery-orientated PGs and Ts are more predisposed toward self-assessment and/or FA. They believe that learning is stimulated by feedback, self-reflection, coming back on issues and personal coaching.

To me, all feedback is valuable; I think you can use all information one way or another on your way to medical expertise (PG2-P).

To do postgraduation, one has to be very well motivated. I sincerely wonder whether more SA will increase performance for this group of motivated people (T-P).

Learning environment

The perception of the learning environment was another important feature when active engagement in assessment for learning was discussed. The term "learning environment" in this study means encouraging Ts, clear assessment procedures and supportive learning and work culture.

Committed Ts

PGs frequently stressed the importance of committed supervisors, who are interested in teaching as well as in developing their teaching skills.

You can notice which supervisors are really teaching-minded: They tend to teach the T courses, prepare themselves and give structured feedback (PG2-O).

When Ts were involved in-training assessment, mentoring skills became another precious asset. It transpired that the ability to approach PGs with a genuine interest in their long-time progression, both in their career and private lives was sorely missed by the PGs.

I feel that T should be interested in his PGs and should be well informed on their progress and the competencies that they have achieved (PG2-P).

Clear standards and consequences

Furthermore, both PGs and Ts expressed the need for clear standards and clarity on the consequences of substandard performance. Several PGs mentioned devaluation and/or disregard of feedback as a result of the absence of these.

If you are being assessed with the purpose to stimulate learning and the result of the assessment is without consequences, the impact will be disappointing (PG2-O).

Acknowledgment and appreciation

Acknowledgment of the importance of clinical teaching, dedicated teaching time for PGs and Ts was stressed as an important enabler of FA.

And you need dedicated staff and time, because in present setup, you have to go and do something else before you even had the opportunity to give feedback (T-P).

Appreciation/reward gives you stimulus to excel (PG2-O).

Credibility of feedback

Not all feedback automatically translated into learning. For this, the credibility of both feedback content and feedback giver were of paramount importance. When these were not credible enough, feedback was often rejected and consequentially did not result in the intended learning. The credibility of feedback content depended on issues like authenticity (does the feedback relate to a representative, directly observed doctor-patient encounter), and whether feedback can be judged against a clear, well accepted standard (e.g. guideline, latest research).

There is no problem to get some advice of a T on a patient problem; however, usually I get one, without him seeing the patient. I would like to get some structured feedback after being observed with the patient (PG1-O).

Apart from feedback content, personality traits and feedback strategies were other important determinants of the credibility of Ts. Feedback from a T who is perceived as a role model, well respected, enthusiastic passionate about his subject, encouraging to PGs was valued most. Especially, if this person was also able to provide structure during feedback sessions and remembered when and how to come back on issues.

Especially someone whom I personally regard as an exemplary doctor. If I see he/she is a professional in a way that I would like to be in the future. That's the person from whom I prefer to get feedback (PG3-P).


   Discussion Top


After FGD of PGs and Ts on perception of FA the themes emerged were, harsh reality of present assessment structure, individual perspectives on feedback of trainees and supervisors, a supportive learning environment and credible feedback.

Harsh reality of present summative assessment structure

Assessment has been taken rather casually in the present postgraduate medical education program. Ignoring educational principles while assessing students, merely because it results in more work, seriously compromises the utility and sanctity of assessment. [7] Faculty development programs and hands-on training workshops of the faculty on FA can bring transparency and credibility in the assessment of PGs.

Individual perspective on feedback

The individual perspective on feedback in this study is determined by both ownership and achievement goal orientation. Ownership is an internal drive to make the most of postgraduate training and act accordingly. [8] In our study, ownership of PGs plays a central role in both the motivation of a PGs to ask for feedback and the Ts willingness to start giving feedback. [9],[10] Progressive independence in the training years increases the awareness of personal responsibility along with an active approach of PGs to their learning pathway. However, supervisors expect ownership right from the start of training. It is, therefore, essential that the importance of ownership is explicitly discussed with junior PGs right from the start of training.

Ts revealed little information about their motivation to get actively engaged in FA. Available literature suggests that both a feeling of responsibility for the training of future doctors and beliefs arising from the Ts achievement goal orientation play a role. [11] This is in accordance with our finding that goal orientation is a main determinant of the assessment preference of both PGs and Ts. Whereas people with a mastery goal orientation tend to focus on acquiring and developing competence, taking all feedback as an opportunity to improve their learning, the focus of people with a performance-orientation tends to be on demonstrating one's competence and outperforming others; the latter usually valuing the clear standards and consequences of SA. [12] However, even though goal orientation itself may be difficult to modulate, awareness of both PGs and Ts of their personal achievement goal orientation should be stimulated as this will aid in customizing assessment and preventing frustrations on both sides.

Supportive learning environment

Supportive learning environment acts as an important facilitator in engaging in FA. The need for dedicated assessment moments and teaching time is must because time pressures induce a surface learning approach. [13]

Both PGs and Ts expressed a need for clear standards of performance in the combination with clear consequences for substandard performance. Here, the complex situation is that a doctor on the everyday work floor requires performances which are multi-dimensional, making it very difficult to explicate what good performance is, with the risk of getting lost in detailed, unrealistic lists full of desirable attitudes and skills. [14],[15] Meanwhile, there is a need for a continuing discussion on the minimal level of performance of a PG along with professional values and good clinical practice.

Credibility of feedback and/or feedback giver

Our participants particularly emphasize the importance of credible feedback and feedback givers as perceived by PGs and the importance of authentic assessment as perceived by Ts. However, as Ende already points out in 1983: Without feedback young doctors tend to develop a system of internal validation that excludes validation from external sources. [16] There is evidence that physicians are poor at self-assessment, making it imperative to develop a system of external evaluation of learner performance that participants trust and use. [17] FA is more than just giving feedback on a single occasion. For FA to exert an effect on learning an action plan follow-up and an opportunity to demonstrate improvement should be part of the process . [18]

Combining the perspectives of both trainees and supervisors made it possible to explore the roles of both stakeholders in the process. Using the focus group technique has provided us with rich quality data and made the complexity and multi-dimensionality of FA in daily clinical practice evident. The limitations of the study could be participants with strong views who share more information than others. This bias has been minimised by questions, answers, and reporting but still moderator bias to an extent is unavoidable.


   Conclusions Top


The educational impact of FA is multidimensional and actively engaging in assessment for learning is quite a challenge to both PGs and Ts. Honest and fair assessment structure will encourage PGs to excel. Ownership and achievement goal orientation determine active engagement in assessment for learning. Scheduled assessment moments and clear standards, procedures, and consequences facilitate assessment for learning.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

 
   References Top

1.
Schuwirth LW, Van der Vleuten CP. Programmatic assessment: From assessment of learning to assessment for learning. Med Teach 2011;33:478-85.  Back to cited text no. 1
    
2.
Irvine DH. Everyone is entitled to a good doctor. Med J Aust 2007;186:256-61.  Back to cited text no. 2
    
3.
van der Vleuten CP, Schuwirth LW. Assessing professional competence: From methods to programmes. Med Educ 2005;39:309-17.  Back to cited text no. 3
    
4.
Satpathy A, Kneebone R. Workplace-based assessment in surgical training: A qualitative study. Bull R Coll Surg Engl 2011;93:1-5.  Back to cited text no. 4
    
5.
Dijksterhuis M, Schuwirth L, Braat D, Scheele F. What′s the problem with the mini-CEX? Med Educ 2011;45:318-9.  Back to cited text no. 5
[PUBMED]    
6.
Cross V, Hicks C, Parle J, Field S. Perceptions of the learning environment in higher specialist training of doctors: Implications for recruitment and retention. Med Educ 2006;40:121-8.  Back to cited text no. 6
    
7.
Singh T, Anshu, Modi JN. The quarter model: A proposed approach for intraining assessment of undergraduate students in Indian medical schools. Indian Pediatr 2012;49:871-6.  Back to cited text no. 7
    
8.
Kusurkar RA, Ten Cate TJ, Vos CM, Westers P, Croiset G. How motivation affects academic performance: A structural equation modelling analysis. Adv Health Sci Educ Theory Pract 2013;18:57-69.  Back to cited text no. 8
    
9.
Veloski J, Boex JR, Grasberger MJ, Evans A, Wolfson DB. Systematic review of the literature on assessment, feedback and physicians′ clinical performance: BEME Guide No 7. Med Teach 2006;28:117-28.  Back to cited text no. 9
    
10.
Watling CJ, Kenyon CF, Zibrowski EM, Schulz V, Goldszmidt MA, Singh I, et al. Rules of engagement: Residents′ perceptions of the in-training evaluation process. Acad Med 2008;83:S97-100.  Back to cited text no. 10
    
11.
Cleveland JN, Lim AS, Murphy KR, editors. Feedback Phobia? Why Employees do Not Want to Give or Receive Performance Feedback. Cheltenham: Edward Elgar; 2007.  Back to cited text no. 11
    
12.
Senko C, Hulleman CS, Harackiewicz JM. Achievement goal theory at the crossroads: Old controversies, current challenges, and new directions. Educ Psychol 2011;46:26-47.  Back to cited text no. 12
    
13.
Struyven K, Dochy F, Janssens S, Segers M, Dochy F, Cascallar E. Students′ perceptions about new modes of assessment in higher education: A review. In: Gijselaers WH, Wilkerson LA, Boshuizen HP, editors. Optimising New Modes of Assessment: In Search of Qualities and Standards. Netherlands: Springer; 2003. p. 331-47.  Back to cited text no. 13
    
14.
Yorke M. Formative assessment in higher education: Moves toward theory and the enhancement of pedagogic practice. High Educ 2003;45:477-501.  Back to cited text no. 14
    
15.
Hatem CJ, Searle NS, Gunderman R, Krane NK, Perkowski L, Schutze GE, et al. The educational attributes and responsibilities of effective medical educators. Acad Med 2011;86:474-80.  Back to cited text no. 15
    
16.
Ende J. Feedback in clinical medical education. JAMA 1983;250:777-81.  Back to cited text no. 16
[PUBMED]    
17.
Nothnagle M, Anandarajah G, Goldman RE, Reis S. Struggling to be self-directed: Residents′ paradoxical beliefs about learning. Acad Med 2011;86:1539-44.  Back to cited text no. 17
    
18.
Black P, Wiliam D. Assessment and classroom learning. Assess Educ Princ Policy Pract 1998;5:7-74.  Back to cited text no. 18
    



 
 
    Tables

  [Table 1], [Table 2]



 

Top
 
 
  Search
 
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
    Abstract
   Introduction
   Subjects and Methods
   Results
   Discussion
   Conclusions
    References
    Article Tables

 Article Access Statistics
    Viewed1767    
    Printed28    
    Emailed0    
    PDF Downloaded331    
    Comments [Add]    

Recommend this journal