|Year : 2017 | Volume
| Issue : 5 | Page : 27-32
Critical review of “Family health advisory services” assessment in MBBS training program in community medicine
Kiran Goswami1, H Salve1, S Malhotra1, Y Kumar2
1 Centre for Community Medicine, AIIMS, New Delhi, India
2 CMET, AIIMS, New Delhi, India
|Date of Submission||04-May-2017|
|Date of Acceptance||20-Oct-2017|
|Date of Web Publication||20-Dec-2017|
Dr. Kiran Goswami
Room No 30, CCM, Old OT Block, AIIMS, Ansari Nagar, New Delhi
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Context: Family Health Advisory Services (FHAS) posting as well as its assessment is resource demanding but fails to enjoy priority. Study focuses on a holistic overview of the assessment process to understand need for change. Aims: The aim of this study is to identify perceived gaps in current assessment practices related to FHAS posting. Settings and Design: A cross-sectional mixed method study among all the V semester students currently undergoing assessment for the posting, past students (selected VII semester students and interns), preceptors (supervising residents – postgraduate students in department and senior resident, health assistants, medical social service officer), and involved faculty. Subject and Methods: Self-administered questionnaire, in-depth interview, focus group discussions (two) as well as observations using checklist were used for data collection and triangulation. Statistical Analysis Used: Quantitative data used in this study were statistical measures of central tendency and dispersion. Qualitative data transcript repeatedly read to identify underlying common themes, compared to draw inference. Results: There was a lack of guidelines and communication regarding assessment. Formative assessment was not performed and replaced by one time end assessment. All components of learning were not assessed. End-posting assessment was not standardized and unrelated to learning objectives. Award of scores was skewed toward right for intervention and toward left for analysis and community diagnosis. Conclusions: There is a need to focus on proper implementation of programme to strengthen formative assessment. Assessment should be relevant to learning objectives of posting. Faculty has to lead by example.
Keywords: Community medicine posting, family health advisory services assessment, field assessment, students' opinion
|How to cite this article:|
Goswami K, Salve H, Malhotra S, Kumar Y. Critical review of “Family health advisory services” assessment in MBBS training program in community medicine. Int J App Basic Med Res 2017;7, Suppl S1:27-32
|How to cite this URL:|
Goswami K, Salve H, Malhotra S, Kumar Y. Critical review of “Family health advisory services” assessment in MBBS training program in community medicine. Int J App Basic Med Res [serial online] 2017 [cited 2020 May 28];7, Suppl S1:27-32. Available from: http://www.ijabmr.org/text.asp?2017/7/5/27/221305
| Introduction|| |
Goal of medical education is to ensure that a medical graduate acquires public health competencies. The objectives of medical education include acquiring of skills to provide patient-centered primary health care at community level. Modifying and introducing learning experiences to expose students to real-life situations in community has been emphasized time and again.,,, Accordingly, MBBS curriculum includes community-oriented teaching–learning strategy with communities serving as “laboratories” for skill learning. Community medicine departments have taken major responsibility for community orientation of medical undergraduates. In most of medical colleges, a structured family or community attachment program is implemented.
At institute, structured family posting is a longitudinal weekly follow-up of families in community for a period of 9 months in the 4th and 5th semester of MBBS in form of Family Health Advisory Services (FHAS). The objective of the program is to study the changing health status of family members and health problems in family over a period of time to understand the interplay of environmental and social factors. A study of chronic morbidities aims to give them an insight to factors related to health-seeking behavior and compliance. Students develop confidence in communication with community under close guidance of preceptors. Preceptors are junior residents (mainly second and third semester postgraduate students in community medicine) as well as field workers (having experience of more than 7 years) and each preceptor works with 6–10 students.
During the posting, students are posted for 3 h a week in department. The posting involves classroom discussion and planning and community-based data collection and interventions as its two components. Each student is allotted 2–3 families. At the end of six structured exercises covering demographic, social, environmental, nutritional, under five care, and family welfare, they compile the data and analyze it to make community diagnosis. An appropriate community intervention program is planned and implemented to address identified issues under the guidance of faculty for Health Communication and Medical Social Services officer. At the end posting, a viva is conducted by pair of faculty in four batches.
This is very resource-demanding activity both for students and departmental faculty. FHAS assessment scores form a third of internal assessment and 17% of the total marks for the subject of community medicine at institute. FHAS assessment includes formative as well as summative assessment [Table 1]. Classroom participation is marked by senior resident. It reflects attendance of students in briefings for FHAS. Weighted scores are given on basis of attendance.Score for other components of formative assessment is marked by preceptors who are immediate supervisors of students for monitoring application of skills in community.
|Table 1: Components of existing family health advisory services assessment|
Click here to view
Informal feedback of the last few batches (exit interviews) revealed dissatisfaction of students with FHAS assessment. It was observed that some good students fared poorly in FHAS assessment. On discussion, students were found to be dissatisfied by current assessment and reported it as very “casual” and “ambiguous.” A strong assessment system motivates students to learn, and hence, there was a need to assess the current system.
A review of literature supported the existence of problems in assessment., If the current student assessment does not cover all domains (clinical, performance, and communication skills) and the contents do not match enlisted objectives, credibility of assessment is lost. The good thing is that problem can be followed with remedial action and scope for improvement exists.
Aim and objectives
- Identification of gaps in the current practices of FHAS assessment during Community Medicine posting as perceived by students, preceptors/supervisors, and faculty
- To identify remedial action for of FHAS assessment during Community Medicine posting.
| Subject and Methods|| |
A cross-sectional study with mixed method was carried out toward end of the FHAS posting in our department. Protocol was finalized with the guidance of FAIMER faculty and fellows, co-investigators, and head of the department. Ethical clearance was obtained from the Institute Ethics Committee.
Subjects included MBBS students of V and VII semester, interns, preceptors, and faculty involved in FHAS for the current year (2016). All students of VI semester were included in the study. Other students (VII semester and interns) were included as convenient. Batch of 18 VII semester students posted for rural posting as well as 11 interns posted in department were included with the aim of data triangulation.
Data were collected using self-designed pretested instruments, validated with help from FAIMER colleagues [Figure 1]. Information from each participant was collected only once.
Opinion of students of V th and VII th semester, interns, and preceptors was assessed from their scores on a scale of 1–10 for 15 statements related to characteristics of FHAS assessment. A cutoff value of 7 and above was decided to suggest an agreement to a particular statement. Proportion of students expressing agreement was tabulated for interpretation. Distribution of marks awarded was studied to identify pattern of scoring in different components by variety of people involved and address the issue of perceived concern about effect of unrelated factors on scores.
Data collected from participants were entered in Excel and analyzed after cleaning. Quantitative data from students, scores, and observation checklist were expressed in measures of central tendency, and dispersion. Qualitative data transcript from Interviews and FGDs was repeatedly read to identify underlying common themes and organized under common domains. Observations among stakeholders were compared to draw inference and get a comprehensive view on concerned issues.
| Results and Discussion|| |
The students were introduced to community under close guidance of preceptors. It was observed that junior residents had a high turnover during posting. Most of the times, preceptor happened to be a newly joined first year resident. Preceptors are verbally briefed about assessment at beginning of FHAS, but no orientation was planned for new preceptors joining in between. No written guidelines or training was imparted to preceptors for assessment.
Preceptors mark students for participation in field activities including data collection, communication with family, and family intervention. Summary score had to be submitted by preceptors at end of the posting based on continuous field assessment and monitoring of records. However, it was observed that marking for field activities and record books was done only at the end of posting. Although preceptors gave marks to the students for field work, many students reported that they are not assessed for attitude and skills (communication, interviewing, examination skills, interventions, and efforts at family level). Some students suggested that feedback from family should be a part of formal assessment. Faculty also supported field-based assessment instead of a theoretical end-posting viva of FHAS.
There is an inherent communication gap between students and residents. Almost all the residents are from other medical colleges with variable training. Residents said that the undergraduate students are comparatively more bright and analytic and ask for more information. Assessment of posting carries low priority as they get passing marks easily unless they deliberately put effort to fail.
Students, on the other hand, had concern about quality of field supervision and guidance from preceptor. Timing of end assessment is close to the second professional examinations resulting in low priority.
Students, preceptors as well as faculty identified lacuna in regular record checking. It is limited to end posting, resulting in forging the data for sake of completion of records. Hence, every record is viewed suspiciously by faculty.
Two more problems related to record books were identified - The interview schedules for data collection are framed and discussed in English. Over the years, the proportion of students from South India has increased to 30%–50% in the institute. Hence, communicating in Hindi becomes a major challenge. Students told that they attend briefing sessions for the sake of marking attendance. Most of the times, it is not explained why a question is being asked and how it is to be probed. Briefing sessions are often delegated to residents who just read the questions, open-ended questions are asked as closed-ended ones due to lack of clear instructions. Preceptors, on the other hand, blamed students for not attending briefings.
Students opined that there is no standardization of process of collecting information needed for community diagnosis. Data entry sheet is not discussed with briefing and not available to all due to the absence of internet facility in LT. By the time of compilation for end-term presentation, they do realize futility of putting efforts to analyze data or its future utility.
Medical social service officer and faculty of health education help students in organizing community-level intervention. Average of marks awarded by both for intervention is considered in assessment. Scoring for community interventions is only based on students' participation in skit/role play. Students and participants opined that students scored heavily for this activity just by showing their face during planning and execution phase.
Regarding end-posting viva, some students expressed concern about discriminatory nature of marks.
Faculty took end-posting viva in four groups with two in each group. Duration of viva was 8–10 min for most students. Few reported viva time as 4 min and one reported 13 min. Preceptors observing the viva session perceived adequacy of time in viva.
Environment during viva was reported as nonthreatening (5–7 on scale of 10) by students. During direct observation, about half the students were observed to be nervous enough to have interference in fair assessment. Examiners were reported to be irritated in one instance and sarcastic to three students. One of the examiners mentioned during in-depth interview that threatening environment did exist during viva due to which students were disturbed and could not concentrate to give their best.
Viva questions were reported to be relevant to learning during FHAS posting. Difficulty level was higher in one set of examiners (score 7 compared to 4.5 for others on a scale of 1–10). Students complained that some faculty do not accept answers even if they are correct as per book. Faculty agreed that viva is quite subjective as there are no guidelines on content coverage during viva. Need for standardization of scores at end-posting viva was suggested by most of faculty. Out of eight faculty members involved in FHAS (six involved for more than five years), only one had received training for teaching. Three of them had attended <3 days' workshop at MEU for MCQ framing. All except one showed interest in training but felt it is useful in the first 2 years of joining.
Pattern of awarded scores differed for different components of assessment [Table 2]. Range of marks was high for records and end-posting viva. Highest scores were awarded for intervention and low scores for analysis and presentation. Although both these activities were end assessment activities, a very high standard deviation was observed in analysis and presentation, suggesting dispersion with values at two extremes.
|Table 2: Distribution of marks awarded for different components of family health advisory services assessment|
Click here to view
Preceptors mentioned that despite provision of structured marking sheet, they are questioned if they give high or low marks to students, and hence, they play it safe by giving uniform and at least passing marks.
Opinion of 60% of participants (students, interns, and preceptors) regarding FHAS assessment was that it was fair, transparent, and satisfactory [Table 3]. The domains scored low were field activities (57%) and gain in skills (62%). Knowledge content dominated assessment (71%).
|Table 3: Opinion about characteristics of family health advisory services assessment among participants*|
Click here to view
However, it was not only FHAS-related activities but also other factors that affected scores (71%) as per students. There was a problem of lack of uniformity in application of criteria for assessment (44%) as well as award of marks (37%). Qualitative data observed that adequate time was not given to candidates for term-end viva (24% students). Of total, 31% students did not think that assessment helped them to identify and improve in their subject-related weak areas.
Preceptors were observed to have a more favorable view toward current assessment except for award of nonuniform award of marks and effect of unrelated factors on marking.
Undergraduate log book of the department was only written source of information on FHAS. Although students were told to improve their learning during community diagnosis presentation and end-term viva voce, no structured feedback was formally provided to them. Less than half, the observations on viva mentioned provision of feedback on performance. Preceptors and faculty agreed on need to provide regular feedback but mentioned time constraint as a barrier.
Suggestions for improvement in assessment were focused mostly on improvement in implementation of posting and formative assessment in field.
We observed that leadership of faculty determines the importance of activity among students and preceptors. This warrants change in casual attitude of faculty toward fieldwork supervision and classroom briefing. Their commitment toward supportive supervision to students as well as residents will not only help in better implementation of FHAS activity but also enthuse interest among students about the activity.
Written FHAS guidelines about objectives, process of implementation, and assessment of students are essential for sensitization of all stakeholders involved in the activity for years to come. Guidelines on formative assessment need to be clear and implementation needs a regular supervision. Emphasis has to be on field-based assessment with testing of skills.
There has to be consensus among faculty as well as their training to develop and implement a relevant, field-based, ongoing assessment system with feedback for enhanced learning.
Classroom briefings should include clarity on why data are being collected, how to communicate to get reliable information, and how to enter data in excel sheet. Pro forma needs to be made in bilingual format. Data entry should be carried out and cross-checked by preceptors on regular basis.
Preceptors are link between students and faculty. They are primary support for field based activities. There is a need to train them to strengthen supervisory and assessment skills. It needs refresher trainings as turnover is high.
Weightage of community-based intervention should be decreased, and it should be made more explicit. Intervention at family level needs to be included in assessment to motivate students. As learning objectives of the posting are best assessed by field-based assessment, end-posting viva needs to be replaced by it. The feedback from peers, preceptors, and allotted families should be a part of assessment.
| Conclusions|| |
Combination of formative and summative approach was identified as perceived strength of FHAS assessment. Assessment had been planned to cover field visits, record keeping, presentation of findings, attendance recording, participation in group activities of community diagnosis, community level intervention, and viva performance. However, the implementation was far from what was planned. There was no formative assessment at all. Over the years, changes have occurred in students' profile and their expectations, necessitating a change to improve learning in this opportunity by changing nature of assessment. It needs to be formative and field based. Ongoing feedback is a must to not only increase the learning but also justify inputs from department, institute, students as well as community itself.
Resources are being used but the output lacks due to half-hearted implementation. There needs to be written document to ensure a common basic understanding. Orientation and reorientation of preceptors have to be a part of this posting. Faculty leadership is the most essential component as the implementation of program itself was observed to be deficient. Faculty development programme can help bridge the gap between what is expected and what is currently practiced. Perhaps the most important factor is the commitment of faculty to not let the posting become a “missed opportunity.”
All faculty members were sensitized for study for the past 1 year and that could have changed their behavior during assessment.
Observation of viva sessions was carried by junior residents after briefing, but no standardization was done. Interobserver variation is possible. Only a third of end-posting assessment observations recorded. Furthermore, JR could be hesitant to remark adversely about faculty behavior.
Qualitative data number limited as all have been included.
I acknowledge contributions of my past Head of the Department Dr. CS Pandav, Departmental Faculty and Residents, FAIMER Faculty and Faimer fellows. Most important, all the students and interns who were forthcoming and answered sincerely to discover unexpected results and supported me at every step deserve my sincere thanks.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Guidelines for Preventive and Social Medicine/Community Medicine/Community Health Curriculum in the Undergraduate Medical Education, World Health Organization SEA-HSD-325. New Delhi; 2010. Available from: http://www.apps.searo.who.int/PDS _DOCS/B4451.pdf
. [Last accessed on 2017 Oct 02].
Guidelines for Preventive and Social Medicine/Community Medicine/Community Health Curriculum in the Undergraduate Medical Education. WHO SEA; 2010. Available from: http://www.apps.searo.who.int/PDS_DOCS/B4482.pdf
. [Last accessed on 2017 Oct 02].
Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N, Timothy E, et al
. The Lancet Commissions. Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet 2010;376:1923-58.
Shrivatava JB. Report of the Group on Medical Education and Support Manpower. Ministry of Health and Family Welfare, Government of India; 1975.
Nongkynrih B, Anand K, Kusuma YS, Rai SK, Misra P, Goswami K, et al.
Linking undergraduate medical education to primary health care. Indian J Public Health 2008;52:28-32.
] [Full text]
Sood R, Adkoli BV. Medical education in India – Problems and prospects. J Indian Acad Clin Med 2000;1:211-2.
Sood R, Singh T. Assessment in medical education: Evolving perspectives and contemporary trends. Natl Med J India 2012;25:357-64.
Epstein RM. Assessment in medical education. N Engl J Med 2007;356:387-96.
Gopalakrishnan S, Kumar PG. Community medicine teaching and evaluation: Scope of betterment. J Clin Diagn Res 2015;9:JE01-5.
[Table 1], [Table 2], [Table 3]