|Year : 2017 | Volume
| Issue : 2 | Page : 78-82
Assessment of a standardized rubric to evaluate student presentations
Hassan Mohamed Abouelkheir
Department of Oral and Maxillofacial Surgery and Diagnostic Sciences, Riyadh Colleges of Dentistry and Pharmacy, Riyadh, Kingdom of Saudi Arabia
|Date of Web Publication||25-Jul-2017|
Hassan Mohamed Abouelkheir
Department of Oral and Maxillofacial Surgery and Diagnostic Sciences, Riyadh Colleges of Dentistry and Pharmacy, P.O. Box 84891, Riyadh 11681
Kingdom of Saudi Arabia
Source of Support: None, Conflict of Interest: None
Introduction: Design of a standardized rubric for measuring student performance in presentations is a critical issue.
Materials and Methods: Thirty male students (Group I) and thirty female students (Group II) participated in this study. Likert scale was used (excellent, good, average, and unsatisfactory). It was used against the itemized criteria (introduction, content, slide arrangement, written work, delivery and use of references).
Results: high Cronbach's alpha reliability of the rubric was found in either Group I (0.945) or Group II (0.878) where it is slightly higher in the Group I at the 5% level. Corrected item-total correlation for delivery variable in Group I is very weak (0.222) while correlation for reference variable in Group II was averaged (0.525).
Conclusion: Group II showed higher delivery performance than Group I as Group I had a lack of English language proficiency, while in Group II, they miss reference citation which can be corrected.
Keywords: Assessment, multimedia, presentations, rubric, students
|How to cite this article:|
Abouelkheir HM. Assessment of a standardized rubric to evaluate student presentations. Saudi J Oral Sci 2017;4:78-82
| Introduction|| |
Evaluations are important in the process of teaching and learning. In health education, assessment of performance is essential for student's evaluation.
On Miller's pyramid, a framework used in medical education for measuring learner outcomes, i.e., “knows” is placed at the base of the pyramid, followed by “knows how,” then “shows how,” and finally, “does” is placed at the top.
An evaluation rubric for grading the presentations was designed to allow faculty evaluators to objectively score student performances in the domains of presentation delivery and content. Passing a presentation-based course was contingent upon this high-stake evaluation. The reliability and validity of the rubric used and the evaluation process need to be clarified and justified for the benefit of students.
Scoring rubrics are descriptive scoring schemes that are developed by teachers or other evaluators to guide the analysis of the products or processes of the student's efforts.
Rubrics include two elements: (a) A statement of the criteria to be evaluated and (b) an appropriate and relevant scoring system. Rubrics can be classified as either holistic or analytic. Holistic rubrics award a single score based on the student's overall performance, whereas analytic rubrics give multiple scores along several dimensions.
In analytical rubrics, the scores for each dimension can be summed for the final grade. The advantage of the holistic rubric is that papers can be scored quickly. The analytical rubric provides more detailed feedback for the student and increases consistency between graders.
Furthermore, when rubric is used as the basis for evaluating student performance, it is a type of measurement instrument and as such, it is important that the rubric exhibits reliability (i.e., consistency of scores across repeated measurements) and validity (i.e., the extent to which scores truly reflect the underlying variable of interest). Therefore, the present study will analyze the standard rubric used for the evaluation of students' presentation courses, i.e., is it reliable or need modification to improve the assessment?
The aim of this study is to measure the reliability of the rubric used in student's Powerpoint evaluation. In addition, the study compares the reliability of the rubric used by male group students and female group students of the same level (Level 12) in undergraduate dental students in Riyadh Colleges of Dentistry and Pharmacy as a private dental college in Riyadh province.
| Material and Methods|| |
Sixty students of Riyadh Colleges of Dentistry and Pharmacy in Level 12 participated in this study. They are divided into two groups:
- Group I: Thirty male students at Level 12 (the last semester before graduation)
- Group II: Thirty female students at Level 12 (the last semester before graduation).
All students were participating in the CPC-622 course which deals with problem-based cases on oral and maxillofacial radiology where students were divided into small groups of 4–5 students. Each group will get a problem-based learning radiological case from the course director and they have to prepare the case within 1 week. There are two group presentations every week. They used a standard rubric to the checklist [Appendix 1 [Additional file 1]].
Each presentation takes 20 min where all students of the same groups share presentation in turn. Globing rating scale is 1–4.
4 = Excellent
3 = Good
2 = Average.
1 = Unsatisfactory.
The itemized criteria are as follows:
- Slide arrangement
- Written work
- Use of references.
Student's guidelines about detailed previous checklist are handed over to students before preparation of presentations.
Analysis of rubric for measuring reliability was calculated by using (Cronbach's alpha) for the assessment of power point presentation of students. Ethical clearance was obtained from the Ethical Committee for research of Riyadh Colleges of Dentistry and Pharmacy with Institutional Review Board (IRB) approval number (RC/IRB/2016/148).
Reliability of scoring rubric is calculated by using SPSS- statistical program version 22(IBM Corp., Armonk, NY, USA).
| Results|| |
[Table 1] shows high Cronbach's alpha reliability, in either males (0.945) or females (0.878) where it is slightly higher in the male group at 5% level.
[Table 2] shows the information regarding responses by the male group of individual variables and provides information regarding the mean and standard deviation for responses to each variable. The delivery variable indicates a higher score (2.87) which means participants were in agreement, but standard deviation (0.681) shows a wide variety of responses.
[Table 3] shows the information regarding responses by the female group of individual variables and provides information regarding the mean and standard deviation for responses to each variable. The delivery variable indicates a higher score (3.20) which means participants were in agreement, but standard deviation (0.925) shows a wide variety of responses too.
[Table 4] shows that corrected item-total correlation for delivery variable in male group is very weak (0.222). Therefore, Cronbach's alpha is 1.00 if item had been deleted, this means if delivery variable is removed from the scale, the reliability of scale scores will reach 1.0, which is a perfect reliability.
[Table 5] shows that corrected item-total correlation for reference variable in female group is average (0.525). Therefore, Cronbach's alpha, if item deleted, is 0.872. On the other hand, delivery variable, where corrected item-total correlation, is 0.764, and if the delivery variable is removed from the scale, the reliability of scale scores will reach until 0.819.
| Discussion|| |
The rubric-based method provides a more global assessment of performance than the checklist-based method because it allows for the assessment of critical thinking and professional behavior in addition to knowledge and presentation skills. The rubric evaluates professional attire, interactions with small group peers and instructors, approach to group participation, and preparing for case presentation.
Jonsson and Svingby  proposed that rubrics can promote a higher quality of learning by focusing attention on the assessment criteria in elaborating the projects and that they help students to perform a much more precise self-assessment of their own work. They also allow teachers to make more judgments about the level of acquisition of complex skills and provide more specific feedback about the student's performance.
In the present study, high reliability of rubric scoring in both male and female groups was found. These results were in agreement with the study of Peeters et al. who used a rubric scoring of four rating scale categories and also found a high reliability. They also concluded that rubric scoring of four rating scale categories was better for measurements rather than 10-point Likert scale, here the full 10-point scale can be used in patient care settings such as in quantifying pain.
Therefore, Five-point scale is commonly used, but an odd number of categories can be problematic for interpretation and is not recommended.,
In the present study, corrected item-total correlation for delivery variable in male group was very weak (0.222). Therefore, Cronbach's alpha is 1.00 if item is not deleted. This means that individual delivery item which constitutes English, spelling, grammar, and voice tone were weak between individuals in each group. The same delivery item in female groups was at an average level (0.525). This means female groups have better English proficiency than male groups.
This was in agreement with a study by Kaliyadan et al. who found a significant correlation between the score in the English language assessment and the final summative part of the medical content assessment of the preparatory year of medical school at King Faisal University, Hofuf, Saudi Arabia, from 2011 to 2013. For the Saudi medical students, they also concluded that English language proficiency is not only important for their basic undergraduate course, but also for higher studies in the future.
Salem et al. conducted a cross-sectional study at the Faculty of Medicine, King Fahad Medical City, King Saud Bin Adulaziz University for Health Sciences, Riyadh, KSA. Medical students of preclinical 2nd and 3rd year participated in the study, during the academic year 2010–2011. They studied the sociodemographic factors, interest, and motivation to study medicine, academic, and educational factors as well as socioeconomic and cultural factors and their relation to cumulative grade point average (CGPA) score of the students.
They found that only four factors have a significant association with CGPA. These include gender, marital status, interest of students to study medicine, and the type of transportation used to reach the faculty of medicine. Female students scored significantly higher CGPA than male students. In addition, female students' age was significantly lower than their male counterparts. This is because males with poor CGPA have to repeat certain courses that may affect their progress. The present study showed also some male students who repeated certain courses and this delay them to keep pace with their patches. This is also in agreement with other studies.,
In the present study, female students showed higher delivery performance than male students. This is also in agreement with the results of Salem et al. as their study showed that female students achieved high significant performance in all courses and they have a better grasp of English. Another study from the United Arab Emirates has shown that students' competence in English is the most important factor that affects the academic performance.
Salem et al. also concluded that the mean of transportation to faculty has a significant influence on CGPA as male students drive their own cars and they spend more time on social activities and family responsibilities. On the other hand, females are not allowed to drive in Saudi Arabia and this limits their movement and give them more time to study and improve their CGPAs.
Therefore, more effort should be made to improve language-learning strategies rather than the time allocated for learning a language to ensure better proficiency.
Javid et al. proposed learning strategies for English learners and developed instructions for less successful learners that would help them become more successful in their language study. Zaid and Alamir and Fatemipour , suggested that more integrative English training combining reading, writing, speaking, and listening skills in authentic medical contexts will be important in ensuring the students actually transfer the benefit of language training to their academic performance. Therefore, role-plays and scenarios similar to actual medical consultations would go a long way in improving relevant medical communication skills in English.
Although students had more time to prepare presentations and assignments in radiological problem-based cases, English proficiency was not a major influencing factor even they work together in small groups. Therefore, recommendations for continuous and well-planned activities such as book clubs and reflective essays may improve English skills of medical students and these activities should be added as credit points for students to motivate them for further development.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990;65 9 Suppl:S63-7.
Brookhart SM. The art and science of classroom assessment: The missing part of pedagogy. ASHE-ERIC Higher Education Report. Vol. 27. Washington, DC: The George Washington University, Graduate School of Education and Human Development; 1999.
Peat B. Integrating writing and research skills: Development and testing of a rubric to measure student outcomes. J Public Aff Educ 2006;12:295-311.
Moskal, BM. Scoring rubrics: What, When and how? Pract Assess Res Eval 2000;7:1-8.
O'Brien CE, Franks AM, Stowe CD. Multiple rubric-based assessments of student case presentations. Am J Pharm Educ 2008;72:58.
Jonsson A, Svingby A. The use of scoring rubrics: Reliability, validity and educational consequences. Educ Res Rev 2007;2:130-44.
Peeters MJ, Eric GS, Gregory ES. Instructional design and assessment; A standardized rubric to evaluate student presentations. Am J Pharm Educ 2010;74:1-8.
Miller GA. The magical number seven plus or minus two: Some limits on our capacity for processing information. Psychol Rev 1956;63:81-97.
Weems GA, Onwueghbuzie AJ. The impact of midpoint responses and reverse coding on survey data. Measure Eval Couns Develop. 2001;34:166-76.
Kaliyadan F, Thalamkandathil N, Parupalli SR, Amin TT, Balaha MH, Al Bu Ali, WH. English language proficiency and academic performance: A study of a medical preparatory year program in Saudi Arabia. Avicenna J Med 2015;5:140-4.
] [Full text]
Salem RO, Al-Mously N, Nabil NM, Al-Zalabani AH, Al-Dhawi AF, Al-Hamdan N. Academic and socio-demographic factors influencing students' performance in a new Saudi medical school. Med Teach 2013;35 Suppl 1:S83-9.
Haist SA, Wilson JF, Elam CL, Blue AV, Fosson SE. The effect of gender and age on medical school performance: An important interaction. Adv Health Sci Educ Theory Pract 2000;5:197-205.
Greenfield S, Parle J, Holder R. The anxieties of male and female medical students on commencing clinical studies: The role of gender. Educ Health (Abingdon) 2001;14:61-73.
Harb N, El-Shaarawi A. Factors Affecting Student Performance. MPRA No. 136. 2009. p. 17-46.
Javid CZ, Al-Hubati TS, Uthman A. Effects of English language proficiency on the choice of language learning strategies by Saudi English-major undergraduates. Engl Lang Teach 2013;6:35-47.
Zaid MA, Alamir AH. An exploration of the English language skills in relation to overall EFL proficiency level of medical students in an EAP course. J Arabic Hum Sci Qassim Univ 2010;3:1-17.
Fatemipour H. The effectiveness of reflective teaching tools in English language teaching. J Mod Thought Educ 2009;4:73-90.
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]