Farhad Jafari¹, Mehdi Hadavand¹ Mahmood Samadpour¹, Fatemeh Azami¹ and Seyed Ehsan Beladian Behbahan²*

¹Department of Health and Social Medicine, School of Medicine, Shahed University, Tehran, Iran.

²Department of Health and Social Medicine, School of Medicine, Shahidbeheshti University, Tehran, Iran.

DOI : https://dx.doi.org/10.13005/bpj/419

Abstract

The quality of education provided during the study, is one of the services quality determinants of graduates. The defects may be resolved through assessment of such educations. It is of great importance to evaluate the scores achieved by students in pre-internship comprehensive exams that all universities participate in them. This descriptive-analytical study, examined 375 medical.. The related factors and the data were analyzed by SPSS, Pearson’s correlation and regression tests. Of 375 subjects entered the university over 11 years, 56.3% (n=211) were female, 71.6% (269) were living in dormitories and 91.7% (343) were admitted following the first innings of test. The mean score of pre-internship exams was 113.5±20. Its score correlated with gender, place of residence, being with family, history of failure in exam, basic sciences exam’s score, GPA of medical externship and Physiopathology courses. Using linear regression and evaluating various variables, the most prediction validity was obtained with medical externship course GPA (R2=0.543) and the Physiopathology course GPA (R2=0.433), respectively. Furthermore, the regression line equation with the Backward procedure was as follows: Y= 8.5 (medical externship GPA) + 0.297 (score of basic sciences course)-53.53. Based on the results, the pre-internship exam scores can be predictable with a high validity.

Keywords

Pre-internship Comprehensive Exam; Evaluatio; Prediction; Medical Student

Download this article as: 

Copy the following to cite this article: Jafari F, Hadavand M, Samadpour M, Azami F, Behbahan S. E. B. Evaluation of Pre-Internship Comprehensive Exams Scores and their Predictive Factors. Biomed Pharmacol J 2013;6(2)

Copy the following to cite this URL: Jafari F, Hadavand M, Samadpour M, Azami F, Behbahan S. E. B. Evaluation of Pre-Internship Comprehensive Exams Scores and their Predictive Factors. Biomed Pharmacol J 2013;6(2). Available from: http://biomedpharmajournal.org/?p=2735

Introduction

Medical education is a public trust. General medicine course is of great importance due to the heavy responsibility for graduates in this field. Thus, in many countries including Iran, volunteers are evaluated in different aspects (1). Value appointment or judgment about one’s value is named appraisal (2). Appraisal ability of student and graduation of medicine in theoretical and practical compasses can use not only in determining the effectiveness, but also in planning as educational data for the future (3). In the 1950s, rapid and extensive changes occurred in assessment methods in the field of medical education. During this period, several assessment methods were established and implemented (4). The proposed methods were primarily based on assessing clinical skills (history taking and physical examination of the patient), communication skills, process skills and professional expertise (5). However, experience has proved that the clinical learning is preferred in comparison with theoretical medical education. However, little work has been done to evaluate the clinical experiences (6).

There are many differences between students in terms of experiences and awareness (7, 8) such that the differences can be seen even after graduation (9, 10). Given the same educations provided, these differences are caused by other factors, which should be identified to address deficiencies. Generally, medical exams are poor predictors of the success of students (11). Nowadays, the medicine emphasis is on thinking skills, analysis, question, problem solving and communication with the public (12).

Obviously, any comment about the causes of success or lack of success requires scientific scrutiny. Rather, all effective variables relevant to students, faculties and departments, educational programs and comprehensive exams should be thoroughly studied. The educational system of medical departments, examines education process through a two-step procedure by comprehensive tests of Basic Sciences and Pre-internship. Students who achieve the required standards and pass the integrated designed tests- ideally with enough validity and reliability to measure students’ knowledge- are allowed to enter higher education levels. Although the multiple-choice questions have an appropriate reliability, this fact cannot be denied that such exams evaluate one of students’ capabilities, i.e. the clinical information (2, 13, 14). In a study by Nasri that cover student’s evaluation about these two exams, the student mentioned the Pre-Internship exam as the more important one in knowledge evaluation (15).

Nowadays in some countries, including Iran, pre-internship comprehensive exam held after the Physiopathology courses and 18 months of clinical externship. Once the students passed the exam, they will gain experience as intern besides professors and residents, If they can get essential abilities during this 2 years period, general medicine certificate gave.(16)  so they can apply their knowledge to treat patients. The exam is a multiple-choice test question. Given the courses provided within a period, 200 questions are designed for a pre-internship exam. The minimum passing grade is 70% of the average score of 5% of students who get the highest scores in the country. So far, 45 pre-internship comprehensive exams have held.

The scientific and practical backgrounds of this quality achieved through the education process. However, the factors relevant to students are also very important. The present study also aims to examine students (Inputs), education process (Process) including faculty, department, and curriculum as well as the result of pre-internship comprehensive exam (Outputs).

Although few studies have been carried out on these exams, most of them are limited to a period of one or three years, so they do not provide a possibility to predict the pre-internship exam scores using the basic sciences test scores and GPA of medical externship course. Therefore, the present study has the aim to examine the pre-internship test scores of students in Shahed University of Medical Sciences over ten innings in order to determine the factors associated with pre-internship test results. On the other hand, a mathematical model was proposed to predict exam scores.

Materials and Methods

This is a documentary research of cross-sectional type. The objective of the present study is to examine the pre-internship exam and its predictors. The participants were medical students in the Shahed University. They entered the University during a period of ten years. They have participated in ten innings of pre-internship comprehensive exam following the basic sciences, Physiopathology and medical externship courses.

The main criteria for inclusion was being a student in medical school and participating in one inning of the pre-internship comprehensive exams. Students who have not participated in any exams for some reasons as well as those have withdrawn or have dismissed from school, excluded. Thus, the population consisted of 375 medical students who participated in 413 innings of pre-internship comprehensive exams during ten consecutive periods. The sample was selected by use of census method.

Required data including high school GPA, different grades in  entrance exam, history of failure, GPA of basic sciences, the score of basic sciences comprehensive exam, GPA of Physiopathology, medical externship GPA, the number of participants in each exam, the score of pre-internship exam and other demographic information such as age, gender, quotas, location, condition of life, were collected referring Sanjesh Organization, academic records of students as well as the Ministry of Health and Medical Education. The data entered into information forms. To preserve dignity of the studied population, the name of individuals was not mentioned both in the positive and negative direction. Basically each person was given a code number in information forms. Data was analyzed by use of SPSS-16. Central and distribution indexes as well as appropriate tables were used to describe the data as required. The data were analyzed using t-Test, Pearson correlation test and linear regression methods.

Results

Total of 375 medical students in Shahed University were studied. Of this, 56.3% were females (n = 211) and 43.7% (n = 164) were male. The mean age on arrival at university was 19.41±1.9 years. The youngest and oldest students aged were17 and 29 years old, respectively. The mean high school GPA was 16.8±2.82. The GPA of basic sciences course was 14.77±1.51. The mean GPA of Physiopathology course was 14.12±1.5. The average GPA of medical externship course was 15.6±1.2. Of 375 medical students, 44.5% (n=167) had been accepted with  Shahed quota, 71.6% (n=269) were living in dormitories and 28.6% (n=106) were not living in dormitories. Among them, 91.7% (n = 343) were admitted in the first innings of the test. The mean score of pre-internship comprehensive exam was 116.95±21.84. The mean scores of women and men were  114.9± 19 and 111.7±21, respectively. Table 1 shows the frequency distribution of pre-internship test raw scores.

Table 1: The frequency distribution of pre-internship tests raw scores

Comprehensive test scores	Frequency (%)	Cumulative frequency
<100	99 (26.4)	26.4
100-109	61 (16.3)	42.7
110-119	70 (18.6)	61.3
120-129	64 (17.1)	78.4
130-139	41 (10.9)	89.3
≥140	40 (10.7)	100
Total	375 (100)	100

 

Pearson’s correlation test showed a significant correlation between age and pre-internship exam score (r=0.292, P=0.04), even though the mean age of students failed the test (19.50%) was slightly higher than those passed the exam successfully (19.40). The pre-internship exam mean score of those accepted through various quotas was 103.7±17. This was lower than the mean score of free students 121.7±18. A correlation was found between the scores and quota (P=0.001). Furthermore, there was a significant correlation between a history of failure in exam and pre-internship exam score, so that the score of those without a failure history (118±18) was higher than those with a history of failure (97±15) (P=0.045). The scores of students who lived with their families (116±17) were higher than those who lived apart from their families (109.5±16). Moreover, the scores of students lived in dormitory (106±16) were lower than those who lived in the house (112±17).

According to Pearson’s correlation test, there was a significant positive correlation between pre-internship exam score and basic sciences test score (r=0.669, P<0.001). Furthermore, a significant relationship was found between pre-internship test score and high school GPA (r=0.491, P<0.001). A significant correlation was found between pre-internship test scores and GPA of Physiopathology course (r=0.658 P<0.001). Furthermore, there was a significant positive correlation between the pre-internship test score and entrance exam scores (Table 2). It can be seen, that the strongest correlation was found with math score.

Table 2: The correlation between pre-internship test scores and correct answers to entrance exam

	English	Math	Biology	Physics	Chemistry	Geology
pre-internship test scores	0.001>P r=0.248	0.001>  P 0.319=r	0.001>P 0.284=r	0.001>P 0.306=r	0.001>P 0.309=r	0.001>P 0.170=r

 

Regression analysis were  used to determine the validity and capability of variables for predicting pre-internship exam scores as well as to determine the association of various factors with pre-internship comprehensive test scores. Y and X denote the dependent variable (pre-internship test score) and independent variables, respectively. The equation for the regression line is as follows:

The regression line equations for various variables are as it shown in Table 3:

Y= β₀ + β₁X + β₂X₂ +_—-+ β_nX_n

According to Table 3, in all cases, P<0.001.

As it can be seen, the highest coefficient of determination (R²) was obtained for medical externship GPA, basic sciences comprehensive test score and Physiopathology GPA. Among entrance exam grades, the highest R-squared obtained for mathematics. Two models with highest coefficient of determination (adjusted R-squared) were proposed using stepwise regression for Physics entrance exam score, GPA of basic sciences course, the Chemistry, Biology, Mathematics, English scores, basic sciences comprehensive test score, Pathophysiology GPA and medical externship GPA.

Table 3: The regression line equations

R2=0.241	Diploma GPA = X1	(X1) 3.54+57.382=Y
R2=0.081	X2=English score in entrance exam	(X2)0.151 +101.16 = ‎Y
R2=0.094	score in entrance exam Physics= X3	(X3) 0.196+102.153= Y
R2=0.095	X4= Chemistry score in entrance exam	(X4) 0.242+95.452 = Y
R2=0.102	X5= Math score in entrance exam	(X5) 0.242+101.757= Y
R2=0.081	X6= Biology score in entrance exam	(X6) 0.225+97.40 = Y
R2=0.029	X7=Geology score in entrance exam	(X7) 0.160+105.420= Y
R2=0.382	X8=GPA of Basic Science course	(X8) 10.317+34.715= Y
R2=0.447	Basic Science test score = X9	(X9) 0.623+40.58= Y
R2=0.433	GPA of physiopathology course =X10	(X10) 8.48+6.34 – = Y
R2=0.543	GPA of medical externship course =X11	(X11) 11.85+71.29-=Y

The first model

R²=0.507

Regression line equation: Y=-45.191+8.6 (medical externship GPA) + 0.192 (basic science test score)

Second model

R²=0.474

Regression line equation: Y=-53.685+10.5 (medical externship GPA)

Ignoring the scores of entrance exam, the backward regression proposed three following models with the highest coefficient of determination (R²=0.606) only using basic sciences comprehensive test score, Pathophysiology GPA, medical externship GPA and GPA of basic sciences course as variables:

Model 1:

Y=-50.11+0.322 (X₁) – 0.716 (X₃) + 8.55 (X₄) – 1.17 (X₅)

where

X₁ = the basic sciences test score

X3 = Pathophysiology GPA

X4 = medical externship GPA

X5 = basic sciences GPA

Model 2

Y=-51.05+0.33 (X₁) +9 (X₂) – 0.95 (X_5­)

where

X₁ = the basic sciences test score

X₂ = medical externship GPA

X5 = basic sciences GPA

Model 3

Y=-53.54+0.297 (X₁) +8.5 (X₂)

where

X₁ = the basic sciences test score

X₂ = medical externship GPA

Discussion

The quality of education courses ensure the quality of the services provided by graduates. Educational services provided according to criteria considered by educators. However, continuous monitoring these services is necessary due to changes and improvements in the education system, knowledge of students and the needs of the community. This may help solving the failure of parts of education system or evaluation system.

In this study, most students are able to earn a passing grade, which is higher than those pass the basic sciences comprehensive exam. There are several reasons for this. First, a number of poor students basically do not participate pre-internship comprehensive exam following failure in basic sciences test or even in Physiopathology and medical externship courses. In other words, poor students are filtered and stronger students will participate in the test. Furthermore, since the course immediately following the basic sciences course is more practical at bedside, the pre-internship exam topics and questions have more close affinity with the whole medical field, the educations are more visible and students are even more motivated to learn.

Pre-internship test raw score was higher than that of basic sciences comprehensive exam. Women were more successful in pre-internship exam than men were. The mean raw score of women was higher than men. This result is consistent with the results of Ferguson (17). Furthermore, the results of Yousefi (18), Zahedi Asl (19) and Kadkhodaei (20) showed the better academic achievement of women. However, Mohammadi (21) found no significant differences between boys and girls in terms of pre-internship exam score. Zahedi Asl (19) found no correlation between educational status and age. According to Kadkhodaei (9), the age is inversely related to GPA. One of the possible reasons for the lack of correlation is nearly the same age of participants. About 50% of the students admitted using special quotas.

Most students lived in dormitories. It may be concluded that they are equally non-native. According to other similar studies, living in dormitories and being non-native increase the risk of failure (20, 22-24). Since the lifestyle (being with family or being apart from the family) had a significant relationship with the success in exam, the role of family support can be realized. In this study, pre-internship exam score was associated with high school GPA. Khaksari (24), Mohammadi (21) and Dehbozorgi (23) also found a positive correlation between high school GPA and pre-internship comprehensive test result.

A significant correlation was found between the scores of pre-internship exam and basic sciences test. Given a same correlation coefficient of 0.669, a stronger correlation was observed compared to Mohammadi’ study (21) who found a correlation coefficient of 0.598. However, both results are in good agreement. Furthermore, sample size in this study, in nearly four times of the sample size in Mohammdi’s study. As the regression analysis showed, the most prediction validity of pre-internship exam score observed with medical externship GPA and then Physiopathology GPA.

Generally, the pre-internship exam scores are related to the following factors: gender, place of residence, living with family or away from them, entrance exam scores, a history of failure, scores of basic sciences tests, high school GPA, basic sciences GPA, and medical externship and Physiopathology GPA. According to the results of the present study, the regression line equation may predict the score of pre-internship exam with a validity of R²=0.606 using the score of basic sciences test and medical externship GPA.

Conclusion

Pre-internship exam score was related to gender, place of residence, being with family, a history of failure in basic sciences exam and GPA of externship and Physiopathology courses. The most reliable prediction obtained using GPA of externship and pathophysiology courses. The results showed that the score of pre-internship exam could be predicted with a high validity. Given the proper capability of the regression equation to predict the scores of pre-internship comprehensive exam, it is recommended to identify high-risk students and provide provisions to strengthen their scientific knowledge. Moreover, the factors associated with the success or failure of students in pre-internship exam were identified. Based on these findings, proper planning can be done in this regard to obtain higher scores in such tests.

Acknowledgments

The authors wish to thank the staff of Sanjesh Organization, Specialized Sanjesh Center of Ministry of Health and Medical Education, staff and authorities of Medicine Department of Shahed University for their invaluable helps during various stages of this work.

References

1. Turner JL, Dankoski ME. Objective structured clinical exams: a critical review. Fam Med. 2008;40(8):574-8.
2. Javadi M. Assessment of the Result of Basic Sciences Exams of Medical Students in Qazvin University. The Journal of Qazvin University of Medical Sciences. 2001;18:69-75.
3. Alizadeh Naeini M, Kadivar M, Ghayumi M, Alavi Z, Vaseghi N, Karamifar N, et al. Evaluation of the Quality of Theoritical and Practical Skills in Medicine Student (6th year) of medical University of Shiraz and its Comparisonwith Writen Exam Score for Pre-Internship in 1383. wwwSIDir. 2004.
4. Norcini JJ. Current perspectives in assessment: the assessment of performance at work. Medical education. 2005;39(9):880-9.
5. Norcini JJ, McKinley DW. Assessment methods in medical education. Teaching and teacher education. 2007;23(3):239-50.
6. Jolly BC, Jones A, Dacre JE, Elzubeir M, Kopelman P, Hitman G. Relationships between students’ clinical experiences in introductory clinical courses and their performances on an objective structured clinical examination (OSCE). Academic Medicine. 1996;71(8):909-16.
7. Lockwood D, Goldman L, McManus I. Clinical experience of clerks and dressers: a three-year study of Birmingham medical students. Journal of the Royal Society of Medicine. 1986;79(1):38.
8. LOCKWOOD DN, Goldman L, McManus I. Surgical dressers: the theatre experience of junior clinical students. Medical education. 1986;20(3):216-21.
9. Dent T, Gillard JH, Aarons EJ, Crimlisk HL, Smyth-Pigott PJ. Preregistration house officers in the four Thames regions: I. Survey of education and workload. BMJ: British Medical Journal. 1990;300(6726):713.
10. Langdorf MI, Strange G, Macneil P. Computerized tracking of emergency medicine resident clinical experience. Annals of emergency medicine. 1990;19(7):764-73.
11. Peers IS, Johnston M. Influence of learning context on the relationship between A‐level attainment and final degree performance: a meta‐analytic review. British Journal of Educational Psychology. 1994;64(1):1-18.
12. Zamanzad B, MoeziM, Shirzad H. The Evaluation of Satisfaction and Stajer and Intern Assessment of Educational Quality of Clinical Courses in Medical University of Shahrekord. Medical Sciences University of Semnan. 2007;1(9):13-20.
13. Sloan D, Donnelly M, Johnson S, Schwartz R, Strodel W. Use of an Objective Structured Clinical Examination (OSCE) to measure improvement in clinical competence during the surgical internship. Surgery. 1993;114(2):343-50; discussion 50-1.
14. Schwanz RW, Donnelly MB, Sloan DA, Johnson SB, Strodel WE. The relationship between faculty ward evaluations, OSCE, and ABSITE as measures of surgical intern performance. The American journal of surgery. 1995;169(4):414-7.
15. Nasri K, Kahbazi M, Nasri S. Medical Students’ Viewpoints toward Basic Sciences and Preinternship Comprehensive Exams in Arak University of Medical Sciences Iranian Journal of Medical Education 2010;10(1):82-90.
16. Rudbari M, Movahed S. The Study of Effective Factors on the Result of Pre-Internship Exam for Medicine Student at Medical University of Zahedan. Developmental Steps in Medical Education. 2005;1(2):94-103.
17. Ferguson E, James D, Madeley L. Factors associated with success in medical school: systematic review of the literature. Bmj. 2002;324(7343):952-7.
18. Yousefi Mashouf R, Saeedi Jam M. Evaluation of medical students in the basic science courses, Hamedan University of Medical Sciences 1989-94. Journal of school of Medicine of Tehran University. 2000:76.
19. H. ZA. causes of educational dropout of students. Report of the first workshop on counseling. 1997;2: 1-21.
20. Kadkhodaie H, Farahadi A, AN. H. Factors affecting students’ academic performance at Lorestan University. Proceedings of the Second National Conference on Student Mental Health. 2002:46.
21. MohammadiM, J. A. Predictive validity of the comprehensive basic science examination (CBSE) for success assessment of comprehensive pre-intership examination (CPIE) in medical students. Iranian Journal of Medical Education. 2002;2(0):40-.
22. Roudbari M, ShariatiR. The Role of Students’ Educational and Demographic Factors in Basic Sciences Examination in Zahedan University of Medical Sciences. Iranian Journal of Medical Education. 2002;2 (1):27-35.
23. Dehbozorgi GH, HA. M. causes of educational dropout in 1993 and 1992 medical students at Shiraz University. Iranian Journal of Medical Education. 2002;2 (0):59-60.
24. Khaksari M, Smaeelzadeh M, Jafarpour P, Taleghani F. Proceedings of the Second National Congress of Medical Education. 1995;2:43-4.

(Visited 370 times, 1 visits today)