Teaching and Learning in Nursing
Volume 2, Issue 3 , Pages 80-84, July 2007

The predictive accuracy of Health Education Systems, Inc., examinations for associate degree nursing students

  • Mary J. Yoho, PhD, RN

      Affiliations

    • Elsevier Review and Testing, Houston, TX 77042, USA
    • Tomball College, Tomball, TX 77375, USA
    • Corresponding Author InformationCorresponding author. 11011 Richmond Avenue, Suite 450, Houston, TX 77042, USA. Tel.: +1 713 346 6913 (office); fax: +1 713 346 6977.
    • ,
  • Anne Young, EdD, RN

      Affiliations

    • Texas Woman's University, College of Nursing, Houston, TX 77030, USA
    • ,
  • Carolyn Adamson, PhD, RN

      Affiliations

    • Texas Woman's University, College of Nursing, Houston, TX 77030, USA
    • ,
  • Robin Britt, EdD, RNC, WHCNP

      Affiliations

    • Texas Woman's University, College of Nursing, Houston, TX 77030, USA

Article Outline

Abstract 

This longitudinal study was conducted to determine the accuracy of Health Education Systems, Inc. examinations in predicting associate degree nursing (ADN) student success. The Admissions Assessment reading comprehension examination was predictive of student success as measured by the study population's Mid-Curricular (MC) scores, and MC scores were predictive of Exit Exam (E2) scores. The E2 was 94.83% accurate in predicting the success of the National Council Licensure Examination for Registered Nurses. Findings of this study support the use of standardized testing within an ADN curriculum.

Keywords: Health Education Systems, Inc., National Council Licensure Examination for Registered Nurses, Associate degree nursing

 

Back to Article Outline

1. Introduction 

The U.S. Department of Labor (2005) estimated that 29.4% more new nurse graduates will be needed to fill registered nurse (RN) positions by 2014. Increasing the number of RNs in an effort to ameliorate the nursing shortage is dependent on several factors: the number of applicants accepted into the entry-level RN programs, the number who are retained in the programs, and the number who successfully pass the National Council Licensure Examination for Registered Nurses (NCLEX-RN). Johnson (2003) reported attrition rates as high as 70% in schools of nursing and attributed these rates to students' lack of preparation for rigorous nursing curricula and their need to engage in outside employment while in the nursing program. Of those who do complete nursing programs, approximately 12% fail the NCLEX-RN (National Council of State Boards of Nursing [NCSBN], 2006), thereby delaying or, in some cases, eliminating their entrance into the RN workforce.

The faculty at a Southwest Texas Associate Degree Nursing (ADN) program turned to the use of standardized nursing examinations to assist them in identifying students who have the academic preparation necessary to succeed in the program, those who are at risk for failure once enrolled in the program, and those who will likely require remediation to be successful on the NCLEX-RN. The purpose of this study was to determine the predictive accuracy of three standardized tests produced by Health Education Systems, Inc. (HESI).† Specifically, the ability of the HESI Admissions Assessment (A2) examination to predict performance on the HESI Mid-Curricular (MC) examination, the ability of the MC to predict performance on the HESI Exit Exam (E2), and the ability of the E2 to predict NCLEX-RN success were examined.

Back to Article Outline

2. Review of the literature 

Predicting student success in nursing programs and predicting NCLEX-RN success are a challenge for nurse educators. Numerous research studies have attempted to identify predictors of success, as well as methodologies for applicant selection, retention, and graduation. Although some studies found no significant relationship between nursing school grade point average (GPA) and NCLEX-RN success (Briscoe & Amena, 1999, Daley et al., 2002, Haas et al., 2004, Hardin, 2005), others have found that the nursing didactic course GPA was predictive of NCLEX-RN success (Collins, 2002, Drake, 1996). Some researchers reported that nursing prerequisite academic course GPAs were predictive of NCLEX-RN success (Milan, 1997, Swenty, 1998), and one author reported that pharmacology course grades were predictive of NCLEX-RN success (Percoco, 2001).

Numerous research studies have focused on the use of standardized examination scores to predict student success and to identify students' remediation needs, as well as to evaluate the nursing curriculum. Preentrance math and reading comprehension skills reported by the Nursing Entrance Test and the Registered Nursing Entrance Examination (Gallagher, Bomba, & Crane, 2001), as well as the Nelson Denny Reading Test (Cloud-Hardaway, 1988), were found to be significantly related to NCLEX-RN success. Higgins (2005) reported that assessment testing throughout the nursing curriculum reduced attrition and increased NCLEX-RN success. Hardin (2005) reported that the A2, the HESI-customized MC, and HESI specialty examinations were useful for predicting success in ADN nursing programs and that the E2 was useful for predicting NCLEX-RN success.

Comprehensive examinations are frequently administered to students during the final semester or quarter of the nursing curriculum to assess students' ability to succeed on the licensing examination. Daley et al. (2002) reported that E2 scores were 100% accurate in predicting NCLEX-RN success (N = 224) in a baccalaureate nursing program (bachelor of science in nursing [BSN]). In five studies of ADN, BSN, and diploma students (N = 27,809), the E2 was found to be 96.36% to 98.46% accurate in predicting NCLEX-RN success (Lauchner et al., 1999, Lewis, 2005, Newman et al., 2000, Nibert & Young, 2001, Nibert et al., 2002).

Back to Article Outline

3. Methodology 

A descriptive, longitudinal, correlational design was used to assess the predictive accuracy of HESI examinations, specifically the A2, the MC, and the E2. Predictive accuracy was defined as achieving the faculty-designated score on the subsequently administered HESI examination or success on the first attempt at taking the NCLEX-RN when the faculty-designated score was achieved on the previously administered HESI examination. This study assessed the accuracy of the math and reading comprehension A2 scores in predicting the ADN student population's MC scores, the accuracy of the MC scores in predicting E2 scores, and the accuracy of E2 scores in predicting NCLEX-RN success.

The study sample consisted of a cohort of 139 students who took the A2, the MC, the E2, and the NCLEX-RN between August 2002 and October 2004. Students who left the program during the study period were included in the analysis up to the point of their exit from the program.

Faculty at the participating ADN school of nursing selected 70% as the minimally acceptable score for the A2 math and reading comprehension examinations. The faculty established 850 as the minimally acceptable HESI score for the MC and the E2.

Back to Article Outline

4. Description of instruments 

4.1. HESI A2 

At the time this study was conducted, the A2 consisted of six academically oriented examinations: math, reading comprehension, grammar, vocabulary, science (chemistry and biology), and anatomy and physiology, as well as three personally oriented examinations: a learning styles assessment, a personal inventory, and a behavioral inventory.‡ In addition to a percentage score on each of the academically oriented examinations, the A2 also provides a composite score. This composite score is the average percentage score for all academically oriented examinations the examinee completes.§ Only the math and reading comprehension examinations were administered to the study population. The Kuder–Richardson formula 20 (KR20) is used to calculate the estimated reliability for each examination. The KR20 for the math examination that was administered to the study population was 0.93, and the KR20 for the reading comprehension examination that was administered to the study population was 0.90. Content validity of the A2 was established by experts in math for the math examination and experts in reading for the reading comprehension examination.

4.2. HESI MC 

The MC was a custom examination designed to evaluate students' ability to solve clinical problems related to the content presented in the first half of the participating ADN program. The MC was administered to the study population at the end of the first year of the 2-year program. Reliability estimates for this examination are based on prior administrations of the test items contained on the MC and the item analysis data obtained from these administrations. The KR20 for the MC that was administered to the study population was 0.92. Content validity for test items contained on HESI examinations, including the MC that was administered to the study population, is established by expert nurse clinicians who write the test items presented on HESI examinations and the ongoing evaluation of these test items in terms of their relevance to nursing practice. Construct validity for the MC is inferred because HESI test items are developed according to the concepts presented by Morrison, Nibert, and Flick (2006) for writing critical thinking test items for nursing examinations. Additionally, the test blueprint for the MC was based on the course syllabi for the first year of the curriculum. Faculty reviewed test items provided by HESI in terms of their applicability to the course content covered in the first year of the curriculum and approved questions for inclusion in the examination.

4.3. HESI E2 

The E2 was administered to the study population 6 weeks prior to graduation, and the estimated reliability, or KR20, for the first version of the E2 that was administered to the study population was 0.90. Content validity for the E2 is established just as it is for the MC, through ongoing evaluation of the HESI test items by nursing experts. Construct validity is inferred because the test items are written to reflect current nursing practice (Morrison et al., 2006), and the content design for the E2 is based on the NCLEX-RN test plan published by the NCSBN (Morrison et al., 2004, National Council of State Boards of Nursing, 2007). Criterion-related validity is established by research studies that examine the accuracy of E2 scores in predicting NCLEX-RN success, and five such studies found that the predictive accuracy of the E2 ranged from 95.67% to 99.50% for ADN students (Lauchner et al., 1999, Lewis, 2005, Newman et al., 2000, Nibert & Young, 2001, Nibert et al., 2002).

4.4. NCLEX-RN 

The NCLEX-RN is a computer-adaptive examination that is administered to candidates for RN licensure. This examination is designed to measure entry-level competence, and scores are reported as pass or fail. Reliability of the NCLEX-RN is evaluated with a decision consistency statistic, which is psychometrically reliable between .87 and .92 (NCSBN, 2007). Test items for this licensing examination are developed and reviewed by nurse experts with backgrounds in entry-level practice settings. A practice analysis study is conducted every 3 years to determine the activities that are expected of entry-level nurses, and findings from these studies are used to substantiate the value of test items contained on the licensing examination. This process of test item development and review is used to establish content validity for the NCLEX-RN (NCSBN, 2007).

Back to Article Outline

5. Findings 

Of the 139 students who took the A2, 135 (97.12%) achieved the faculty-designated minimally acceptable A2 composite score of 70% or higher. On the A2 math examination, 128 of the 139 students (92.09%) achieved the minimally acceptable score of 70% or higher, and on the A2 reading comprehension examination, 121 (87.05%) achieved the minimally acceptable score of 70% or higher. Students were counseled and referred for remediation as needed based on their A2 scores.

At the end of the first year, 101 of the original 139 student cohort (72.66%) remained in the program, and all 101 students took the MC. Of these 101 students, only 22 (21.78%) achieved the faculty-designated minimally acceptable score of 850 or higher on the MC. Of the 128 students who made at least a minimally acceptable A2 math score, only 20 (15.63%) achieved a satisfactory MC score, and of the 121 students who made at least a minimally acceptable reading comprehension score, only 12 (9.92%) achieved a satisfactory MC score.

In the final semester of the ADN program, 77 of the original 139 student cohort (55.40%) remained in the program, and all 77 took the E2. All 22 students (100%) who achieved the faculty-designated minimally acceptable MC score of 850 achieved the faculty-designated minimally acceptable score of 850 or higher on the E2. Of the 77 students who took the E2, 58 (75.32%) achieved the faculty-designated minimal HESI score of 850 or higher on their first attempt at taking the E2. Of the 58 who made satisfactory E2 scores, 55 (94.83%) were successful on their first attempt at taking the NCLEX-RN.

Pearson correlations were calculated to determine the relationship between students' A2 math scores and their MC scores and between students' A2 reading comprehension scores and their MC scores. Although A2 math scores were not significantly correlated to MC scores (r = .129), A2 reading comprehension scores were positively correlated to MC scores (r = .412; p = .01). A Pearson correlation was also calculated to determine the relationship between MC scores and E2 scores. Findings indicated that MC scores were positively correlated with E2 scores (r = .617; p = .01), and the first version of the E2 was 94.83% accurate in predicting NCLEX-RN success (see Table 1).

Table 1. Pearson correlations of HESI examinations and predictive accuracy of the HESI E2
ExaminationCorrelation with examination scorePearson correlation
Math A2MC.129
Reading comprehension A2MC.412⁎
MCE2.617⁎

Note. The E2 was 94.83% accurate in predicting NCLEX-RN success.

⁎p < .01.

Back to Article Outline

6. Discussion 

Only two A2 examinations, math and reading comprehension, were administered to the study population, and it is possible that greater predictability of student success could have been achieved by administering additional examinations that are provided by the A2. The study population's reading comprehension scores were significantly correlated with their MC scores, and it is possible that the other two A2 English language examinations, vocabulary and grammar, could have provided additional predictive value in terms of assessing student success. Admission requirements for applicants to the ADN program that participated in this study included satisfactory completion of a college-level anatomy and physiology course. Therefore, scores on the A2 science examinations, particularly the anatomy and physiology examination, might have been useful in assessing students' ability to succeed in the ADN program. The student population's math scores were not significantly correlated with their MC scores. This finding could have been related to allowing the use of calculators to compute the math problems presented. In 2005, after the study population took the A2 math examination, the test blueprint for this examination was changed to include more thought problems that require critical thinking math skills to answer. Further research is needed to determine if this change to the A2 math examination test blueprint has improved its usefulness in assessing student success.

Only 21.78% of the study population achieved a score of 850 or higher on the MC. Two possible explanations for this population's poor MC performance were considered by the faculty. First, no consequences were associated with MC outcomes. MC scores were only used to assess students' academic status midway through the program and to identify their remediation needs. Therefore, it is possible that students did not take this examination seriously, nor did they prepare adequately for the examination. Consequently, A2 findings in relation to the students' MC scores may have been spurious because the MC scores may not have reflected the students' actual capabilities. The second possible explanation for the students' low MC scores might be related to the faculty-determined benchmark score for the A2, which was 70%. HESI recommends a score of 75% for students entering an RN program. It is possible that the 17 students who scored between 70.00% and 74.99% needed additional remediation and support to be successful in the nursing program. This possibility is supported by the fact that the attrition rate for this cohort was 44.60%. Of the original 139 students, 62 did not complete the program in 2 years; 38 withdrew in the first year of the curriculum, and 24 withdrew in the second year of the curriculum.

The student population's MC scores were 100% accurate in predicting their E2 scores. This finding suggests that administration of standardized examinations within an ADN program provides predictive information about students' ability to succeed in a nursing program, as well as information about their need for additional support or remediation. However, the high degree of accuracy (100%) of the MC in predicting E2 scores may have been related to the fact that only 21.78% of the student population achieved satisfactory MC scores. It would be worthwhile to assess the predictive accuracy of the A2 in terms of MC scores, as well as the predictive accuracy of the MC in terms of E2 scores, in a student population whose nursing program attaches consequences to MC scores. The testing policy designed by ADN faculty at the participating program did associate consequences with students' E2 scores. Students who did not achieve the faculty-designated benchmark score of 850 were required to participate in individualized remediation. This remediation focused on content areas for which the student scored below 850. Following remediation, students were required to retest with another version of the E2. To progress to graduation, students were required to follow this remediation and retesting process until the faculty-designated benchmark E2 score of 850 was achieved. This consequence associated with obtaining the benchmark E2 score may have been related to the high degree of accuracy (94.83%) that the first-version E2 had in predicting NCLEX-RN success. Of the 77 students who took the E2, 58 (75.32%) achieved the faculty-designated benchmark score the first time they took the E2. All 19 students who were unsuccessful in achieving the benchmark score on the first version of the E2 were successful on the second or third version of the E2. Therefore, all 77 students who matriculated to the last semester of the program achieved the faculty-designated benchmark score of 850 within three administrations of different versions of the E2 prior to graduation, and all 77 students graduated from the program. Students prepared carefully for the E2 and were remediated based on weaknesses identified in their E2 scoring reports. It is probable that the faculty-designed testing policy that associated consequences with E2 scores—in this case, progression to graduation—was instrumental in this ADN cohort achieving an NCLEX-RN pass rate of 92.21%.

Back to Article Outline

7. Summary 

The A2 reading comprehension examination was predictive of student success as measured by the study population's MC scores. However, it is likely that administration of additional A2 examinations would have increased the value of the A2 in predicting student success. Because the attrition rate was high for this study population (44.60%), the faculty is considering increasing the A2 benchmark score from 70% to 75%, which is the recommended score for admission to RN schools of nursing. Although this population's low MC scores may have been influenced by the low faculty-designated benchmark for the A2 and the fact that no consequences were associated with students' outcomes on this examination, the MC was highly predictive of E2 scores, and the E2 was highly predictive of NCLEX-RN success.

Nursing faculties take seriously their charge to design and revise their school's testing policy. The findings of this study provide worthwhile data for faculty to consider when fulfilling this function and suggest that administration of standardized examinations within an ADN curriculum can assist faculty in identifying student weaknesses so that remediation efforts can be focused on students' individual learning needs.

Back to Article Outline

References 

  1. Briscoe V, Amena M. The relationship of academic variables as predictors of success on the National Council Licensure Examination for Registered Nurses (NCLEX-RN) in a selected associate degree program. The Association of Black Nursing Faculty Journal. 1999;10(4):80–84
  2. Cloud-Hardaway S. Relationship among “Mosby's Assess Test” scores, academic performance, and demographic factors and associate degree nursing students. Dissertation Abstracts International. 1988;49(07):2567B;[UMI No. 8817018]
  3. Collins P. Predicting a passing outcome on the National Council Licensure Examination for Registered Nurses by associate degree graduates. Dissertation Abstracts International. 2002;64(01):142B;[UMI No. 3077374]
  4. Daley L, Kirkpatrick B, Frazier S, Chung M, Moser D. Predictors of NCLEX-RN success in a baccalaureate nursing program as a foundation for remediation. Journal of Nursing Education. 2002;42(9):390–398
  5. Drake C. The predictive validity of selected achievement variables relative to a criterion of passing or failing the National Council Licensure Examination (NCLEX) for nursing students in a two-year associate degree program. Dissertation Abstracts International. 1996;57(01):182A;[UMI No. 9614016]
  6. Gallagher PA, Bomba C, Crane L. Using an admissions exam to predict student success in an ADN program. Nurse Educator. 2001;26(3):132–135
  7. Haas R, Nugent K, Rule R. The use of discriminant function analysis to predict students' success on the NCLEX-RN. Journal of Nursing Education. 2004;43(10):440–446
  8. Hardin, J. (2005). Predictors of success on the National Council Licensing Examination Computerized exam (CAT-NCLEX-RN) in associate degree nursing programs: A logistic regression analysis. Unpublished doctoral dissertation. Texas A&M University, Commerce.
  9. Higgins B. Strategies for lowering attrition rates and raising NCLEX-RN pass rates. Journal of Nursing Education. 2005;44(12):541–547
  10. Johnson A. Panel urges revamp of nursing schools. Retrieved December 28, 2006, from www.nursezone.com2003;
  11. Lauchner K, Newman M, Britt R. Predicting license success with a computerized comprehensive nursing exam: The HESI Exit Exam. Computers in Nursing. 1999;17(3):120–128
  12. Lewis C. Predictive accuracy of the HESI Exit Exam on NCLEX-RN pass rates and effects of progression policies on nursing student exit exam scores. Dissertation Abstracts International. 2005;66(11):154B;[UMI No. 3195986]
  13. Milan C. Identifying predictors of performance for the associate degree graduate nurse on the National Council Licensure Examination for Registered Nurses. Dissertation Abstracts International. 1997;58(08):4143B;[UMI No. 980575]
  14. Morrison S, Adamson C, Nibert A, Hsia S. HESI exams: An overview of reliability and validity. CIN: Computers, Informatics, Nursing. 2004;22(4):220–226
  15. Morrison S, Nibert A, Flick J. Critical thinking and test item writing. 2nd ed.. Houston, TX: Health Education Systems, Inc; 2006;
  16. National Council of State Boards of Nursing . NCLEX statistics from NCSBN. Retrieved December 28, 2006, from http://www.ncsbn.org/461.htm2006;
  17. National Council of State Boards of Nursing . NCLEX statistics from NCSBN. Retrieved January 3, 2007, from http://www.ncsbn.org/Reliability.pdf2007;
  18. Newman M, Britt R, Lauchner K. Predictive accuracy of the HESI Exit Exam: A follow-up study. Computers in Nursing. 2000;18(3):132–136
  19. Nibert A, Young A. A third study on predicting NCLEX success with the HESI Exit Exam. Computers in Nursing. 2001;19(4):172–178
  20. Nibert A, Young A, Adamson C. Predicting NCLEX success with the HESI Exit Exam: Fourth annual validity study. CIN: Computers, Informatics, Nursing. 2002;20(6):261–267
  21. Percoco T. Variables predictive of program and NCLEX success for associate degree nursing students. Dissertation Abstracts International. 2001;62(08):3358B;[UMI No. 3023360]
  22. Swenty C. Use of the RNEE, ACT, and grades as predictors of success in associate degree nursing programs. Masters Abstracts International. 1998;36(05):1333;[UMI No. 1389177]
  23. U.S. Department of Labor . Occupations with the largest job growth, 2004–2014. Retrieved December 28, 2006, from http://www.bls.gov/emp/emptab3.htm2005;
  • † HESI was acquired by Elsevier in January 2006.
  • ‡ The A2 currently consists of seven academically oriented examinations. The science examination was divided into two separate examinations: chemistry and biology. Also, the number of personally oriented examinations was reduced to two: learning styles assessment and personal inventory.
  • § The A2 currently produces two subset scores, the English Language subset score, which is the average of the examinee's reading comprehension, grammar, and vocabulary scores, and the Science subset score, which is the average of the examinee's chemistry, biology, and anatomy and physiology scores.

PII: S1557-3087(07)00036-4

doi:10.1016/j.teln.2007.04.004

Teaching and Learning in Nursing
Volume 2, Issue 3 , Pages 80-84, July 2007

Article Tools