Reflection & Dialogue #7 - High-Stakes Testing, December 2010

High-Stakes Testing
December 2010 

Introduction

The requirement to pass a standardized test as a prerequisite for student progression in nursing programs is a growing and intensifying trend. Because the results can block graduation or deny eligibility to take the NCLEX^TM licensing exam, such testing is called "high stakes." The issues around high-stakes testing have crystallized around NCLEX predictive testing and the serious consequences they have on progression decisions. 

Though licensure exams such as NCLEX are themselves high-stakes, for students, faculty, and schools of nursing, there is no debate about the need to protect the public through standardized evaluation measures of nursing competence. But there are no universally accepted standards for how predictive tests and related policies should be implemented. In order to address this concern, in June 2010, leaders in nursing education, nursing practice, health care, and higher education, comprising the NLN Presidential Task Force on High-Stakes Testing, began to develop policy guidelines for use of end-of-program testing, exploring current thinking and making recommendations for policy guidelines and practices in nursing education programs.

The recommendations in this Reflection & Dialogue incorporate the NLNs core values and strategic mission as an overall frame and consider multiple measures for competency evaluation.

Considerations

• Though test results can be useful in various ways — providing students with information about their knowledge compared to other students using national norms, and helping faculty identify curricular strengths and weaknesses — it is important to guard against using standardized tests to predict NCLEX performance. Requiring a predetermined score for students to graduate and/or to take the NCLEX to ensure that program pass rates remain at state board-prescribed levels can be detrimental to those who have successfully passed all components of the nursing program. Those students may be given no other option than to take the exit examination repeatedly until they achieve the programs designated passing score. 
  
  
• Most commercially available standardized predictive tests provide individual student scores that are linked to a probability of passing the NCLEX-RN. Research has shown that while predictive tests often work well in identifying high-performing students likely to pass the licensing exam, they are much less precise in identifying individual students who will fail the NCLEX (Spurlock, 2006). This distinction in describing the accuracy of a test is especially important when policies that prevent progression or graduation are in place. 
  
  
• Predicting success and failure are important aspects of testing but not at the expense of a students ability to flourish. The use of a single test to determine a students progression or graduation can have a profound deleterious effect on that individual. There are documented studies, for example, where standardized tests themselves show differential functioning for white vs. non-white testers (Stout, 2002; Helms, 2005; Helms, 2006; Solorzano, 2008). The main issue seems to be how data from these tests are used, which speaks more to the predictive fairness issue. Using tests outside of their intended purposes, negatively impacts all test takers especially minority students who can be faced with social bias, stereotype threat, poorer early preparation, etc. 
  
  
• Standardized tests are designed to measure learning and testing abilities at a point in time. Graduates have reported that employers are requesting standardized test results prior to taking the NCLEX and use them in hiring decisions. Additional high stakes are engendered for students when potential employers use tests for unintended purposes and incorrectly such as comparing candidates who were tested at different times in their programs.
  
  
• Students who have suffered negative consequences for performing poorly on standardized tests have filed suit against their nursing programs using a variety of legal bases. Grounds may include breach of contract, lack of due process, and even educational malpractice. For example, students have used the college catalog or program handbook, which may be viewed as a contract between the school and the student, as part of their legal defense. If the catalog or handbook did not include policies about standardized testing, along with potential consequences for progressing, the student could claim that the school breached the contract. Students may claim lack of due process if changes were made to progression requirements while they were enrolled in the nursing program. And if standardized tests were inappropriately placed and utilized, the student could base a legal action on educational malpractice.

Recommendations

Central to these recommendations is a commitment to fair testing practices to ensure that both tests and the decisions based on tests are valid, supported by solid evidence, and fair to all test takers regardless of age, gender, disability, race, ethnicity, national origin, religion, sexual orientation, linguistic background, testing style, ability, or other personal characteristics. Fairness, in this context, means that all test-takers are given a comparable opportunity to demonstrate what they know and how they can perform in the learning area being tested (Code of Fair Testing Practices in Education, 2004).

The recommendations are also developed within the context of an overall need for testing. Faculty feel a tremendous obligation to assess students abilities and assure that they are competent to practice nursing, while recognizing that current approaches to learning assessment are limited and imperfect. Faculty are mindful that tests and evaluative measures are used not only to evaluate student achievement, but, as importantly, to support student learning, and evaluate and improve teaching and program effectiveness.

Recommendations for Faculty

• In making decisions about use of standardized tests, faculty should require comprehensive information about the test, including evidence that the tests under consideration have been developed to minimize cultural bias. Information about norms and norming procedures, reliability and validity should also be reviewed by faculty before making a decision to use any standardized test. 
  
  
• In developing policy based on test results, faculty should include the core principle that multiple sources of evidence are fundamental to evaluate basic nursing competence. This is especially true when high-stakes decisions are based on the assessment.
  
  
• Faculty should teach students about the purpose, use, and results of standardized tests and testing methods within a program. Also, students should receive information about what precautions to take when potential employers request their standardized test results.

Recommendations for Deans, Directors, Chairs

• Provide appropriate resources for faculty and staff to develop knowledge and skill in using multiple approaches to assessing student learning and nursing competence. 
  
  
• With faculty, review standardized testing practices and progression policies based on test results to ensure alignment with fair testing practices.
  
  
• Provide leadership in creating a climate that supports quality improvement, a climate where results from learning assessment, including standardized test results, can be used in the aggregate to inform program improvements.

Recommendations for the National League for Nursing

• Develop and promulgate Fair Testing Guidelines for Nursing Education based on best available evidence on testing practices from nursing and other disciplines.
  
  
• Provide faculty development programming on use of these fair testing guidelines in the development of program testing and progression policies.
  
  
• Explore with NSNA (National Student Nurses Association) the development of a faculty and student bill of rights and responsibilities for testing.
  
  
• Promote research and development on fair testing practices.
  
  
• Promote the development and validation of new approaches to assessment of the knowledge, skills, and competencies essential for quality nursing practice.

Colleagues, these are our thoughts at this point, please join us in this dialogue and of course, the reflection…

1. What are faculty responsibilities in employing fair testing practices and in assuring that their graduates are competent to begin the practice of nursing? How can faculty contribute to the development of stronger, more comprehensive measures of nursing competency?



2. What national standards would both help faculty in your school determine progression and graduation policies and provide legal protection for all parties involved (program, faculty, student, and the public)?



3. Given that NCLEX results have enormous high-stakes consequences to a program when graduates do not pass on the first try, how can students contribute both to their own success and to their programs success vis a vis NCLEX results?

---

References

Joint Committee on Testing Practices, (2004). Code of fair testing practices in education. Retrieved from www.apa.org/science/programs/testing/fair-testing.pdf

Helms, J. (2005). Stereotype threat might explain the black-white test-score difference.American Psychologist, 60(3), 269-270. doi:10.1037/0003-066X.60.3.269

Helms, J. (2006). Fairness is not validity or cultural bias in racial-group assessment: A quantitative perspective. American Psychologist, 61(8), 845-859.

Solórzano, R. (2008). High stakes testing: Issues, implications, and remedies for English language learners. Review of Educational Research, 78(2), 260-329.

Spurlock, D. R. (2006). Do no harm: Progression policies and high-stakes testing in nursing education. Journal of Nursing Education, 45(8), 297-302.

Stout, W. (2002). Psychometrics: From practice to theory and back: 15 years of nonparametric multidimensional IRT, DIF/test equity, and skills diagnostic assessment. Psychometrika, 67(4), 485-518.