DO NO HARM- HIGH STAKES TESTING AND STUDENTS WITH LEARNING DISABILITIES A report by Disability Rights Advocates Background - Disability Rights Advocates and the Struggle for Equal Rights for Students with Disabilities Disability Rights Advocates (DRA) is a non-profit public interest law center located in Oakland, California, dedicated to advancing the rights of people with disabilities. In just a few short years, Disability Rights Advocates and others in the learning disability community have borne witness to several significant advances in the ongoing struggle for equal opportunity and dignity. The foundation for many of the achievements was laid in 1997, when a brave group of college students with learning disabilities won a class action trial against Boston University for its discrimination against the learning disability community. The case, in which DRA was lead counsel, resulted in a comprehensive order from a federal judge and provided a roadmap for the safeguards which colleges and universities must provide to ensure that students with learning disabilities obtain reasonable accommodations, opportunities and dignity. That same year, an equally courageous woman, Marilyn Bartlett, along with her attorney and clinician, JoAnne Simon and Dr. Rosa Hagin, won another federal trial against the New York State Board of Bar Examiners, in a case which also set the standards for evaluating learning disabilities and providing reasonable testing accommodations. Soon thereafter, DRA teamed up with the International Dyslexia Association and a coalition of learning disabled individuals in California to force the California Bar to overhaul its accommodation procedures on the State's bar exam so that law school graduates with learning disabilities could have a fair opportunity to obtain their professional licenses to practice law. More recently, DRA worked with IDA and others to reach a settlement with the Educational Testing Service to discontinue the discriminatory practice of "flagging" students with disabilities who use accommodations on ETS examinations such as the GRE and GMAT. In addition, as part of this case, a Blue Ribbon Panel of disability and testing experts will review the appropriateness of flagging on the SAT-that "Big Test" which is owned and controlled by the College Board. Finally, we are pleased to announce the issuance of the report from the Blue Ribbon Panel of experts convened by DRA and the Oregon Department of Education to address the interplay between standardized examinations and students with learning disabilities. The expert report describes in detail the accommodations, alternate assessments, appeals procedures, and other safeguards which must be implemented for the statewide assessment system to comply with the law and guarantee educationally sound opportunities to students with learning disabilities. Although there is still much work to be done, these victories mark important developments in the continuing movement to bring equal opportunity to individuals with learning disabilities in the schools, the professions and beyond. Table of Contents I. Introduction II. The Interplay of Learning Disabilities and Standardized Tests Learning Disabilities Such as Dyslexia Severely Impair a Person's Ability to Read and Visually Process Information Many Students with Learning Disabilities Also Have Great Difficulty with Spelling, Punctuation and Grammar, as Well as with the Physical Act of Writing The Tests Often Test the Disability, Rather than the Ability of the Student Handwritten Tests Can Unfairly Disadvantage Students with Learning Disabilities Many Multiple Choice Tests Unfairly Impact Dyslexic Students Because They Do Not Provide Sufficient Context III. Numerous Federal Laws Protect Students with Learning Disabilities in the Testing Context Students with Learning Disabilities Must Be Provided with Equal Access to Educational Programs, Services and Activities Students with Learning Disabilities Who Can Benefit from a Reasonable Accommodation must Be Provided All States Are Legally Required to Have in Place Alternate Assessments for Any Disabled Student Who Is Disadvantaged by an Assessment System All Learning Disabled Students Are Supposed to Have Either an Individualized Education Program (IEP) or Section 504 Plan That Provides for Their Individual Needs Constitutional Due Process Requires That High Stakes Assessments Be Phased in over a Sufficient Amount of Time High Stakes Assessments Must Be Valid IV. Implementing the Law, Lessening the Disadvantage Adequate Accommodations are One Way to Lessen the disadvantage for some students with learning disabilities Students Should Not be penalized for using accommodations on the assessments Alternate Assessment Systems Are Necessary Due to Testing Limitations and the Nature of Learning Disabilities Who Should Take the Alternate Assessment There Are a Number of Methods by Which Learning Disabled Students Can Be Alternately Assessed Students Should Not Be Penalized for Participating in an Alternate Assessment There Must Be Clear and Easily Accessible Appeal Processes V. High Stakes Assessments in Several States Alaska California Oregon Sidebars 1. Thirteen Core Principles to Ensure Fair Treatment of All Students, Including Those with Learning Disabilities, with Regard to High Stakes Assessments Page 15 2. Essential Elements of a Fair and Non-discriminatory Assessment System for Students with Learning Disabilities Resources Endnotes Introduction to Appendix Appendix: Blue Ribbon Panel Majority Report Blue Ribbon Panel Minority Report I. Introduction In recent years, a trend toward high-stakes testing in public schools has swept the country. As of Fall 2000, twenty-three states require students to pass a high school "exit exam" to receive a high school diploma; another seven states plan to adopt exit examinations within the next three years. Many other states use tests to make other types of high-stakes decisions, such as decisions about whether a student is eligible for scholarships, advanced placement, and honors classes. Approximately thirteen states use standardized tests to determine whether a student is promoted or retained. Some states have proposed using test results to determine eligibility for state universities or even employment. The results can be catastrophic. Whether a student is retained in a grade is the strongest predictor of whether that student will drop out of school. High-stakes testing achieved particular prominence during the election campaign of 2000 with candidates for both major parties embracing standardized testing as a fundamental element of educational "reform." Both presidential candidates argued that increased standardized testing is essential to hold schools accountable and to raise educational standards, and both candidates pledged to institute greater standardized testing if elected president. Nowhere in the debate, however, was there recognition of the potential negative consequences of high-stakes standardized testing, nor was there a discussion of the safeguards that must be in place in order to ensure that standardized testing is carried out in a fair and nondiscriminatory manner. The reality is that many of these tests are not properly developed or implemented, leading to high failure rates, increased numbers of students dropping out of school, and loss of self-esteem and educational advancement. In Oregon, for example, approximately 95 percent of disabled students failed a recent round of testing. Often the tests are hastily or sloppily devised, with insufficient attention given to the groundwork necessary to develop a technically sound assessment. These tests thus are often of very questionable validity for students with disabilities, and most states have not given sufficient consideration to either appropriate accommodations or alternative assessments for disabled students. In addition, because implementation of the high-stakes tests is frequently rushed, students are not given enough time to prepare. This paper focuses on the needs and rights of one large population-students with learning disabilities-with regard to high-stakes standardized tests1. Experience has demonstrated that when high- stakes examinations are implemented, the needs and rights of learning disabled students are usually overlooked. Section II of this paper analyzes the nature of learning disabilities and how the design of many standardized tests discriminate against students with those disabilities. In theory, these students are protected by federal law from discrimination on these examinations. Accordingly, high-stakes standardized testing should not be imposed on students with learning disabilities unless those tests have been properly validated for their use and for those students. In addition, accommodations and alternate assessment systems must be made available to disabled students. Because development and implementation of these systems take many years, sufficient lead-in time also must be provided. In practice, these protections are often ignored. This paper can help educators and parents of students to learn about the procedures that can help students with learning disabilities with respect to these assessments. Section III contains a brief analysis of the federal laws that protect students with learning disabilities in the testing context. Section IV examines what states and educators can and must do in order to ensure that students with learning disabilities are treated fairly with respect to a high-stakes assessment system. Finally, Section V looks at how high-stakes tests have been implemented in several states across the country, including Oregon, in which a lawsuit brought by learning disabled students led to a comprehensive evaluation of that state's assessment system by a panel of experts in testing and learning disabilities. II. The Interplay of Learning Disabilities and Standardized Tests Learning Disabilities Such as Dyslexia Severely Impair a Person's Ability to Read and Visually Process Information Learning disabilities stem from neurological and sometimes heritable differences in brain structure, and can dramatically impact the manner and duration in which persons with learning disabilities read, write, learn and take tests. They cannot be cured. Virtually all persons with learning disabilities attempt to compensate for their disabilities on their own, and the strategies they acquire painstakingly over a lifetime are inevitably imperfect. An example of such a strategy is running a finger along the page as a person reads to help keep their place. Other strategies include reading aloud, recognizing words by recognizing their shape or guessing from context, and reading the same sentence over and over again. These strategies help people with learning disabilities decode written information. However, they are a compensation and not a solution, and will most likely not ever allow a reader with a learning disability to read as quickly as a person without a learning disability. Many Students with Learning Disabilities Also Have Great Difficulty with Spelling, Punctuation, and Grammar, as Well as with the Physical Act of Writing. In addition to their difficulties with reading, many students with dyslexia and other learning disabilities have problems with spelling, punctuation, and grammar. These problems are not the same as those of a non-disabled students who has some problem with spelling. Because some learning disabled students have difficulty with phonological codes that guide letter and word formation, spelling often becomes tangled with letter transpositions, dropped endings, or word substitutions. In contrast to someone who is simply a poor speller, learning disabled students often spell the same word differently within the same paper. "Other errors include omission of low information load words, such as auxiliaries, modals, prepositions, pronouns, or conjunctions; distorted word order; lack of punctuation; limited word choice, and problems using multiple meanings of words."2 In addition, many students with learning disabilities have difficulty with the act of writing itself due to problems integrating visual and fine motor functions. Sometimes referred to as dysgraphia, this learning disability is characterized by inconsistent spacing between words and letters and by extreme upward or downward slants on an unlined page. For these students, handwriting becomes an impediment to communication. The Tests Often Test the Disability, Rather than the Ability of the Student Standardized tests assume that each student taking the test will read in the same manner. However, a student with a learning disability cannot process words like other students. Every time a learning disabled student reads a word, it is as if he or she is encountering that word for the first time. Reading is thus a slow and tiring process. On these tests, the student is struggling with reading the test, not answering the questions. The students' disability, not his or her ability, is being assessed. One reason that high-stakes assessments have a discriminatory impact on students with learning disabilities is because often when the tests were developed, little no attention was given to how the tests would impact learning disabled test takers. The sample population that is used by test developers to set the average scores for the tests usually does not include students with disabilities. When disabled students are included in the sample population, it is often unintentional, and the performance of these individuals is not separately tracked. Most testing publishers also do not give students with disabilities accommodations they need when testing a sample population, thus leading to a dearth of information and research about the true effect of an accommodation on a testing situation. Handwritten Tests Can Unfairly Disadvantage Students With Learning Disabilities Several states have also required tests to be handwritten, and learning disabled students are not allowed to use computers or typewriters. Not only is it hard to understand any rationale for this requirement in this modern age of technology and computers, but many students with learning disabilities require a computer or wordprocessor as a writing tool because their disability impairs the physical act of writing. Moreover, many of these students have been provided with such technology during their education as part of their legally mandated Individualized Educational Program or Section 504 Plan, and they will be inadequately prepared if required to suddenly handwrite their examinations on high-stakes assessments. Many Multiple Choice Tests Unfairly Impact Dyslexic Students Because they Do Not Provide Sufficient Context Most assessment systems have multiple choice sections. Multiple choice tests generally do not provide sufficient context for learning disabled students, and dyslexic students rely heavily on context as a compensating mechanism to identify words. The disadvantage of multiple choice tests could be eliminated for many learning disabled students through use of audio tests or readers. III. Numerous Federal Laws Protect Students with Learning Disabilities in the Testing Context Federal civil rights statutes protect students with learning disabilities in the educational context.3 The Individuals with Disabilities Education Act, 20 U.S.C.  1400 et seq. ("IDEA"), Section 504 of the Rehabilitation Act of 1973 ("Section 504"), and the Americans with Disabilities Act of 1990 ("ADA") all prohibit schools from discriminating against students on the basis of their disabilities. These laws, and the regulations under them contain specific prohibitions relevant to the high-stakes standardized testing and its impact on students with learning disabilities. Students With Learning Disabilities Must be Provided with Equal Access to Educational Programs, Services and Activities Learning disabled students are guaranteed equal access to all of the benefits of programs, services and activities offered by a school and school district. This includes programs and activities such as diplomas, honors programs, graduation, advanced placement classes and summer programs, that are linked, even in part, to performance on a standardized assessment. In particular, federal laws prohibit educational entities from: (1) Denying learning disabled students the opportunity to participate in or benefit from aids, benefits and services provided by educational entities;4 (2) Providing disabled students with benefits that are different or not as effective in affording equal opportunity to obtain the same results, to gain the same benefits, or the reach the same levels of achievement as that provided to others;5 (3) Imposing eligibility requirements that screen out or tend to screen out an individual with a disability or any class of individuals with disabilities from fully and equally enjoying any service, program, or activity offered by schools and educational entities;6 (4) Utilizing criteria or methods of administration that have the effect of subjecting disabled students to discrimination on the basis of disability or that have the purpose or effect of defeating or substantially impairing accomplishment of the objectives of an educational entity's program with respect to individuals with disabilities;7 and (5) Operating a certification program that subjects students to discrimination on the basis of disability.8 A state cannot, for example, require that all students run a 100 yard dash as part of a high school "exit exam" because such a test discriminates against students with mobility disabilities. The state would have to exempt mobility disabled students from that portion of the exam, or otherwise accommodate them by assessing them by an alternate method. Similarly, a state cannot legally implement a standardized assessment that directly tests a student's disability, such as dyslexia, by heavily emphasizing spelling or the ability to read quickly without (a) exempting dyslexic students from that portion of the test, (b) providing a reasonable accommodation such as a spell-checker, and/or (c) providing an alternate assessment as discussed below. Students with Learning Disabilities Who Can Benefit From a Reasonable Accommodation Must Be Provided with It Federal laws also require that students with learning disabilities be provided with reasonable accommodations and auxiliary aids and services when those accommodations will help avoid discrimination or unfair treatment on the basis of disability.9 Examples of reasonable accommodations that students with learning disabilities might require for some tests include extended time, the use of computers or wordprocessors (with spell-checkers for dyslexic students), dictation machines, the use of a scribe or reader, and audio test materials.10 These laws guarantee students with learning disabilities the right to reasonable accommodations on standardized assessments so that the examinations accurately measure the skills they are designed to measure, rather than the limitations caused by the disability. Dyslexic students who are required to handwrite the examinations will perform poorly because their learning disability will prevent them from expressing their actual math or writing skills. Schools also may not penalize students for using reasonable accommodations. Federal laws prohibit educational entities from using policies that deter learning disabled students from applying for testing accommodations.11 All States are Legally Required to Have in Place Alternate Assessments for Any Disabled Student Who Is Disadvantaged by an Assessment System Amendments to the Individuals with Disabilities Education Act (IDEA) required all educational entities to have, by July 2000, alternate assessments for disabled students who cannot participate in standardized assessments.12 Although some states have designed alternate assessments limited only to those students with severe developmental disabilities, this is not sufficient. For example, the U.S. Department of Education commentary on the alternate assessment requirement states that "alternate assessments need to be aligned with the general curriculum standards set for all students and should not be assumed appropriate only for those students with significant cognitive impairments."13 The requirement for alternate assessments by July 2000 only confirmed an existing obligation. Even if Congress had not passed these IDEA amendments, alternate assessments for learning disabled students would still be required because federal civil rights laws impose on schools and States the obligation not to discriminate against students with learning disabilities. Once a State decides to administer an assessment to students, the State also has an obligation to ensure that learning disabled students are fairly assessed and not discriminated against. A necessary part of that process is providing alternate assessments to those students who, by virtue of their learning disabilities, will not be fairly assessed on part or all of an assessment system. There is no educational or legal basis for limiting the availability of alternate assessments. An alternate assessment should be available to all students for whom all or part of the general assessment is not appropriate, and should have no pre-set artificial caps limiting the alternate assessments to a certain percentage of the population. Instead, the governing principle should be to design assessments which ensure that students with learning disabilities are evaluated in a manner that is fair. All Learning Disabled Students Are Supposed To Have Either an Individualized Education Program (IEP) or Section 504 Plan that Provides for their Individual Needs Federal law requires that students with learning disabilities have either an Individualized Education Program (IEP) or Section 504 Plan that specifies a student's individual needs. These plans are required by law to be created through an interactive process between educators, parents and their students. An IEP for a student with dyslexia, for example, might describe a special curriculum for the student to study, as well as the need for a computer and/or wordprocessor for homework and tests. Amendments to IDEA provide that how and whether a student will participate in a general assessment or an alternate assessment of any accommodations that a student will use, must be addressed in an IEP.14 If the IEP team determines that a student will not participate in the general assessment, the IEP must contain a statement to that effect, as well as how that student will be assessed.15 Although Section 504 of the Rehabilitation Act does not contain an analogous requirement for Section 504 plans, 504 plans provide an appropriate forum for making decisions about participation in assessments. Both IDEA and Section 504 regulations also require grievance procedures that incorporate appropriate due process standards and provide for the prompt resolution of complaints of discrimination. Students and their parents must be able to challenge decisions about what accommodations a student will be able to use, as well as the decision regarding participation in alternate assessments. Constitutional Due Process Requires that High-stakes Assessments be Phased In Over a Sufficient Amount of Time Due process guaranteed by the United States Constitution requires that assessments may not be fundamentally unfair or arbitrary. Realistically, a fair phase-in period will almost always require a number of years to implement. Courts have found assessments unconstitutional when students were held accountable under high-stakes assessments when they were not given adequate notice and sufficient opportunity to prepare for the tests.16 At a minimum, unless students have actually been taught the material that is tested, it is fundamentally unfair to hold them accountable. Courts have held that students with disabilities may require more advance notice and more substantial opportunities to prepare for such testing than may be otherwise necessary.17 This is so because, unlike students without disabilities, students with individualized education plans do not necessarily focus their academic efforts on the precise goals set by the school or district goals - they concentrate instead on meeting the personalized educational goals established for them. They must, therefore, have substantial notice, sufficient exposure to most of the contents of the test, and time to have the test objectives adequately incorporated into their IEPs. In order to ensure that students with learning disabilities have sufficient notice and preparation, there should be a lead time of at least four to six years before high-stakes consequences are attached to assessments. High-stakes Assessments Must Be Valid It is a basic principle of test validity that a test must actually measure what the test is intended to measure. A test is not valid in itself, but is valid only for a particular purpose. A test that is valid for one purpose may be invalid for another. As noted in a recent report by The National Research Council of the National Academy of Sciences, "[t]ests that are valid for influencing classroom practice, "leading" the curriculum, or holding schools accountable are not appropriate for making high-stakes decisions about individual student mastery unless the curriculum, the teaching, and the tests are aligned." (Emphasis added.)18 The responsibility for test validity lies with both the test developer and the test user. In particular, the test user must determine that there is evidence to support intended uses for a test before using it in a particular way.19 Unfortunately, many assessments are used to make high-stakes decisions without any evidence that it is appropriate to make these decisions based on the tests. Another problem is that although the tests are often required by the states, the states often give school district and schools the discretion to use the tests in any manner the districts wish. As a result, districts and schools often use assessments for purposes that were unintended and not validated by the test developer. Given the widespread absence of specific validity, the frequent failure to provide both reasonable accommodations and alternate assessments, and the general failure to align curriculum and teaching with tests for disabled students, it will often be unfair, illegal, and educationally unsound to impose high-stakes individual consequences for standardized tests on students with disabilities. IV. Implementing the Law, Lessening the Disadvantage Adequate Accommodations are One Way to Lessen the Disadvantage for Some Students with Learning Disabilities Decisions about the accommodations a student will need to participate in an assessment system should be made by his or her IEP or Section 504 team, with the full participation and involvement of the student's parents. These are the people who know and understand a student's needs best, and are best suited to make individualized decisions about participation in the assessment system. For example, an automatic spell-checker will only be an appropriate accommodation for dyslexic students who have some degree of proficiency with spelling.20 Many dyslexic students will not be able to select the properly spelled word because their disability so impairs their ability to identify words. For these students, an automatic spell-checker will be of little use, and for these students the IEP or Section 504 team may decide that they should be exempted from this portion of the assessment, or assessed in an alternate way. Students Should Not Be Penalized for Using Accommodations on the Assessments The test results of disabled students who participate in the assessment with accommodations should count for whatever purposes the assessment system has been validated. Similarly, while it may be desirable to maintain data on the participation of learning disabled students in the assessment, the scores of these students who participate with accommodations should not be "flagged" in any way that will have a stigmatizing effect. Test publishers often have not conducted adequate research on how accommodations affect test validity. As a result, test publishers label a number of accommodations as "non-standard," or "modifications," often because it is not clear how they affect test validity. Schools may withhold the benefits of doing well on a standardized exam from a student who uses a non-standard accommodation. However, penalizing a student for using a non-standard accommodation is comparable to not allowing them to participate in the test at all. It is unfair and discriminatory to penalize a student with a disability for using a needed accommodation on an assessment simply because the test publisher has not conducted the necessary research about the effect of the particular accommodation on the test. Alternate Assessment Systems are Necessary Due to Testing Limitations and the Nature of Learning Disabilities Many learning disabled students will not be fairly tested if they do not take an alternate assessment, regardless of the accommodations provided because the learning disabled student is inherently disadvantaged by many standard assessments. First, many tests, in fact, assess the student's disability or test-taking ability. Because students with reading disabilities such as dyslexia process information differently than non-disabled students, accommodations will simply not be enough to level the playing field on a test that assumes all students are reading, writing, and learning in the same manner. Second, the compensating strategies that students with learning disabilities use are at odds with the nature of many tests. Learning disabilities are permanent conditions and learning disabled students often have to use a number of compensation clues. These strategies may be useless on a test that does not provide enough text for these clues to be useful. As a result, providing more time on these examinations or allowing a learning disabled student repeated opportunities to take an assessment are not solutions because they ignore the permanent nature of learning disabilities and incorrectly assumes that a learning disabled student will be able to correct mistakes caused by his disability. Moreover, the learning disabled student will be far more susceptible to the stresses during test taking than a non-learning disabled student. Negative side effects of these stresses include diversion from the material, high energy consumption, anxiety and frustration at levels not encountered in other students - all of which can further inhibit performance. Accordingly, more than one alternate assessment must be developed. There must be an alternate assessment developed for students who are not working to strictly academic standards. To be fair, there must also be an alternate assessment based on the same standards of the regular assessment. Many learning disabilities do not necessitate a different set of standards, but they do require that the student learn them and be tested in them in a manner better suited to their needs. Who Should Take the Alternate Assessment? The IEP or 504 team and the parent or guardian of the student should jointly decide what accommodations and assessment the student will take. In order to make an informed decisions, parents should always have access to clear information about the test, the test purpose, and options concerning the test. To ensure that this occurs, the State Department of Education needs to make that information available and to have a system in place to get the information to parents and teachers. The student should participate in decisions on accommodations and assessment systems when the team and the parents find it appropriate. It is extremely important that these decisions be individualized and that the best interests of the student be the highest priority. Decisions should not be governed by arbitrarily set caps on the number of students taking the alternate assessment. Caps will prevent students from getting the services they need, in addition to other permanent damages, such as losing the right to advance a grade, graduate or attend college. Furthermore, the sometimes quoted cap of 1-2% on students taking the alternate assessment incorrectly assumes that only students studying functional curricula need alternate assessments. The decision of whether to take the alternate assessment also should not rely on pre-fabricated checklists. While the type of curriculum should certainly be taken into account, determining the way a student is assessed must be based on several factors, including: A. The nature of the student's disabilities. 1. How does the disability effect the ability to take standardized tests? 2. Does the test penalize the student for his or her learning disability? 3. If the student has an emotional disability, what will be the effect and pressure of taking a standardized test on the demonstration of her ability? B. The accommodations the student receives in the classroom. 1. Are these accommodations applicable to test taking? 2. If not, what other accommodations are available that can replace those classroom accommodations? 3. What sort of accountability (school or student) are the test results used for? C. The standards being tested D. The student's previous exposure to testing. 1. For students with emotional or behavioral disabilities, to what extent has the student faced the pressures of rigorous testing? 2. Does the test design disadvantage the student with his disabilities? For instance, if the test is multiple choice, a student with dyslexia who relies on context may not be fairly assessed with any accommodations and may require an alternate assessment. E. The accommodations listed on the IEP or Section 504 Plan. F. The advantages and consequences of performance on the test. G. Whether the exam or the standards being assessed directly test the student's disability. (If the answer is yes, then the student should receive an alternate assessment for at least that portion of the exam). There are a Number of Methods by Which Learning Disabled Students can be Alternately Assessed A number of different performance assessment techniques are available in developing an alternate assessment. These include: 1. Portfolio-based assessments. These have been developed in Britain, Australia, and the U.S. For example, Vermont has instituted statewide assessment programs in math and writing based on student portfolios, which are files or folders that contain a collection of the student's work. 2. Interviews and oral presentations, which allow a student to verbalize their knowledge. 3. Constructed responses, which require a student to produce his or her own answer rather than select from an array of answers. 4. Experiments, which can test how well a student understands scientific concepts. 5. Projects, which can include demonstrations of skills and knowledge requiring a broad range of competencies. These are often interdisciplinary. This is only a partial list. A plan for an alternate assessment can combine these and other methods. In fact, having only one alternate assessment will likely not be adequate given the broad range of disabilities, ages, and information which need to be addressed. One possibility is a flexible array of alternate assessments from which the IEP or Section 504 team chooses the most appropriate for a particular student.21 Students Should not be Penalized for Participating in an Alternate Assessment So that learning disabled students are not penalized or stigmatized for participating in the alternate assessment, the alternate assessment also must be evaluated by a method that is equal to and indistinguishable from the system used in the mainstream assessment. Those students who participate in the alternate assessment should be eligible for all of the programs, services and activities as students who participate in the general assessment system. If the score is "flagged" or uses a different measurement, colleges, employers, and anyone else who uses this data will be able to discriminate against the learning disabled student. There Must be Clear and Easily Accessible Appeal Processes Students and their parents must be provided with processes by which they can appeal the denial of accommodations or the denial of a request to participate in an alternate assessment. In addition, given the high-stakes nature of the some of the tests, there must be a process for appealing the results of assessments for all students. Although the parent or parents should participate in the decision making process as part of the IEP or Section 504 team, they may disagree with the final result. The IEP and Section 504 processes are forums for such appeals, but are often unwieldy, and thus it is recommended that states set up an appeals process in addition to the IEP and 504 appeals processes for dealing with appeals concerning high- stakes assessments. V. High Stakes Assessments in Several States The experience of several states illustrates the problems that can arise for learning disabled students when high-stakes assessments are implemented. Alaska In 1997, the Alaska legislature enacted a law requiring all Alaska high school students to pass an exit exam in order to receive a high school diploma. Students who do not pass the examinations will receive a Certificate of Attendance. Under current law, the exam takes effect in 2002, and thus students graduating in 2002 are the first students required to pass the test. The exam was administered for the first time to students in the spring of 2000, raising serious concerns about the short lead in period for students to prepare for the tests. Recently, the Governor of Alaska, the State Board of Education, and other groups have advocated extending implementation of the exit exam to 2004 due to concerns that the exit exam has not been properly aligned to the curriculum and that if the current schedule is maintained, a large percetage of students would not receive a diploma in 2002. The Alaska exit exam raises additional concerns for students with disabilities because the State has disallowed certain accommodations such as spell checkers, grammar checkers and calculators. The list of allowable accommodations is quite limited, and a student must petition his or her IEP or Section 504 team in order to use an accommodation that is not on the State's approved list. California Under the direction of Governor Gray Davis, California will administer its new high school exit exam for the first time during the spring of 2001. The exam, touted by Governor Davis as the toughest in the nation, tests such challenging topics as algebra, despite the fact that algebra is not currently a graduation requirement. The State has announced that it intends to add geometry and statistics to the exit exam in the future. There are serious concerns about an insufficient phase-in period and many other deficiencies. Although the exit exam is being implemented during the 2000-2001 school year, the State acknowledges that there is no alternate assessment for students with disabilities. Moreover, as of the time of this analysis there is mass confusion concerning accommodations (there are no final guidelines about what accommodations are allowed on the exit exam), no guarantee of curricular alignment and no appeals process. Oregon In 1999, Oregon implemented a high-stakes assessment system under which sophomores were required to pass a series of tests in order to achieve a "Certificate of Initial Mastery." Schools and districts made a number of high-stakes decisions based on a student's performance on these tests, including decisions about whether a student could graduate from high school and whether a student was eligible for scholarships. A group of learning disabled students and their parents filed suit challenging the assessment as discriminatory in a number of ways. The list of allowable accommodations was extremely limited, and as a result many learning disabled students were denied accommodations on the tests that were listed in their IEPs or Section 504 plans. Another problem with the examinations was that there was no alternate assessment appropriate for students with learning disabilities. In an effort to facilitate settlement, the parties in that case appointed a "blue ribbon panel" of experts in learning disabilities and testing to evaluate Oregon's system and make recommendations on how students with learning disabilities can be fairly included. This is the first time a high-stakes assessment system has been independently examined by experts to determine the impact on learning disabled students and make proposals to ensure that students with learning disabilities are protected. The panel's report is attached, along with Disability Rights Advocates' brief commentary on the panel report. In sum, the panel system recommended that Oregon institute comprehensive reforms in order to ensure that students with learning disabilities are not discriminated against. Thirteen Core Principles to Ensure Fair Treatment of All Students, Including Those with Learning Disabilities, with Regard to High Stakes Assessments Endorsed by The International Dyslexia Association, National Center for Learning Disabilities, and Learning Disabilities Association of America 22 1. The needs and rights of students with learning disabilities must be vigorously protected to ensure that these students (a) have an equal opportunity to participate in and attain all of the benefits of high stakes assessment programs and (b) to ensure that they are not disadvantaged or discriminated against on the basis of disability with regard to such assessments. 2. Students identified as having learning disabilities must have access to the general education curriculum throughout their school years and to general education courses and curriculum in secondary school to ensure that they will be best able to demonstrate their intelligence, abilities, knowledge, and skills on high stakes assessments. 3. In designing and implementing high stakes assessments, educators and administrators must protect the rights and needs of students with learning disabilities and ensure that the assessments do not discriminate against students with learning disabilities. 4. In designing and implementing remediation options for students failing or performing poorly on high stakes assessments, educators and administrators must protect the rights and needs of students with learning disabilities and ensure that the remedial assistance available does not disadvantage or discriminate against students with learning disabilities. 5. Political and administrative considerations, such as how the inclusion of students with learning disabilities in high stakes assessments will affect reporting of scores of schools and districts, must not be allowed to (a) override the rights and needs of students with learning disabilities, (b) adversely affect the benefits to learning disabled students once included, or (c) place the providers of special education at a disadvantage because of administrative or funding consequences of non-participation. 6. As determined during the Individualized Education Program and Section 504 processes, and in compliance with the needs of each individual student, students with learning disabilities must be provided with all necessary accommodations for their learning disabilities on high stakes assessments. 7. Students with learning disabilities must be provided the same accommodations on the assessments that they have used during their educational careers. 8. Within the IEP and Section 504 processes there must be fair, neutral and clear processes by which students with learning disabilities and their parents can appeal decisions about the accommodations provided on the assessments. The availability of these appeal processes should be clearly communicated to students and their parents. 9. There must be a meaningful alternate assessment system available to children who are disadvantaged by the high stakes assessments as a result of a learning disability. 10. There must be procedures in place to ensure the fair assessment of students with learning disabilities who take alternate assessments, as well as the fair assessment of those who take standard assessments with accommodations. 11. Whether a student takes a standard assessment (or portion thereof) or is assessed under an alternate assessment (or portion thereof) should be decided by a student and his or her parents in concert with school personnel, and should be addressed during the Individualized Education Program and Section 504 processes. 12. The number of students who participate in an alternate assessment should not be artificially limited to a certain percentage, but should be available to all children whose Individualized Education Programs or Section 504 Plans show that they would benefit. 13. States and school districts should accumulate data and encourage research on the effects of high stakes assessment on the education of all children, including those who receive accommodations and those who participate in alternate assessments. Essential Elements of a Fair and Nondiscriminatory Assessment System For Students with Learning Disabilities  Lead Time: Sufficient advance time (4-6 years) so that students with learning disabilities will not be negatively impacted by an assessment system.  Validity: An assessment system which actually tests what it was designed to measure and only for purposes and categories of students that have been validated.  Testing What Was Taught: Guarantees that students are actually tested on material that they have learned in the classroom.  Accommodations: Allowable accommodations that include any accommodation previously used and is listed in a student's IEP or Section 504 Plan.  Parent Involvement: Decision making about a student's participation in the assessment system by IEP and Section 504 teams.  Test Ability, Not Disability: Examinations that do not test a student's learning disability (e.g., tests that are required to be handwritten or that heavily emphasize spelling).  Limit on High Stakes Test Use: Rarely, if ever, should standardized tests be given high-stakes consequences for students with disabilities.  Alternate Assessments: Alternate assessments that allow learning disabled students to demonstrate their knowledge, rather than the effects of their disabilities.  Appeal Procedures: Procedural safeguards to ensure that students are able to contest decisions about accommodations, alternate assessments, and scores on assessments.  Information: Provision to students and parents of clear and accurate information about the assessment system, accommodations, alternate assessments, and appeals.  No One Measure: Multiple measures of student performance utilized in the assessment system Resources Department of Education Office of Civil Rights (202) 205-8679 Department of Justice (800) 514-0301 (voice) (800) 514-0383 (TTY) The following are primarily resources for systemic problems and not individual or IEP issues: Disability Rights Advocates 449 15th Street, Suite 303 Oakland, CA 94612 (510) 451-8644 (voice) (510) 451-8511 (fax) (510) 451-8716 (TTY) Fair Test 342 Broadway Cambridge, MA 02139 (617) 864-4810 (voice) (617) 497-2224 (fax) Website: www.FairTest.org, e-mail: FairTest@aol.com The International Dyslexia Association, International Office 8600 LaSalle Road, Chester Building, Suite 382 Baltimore, MD 21286-2044 Messages (800) ABCD123 (410) 296-0232 (voice) (410) 321-5069 (fax) Learning Disabilities Association of America 4156 Library Road Pittsburgh, PA 15234 (412) 341-1515 (voice) (412) 344-0224 (fax) National Center for Learning Disabilities 381 Park Avenue South, Suite 1420 New York, NY 10016 (212) 545-7510 (voice) Endnotes to "Do No Harm" 1. Students with learning disabilities are only one of many groups being discriminated against by these high-stakes standardized tests. Students with physical disabilities, students who do not speak English as their primary language and students of color have also suffered educational damage. See, e.g., Graduation and Promotion Testing: Potential Benefits and Risks for Minority Students, English Language Learners and Students with Disabilities. Poverty and Race, Sept.-Oct. 2000, Vol. 9, No. 5, p. 1. 2. Sills, Caryl K., Success for Learning Disabled Writers Across the Curriculum, 43(2) College Teaching 66 (1995). 3. In addition to federal law, there are numerous state laws that provide protection for students with learning disabilities with regard to high-stakes testing. An analysis of these laws is beyond the scope of this paper. 4. See 42 U.S.C.  12182(b)(1); 28 C.F.R.  130(b)(1)(i)-(ii). These citations are to both the federal laws and the implementing regulations. 5. See 42 U.S.C.  12182(b)(1); 28 C.F.R.  25.130(b)(1)(iii)-(iv). 6. See 42 U.S.C.  12182(b)(2)(A); 28 C.F.R.  130(b)(8). 7. See 42 U.S.C.  12182 (b)(D); 28 C.F.R.  35.130(b)(3). 8. See 42 U.S.C.  12182(b)(2)(i); 28 C.F.R.  35.130(b)(6). 9. See 42 U.S.C.  12182(b)(2)(A)(ii)-(iii); 28 C.F.R.  35.130(7). 10. These accommodations may not be sufficient to level the playing field. For example, extended time may not make any difference for dyslexic students on multiple choice tests because multiple choice tests generally do not provide sufficient context to allow these students to decode and identify words. Similarly, automatic spell-checkers will only assist those students who have enough spelling proficiency so that the student can identify the properly spelled word out of a list of words. 11. See, e.g., 42 U.S.C.  12182(b)(1)(D) & (b)(2)(A); 28 C.F.R.  35.130(b)(8); 34 C.F.R.  104.4(b)(4); 45 C.F.R.  84.13; Guckenberger v. Boston University, 974 F.Supp. 106, 134-38 (D. Mass. 1997) (Boston University violated the ADA and Section 504 by imposing restrictive policies for requesting reasonable accommodations). 12. See 20 U.S.C.  1412(a)(17)(A); 34 C.F.R.  300.138. 13. 64 Fed. Reg. 12564-65 (March 12, 1999). 14. 34 C.F.R.  300.347(a)(5)(i). 15. 34 C.F.R.  300.347(a)(5)(ii). 16. See Debra P. v. Turlington, 644 F.2d 397 (5th Cir. 1981). 17. See Brookhart v. Illinois State Board of Education, 697 F.2d 179 (7th Cir. 1983). 18. Heubert, Jay P. & Hauser, Robert M. (eds.), High Stakes: Testing for Tracking, Promotion and Graduation (Committee on Appropriate Test Use, National Research Council 1998). 19. See generally, The Standards for Educational and Psychological Testing (AERA, APA, NCME, 1999), which set forth the professional standards for test construction and usage. For example, Standard 11.2 provides, "When a test is to be used for a purpose for which little or no documentation is available, the user is responsible for obtaining evidence of the test's validity and reliability for this purpose." Similarly, Standard 11.4 states, "The test user should have a clear rationale for the intended uses of a test or evaluation procedure in terms of its validity and contribution to the assessment and decision-making process." For example, Standard 1.1 provides, "A rationale should be presented for each recommended interpretation and the use of test scores, together with a comprehensive summary of the evidence and theory bearing on the intended use or interpretation." Standard 1.2 states, "The test developer should set forth clearly how test scores are intended to be interpreted and used. The population(s) for which a test is appropriate should be clearly delimited, and the construct that the test is intended to assess should be clearly described." The comment to Standard 1.2 states that "No test is valid for all purposes or all situations. Each recommended use or interpretation requires validation and should specify in clear language the population for which the test is intended, the construct it is intended to measure, and the manner and contexts in which test scores are to be employed." 20. For these reasons, two currently available accommodations, the Franklin Speller and the dictionary, are even less helpful for a dyslexic student, because in order to use either the speller or a dictionary, a student must first realize that he or she has spelled a word incorrectly, and then must know how to look up the correctly spelled word. 21. Some states have proposed reviewing a student's IEP folder as a method of alternate assessment. While this could be incorporated into an assessment system, it is far too simplistic to serve as an alternate assessment on it own and excludes the student from the assessment. 22. LDA also notes five additional points about the 13 Core Principles: (1) State-wide assessment is a very complax issue because each state develops its own system and process. Thus, some of the principles may not be appropriate for generalization to all states; (2) Since many states and school districts are in the initial stages of implementation, new areas of concern may emerge that have not been addressed in these principles; (3) The term "alternative assessment" is a complex term that currently has many meanings. Although it is addressed in IDEA and its regulations, it is a term that is not yet clearly defined. The role of alternative assessment in ensuring fair treatment for students with learning disabilities requires continued exploration at many levels; (4) The use of the term "disadvantaged" tends to increase confusion and misunderstanding between school difficulties created by learning disabilities and those created by economic, cultural, of environmental disadvantage. It should be avoided wherever possible, unless its use is a legal necessity; and (5) As state-wide assessments are implemented, an expanded range of issues beyond those in the principles will need to be addressed. The role of test environment, modifications, graduation requirements, and diploma options are just a few examples. DISABILITY RIGHTS ADVOCATES INTRODUCTION TO BLUE RIBBON PANEL REPORT The Blue Ribbon Panel jointly appointed by the parties in Advocates for Special Kids et al. v. Oregon State Board of Education et al., conducted a comprehensive evaluation of Oregon's Statewide Assessment System (OSAS) to determine whether the OSAS discriminated against students with learning disabilities. The panel's report, which is attached,+ contains detailed findings about how the OSAS currently unfairly impacts students with learning disabilities. The report also makes extensive recommendations to the Oregon Department of Education on ways in which the OSAS can be improved so that students with learning disabilities are treated fairly and lawfully. While some of the panel's findings and recommendations are specific to Oregon's assessment, most of the recommendations are generally applicable to high stakes assessments. Of particular relevance to other states that are implementing high stakes assessments, the Blue Ribbon Panel recommended that students should be provided with any accommodation that is part of their education plan, unless the State can show by research evidence that the accommodation invalidates the score interpretation. The panel also concluded that an alternate assessment, alternative assessment, and/or alternate scoring procedures must be available to students with learning disabilities whenever the assessment tests an area of a student's learning disability. Importantly, the panel recommended that assessments must be validated for students with learning disabilities, and that the assessment should not be used for high stakes consequences for students with learning disabilities. Another key recommendation by the panel is that there must be a readily- accessible appeals process available to students by which they can appeal decisions about the denial of accommodations and whether a student will participate in the standard assessment or an alternate assessment. The panel's report represents the first time that an independent body of experts has analyzed a state's assessment system to determine its impact on students with learning disabilities. The panel report will be of invaluable assistance not only to the Oregon Department of Education, but to all states regarding how to ensure that assessments do not discriminate against students with learning disabilities. Report of the Blue Ribbon Panel: Students with Learning Disabilities and the Oregon Statewide Assessment System Executive Summary During January - December 2000, a Blue Ribbon Panel has carried out its charge to review the Oregon Statewide Assessment System to determine the extent to which its provisions and implementation ensure that students with learning disabilities have equal opportunity to participate in and attain all the benefits of the statewide assessment program. The Oregon Department of Education has developed a thoughtful and innovative program for which it deserves public and professional commendation. The Panel has not attempted to substitute its policy preferences for those of ODE, but rather to confine its recommendations to what it deems necessary to ensure legally required and educationally sound opportunities for students with learning disabilities. This Executive Summary consists of a statement of underlying principles followed by a list of specific recommendations in the following areas: Accommodations, Alternate Assessment, Problem Resolution and Appeals Process, IEP Process, and Use of Test Results. The body of the report contains a summary of current practices and panel observations within these areas, as well as additional detail regarding each recommendation. Underlying Principles  Students with learning disabilities should be provided a statewide assessment option that leads to the CIM (i.e., with or without approved accommodations, or through an alternate or alternative assessment).  The OSAS assessment should not directly assess a student's learning disability. ODE should provide alternative assessment, alternate assessment, and/or alternate scoring procedures for students with learning disabilities in a specific area that is assessed by OSAS and that is an essential requirement for meeting or exceeding the benchmark for obtaining a CIM so that a student's disability is not directly assessed  The current Oregon list of allowable accommodations for use by students with learning disabilities is too narrow. Accommodations should be allowable, valid, and scored if they are consistent with instructional and classroom accommodations included within a student's IEP--unless and until research invalidates the construct and purpose of the OSAS.  The CIM should be scored, reported, and awarded separately by subject. . Scoring across all subtests inherently and unavoidably has an adverse effect on students with learning disabilities.  The assessment waiver process, within the IEP process, should be specifically and narrowly defined. This process should be reserved only for students whose parents/guardians have requested nonparticipation in the OSAS.  The Problem Resolution and Appeals Process should be a three tiered approach that incorporates both informal (Tier one) and formal problem solving (Tier two) at the local level, and if the problem is not resolved, formal appeal procedures at the state level (Tier three). Within this process, the reason(s) for appeal should be broadened to include all aspects of the OSAS beyond scoring concern.  ODE should expand information for parents and school personnel and provide purposeful and systematic training regarding the OSAS to include the full array of participation options for student with learning disabilities, guidelines for the allowable accommodations consistent with instructional and classroom assessment accommodations included in a student's IEP, alternative and alternate assessment(s), and the Problem Resolution and Appeals Process. Purposeful and systematic training is needed regarding the above for parents, advocate organizations, administrators, IEP teams, and other educators.  On-going research and study are needed in areas such as accommodations that invalidate the construct of OSAS assessments; the nature of accommodations used and needed by students with learning disabilities; and psychometric analyses assuring the reliability, validity, and fairness of the OSAS for students with learning disabilities.  Consistent with ODE policy, OSAS should not be used for high stakes consequences for students. In order to avoid unfair consequences for students with learning disabilities, ODE should track potential misuses of the OSAS and be proactive in its efforts to discourage inappropriate uses of the CIM by school district Section 1 - Introduction Background and Charge A Blue Ribbon Panel was formed in order to recommend resolution between the parties in ASK v. ODE. The primary issue in this lawsuit is the alleged discrimination against students with learning disabilities caused by the development and implementation of the Oregon Statewide Assessment System (OSAS). Specifically, the Blue Ribbon Panel was charged with reviewing the OSAS as it relates to students with learning disabilities, and to make recommendations to the Oregon Department of Education (ODE) on policies and procedures designed to ensure that students with learning disabilities: (1) have equal opportunity to participate in and attain all of the benefits of the statewide assessment program, and (2) are not denied any educational benefits otherwise available to eligible students based on their participation in the statewide assessment program. The Panel notes that issues and concerns regarding the equal opportunity and benefit attainment for and by students with learning disabilities eligible under IDEA may also apply to the broader group of students with learning disabilities covered by Section 504 of the Vocational Rehabilitation Act of 1973. Although not clearly defined in its mandate, the Panel encourages the parties to consider the Report recommendations for both learning disabled students under IDEA and Section 504. Substantive recommendations of this Report are deemed by the Panel relevant to not only students with learning disabilities, but to all students with disabilities. Panel Members Four researchers and educators from across the country were selected as members of the Blue Ribbon Panel. In addition to these four individuals, Jacob Tanzer served as moderator of the Panel. In this role, he facilitated, supported, and participated in the work of the Panel through its background review of written materials, on-site discussions with the ASK parties, Panel discussions, and formulation of Panel findings and recommendations. Following are the backgrounds of Mr. Tanzer and the four members of the Blue Ribbon Panel: Jacob Tanzer - Jacob Tanzer is a lawyer currently in private practice of counsel to the firm of Ball Janik LLP in Portland, Oregon, specializing in business litigation, government law and mediation. He formerly served as a judge of the Oregon Court of Appeals and a justice of the Oregon Supreme Court. In his earlier career, he served as a federal state and local government lawyer and as the founding director of the Oregon Department of Human Resources. Judy Elliott, Ph.D. - Dr. Elliott is currently the Assistant Superintendent of Special Education, Long Beach Unified School District, Long Beach, California. She was formerly a Research Associate at the National Center on Educational Outcomes, University of Minnesota. In her current and past positions, Dr. Elliott assists school districts and state education departments in their efforts to update and realign curriculum frameworks, instruction, and assessments to include all students. She also serves as a national consultant and staff development professional to school districts and organizations, with a focus on effective instruction, intervention and teacher assistance teams, behavior management, and inclusive accountability and assessment. George Engelhard, Ph.D. - Dr. George Engelhard is a professor of educational measurement and policy at Emory University. He received his Ph.D. from the University of Chicago in the MESA (measurement evaluation and statistical analysis) program with a concentration in the sociology of education. His research has focused on a variety of topics including the roles of assessment in improving educational processes, Rasch measurement theory, history of measurement theory, and issues in the assessment of written composition. Among his current projects is research on models of judgment and educational assessment, the examination of assessor effects within the context of teacher assessment, standard setting on high stakes performance assessments, and psychometric issues related to the assessment of students with learning disabilities. Dr. Engelhard has published over 70 articles, test reviews, and chapters. Professor Engelhard is co-editor of the series of volumes on Rasch measurement entitled: "Objective Measurement: Theory into Practice". He is a member of the Measurement Research Advisory Panel for the National Board for Professional Teaching Standards and a former President of the Georgia Educational Research Association (1992-93). Dr. Engelhard has served on statewide technical advisory committees on educational measurement policy in Ohio, Michigan, Georgia, Oregon, Washington, and Texas. He has received a variety of awards, such as the National Academy of Education/Spencer Foundation Fellowship Award, a Lilly Post-Doctoral Teaching Award, and a Writing Across the Curriculum Project Award. Judith Schrag, Ed.D. - Dr. Schrag has over 25 years of administrative experience at the local, state, and federal levels in programs and services for students and youth with special needs, including students with learning disabilities. Dr. Schrag is the former Director of the Office of Special Education Programs for the U.S. Department of Education. She was also Assistant State Superintendent of Public Instruction for Washington State responsible for a number of special programs, including special education. She was the State Director of Special Education within the State of Idaho, a local special education administrator, and a general and special education classroom teacher. Currently, Dr. Schrag is Co-Principal of Education and Human Services Group. Susan A. Vogel, Ph.D. - Dr. Vogel is Professor, Department of Literacy Education and Presidential Research Professor at Northern Illinois University. She has published over 50 articles, four books, several handbooks, and many book chapters on adolescents and adults with learning disabilities in higher education and beyond with a specific focus on factors contributing to success, postsecondary support services, faculty attitude toward students with disabilities, and most recently, on outcomes for those with severe problems of literacy. Oregon's Education Act Provisions As stipulated by the Educational Act for the 21st Century, the State Board of Education shall "prescribe the standards.that a student should meet in order to obtain a Certificate of Initial Mastery (CIM). The CIM is based on a series of performance-based assessments and content assessments benchmarked to mastery levels at approximately grades 3, 5, 8, and 10. [ORS 329.465(2)] The Act requires the statewide assessment system to involve criterion-referenced assessments, including performance-based assessments, content-based assessments and other valid methods to measure the academic content standards and to identify students who meet or exceed the standards for each mastery level leading to the CIM and the Certificate of Advanced Mastery (CAM). [ORS 329.485(1)(a)] Following are provisions of the Educational Act for the 21st Century and the Oregon Administrative Rules implementing this Act that directly impact all students, including those with learning disabilities: 1. Oregon Educational Standards. The Oregon content standards cover the core or essential concepts in reading, writing, speaking, and mathematics leading to a CIM and the CAM. These standards apply to all students, including those with and without disabilities. 2. Assessment Options for Students, Including those with Disabilities. In the 2001 assessment, the Oregon Statewide Assessment System (OSAS) framework will include several options for test participation for students with disabilities.  Standard administration with or without accommodations.  Modified test administration in which the student takes one of the benchmark assessments with modification(s).  Standard administration-challenge benchmarks in which students may challenge a higher or lower benchmark assessment.  Extended assessments for students performing below Benchmark 1 (3rd grade) involving curriculum-based measures (CBM) in the areas of emerging reading, mathematics, and writing  Career and Life Role Extended Assessment System (CLREAS)-a functional observational assessment of life skills in the context of life routines.  Optional or alternative ways to demonstrate mastery of the standards for the CIM rather than demonstrating mastery through one or more of the statewide assessments. 3. Certificate of Initial Mastery (CIM). The CIM is achieved by students in the 10th grade. It is based on a series of performance-based assessments and content assessments benchmarked to a mastery level at approximately grades 3, 5, 8, and 10. However, students do not need to pass benchmarks at grades 3, 5, and 8 to receive a CIM. Rather, in order to get the CIM, a student should pass the CIM content and performance standard assessments (i.e., multiple-choice assessments, performance assessments, and classroom samples) that are typically administered at the 10th grade. It was not the intent of the legislature or the State Board of Education that awarding the CIM takes the place of or is required in order to receive a high school diploma. The use of the CIM and the proposed CAM are intended to be value-enhanced vehicles for communicating the skill level of students with and without disabilities. The CIM requirements have recently undergone change and redefinition, and pending State Board of Education approval in September 2000, the CIM would be offered in each of the curriculum areas currently being assessed. A student who meets or exceeds performance standards at the 10th grade level in any one of the curriculum areas can receive a CIM in those specific curriculum areas. A student will also have the opportunity to demonstrate the skills and knowledge at any time between grades 9 and 12 and to demonstrate performance at the 10th grade level in any or all of the curriculum areas being assessed. 4. Certificate of Advanced Mastery (CAM). A CAM is proposed for students who achieve at a higher standard in the 12th grade. It is still in the developmental process. The ODE anticipated that the redesign would be completed in Fall 2000, at which time a feasibility study will be conducted to determine the capability of schools for implementation. Assuming positive information from the feasibility study, the CAM will be developed more completely and piloted and fully implemented within the schools in 2004- 2005. 5. Multiple Assessment Procedures. The OSAS includes multiple assessment procedures: multiple choice tests for students that reflect discrete content standards, on-demand performance tasks that reflect performance standards, and classroom work samples that provide evidence of the opportunity to learn. All of the Oregon State Assessments are power tests (i.e., untimed tests). The OSAS offers multiple opportunities for students to demonstrate their level of performance. No single test is the decision-maker for any individual student. 6. Challenge Process. Students may challenge the Oregon standards that are at one grade level, but performing at a different grade level (lower or higher). Specifically, they may take the state assessment at a different grade level (higher or lower) than the grade they are currently in (similar to any out-of-level test). 7. Role of the IEP Related to Inclusion of Students With Disabilities in Statewide Assessment. The State Board of Education has adopted a statewide policy of placing the responsibility of determining assessment accommodations with the student's Individual Education Program (IEP) team. The ODE has developed a state, common IEP form for use by school district IEP teams for documenting the type(s) of assessment accommodations to be provided for each student. 8. Testing Accommodations and Modifications. According to OAR 581-105-0068, testing accommodations are changes in how a test is presented to or how a person responds to a test. Testing accommodations include a variety of alterations in presentation format, response format, setting in which the test is taken timing, or scheduling. These alterations do not substantially change the level, content, or performance criteria of the test. The changes are intended to provide a student equal access to learning and equal opportunity to demonstrate what is known. Testing modifications result in a substantial change to what a test measures or the level of difficulty of the test. The modification is provided to enable students, including those with learning disabilities, to participate meaningfully in the OSAS. Testing modifications result in changes in the difficulty level and the content of the measure. Each year, an internal Accommodations Panel reviews the list of state-approved modifications and makes recommendations regarding any changes in expanding the allowable testing accommodations (i.e., shifting a modification to an accommodation) based on research conducted within Oregon or in the nation. Draft state guidelines have been developed to assist school districts in determining the difference between a testing accommodation and a modification. Following a local review of requests from a school, school districts may submit requests to this State Accommodations Panel for consideration of testing modifications to be considered as accommodations. 9. Oregon Alternate Assessment. The State Board of Education has adopted standards in an area called Career Related Learning for the CAM as well as the subject content areas for the CIM. The Area of Career and Life Roles includes personal management, problem solving, communication, teamwork, organizations and systems, employment foundations, and career development. Currently, measures are being developed for these standards. Although Oregon has been committed to a single comprehensive assessment system that includes all students, the implementation of the IDEA alternate assessment involves an assessment of the Career Related Learning standards adopted by the State Board for the CAM. Oregon's alternate assessment, called the Career and Life Role Extended Assessment System (CLREAS) measures the lower end of the Career Related Learning standards. Students assessed on the CLREAS may be also working toward functional academics in reading, math, science, or social studies. This "alternate assessment" for purposes of IDEA requirements has been recently piloted for implementation during the 2000-2001 school yea and do not lead to a CIM. Through research in collaboration with both the University of Oregon and Portland State University, curriculum based measures have been developed in the areas of emerging reading, mathematics, and writing aligned to the Rasch Unit (RIT) scale. Previously referred to as Extended CIM, these are now called Curriculum-Based Measures: Extended Writing, Extended Reading, and Extended Mathematics. In effect, this change will allow students with disabilities performing below Benchmark 1 (3rd grade), to take these Curriculum-Based Measures beginning with the 2001 assessment. However, performance on these Curriculum-Based Measures do not lead to awarding of the CIM, although it is the bottom level of the CIM benchmarks. 10. Waiver Review. This review is reserved for students who parents request nonparticipation in the OSAS. 11. Appeals Process. Although not fully developed, the intent of the two-tiered appeals procedure is for the State to review local requests dealing with challenges related to re-scoring and consists of local and state appeals procedures. Within the body of this Report, various strengths, weaknesses, issues, and recommendations are outlined related to participation or inclusion of students with learning disabilities within the OSAS. It should be noted that during the period of Panel deliberations, several changes and refinements were made and/or identified by the Oregon Department of Education. Therefore, the Panel notes that the OSAS participation options are still in transition. Panel Work The Blue Ribbon Panel carried out its work from January - August 2000. During this time, an orientation meeting and a subsequent meeting were held with the parties in order to receive background information about the development and implementation of the OSAS, as well as specific issues and concerns. The Panel held two additional meetings and carried out other work via email, telephone, and fax. The Panel members reviewed a number of documents in order to obtain a thorough understanding of the development and administration of the Oregon OSAS. A summary of these documents can be found in Appendix A. ODE has formulated a thoughtful, sophisticated assessment program that is in many ways exemplary. Throughout its work, the Panel has not attempted to substitute its policy preferences for those made by ODE insofar as they are consistent with legal mandates requiring equal opportunity for students with disabilities. When observed that there could be actual or potential limitation upon equal opportunity and benefit attainment for and by students with learning disabilities, we have offered recommendations. Where we have perceived a problem, we have offered the recommendations considered most efficacious. Section 2 - Accommodations In 1992, ODE began its annual review of the test administration manual regarding participation and accommodation policies for the OSAS. Shortly after, the ODE obtained a Federal research grant to begin addressing the use of accommodations on large-scale assessments. In 1996, a national advisory panel was established to guide the ODE's policy decision making regarding assessment accommodations. Concurrently, ODE has been an active participant in the National Center on Educational Outcomes and Council of Chief School Officers' State Collaborative on Assessment and Student Standards (SCASS) for special education students. Thirty-five states participate in the SCASS. ODE has taken great steps to develop a thoughtful assessment system that is inclusive and thorough. It has gone to great lengths to work and collaborate with nationally known researchers in the area of assessment of students with special needs and limited English proficient students. Although ODE is being challenged legally for certain aspects of the OSAS related to students with learning disabilities, this Panel commends ODE for one of the most comprehensive assessment systems in the nation. Current Practice Oregon is required, by Federal mandate, to provide students with disabilities with appropriate accommodations on district and state assessments. The Individual Education Planning (IEP) team makes assessment accommodation decisions. The central concern in ASK vs. ODE is whether students with learning disabilities have been denied non-approved accommodations and whether they have equal opportunity to participate in and attain all of the benefits of the statewide assessment program. Although the ODE has taken commendable effort to use existing, but limited research on assessment accommodations to define its policies, it is being challenged in its denial of the use of non-approved accommodations for the CIM. For example, the ODE has developed a list of approved accommodations based on research conducted both within and outside of the state. This list has been compiled without necessarily field-testing a random sample of learning disabled students with these OSAS accommodations leading to the CIM. Therefore, not all approved and non-approved accommodations have been validated or invalidated, respectively, for the OSAS. Rather, the lists have been extrapolated from extant accommodation research. Currently, a State-based Accommodations Panel meets annually to review accommodation research. Criteria for adding accommodations to the allowed list for the OSAS are as follows: A. Empirical and experimental research to support its use. B. Professional judgment of those with expertise in the field. C. Examination of the evidence of past requests for modifications. Feedback to the Panel indicates that even though the Panel meets annually, no additional OSAS accommodations have been approved (i.e., moved from modification to accommodation status). Panel Observations Federal regulations require that students with disabilities not only should have access to the general curriculum, but should also be included in district/state assessments with accommodations where necessary. Exemption from the OSAS is not an option unless parents obtain a waiver. Therefore, the issue is not whether the student will participate, but how they will participate. With this overall assumption, the Panel offers the following observations:  Assessment accommodations provided for the OSAS should be consistent with those provided during the instructional process, as included on the IEP including assignments and classroom tests.  Although the State has a collection of research on accommodations, there is little or no evidence to suggest that non-listed accommodations necessarily invalidate constructs the OSAS is intended to measure.  The terms "adaptation", "accommodation", and "modification" as used by ODE, lack clarity and may cause inconsistency in their application.  Training throughout the State has focused on IDEA, not on the OSAS specifically and its specific use of accommodations and modifications. Based on the description of training provided for IEP teams, the Panel questions whether they are aware of these terms and the implications of each. It is uncertain if IEP teams are aware of their ability to challenge a student's need for a non-approved accommodation for administration of the OSAS leading to a CIM.  In the current system, students with disabilities are provided IEP-based accommodations during the instructional process. However, for the purpose of the OSAS, accommodations that do not appear on the State approved list renders OSAS results invalid in counting toward the CIM. Recommendations 1. Accommodations should be considered allowable, valid, and scorable if they are used during instructional or on classroom assessment and are listed on a student's IEP until research evidence invalidates the score interpretation. Rather than consider all accommodations first invalid until proven to be valid, ODE should consider all accommodations valid unless and until research provides evidence that an accommodation alters the construct or level of the OSAS measure. According to Memorandum No. 60-1999-2000 (April 22, 1999), the "individualized education program team shall make a determination of any appropriate accommodations and modifications necessary for the student to participate in the Oregon Statewide Assessment Program." The Panel recommends that any assessment related accommodations, both approved and non-approved, that are customarily used in day-to- day classroom instruction by students with an IEP be considered accommodations that are scorable or allowable for individual students, included in group score reports, and lead to awarding of a CIM. If research is conducted that provides evidence that the accommodation should be non-scorable (non- approved accommodation), then future uses of the accommodation may be treated as non-scorable with proper notification of parents, teachers, and other school personnel. ODE should provide students with learning disabilities an assessment option leading to the CIM that allows them to demonstrate proficiency of the content standards.1 2. The State should develop a demographic data sheet to be utilized by school districts for recording accommodations, both approved and non-approved provided to students that are consistent with those used in instruction, as specified in the IEP. Many of the Panel's questions about the use of approved and non- approved accommodations by students with disabilities, (e.g., types of accommodations used and on what measures and number of students who requested them) could not be provided. A demographic sheet should be completed for each student with a disability. This would allow for accurate documentation of the actual provision of all accommodations (e.g., how many students received approved accommodations and requested non-approved accommodations). It also can provide a means to track whether the needed assessment accommodations written on a student's IEP were, in fact, provided. ODE may consider a sample tracking form utilized by the State of Rhode Island (see Appendix B) that would provide districts and ODE needed documentation of student use throughout instruction as well as the information needed to prepare for the OSAS test day. 3. IEPs for all students with learning disabilities should be reviewed in their next annual IEP review or sooner and then annually thereafter for the presence and appropriateness of accommodations to be used in instruction and the OSAS. ODE uses a published list of allowable accommodations. There exists a strong possibility that needed accommodations have not been written into student IEPs simply because their absence from the list implies a determination by ODE that they would invalidate OSAS. Student IEPs should be reviewed to determine whether accommodations reflect what the student needs for instruction and the OSAS. It is the Panel's observation that local districts are unaware of their right to challenge the provision of a needed, but non-approved accommodation for a student with a learning disability who needs one. Any list of approved accommodations published by ODE as a guide for school districts shall not be deemed exclusive by ODE. While this accommodations review can be conducted as part of a student's annual review, those students whose annual meetings do not occur until Spring of 2001 will miss the opportunity to use accommodations, both approved and non-approved, on the Spring 2001 administration of the CIM. Therefore, the Panel recommends the IEP review begin sooner. 4. Each school district should identify a person who is knowledgeable about both IEP requirements and state testing requirements (e.g., test accommodations, psychometrics of the test, and other measurement issues). This person should serve as resource and point person for IEP teams, upon request. In addition, this person should be involved in all implementation of policy and practices related to the OSAS. ODE should provide guidelines regarding district-based roles and those that relate to serving as an ODE assessment liaison. 5. Information regarding all aspects of the OSAS and options for participation by students with learning disabilities should be disseminated to parents, orally and in writing, as members of the IEP team, so that there is expanded awareness and understanding regarding participation of their child, including alternative and alternate assessment(s), and the Problem Resolution and Appeals Process. 6. Training for parents involving all aspects of the OSAS and the range of participation options for students with learning disabilities should be jointly planned and implemented by ODE and key parent training and advocacy groups/organizations. 7. School districts, school sites, and ODE should keep on-going information regarding student test results regarding benchmark (e.g., grade 3, 5, 8) results to ensure accurate records are kept on which students with learning disabilities passed necessary benchmarks ultimately needed for the CIM. For example, students, including those with learning disabilities, may be placed in an assessment based on a locator test. It is imperative that the progress of these students is carefully tracked to ensure they are given the opportunity to pass all required benchmark assessments in a timely manner. In turn, these data can provide teachers the information needed to tailor curriculum and instruction planning for students who are in need of demonstrating specific skills. ODE should provide internal listings of student results of state tests per benchmark years to each school district for distribution to school sites. These lists can include types of assessment accommodations used (approved and non-approved), if any, by students with disabilities. We recommend that results be reviewed for the use of non-approved accommodations that may have precluded students from meeting the standards for the benchmarks. In the past, the test scores of students using non-approved accommodations have been rendered invalid. 8. ODE should continue its research studies on assessment accommodations needed or utilized by students with learning disabilities on the OSAS, including those accommodations that invalidate the construct(s) of the OSAS. To date, ODE is unable to generate data to support the number of students with learning disabilities who requested assessment accommodations that were on their IEP or whether or not they were automatically provided per the IEP. On-going research in this area will provide empirically- based information that directly addresses one of the major issues of this lawsuit-namely what constitutes an approved versus non-approved accommodation. Section 3 - Alternate Assessments In accordance with IDEA, all states should have in place an alternate assessment by July 1, 2000. This assessment is reserved for a small number of students for whom the regular assessment is not appropriate, even with accommodations. In most states, the alternate assessment is being developed for students with significant cognitive disabilities as a separate system of assessment. Oregon, however, has shown strong commitment to a single comprehensive assessment system that includes all students, including those with more significant disabilities. This effort has been driven by the desire to avoid having an assessment that is an alternate to the "real" assessment. The Panel fully supports the State's inclusion of the alternate assessment as an integral part of the OSAS. Current Practice Oregon's State Board of Education has adopted content standards for the CIM and Career Related Learning for the CAM. The domain of Career and Life Roles has several sub categories or strands including personal management, problem solving, communication, teamwork, organizations and systems, employment foundations, and career development. These categories are reported to appear more appropriate for students with more significant disabilities. Although not a part of Career and Life Roles domain, for the purpose of assessing students with more significant disabilities, a Motor Skills domain was added. Oregon's alternate assessment, called the Career and Life Role Extended Assessment System (CLREAS) measures the lower end of the Career Related Learning standards. Students assessed on the CLREAS may be also working toward functional academics in reading, math, science, or social studies. The State is currently addressing these students' needs by building curriculum-based assessments (formerly called the Extended CIM) at the lower end of the reading/literature and mathematics standards. Recent communication from ODE indicated that the name Extended CIM has been changed to Curriculum- Based Measures: Extended Writing, Extended Reading and Extended Math. The Panel supports this change to more accurately reflect these assessments that do not lead to a CIM. These alternate measures will render valid scaled scores for students who are performing below the first benchmark standard. ODE's multiple assessment approach is to be commended. It based these efforts in its attempt to avoid the past practice of exempting students with special needs from the general education curriculum simply because they have disabilities. ODE's practice moves Oregon in the direction of individualizing which standards a particular student with a significant disability should be working toward. Not all students will be working toward a CIM due to the extensive nature of a disability. However, in order to account for and assess these students with a range of disabilities, ODE is continuing its process of building the Curriculum- Based Measures that reflects the functional academic these students are working toward. Neither the CLREAS, nor the Curriculum-Based Measures, formerly called the Extended CIM, leads to the attainment of the CIM. ODE has recently completed a statewide pilot of the CLREAS. The pilot included approximately 450 students with "moderate to severe disabilities" (see Appendix D, CLREAS 1999 Report, for more information). It appears that the State will be more than ready to implement a thoughtful and aligned alternate assessment for students with more significant disabilities for its next testing cycle (spring 2001). Eligibility criteria for CLREAS are as follows: (1) the student's pursuit of the CIM is not appropriate, (2) the student's needs are best addressed and aligned with the Career and Life Role Education standards, and (3) the benchmarks and assessments embedded within the CAM need to be modified or augmented. [The CLREAS is said to be oriented toward the CAM. However, since the CAM does not exist at this time, the Panel is unable to comment on the linkages of the CLREAS to the CAM.] ODE has defined CLREAS as a "measurement system oriented toward career related learning standards that employ controlled tasks in diverse contexts: school, home, work, and community." However, the CLREAS does not reflect students with learning disabilities who are able to meet the standards for the CIM, but need non-approved accommodations to do so. One issue of the lawsuit is that the State's alternate assessment be one that encompasses students with learning disabilities who are able to demonstrate mastery of the standards required for the CIM, but are precluded from attaining it due to the use of non-approved accommodations. In other words, a different assessment option needs to be available not only for students with significant cognitive disabilities, but also for students with learning disabilities. Panel Observations  ODE is to be commended for its efforts to establish one comprehensive assessment system for all students, including students with disabilities. Oregon has taken the standards for all students and made them relevant to students with more significant disabilities whose foundational curriculum is in the area of life skills. The assessment of these standards is entrenched in daily life or functional skills that are critical for their participation in the community and society.  Currently, there is no assessment option leading to the CIM for students with learning disabilities. The use of non-approved accommodations needed by students with learning disabilities to demonstrate proficiency of these standards automatically exempts them from receiving a CIM.  In its effort to extend the current assessment below the first benchmark, the ODE is building Curriculum-Based Measures, previously named the Extended CIM, to assess the lower end of writing, reading, and mathematics standards. In this manner, the ODE is providing for the assessment of students who fall within the current eligibility criteria for the alternate assessment in a manner that allows them to work toward mastery of basic and emerging academic skills. However, these assessments are not necessarily appropriate for students with specific learning disabilities. In most cases, students with learning disabilities are working toward the same standards as those measured by the CIM. Recommendations 1. Students with learning disabilities should be provided as assessment option(s) that leads to the CIM. 2. Another assessment option leading to the CIM, over and above broader accommodations, the Curriculum-Based Measures, and the CLREAS should be provided for students with learning disabilities. Guidelines for what includes a body of student work that demonstrate a student's mastery of the content and performance standards should be explicitly delineated as well as criteria for the review panel to use in considering whether or not a student meets the standards. 3. The waiver process should be specifically and narrowly defined. This process should be reserved only for students whose parents/guardian have requested nonparticipation in the OSAS (e.g., comatose students, medically fragile students). The ODE assumes that all students will participate in the OSAS with or without accommodations or an alternate assessment. Although there are options available for all students in the State, there may be circumstances where parents may waive participation of their child. It is the Panel's understanding that there has been discussion of a waiver process, but to date, there is not one in place. In its presentation to the Panel, ODE discussed the proposed waiver process predominantly targeted to students with disabilities. On occasion, there may be a student whose parents/guardian requests nonparticipation in the OSAS (e.g., comatose students, medically fragile students). In these cases, the ODE needs to provide guidance on how these students will be counted and accounted for. The Panel recommends that the State delineate what the waiver process is and how parents access it. Dissemination of the process should follow its development. 4. School districts, parents, and students should be made aware of the availability of another assessment option leading to a CIM for students with learning disabilities who are in need of a non- approved accommodation for the CIM. Currently, the Opportunities for Participation in the OSAS (Appendix F, Oregon Department Resources to ASK Blue Ribbon Panel Questions) named the Appeal OAR, reflects a different way for students who have mastered the standards required of the CIM to show what they know and can do. Leading to the CIM, this assessment approach, if explicitly outlined and crafted, could suffice as an option for students with learning disabilities. Information about this option for students with learning disabilities should be broadly disseminated to school personnel and parents and within OSAS training efforts. Section 4 - Problem Resolution and Appeals Process The purpose of Problem Resolution and Appeals process is to provide a systematic mechanism to resolve issues that arise in which parents and school personnel disagree with various aspects of the administration of the OSAS. These issues may include scoring of a student's test, the use of test accommodations, and/or other conditions under which the tests are administered, scored, or used. The Problem Resolution and Appeals Process is an essential and integral part of the OSAS administration. In addition, as with other provisions of IDEA involving identification, evaluation, and IEP development and implementation, the procedural safeguards of IDEA (i.e., citizen complaints, mediation, and due process) are applicable and may be utilized in the event that the Problem Resolution and Appeals Process within the OSAS system cannot resolve differences between parents and school personnel. The Panel recommends, however, that the OSAS Problem Resolution and Appeals Process be utilized prior to other more formal IDEA conflict resolution procedures. Current Practice ODE is to be commended for being one of the few states in the nation that has initiated an appeals process within its statewide assessment program. In Section IV - "Decision Making and Appeals Process," Oregon's Report to the Blue Ribbon Panel (pp. 15-17), the appeals process is described as a vehicle in which students may appeal the score they received on any state assessment (referred to both as the Appeals Procedure and the Challenge Review). The Oregon appeals process consists of two tiers. At the first tier, parents submit an informal request to the panel of experts within the school district. If the parents are not satisfied with the response at the district level, or they do not receive a response within 45 days, they can submit a formal appeal in writing to the State Superintendent. Panel Observations:  There is presently exists a two-tiered process of appeal for the OSAS. There is only one reason for an OSAS appeal --request for re-scoring a section of the OSAS. There are, however, other OSAS- relevant issues about which parents may wish to appeal. Additional reasons for appeal could be that the student be allowed to retake a section of the OSAS, that a section of the OSAS be waived without penalty, that a requested accommodation be considered valid and scorable, or that a student should be provided an alternate assessment.  Current information regarding the OSAS for parents does not include a statement of purpose for the appeals process, examples of issues that are appealable, or guidelines to following in requesting an appeal. In addition, there is no information regarding how an OSAS appeals process might trigger the IDEA procedural safeguards in the event issues cannot be resolved through the OSAS appeals process.  Other than the 45-day timeline for a state-level response to the parents, there is not an assurance that the appeals process will be carried out in a timely manner at both the district and state level. Recommendations 1. ODE should broaden the reasons for appeal beyond re-scoring to include all aspects of the OSAS participation for students with learning disabilities. Some possible additional reasons for appeal are that a specific accommodation be considered valid, allowable, and scorable; that a student be allowed to retake a section of the OSAS; or that a student be assessed utilizing an another assessment option. 2. ODE should implement a systematic and timely team approach to resolve OSAS appeals. The Problem Resolution and Appeals Process should involve a three-tiered team approach that incorporates both informal and formal problem solving. Tier one occurs at the local school level where involved parties gather to discuss and attempt to resolve concerns in an informal manner. If unsuccessful, the Tier two, involving a formal district appeals process, is enacted. Here a structured team approach, including both original and new members, work together to resolve specific concern(s). If this process is unsuccessful, Tier three is initiated at the State level. A formal appeal is submitted to the State and an appeals panel is assembled that includes personnel who were not involved in previous decisions related to the specific concern(s). As with other aspects of IDEA, the IDEA dispute resolution procedures are available if OSAS issues cannot be resolved through the three-tiered Problem Resolution and Appeals Process. 3. The team members involved in the Problem Resolution and Appeals Process should included persons with expertise in learning disabilities and use of accommodations. As with other aspects of IDEA, parents should feel free to involve other supportive individuals such as family members, parent advocates for learning disabilities, other parent advocates, and/or other persons with special knowledge of assessment and accommodations, as a part of the IEP team. 4. The Problem Resolution and Appeals Process should be clearly delineated in all information for parents, teachers, and other professional staff regarding participation of students with disabilities within the OSAS. In addition, the OSAS Problem Resolution and Appeals Process should be aligned with the more formal IDEA dispute resolution procedures. Vehicles of purposeful, active, and timely dissemination include, but are not limited to, the Department website, brochures, information packets, as well as a Seeds of Changes - A Parent Handbook or other information specifically provided for parents. This information should also describe how the OSAS Problem Resolution and Appeals Process relates to the more formal IDEA dispute resolution procedures (i.e., complaint process, mediation, and due process) in the event that parent and/or school concerns regarding the participation of a student(s) with learning disabilities within the OSAS cannot be resolved. In addition, information regarding the appeals process should be reviewed at the student's IEP annual review during the required discussion of participation in the OSAS. 5. ODE should carry out broad information dissemination and statewide training regarding the three- tier Problem Resolution and Appeals Process for school personnel, parents, parent advocates, and others who are involved in the decision-making and administration of the OSAS Appeals Process. The main focus of the training should be on informal problem solving strategies (Tier 1) for local level school personnel, as well as other more structured forms of appeal (Tiers 2 and 3). 6. ODE is encouraged to develop a mechanism to monitor and evaluate the efficacy of the Problem Resolution and Appeals Process at each tier. Evaluation should focus on documenting the nature of the problem, participants, corrective activities. Section 5 - IEP Process The IEP process is the central process for determination and documentation of a student's participation in state and district-wide assessments. One of the elements the IEP addresses is how students will be assessed and what accommodations will be provided in the administration of state or district-wide assessments. The Oregon State Board of Education has recognized the importance of the IEP process by adopting a policy which charges the IEP team with determining whether a student with a disability will participate in the OSAS, including appropriate accommodations, both approved and non-approved, when necessary. If a student's IEP team determines that a student will not participate in all or part of the OSAS, the IEP team should document why the assessment is not appropriate for the student and how the student will be assessed. Finally, the State Board of Education Policy Statement stipulates that the result of the OSAS shall not be used to deny any student an opportunity to receive any educational benefits available to other students who satisfy the other criteria established by the state, a district, or other educational agency for such benefits. Current Practice In addition to the above-described State Board of Education policy, ODE has initiated other efforts related to the role of the IEP in assessment decisions:  The State Board of Education has provided procedural guidance in the decision-making and documentation responsibilities of the IEP team through Oregon Administrative Rules (OAR). These rules specify the decisions IEP teams should make regarding the appropriate use of state and district-wide assessments, including the alternate assessment.  The Accommodations Administration Manual includes information regarding testing accommodations for students with disabilities.  A State IEP form has been developed that includes space for documenting the extent to which the student will participate in standard administration of state and district assessments (with and without approved accommodations), the types of state and district assessment modifications to be utilized, and other information regarding state and district assessment participation.  Efforts have been made to work with the State's administrators' organization to free teachers to fully participate in the IEP process.  A collaborative model has been used with statewide organizations such as PTA, teacher organizations, and the State Special Education Advisory Council for dissemination of information regarding the types of test accommodations that can be provided as a part of the IEP process.  Statewide trainings have been conducted by the ODE for teachers, principals, administrators, and curriculum directors within local school districts.  A CD-ROM has been made available to the Oregon Coalition for Parent Education, which includes information regarding IDEA '97 changes related to the inclusion of students with disabilities in district and state assessment.  A paper entitled "Test Access and Adaptation: Guidelines for Assessment of Students With Disabilities and Limited English Proficiency" has been prepared by Dr. Gerald Tindal, University of Oregon, that outlines proposed conditions and definitions of differentiated decisions that IEP teams need to make so that students with disabilities can participate in the OSAS with accommodated, modified, or alternate assessments. Panel Observations The Blue Ribbon Panel commends the State Board of Education and the ODE for implementing the above activities. However, following are several concerns:  ODE's list of allowable accommodations is too limited in order to serve the needs of all students with learning disabilities. The process, carried out by a State panel, is also unclear for determining when additional accommodations will be allowed by the State to accommodate specific needs of students with learning disabilities. To date, the State panel has not increased the list of approved accommodations. See Section 2 of this Report for additional observations and recommendations (i.e., Accommodations and Modifications).  Although the instructions for the use of the State IEP form discuss the need for documenting accommodations needed by individual students, IEP teams are not provided clear guidance regarding the nature of allowable accommodations as well as parameters for determining/identifying additional needed accommodations that are used in the instructional process as documented within the IEP and that could be provided within the OSAS without violating the integrity of the test/assessment constructs.  There is insufficient guidance provided for IEP teams regarding the range of options for OSAS participation for students with learning disabilities-and their implications related to awarding of the CIM.  The State IEP form provides space for the IEP team to indicate whether the student will participate in the standard OSAS administration, with or without accommodations. However, it does not include a way to document other OSAS participation options including changes in standard administration, Curriculum-Based Measures: Extended Writing, Extended Reading, and Extended Mathematics, and the Career and Life Role Extended Assessment System (CLREAS).  Training provided thus far by the ODE for local IEP teams appears to have been subsumed under broader training regarding changes related to the IDEA '97 Amendments. Information and training for IEP teams within local school districts regarding participation of students with disabilities within the OSAS was imbedded within this overall IDEA training. There doesn't appear to have been a purposeful and systematic training plan regarding the participation of students with learning disabilities within the OSAS.  Although CD ROMs were provided to parents via the Oregon Coalition for Parent Education, this training again has been imbedded in overall IDEA '97 training. There doesn't appear to have been a purposeful and systematic training of parents regarding the various OSAS participation options for students with learning disabilities. Recommendations Based on the above observations and concerns, the following recommendations are made by the Blue Ribbon Panel specifically related to the IEP process. 1. Assessment options for students with learning disabilities utilized within the OSAS, as determined within the IEP team decision process, should be broader. In addition to broader allowable accommodations, an assessment option should be provided for students with learning disabilities leading to the CIM. This option is necessary to provide equal access to the OSAS. 2. The State IEP form should be modified to include a checklist of all OSAS assessment options as well as to modify IEP references to assessment exemptions. The Panel recommends that the statewide IEP form be revised to include a list of all the OSAS participation options available for students with disabilities. In addition, The Panel advises the ODE to revise the current statewide IEP form to drop the option of exemption from the OSAS. The IDEA Amendments of 1997 specifically indicates that students with disabilities should participate in either the state/district assessment with or without accommodations or an alternate assessment. According to Federal regulations, the only students who may be exempted from state and district assessments are those convicted as adults under state law and incarcerated in adult prison (34 CFR 300.311(b)(1)). A space should be maintained, however, to document parental choice to have their child opt out of any or all of the assessments. 3. Detailed guidance should be developed and disseminated to IEP teams in making individual decisions regarding the multiple OSAS participation options, detailing which do and do not lead to awarding of the CIM. Guidance is needed for IEP teams regarding the decision making process for determining which OSAS option is appropriate for individual students with learning disabilities. 4. Training should be provided for IEP teams in all aspects of the OSAS and options for participation by students with learning disabilities. Given the complexity of various OSAS participation options for students with learning disabilities, the state needs to be much more aggressive in its IEP team training efforts. The Panel recommends that a schedule of statewide training and materials be developed for use by school districts. School districts can, in turn, conduct site-by-site training. IEP team training should include information and skill building regarding the selection of appropriate assessment accommodations for use with individual student (e.g., the purposes of accommodations, knowledge of the constructs underlying the assessments, details of all OSAS participation options, and accommodations that preserve the integrity of the tests/assessments, and implications for the awarding of the CIM certificate). 5. Information regarding all aspects of the OSAS and options for participation by students with learning disabilities, should be disseminated to parents, as members of the IEP team, so that there is expanded awareness and understanding regarding OSAS participation options for their child, as well as the implications of each for attaining the CIM. Systematic efforts are needed to disseminate guidance and training for IEP teams within local school districts (i.e., teachers, administrators, related services personnel, and parents). Information should include content regarding the following: the nature and content of the assessment system, the IEP team process for determining participation, allowable student accommodations, the range of Curriculum-Based Measures (Extended Writing, Extended Reading, and Extended Mathematics), the Career and Life Role Extended Assessment System (CLREAS), and Problem Resolution and Appeals Process. For example, a brochure on assessment, assessment accommodations-both approved and non-approved, an alternate assessment(s), and assessment options for students with learning disabilities could be developed and disseminated by the ODE, in coordination with all parent and advocacy organizations as well as through local school districts for their dissemination to parents. 6. ODE and key parent training and advocacy groups/organizations should jointly plan training involving the range of participation options, detailing which do and do not lead to awarding of the CIM. 7. Training involving all aspects of the OSAS and the range of participation options for parents of students with learning disabilities should be jointly planned and implemented by the ODE and key parent training and advocacy groups/organizations. This training should include the range of differentiated options for students with learning disabilities to participate in the OSAS, allowable assessment accommodations, and the Problem Resolution and Appeals Process. Section 6 - Use of Test Results One of the mandates of the Panel is to evaluate and make recommendations to the Department on the use of statewide assessment results for students with learning disabilities. ODE recommends that the OSAS be used as follows: 1. Evaluating individual student performance and needs 2. Implementing program improvements 3. Student, school, and district accountability 4. Basis for awarding the Certificate of Initial Mastery (CIM) 5. State reporting The State Board of Education has stated that the OSAS shall not be used to deny any student who otherwise meets the criteria for participation, an opportunity to receive educational benefits available to other students Current Practice Oregon's OSAS is based on a philosophy that reflects a commitment to be as inclusive as possible. Appropriate uses of the statewide assessment test results depend on the collection of lines of evidence regarding the psychometric quality of the ODE assessment for all students. ODE has a technical manual (Technical Digest for the Oregon State Assessment) that provides information regarding the process used to collect evidence regarding the reliability, validity, and fairness of the OSAS. ODE also routinely summarizes psychometric data in annual Technical Bulletins. In evaluating the psychometric quality of ODE assessment program, the three major guidelines for current professional practices are the Test Standards (AERA, APA, & NCME, 1999), Code of Professional Responsibilities in Educational Measurement (NCME, 1995), and Code of Fair Testing Practices in Education (Joint Committee on Testing Practices, 1988). ODE has used these sources to guide its test development and implementation practices. It is beyond the scope of this Panel to provide a detailed evaluation of the OSAS in terms of these guidelines. The Panel believes that the technical characteristics of the OSAS are psychometrically sound for the general population of students in Oregon, and that ODE has collected validity evidence that supports the proposed uses of OSAS. This view is supported by other Panels that have been commissioned by ODE (National Standards Review Team, 1996; Achieve Report, 2000). Although ODE has followed current guidelines for sound psychometric practices including the testing of students with disabilities, the Panel feels that issues related to students with learning disabilities need to be systematically and specifically included in future test development and psychometric analyses. These psychometric analyses should include, but are not limited to, the calculation of reliability coefficients and standard errors of measurement separately for students identified with learning disabilities. Differential item functioning (DIF) analyses should also be conducted using students with learning disabilities as the focal group, and this information presented to Item Bias/Sensitivity Review Panels. The current standard-setting model is psychometrically defensible, but it may have undue negative consequences for students with specific types of learning disabilities. As was shown in a recent study by Haladyna and Hess (2000), the Oregon writing data supports a unifactor interpretation indicating that a unidimensional model is appropriate. This unidimensional model provides an argument for the replacement of a conjunctive model with a compensatory model for writing assessment. This study also favors the use of a compensatory model based on considerations of the reliability of score inferences and rater consistency at the domain level. Empirical studies, such as Haladyna and Hess (2000), should be conducted to explore issues related to the dimensionality and reliability (decision consistency and accuracy) of the assessment. This previous study explored results by gender, ethnic background, and parent education and found that comparable analyses should be conducted for students with learning disabilities. It is of critical importance that the assessment does not directly test a student's disability and make that component an essential requirement for obtaining the CIM. For example, current scoring procedures for the writing assessment (double weighting of particular sections) may exaggerate the adverse effects of the OSAS with students with learning disabilities. The ODE should consider alternative scoring procedures for students with learning disabilities in a particular area that directly measures a student's learning disability. The Panel recognizes that policy makers have the right to define the requirements for a CIM (knowledge and skills); however, it is essential that these policy decisions be data based and informed by technical analyses regarding bias and adverse impact for all students in Oregon. Panel Observations  Although ODE has a commitment to inclusive participation in the OSAS, the implementation of the current system with the distinction between allowable accommodations and modifications (non- approved accommodations) seems to have the unintended consequence of excluding some students with learning disabilities. The Panel recognizes that some school systems may be reluctant to include students with learning disabilities because of a possible drop in average school scores. In order to counteract this concern, ODE should consider providing information separately for different subgroups within a school or district. For example, other states have begun reporting separate and combined scores by race/ethnicity in order to provide data necessary to begin closing the gap between the achievement scores of different subgroups.  It is the Panel's understanding that ODE has decided to adopt a modified-conjunctive standard setting approach for the writing component of the CIM. This new standard setting approach is still "conjunctive" and may have undue impact on students with particular learning disabilities. Also related to the standard-setting process, it appears that a variety of sample papers and impact data were used in the standard-setting process, but no systematic effort was made to include information relative to students with learning disabilities. Future research efforts should be directed toward the consideration of how to appropriately include this information in the standard-setting process, as well as other aspects of ODE assessment development process. The adverse impact of conjunctive and modified conjunctive standard setting within the context of writing is fairly obvious. For example, students with dyslexia, by definition of their learning disability, have spelling disabilities, and, therefore, will automatically have low scores on this writing component. The potential biases and adverse impact of a conjunctive model across the content areas needed to receive the CIM are not as obvious and require additional research. As far as the Panel is aware, Oregon leads the nation in terms of the number of separate components that should be passed conjunctively to obtain certification with high stakes for students. Even though the CIM is not required for obtaining a high school diploma, there are other high-stake consequences associated with the CIM, such as employment opportunities and college admissions. The conjunctive standard-setting model may unfairly preclude students with learning disabilities from receiving a CIM.  ODE conducts an active and exemplary program of psychometric research and statistical analyses. ODE reports can serve as models for other statewide assessment programs. It is well within the current capabilities of ODE to enhance their program of psychometric research to explicitly include special consideration of issues related to the assessment of student with learning disabilities. The research studies by Professor Gerald Tindal (University of Oregon) and Patricia Almond (ODE) provide some of the best empirical evidence that currently exists on student with learning disabilities. The Panel believes that a good assessment system can be made even better based on the recommendations included within this Report. Recommendations 1. ODE should award separate CIMs for each subject to reduce adverse impact on students with learning disabilities allowing them to demonstrate their profile of accomplishments across subjects. Awarding of separate CIMs provide a rich opportunity for all students to provide evidence regarding what they know and are able to do for separate subjects. The use of separate CIMs by subject is necessary to lessen the adverse impact of the OSAS on students with learning disabilities. One of the characteristics of students with learning disabilities is that they exhibit an array of competencies with various strengths and weaknesses that may be masked by the awarding of an overall CIM. The current system creates a domino effect in which students with specific learning disabilities fail one component (for example, dyslectics failing the writing test because they are unable to spell correctly), and therefore fail the overall CIM. Both conjunctive and compensatory scoring mask achievement levels within each subject. Separate CIMS would provide the opportunity for all students in Oregon to provide a richer profile of their accomplishments and allow students with learning disabilities to document that they are functioning at high levels in areas not affected by their specific learning disability. If a component of the OSAS is an essential requirement for meeting or exceeding the standard benchmark for obtaining a CIM, then either an accommodation, alternate scoring, or alternate assessment should be provided. 2. Psychometric analyses carried out by the ODE for assuring reliability, validity, and fairness of the OSAS should be developed and applied to students with learning disabilities similar to those applied to other subgroups (e.g., gender, race, and ethnicity). Future psychometric analyses of ODE assessment data should systematically include issues related to the fair and objective assessment of students with learning disabilities. In all test development activities and psychometric analyses, ODE should treat students with learning disabilities as a distinct group in the same way other subgroups are treated. For example, ODE conducts DIF analyses for various subgroups of students (e.g., gender and race/ethnicity). The Panel recommends that similar analyses be routinely conducted for students with learning disabilities. Other examples might be the reporting of impact data for students with learning disabilities during the standard- setting process or separately to the agency charged with setting the performance standards on the CIM. The underlying principle is that the special needs of students with learning disabilities should be taken into account in all of the psychometric analyses and reporting of technical data in a way that is comparable to current analyses related to gender, race, and ethnicity. Modifications in the current collection of student data should be made as needed to facilitate these analyses. In order to implement this recommendation, the ODE will need to develop an appropriate data reporting system that will support the collection of accurate information related to the type of accommodation/ modification (non-allowable accommodation) used by students with learning disabilities. 3. ODE should develop a record keeping system and database that will support analyses regarding the psychometric quality of the OSAS for students with learning disabilities. In order to maintain the psychometric quality of the OSAS, the ODE should develop procedures for routinely collecting data regarding the accommodations, both approved and non-approved, and types of learning disabilities exhibited by Oregon students. The current data sets do not provide for accurate merging of individual level information on assessment issues and the needs of students with specific learning disabilities. 4. The ODE shou