Difficulties with disability scores

2009 test results released in September inaccurately counted some students as having disabilities when they in fact do not, and some schools' NCLB classifications may change as a result.

The basic process
1. Students were identified based on group membership based on sex, race, disability, and other factors before they were tested in the spring

2. Their individual test responses were scored in early summer.

3. The work of all students was tallied to calculate the percent of students who reached each performance level.

4. The same thing was done with each subgroup.

5. A school needed for all students and for each group to have a specified percentage scoring proficient or above. A school that fell short for any group was counted as not making AYP (short for adequate yearly progress.)

6. The reports released in September listed the percent of students at each performance level in each group, and also announced who did and did not make AYP for the year.

Where the problem hit
In step 1, some students were wrongly counted as having disabilities. It looks like the errors were on students who had qualified for special services in the past, but are no longer classified as needing that help. It also looks like the problem did not happen in all districts, but did happen in multiple places, including Barren, Jefferson, and Scott counties.

To me, that looks like a data system error. Maybe a problem in moving data from our old student software (STI) to the newer Infinite Campus. Or maybe a problem in how the data was taken from Infinite Campus on its way to being used in the 2009 testing files.

The department is still working to find the common factor among the districts where the problem occured and to figure out just what went wrong.

The effects of the problem
Individual student scores look sound.

Total scores for all students in a school also look sound.

Group scores for students with disabilities, though, include some students who don't belong in the group. Correcting that is sure to change some results for that group. Plus, changing the scores for any one group can also change a school's overall NCLB category. Thus far, the schools KDE has checked have not changed AYP status, but they're still working to sort out the complete picture.

The department is still analyzing the problem, but included the basics in their briefing to the Kentucky Board of Education this morning.

There is not yet an estimate on when fully corrected data will be available.

Other articles
The Herald-Leader has coverage here, and Brad Hughes at KSBA has the story here.