K-12 Accountability: Changes to Five Star Ratings

| Post By Susan Perkins Weston |

Kentucky’s new accountability system calls for schools to be rated from one to five stars, based on their performance on a set of dashboard indicators. Earlier today, I shared a quick list of how those indicators have been changed in the most recent (July 27) edition of the proposed regulation. Here, I’ll note recent changes to the five star ratings proposal. The Kentucky Board of Education will hold its second reading of the regulation on Wednesday, August 2, and you can download the full regulation here.

THE MATRIX APPROACH TO FIVE STARS
The proposed regulation now shows a matrix (or table) approach to translating indicator ratings (from very low to very high) into overall ratings of one star to five stars. The matrix concept has been shared widely in the Department of Education’s town halls, presentations, and overview documents, with some minor changes over the months of discussion and public input. It was not included in the previous regulation text, but the July 27 edition includes separate matrix versions for districts, high schools, and elementary/middle schools.

NOT THE SAME MATRIX APPROACH TO FIVE STARS
In past versions of the matrix, a five star rating required very high ratings for most indicators.

The versions included in the proposed regulation change that, saying that:

• Schools can earn five stars with just high rating on most indicators
• Elementary/middle schools can earn five stars even if they have low ratings for growth
• High schools can earn five stars even they have low ratings for transition readiness

These changes will make the star ratings substantially easier to earn. At the end of this post, I’ll share the older and newer matrix versions to allow readers to do their own comparisons.

A MATRIX APPROACH THAT OTHERS WILL BE ALLOWED TO CHANGE
The regulation versions of the matrix say at the top that “standard setting will confirm level of indicator performance necessary for the Star ratings.” That appears to mean that the standard setting participants will have the power to change the matrix rules.

There is also new language that says:

“During the standard setting process, the approximate weights in the following table shall be considered. The proposed ranges in the table indicate the relative emphasis between indicators. The ranges are set to guide Kentucky educators to determine the combination of performance from very high to very low within the indicator during standard setting.”

The table shows weights that could be used for each indicator. For example, at the high school level, the Proficiency indicator is shown with a 15-25 range, and the Graduation indicator is shown with a 5-15 range. The weights look like a formula for combining indicator scores into a single score for the school. I’m puzzled about how the standard setting group or groups could use those weights to change the matrix approach.

On this issue, I hope the August 2 presentation and discussion will provide important clarification on which elements will be decided by the Kentucky Board of Education regulation and which elements will be open to change by the future standard setting process.

A THREE STAR MAXIMUM IF STUDENT GROUP RESULTS ARE WEAK
Even if results for the whole school are very strong, schools will be limited to a maximum of three stars if one of their student groups has troubling results. That approach has been discussed for quite a while, and there are now two different ways the three-star limit can apply.

First, the school can be designated as having a “Gap Issue.” The earlier version of the regulation based the Gap issue designation on “very large” gaps or low performance. The July 27 edition has more precise language:

“A school or district shall be designated as a “Gap Issue School” or “Gap Issue District” for statistically significant achievement gaps or low-performing students. Schools or districts with statistically significant achievement gaps may not be rated above three stars.”

Second, schools that are identified for targeted support and improvement will also be limited to three stars. Under Senate Bill 1, schools will receive that targeted support if any student group has results like the lowest-performing 5% of schools or if any group results has results like the lowest-performing 10% of schools for two years. Under the federal Every Student Succeeds Act (ESSA), those targeted support decisions must be based on all the indicators, and all indicators must be reported separately for each student group (with an exception allowing progress toward English Proficiency to be reported only for English learners).

That second three-star limit based on targeted support is shown in the matrix for elementary/middle schools and the one for high schools. It appears to meet a key ESSA requirements that all states:

• Have an approach to identifying schools where any group is “consistently underperforming”
• Change a school’s rating if a school is identified under that approach (a step ESSA calls “differentiation”)
• Provide targeted support and assistance to schools identified under that same approach

Finally, as promised, here are the matrix versions shown in the July 27 regulation, along with a version from a July 6 overview document from the Kentucky Department of Education