The Children We Leave Behind: Seeing and Engaging The Results

Which student subgroups have the worst achievement gaps, so severe that the group performs like the lowest 1% of all students?  In Kentucky, gaps that deep earn a "focus school" identification under Kentucky's accountability regulation. (Technically, the designation applies when a subgroup has "a score in the third standard deviation below the state average for all students," but that works out to something very like the bottom 1%.)

In this year's list of focus schools, students with disabilities are, by far, the group most likely to have results that earned that designation.   The pie chart above shows that for reading, and matching charts below show that that the pattern is similar for every other subject Kentucky tests.

I don't like what I see in these charts, but I do like being able to see it.  We should not hide from this kind of truth about how we are doing as a state. In each community where a student group is struggling this deeply, we should be talking about the problem and pulling together to solve it.

We should be pulling together to meet children's needs if the group being left so far behind is students with disabilities. We should engage just as intensely if the group is African-American students, Hispanic students, students with limited English proficiency, or students who qualify for free or reduced-price meals: those problems happen in fewer schools, but when they occur they are just as urgent.

Do the schools in your community have a student subgroup that is being left this far behind?  You can find the full list of Focus Schools by going to the Supplemental Data portion of KDE's Open House Portal, clicking the link for Accountability, and scrolling to the bottom of the page.  Please do check, and please do discuss what you find with your neighbors and your local school leaders.  We can change this picture, but only if we see it and engage it.

 

Susan's Opinion: Social Studies without History is a Bad Idea

Proposed new  social studies standards are drawing teacher concerns, according to today's Herald-Leader report.  Teachers say the standards give little guidance on what students should learn each year.

The draft Kentucky Academic Standards for Social Studies is available in the materials for the October 7 Kentucky Board of Education meeting: go to that portal, scroll down to Item IX and then look for Attachment B.

I've read the full document, and I agree with the teachers. My biggest concern is that the draft social studies standards contain no history.

No Valley Forge, no Gettysburg, no D-Day, no Berlin Wall, no falling Twin Towers. No Industrial Revolution and no Depression. No slavery, no segregation, no civil rights movement. No Trail of Tears. No debate over the Bill of Rights and no seventy-year struggle for women's suffrage. And for that matter, no Isaac Shelby, no Isaac Murphy, and no Kentucky at all except for a single reference to the state constitution.

Instead, there's only a section on Historical Thinking, which calls only for overarching skills in thinking about history: chronological reasoning, contextualization and perspectives, historical arguments, and interpretation and synthesis.

This is a big problem for children's learning: no one can use those big skills without having some meaty history to apply them to. Especially, "contextualization" means figuring out how a particular primary document relates to a bigger historical situation--which means you can't do it at all until you've learn a bunch of history.

It's also a big problem for students preparing for citizenship: They really do need to recognize the main outlines of what happened in their state, country, and world before they were born, in order to join the debates about what should happen next. 

And it will be a problem for many who participate in our political community.  Have you heard recently about activists who argue that the new AP U.S. History standards omit too many important parts of our shared heritage? Having also read the APUSH standards, I think those critics are wrong.  But if the same folks criticize these draft Kentucky standards for the same kind of omissions, they will be right.  Those who have worked for decades to ensure that our children understand that our heritage includes the diverse experience and diverse contributions of America's indigenous peoples, by African-Americans, and by more recent immigrant groups, will be just as frustrated.

Finally, these draft standards could be taught equally well in Oregon, Uruguay, or Oman: they're not designed to equip citizens of Kentucky and the United States for the work of participating here. I think that's a mistake.

--Posted by Susan Perkins Weston

EdTrust: Kentucky Accountability Signals Are Weak on Achievement Gaps

 

"Average math proficiency rates of African American students at schools earning a Distinguished rating are lower than average math proficiency rates of white students in Needs Improvement schools."

"Similarly, between 2012 and 2013 in Kentucky, reading proficiency rates for African American students declined at about 40 percent of Distinguished schools with data for this group. Math proficiency rates fell at about the same share of top-rated schools."

Both the chart and the quotes are from a new Education Trust report, "Making Sure all Children Matter: Getting School Accountability Signals Right."  In Kentucky, Florida, and Minnesota, EdTrust checked whether strong ratings school like "Distinguished" reflected strong achievement for students from historically under-served groups.  In all three states, they found good reason for concern that "a high rating despite low performance for some groups paints a false picture of success and allows schools to overlook some students."

I join EdTrust in counting this issue as very important, and I want to add two Kentucky-specific thoughts.

No, the Gap Group does not ensure good signals about results for Kentucky's minority students. In Kentucky's accountability system, a school's overall score includes separate attention to a Gap Group that combines students with low family incomes, disabilities, limited English proficiency, and minority backgrounds.  However, because Kentucky has so many low-income (free/reduced meal) students, results for that one group dominate the combined results. Groups with weaker results than the low-income group effectively disappear.  You can see the disappearance happening in my recent PrichBlog post on "2014 Achievement Gaps," showing that the Gap score matches what's happening for low-income students and hides what's happening for the others. So, no, the Gap Group does not signal the importance of results for other groups of under-served students.

No, the rules on "Focus Schools" do not ensure good signals about results for minority students. Yes, Kentucky's accountability regulation says that if a school has a student subgroup with results like the bottom 1% of students statewide, it must be identified as a Focus School.  And yes, the regulation says that Focus Schools cannot be rated as Distinguished.  But in 2013, the Kentucky Department of Education decided not to follow those rules.  So, no, the Focus School rules do not reliably signal major gap problems. And also, no, major gap problems do not exclude Kentucky schools from being publicly identified as Distinguished.

EdTrust's report concludes:

Our hope is that this analysis will prompt policymakers, advocates, and educators to put equity squarely back on the table in each and every conversation about accountability. The Secretary of Education can re-start that focus by making group performance matter in the upcoming waiver renewal process.

I second that hope and add that the Kentucky Board of Education has independent authority to ensure that all children matter in Kentucky accountability, with or without Secretary Duncan's prodding.

--Posted By Susan Perkins Weston

Kentucky's Jump In Scores: Where Did It Come From?

In Kentucky's accountability system, an overall score combines many kinds of data.  Program reviews are the newest elements, with several kinds of test scores and graduation rates having been considered since 2012.  So when we note that, statewide, scores moved up at the elementary, middle, and high school levels –and moved by more than four points at each level–it makes sense to how much each element contributed.

For example, the statewide elementary overall score grew from 64.2 to 68.7, and the graph below breaks that growth down into parts:

Similarly, the middle school overall score rose from 62.0 to 66.0, with these contributions from different elements:

And high school improvement from 66.8 to 71.3 included this combination of elements.

If you want to think through the arithmetic (not everyone does), the Kentucky Board of Education has set rules on how much each component should count toward the overall score. If you remember a teacher saying "the final exam will be 25% of your grade for the course," you've heard this kind of math before.

For example, the high school formula says that program reviews are worth 23%, and each of the other elements is worth 15.4%.  In the chart above, the 2014 bar shows 23.0 points because statewide, the high school program review score was a perfect 100, and 100 times 23% yields 23 points.  The other elements are smaller because those scores were less perfect, though you can see that achievement, gap group, readiness, and graduation all improved.

More broadly, you can see that program reviews contributed importantly to the growth from 2013 to 2014, contributing 2.1 points for high schools, 3.3 points for middle schools, and 3.2 points at the elementary school level.

2014 Achievement Gaps

Kentucky still has substantial statewide achievement gaps to wrestle, starting with elementary reading.

In addition to seeing that the gaps are large, do note three other parts of the pattern:

• The gap group has the same level of proficiency as the free and reduced-price meal group. (The gap group includes all students in the six other under-served groups shown above.)
• Students with limited English proficiency have the lowest scores, followed by students with disabilities.
• American Indian or Alaska native students have the highest scores of these groups, followed by Hispanic students
• Every sub-group scores below the all students results, and only American Indian or Alaska Native students get within five points.

Here's that same pattern for middle school reading, differing only in there being no groups with a gap smaller than five points.

And it is again for high school reading, again with all gaps larger than five points.

A complete set of graphs would make a massive post, so I'll settle for noting that these patterns are amazingly consistent. Across all six tested subjects (reading, math, science, social studies, writing, and language mechanics) and at all three levels:

• The gap group has the same level of proficiency as the free and reduced-price meal group, except that the two are one point apart in elementary social studies and writing and in high school language mechanics. 

• Students with limited English proficiency have the lowest scores, followed by students with disabilities, except that they switch places in high school mathematics whild still the two lowest scoring groups.

• American Indian/Alaska native students have the highest scores of these groups, followed by Hispanic students, except that they are tied in high school science and writing and switch places in middle school writing.  

• All of these groups have results below the "all students" group, except that American Indian and Alaska native students match all students in elementary social studies and outscore all students in high school social studies.  American Indian and Alaska native students also have results within five points of all students in seven other cases, but are the only group to get that close.

--Posted by Susan Perkins Weston

Note to our readers who receive the blog by e-mail: are the graphs in the post above included in your e-mail versions? Let us know by posting a comment or sending an e-mail to the Prichard Committee office? 

Draft Social Studies Standards!

Next Tuesday, the Kentucky Board of Education will discuss the draft of new standards for Kentucky social studies, and the draft is now publicly available as part of the KBE meeting material.

You can download just the standards or see the full board meeting materials, including information on how the standards were developed and how the state regulation will be amended to include them.

Check them out, and do share your thoughts!

2014 Accountability: Summarizing Some Impressive Results

PrichBlog now has a six-part summary of the strong 2014 statewide accountability results released earlier today, including:

• Overall scores are up more than four points for all three levels of schools
• Achievement and gap group results showed consistent improvement.
• Growth results, with an explanation of why those scores will stay very close to 60 over time.
• Readiness results are up for middle schools and up more for high schools.
• Graduation rates are up, whether a four-year or a five-year rate is used.
• Program review results now average at the fully proficient level for all three subjects at all three levels.

For each school and district, school report cards are also available, using this link and then choosing the reports you want to see.

Overall, it's quite a good year.  Naturally, there's still a great deal of work ahead, and in a few hours, I'll start working on understanding how the new results relate to  classic PrichBlog concerns.  I'll share as I learn.

--Posted by Susan Perkins Weston

Program Reviews as the Newest Accountabilty Element

Program reviews are a new indicator of the quality of student learning opportunities in subjects that Kentucky no longer tests.  For accountability, there are three reviews, focused respectively on writing, arts & humanities, and practical living & career studies.  And, as you can see,when the school scores submitted by all districts are combined into statewide numbers, they average out very high.

It's worth unpacking those results from the statewide briefing packet on the new results a little more.  Individual program review rubrics use a 0 to 12 scale, with 8 being the score for a proficient program.  To get numbers like the ones shown above, the scores on all three reviews are added together and divided by 24 (equivalent to three proficient scores).  Scores above the 100 level are rounded down to the 100 maximum allowed by the state scoring system.

So, the graph above shows that statewide, districts on average are rating their schools proficient or higher on their program reviews.  Individual schools had higher and lower scores, but the averages came out at or above 100 on all three levels.

Below, you can see the results for each subject for the two available years, all using the 0 to 12 scale that applies for individual reviews.  There, too, it's clear that the averages are now at or slightly above the proficient level for all three programs at all three levels.

In 2008, the discussion about program reviews included clarity that the Kentucky Department of Education would need to play a major role in ensuring consistent scoring, along with consistent refusal to estimate the costs.  In 2009, Senate Bill 1 specified "Each local district shall do an annual program review and the Department of Education shall conduct a program review of every school's program within a two (2) year period," again without frank engagement about the price tag.    Since that point, the Department has had no resources to carry out that scale of review, and many discussions have fallen into saying that schools score themselves, even though state law makes the reviews a district obligation. 

Whoever has been leading the process and whoever has been monitoring, I think these results will move us quickly to a serious discussion about what it will really take to ensure that program reviews reflect consistent scoring against high expectations.  That discussion is at least six years overdue.

--Posted by Susan Perkins Weston

Graduation Rates Move Steadily Up

2014 is Kentucky's second year counting adjusted cohort graduation rates as part of accountability scores. Cohort rates tracks students from when they enter high school to when they were expected to graduate.

Last year, school report cards showed a four-year rate, showing how that 86.1% fall 2009 first-time ninth graders graduated by the spring of 2013.  In the 2014 statewide report, there are two numbers. The four-year rate is for the class that started in fall 2010, while the five-year rate is an update on the fall 2009 entrants--showing that another 1.8% of them collected diplomas in the last year.

The statewide school report card notes that "The five‐year adjusted cohort graduation rate is used in accountability calculations," which is an innovation since last year.

--Posted by Susan Perkins Weston

Readiness As An Accountability Component

In the 2014 overall scores released today, readiness is an element for middle schools and high schools.  Those results are broken out below, with some explanation below the graph.

For middle schools, readiness means readiness for high schools success, while for high schol, it means readiness means readiness for college, career, or both.

The middle school results use a single measure: the percent of students who reach benchmark scores in English, mathematics, reading, and science on the eighth grade Explore test.  Explore is a test developed by ACT, Inc., and the benchmark scores were also set by the ACT company.

The high school results reflect multiple measures. Students can show college readiness by meeting  ACT benchmarks set by Kentucky's Council on Postsecondary Education or required scores on college placement tests.   One important detail:  there are CPE  benchmarks for ACT English, mathematics, and reading, but not science, so ACT science is not part of the college readiness definition.  Students can also show career readiness in a variety of ways, and students can be counted as ready for college only, ready for career only, or ready for both.

The chart below shows a little more about Kentucky's high school progress in the past year.  We had a small downturn in the number of graduates, but a clear increase of more than 2,500 students who qualified as college ready and a similar jump in those who are career ready, with a total jump of almost 3,500 in the number meeting one or both standards.  Overall, it's quite an impressive improvement!

 

--Posted by Susan Perkins Weston

Growth Results and Why They Don't Grow

The 2014 overall scores released early this morning include growth results, identified by comparing students' reading and math performance to that of other students with similar scores last year. 

The state Briefing Packet for this year's accountability release says:

The student growth percentile model (SGP) is based on a normative distribution of academic peer group. Since SGP model uses a normative distribution, the percent of students statewide scoring at the typical or higher level will be consistent from year to year at approximately 60 percent. At the individual school level students scoring at typical or higher level range from 14 percent to 89 percent.

I can explain this a little further.   Kentucky's student growth percentile model identifies groups of students who had similar scores on the previous year's assessments (creating those "academic peer groups").  Then, with the current year's results, the bottom 40% within the group are noted as not making typical progress, while the rest are identified as making typical growth.  For all the groups combined, and for the state as a whole, that means the growth component will always come in right below that 60% ceiling.

An individual school can have more than 60% of students making the progress defined as typical for that year--so long as some other school has less than 60%.  Or one school can have less than 60% if another has more.  But when looking at statewide results, it's not a fluke that the results are going to be very similar every year: that's exactly how the model is designed to work.

--Posted by Susan Perkins Weston

Achievement and Gap Details

For the state as a whole, overall scores moved up by four points of more at each level.  That overall score, in turn, reflected a set of component results, available in the statewide school report card, and starting with the two components shown below.

The achievement score uses a weighted formula to combine K-PREP results for reading, mathematics, science, social studies, writing, and language mechanics.  Schools get full credit for students who score proficient or distinguished, partial credit for students who score apprentice, and extra credit if the number of proficient students is higher than the number at the novice level.

The gap group score combines the same six subjects, but looks only at students who make it to proficient or distinguished, without any partial or extra credit, and only at students from historically under-served groups: low income, limited English, students with disabilities, and students from African American, Hispanic, and American Indian/Native American backgrounds.

If we continue this pace of improvement, we can expect to see all  students fully on track for college and career readiness--as shown by reaching a full 100--within:

• 10 years for elementary achievement/all students and 17 years for elementary gap group.
• 18 years for middle school achievement and 29 for middle gap group.
• 29 for high school achievement and 54 for gap group.

To me, those numbers signal that what we're seeing for 2014 is both good and not good enough.

--Posted by Susan Perkins Weston

2014 Overall Results: Wow!

Kentucky's 2014 accountability results, released at midnight, show an impressive one-year step up in how our schools serve our kids, with growth of four or more points at every level.  In Kentucky's accountability system, the overall score combines:

■ Achievement based on K-­‐PREP results in all subjects for all students
■ Gap Group based on K-­‐PREP results in all tested subjects for students who receive free or reduced price lunches, students with disabilities or limited English proficiency, and African-­‐American, Hispanic, and American Indian/Native American students
■ Growth results based on students’ progress in reading and mathematics
■ Readiness results based on Explore, Plan, ACT, and other evidence of readiness for college and career (used only for middle and high schools)
■ Graduation rates (for high schools only)
■ Program review results (added for the first time this year)

This year's improvement isn't all the improvement we need, but it's a really important showing of  how our students and our teachers can make rapid progress!

--Posted by Susan Perkins Weston

Three quick notes for a busy morning...

1.  Student voices, teacher explanations, and learning success, are all part of this great CN2 report on Boyle County strategies!


2. "Exercise is ADHD Medication" at The Atlantic website starts from multiple studies on the brain benefits of physical activity and builds up to justified frustration with schools and communities that still aren't rebuilding students' opportunities to move and become ready to learn.

3. Current federal law mandates reading and math testing every year from grade 3 to grade 8, but recently filed legislation could give states more flexibility, requiring them to test either reading or math in each of those grades.  Representative Stephen Israel (NY) filed the bill with support from the American Federation of Teachers, and details are included in this EdWeek blogpost.