Accountability Changes: A Proposal from the Kentucky Department

| Post by Susan Perkins Weston |

Over recent months, the Kentucky Department of Education has coordinated a wide array of discussions about our next accountability system.  The most recent published proposal based on those discussions was shared with the Kentucky Board of Education at its February meeting. 

Here's a look at how that proposal answers the "six big questions" from my November blog post about accountability change, along with a few notes on topics for further exploration.

1. What should our rising generation know and be able to do?
The proposal does not require revisions to Kentucky’s academic standards.

2. What indicators can we use to track our progress toward those desired results?
The elementary and middle school indicators for school accountability ratings will include:

• Proficiency on state assessments
• Student growth data, using progress toward a student’s annual personal target for improvement
• Achievement gap closure data by income, race, disability status, English learner status, plus a “consolidated group” based on race, disability status, and English learner status
• Transition readiness, focused on learning about non-tested subjects, career fields, and essential skills, using measures that are under development
• Opportunity and access measures that include arts opportunities and standards-based teaching and learning in science, social studies, health, physical education and career studies

For high schools, the indicators used for ratings will include:

• Proficiency on state assessments
• Achievement gap closure data
• Transition readiness, shown by graduation rates and by academic readiness (ACT, SAT, AP, IB, or dual credit) or technical readiness (industry certification, KOSSA, dual credit) or military readiness (ASVAB)
• Opportunity and access measures that consider advanced coursework, arts, writing, global competency/world language, practical living/career studies, and specialized career pathways

In addition to the indicators used for ratings, state reporting will include additional opportunity and access measures (discussed under Question 7).

Topics To Explore
• How will annual personal targets be set?
• When will transition readiness measures be announced?

3. How far and how fast do we intend to raise those indicators?
For students overall, the proposal calls for base goals that reflect recent history, asking for improvement that matches the statewide improvement for the highest scoring student group in 2014-16.

For student groups with lower scores, the proposal calls for additional goals that will cut achievement gaps in half by 2030. Those gaps will be defined by comparing one group to another.

Topics To Explore
• Will there be custom goals for each school?
• Will there be interim benchmarks on the way to each goal?
• If the highest scoring  group declined, how will goals be set?

4. How will we rate (or differentiate) schools each year?
For elementary and middle schools, the proposal calls for public reporting of four performance levels (low, moderate, strong very strong) for:

• Proficiency and growth
• Transition readiness
• Opportunity and access
• Achievement gap closure

For high schools, the same four levels will be use for:

• Proficiency and transition
• Opportunity and access
• Achievement gap closure

Based on a matrix of all of those results, schools will also receive

• An overall school ratings using six categories from outstanding to intervention
• A gap closure designation or a gap issue designation

Topics to Explore
• How will standards be set for the four performance levels?
• How will citizens be included in the standard-setting?
• How will those levels relate to the state’s long-term goals?

5. How will we identify schools for added support?
The proposal calls for:

• Tier I targeted support (early warning) for schools where one or more student groups have results like lowest 10% of schools
• Tier II targeted support for schools where one or more student groups has results like the lowest 5% of schools
• Intervention for schools where results are in the lowest 5% of schools, high schools where graduation rates are below 80%, and schools that have qualified for tier II targeted support for three or more years

Topics to Explore
• How many schools will qualify for targeted support?
• Will all targeted support schools get a gap issue designation?

6. What support will we provide to identified schools?
The current proposal promises an additional document to describe the kinds of support and adds that the “breadth, depth, and intensity of school support will depend in large part on the available resources.”

Topics to Explore:
• When will the support document be available?

7. How will we promote accountability for results not included in the ratings and support rules?
A major innovation in the proposal is a call to report an additional set of indicators that are not counted for the ratings.

For schools, the proposal calls for reporting on:

• Reading and math proficiency as good or better than kindergarten readiness rates for all groups of third-grade students
• Global competency and/or world language exposure for elementary and middle school students
• Proportional identification rates for the Primary Talent Pool and Gifted and Talented services for all student groups
• Proportional out-of-school suspension rates for all student groups
• Chronic absence rate (percent of students who miss 10% or more days in a school year)
• Teachers with appropriate certification
• Teacher turnover rates
• First year teachers as a share of all teachers
• Librarian/media specialists and guidance counselors, with attention to the professional roles they play

For districts, the reporting may also include:

• ALL STAR ratings for the state-funded preschool program
• The percentage of of students served in half-day and full-day kindergarten programs 

New Accountability: SB 1 Amended in Committee

On Thursday, Senate Bill 1, sponsored by Senator Mike Wilson was approved by the Senate Education Committee with amendments in a committee substitute. On Friday, the bill passed the full Senate. House Committee consideration will be the next step.

Along with some smaller changes, the amendments:

• Removed a change to the arts-based high school graduation requirement  
• Deleted a requirement for the new accountability system to include a "band of schools" approach to school improvement
• Revised the rules for identifying schools for targeted support and improvement based on weak results for student groups

To reflect the amendments, the Prichard Committee has updated its bill summary. Key issues include:

• Standards for what students know and can do
• Assessments and approaches to meeting standards
• Accountability steps to ensure progress toward meeting standards
• Other changes (including certified staff evaluations, school council changes, and Department of Education changes)

You can download the complete summary here or the full bill in its current edition here.

Kentucky's Tests Are Harder Now

A new National Center for Education Statistics study provides a sturdy basis for thinking about how our K-PREP assessments, launched in 2012, compare to the earlier Kentucky Core Content Tests.  The study worked out a 2013 NAEP scale score equivalent to scoring proficient on each state's 2013 reading and mathematics assessment, using NAEP's 0-500 scale.  The results for Kentucky are consistently higher than similar findings from 2009, providing a quick, helpful confirmation that Kentucky really is aiming higher.

Sources: 2013 and 2009 comparison studies here, with a hat tip to EdWeek's Curriculum Matters blog.

Readiness Trends (Added Detail!)

In recent years, Kentucky has expanded the evidence we use to show that Kentucky students are ready for college, career, or both, and this post shows some of the details of how that expansion has worked. First, a graph to show what's changed:

 Next, some background about the parts of the graph:

• For 2010, the graph shows the total percent of public high school graduates who met all three ACT readiness benchmarks (English, mathematics, and reading) set by the Council on Postsecondary Education. It combines the students who met those benchmarks while participating in the statewide required 11th grade administration of the test and those who retook the test and reached the benchmarks at a later date.

• Starting in 2011, students who reached the benchmarks on the 11th grade statewide administration are shown separately from those who reached the benchmarks later on.

• Starting in 2012, student success on Compass and KYOTE placement tests, used by universities and KCTCS to assign students to courses, can also be seen. More exactly, students who have not met all three ACT benchmarks can be counted as college ready based on scores from Compass, KYOTE, a combination of subject scores from both tests, or a combination of scores from those two tests and ACT.  

• Also starting in 2012, career ready students are included. For career readiness, students must reach required scores on the Armed Services Vocational Aptitude Battery (ASVAB) or ACT WorkKeys to show academic readiness, and they must also meet needed scores on a Kentucky Occupational Skills Standards Assessment (KOSSA) or earn an industrial certificate to show technical readiness.

The newest results, adding up to 62% of 2014 graduates being counted as ready for college and career, provide the most complete information. The 2014 information includes data from all the assessments Kentucky recognizes as showing readiness. 2014 also reflects an culture in which schools encourage students to prepare for and take additional assessments, especially if their early ACT scores show that they need to improve.

In contrast, the 2010 results are clearly incomplete. The 30% shown as ready reflect only the ACT results. We know that some Kentucky students took each of the other assessments in those years, but their results are not included. Plus, schools had less incentive then to encourage students to try the other ways of demonstrating readiness, so some students may have been ready but not taken any test to make that easy to see.

And, of course, none of the results are superb indicators of the full results we really want for our students. The available assessments can't track students' capacity for sustained work, like using reading skills to research a problem and writing skills to explain a possible solution or taking on a real-world challenge and applying their mathematics skills to wrestle their way to a sound response. They also give very little indication of students' perseverance, teamwork, and other capacities that we know matter deeply for college and career success.

Still, the available information clearly shows Kentucky students as increasingly able to show their readiness as measured by the available assessments, and the same information makes it clear that we have plenty more work to do to make sure every student is equipped for adult success.

Source note: the graph above combines several slides from a December presentation given by the Office of Education Accountability.  See below for the way OEA displayed the same results.

.

New Prichard Report: Progress in Kentucky Education: Higher Standards, Assessments and Teaching

And more important news from Tuesday's release:

LEXINGTON, KY –Kentucky’s academic standards are the focus of a new Prichard Committee report, which shares additional information about the state’s assessment and teacher evaluation systems.

“Progress in Kentucky Education: Higher Standards, Assessments and Teaching” describes the interconnected elements of the state’s education system:

• academic standards that establish what students should know and be able to do as they move from grade to grade toward graduation
• an assessment system that measures how well students and schools are meeting those standards
• a professional growth and effectiveness system to ensure that educators are able to improve their skills every year and equip students for steadily higher levels of performance

The report points out that higher academic standards are critical because Kentucky’s students face a future where jobs will require that they can adapt and learn new skills on a continuing basis; in other words, they must learn how to learn. Kentucky’s state standards help them do that, the report notes.

Making sure schools and students are on the right track is essential, and Kentucky’s statewide assessments provide evidence about the learning that is taking place in schools. The report notes that assessments are particularly important for spotting and addressing achievement gaps between groups of students – a continuing, significant challenge for Kentucky.

The report also emphasizes the relationship between effective teachers and student success, pointing up the need to give educators ongoing feedback and opportunities to develop stronger professional skills. Kentucky’s new Professional Growth and Effectiveness System looks closely at both the quality of what teachers do and the learning that takes place in the classroom.

The system identifies areas where each teacher is already effective and where improvement is needed, and it promises consistent support for making those improvements, the Prichard Committee report points out. It also sets clear criteria and quick timelines for ineffective educators to improve their practice or move out of the profession. A matching system is in place for principals, and similar approaches are being developed for counselors, librarians, instructional supervisors, and other professional education positions.

“It’s clear we are on the right track in Kentucky. Our students are performing at higher levels and teachers are meeting the standards by providing deeper learning opportunities in the classroom. Staying the course and working together will ensure continuous improvement for our students and for our state,” said Stu Silberman, executive director of the Prichard Committee for Academic Excellence.

“Progress in Kentucky Education” was released today along with a special Prichard Perspectives report on  implementing the standards in Kentucky classrooms: “Finding Solutions: Standards Push Students Toward Real-Life Problems.” Together, the reports provide an overview of progress in Kentucky education and how that is reflected in both policy developments and classroom practices.

Both reports are available at prichardcommittee.org.

Big New Idea: Next Generation Instructional Design

Next Generation Instructional Design (abbreviated as NextGen ID or NGID) is an ambitious new education effort underway in Kentucky, and I've spent part of the morning reading up.  Here, I'll offer short basics gleaned from the project webpage, followed by my own italicized notes on why each element seem important.

In daily student experience, NGID learning units will provide new opportunities for "project-based and problem-based learning," designed to develop high levels of knowledge, skill, and understanding and meet Kentucky's academic standards.  

The project/problem part means students themselves will be actively figuring out how to use what they learn, and they'll do the learning in ways that make its usefulness clear from the beginning.  The standards part means it will be the right kind of challenge, equipping them with capacities they can use for further achievement in college, careers, and community participation. Overall, this is a new push toward learning approaches that students often find meaningful, engaging, effective, and (dare I say it?) fun.

In daily teacher experience, these units will provide useful tools and common assignments for helping students learn, designed with built-in ways to check whether the learning is on track.

That's also great.  It's a plan to help educators with the demanding new expectations we've asked them to take on.  "Common assignments" also means they're designed for teachers to be able to collaborate with one another on development, implementation, and examining the resulting work to think about best next steps for students.  And, of course, engaged students are far less likely to cause  behavioral problems, so this sort of approach can also promises help with the whole discipline/classroom management challenge that teachers must address in order to support student learning.

Meanwhile, data on student growth is going to be essential for Kentucky's new Professional Growth and Effectiveness System. The NextGen ID effort will also examine how "student work samples and other artifacts may provide information related to educator effectiveness to complement, supplement or provide alternatives to existing test-based measures of student performance."

 In other words, there's hope that the work students turn in can be used for teacher evaluations.  If this succeeds, it will mean that judgments don't have to rely solely on assessments that measure only a fraction of what we really want students to know and be able to do: it'll be possible to look at fully-applied, hands-on, very-nearly-real-world use of students' knowledge and skills. It's a chance to get most closer to what students have really learned and much closer to what teachers have really achieved.

The work is also new and starting quite small.  Coordinated by the Fund to Transform Education, it involves about 60 Kentucky educators working to think through how the units can be used and what kind of impact they will have, while developing the skills to lead the development of additional units and help colleagues put the units to work across the state.

The good news there is that the plan is to move at the pace that allows deep work and lay good foundations for effective expansion in the years ahead.  The limiting factor is that moving at that pace means this exciting effort launch in all schools right away: it'll take patience and commitment to see how richly we can put it to work statewide.

For added information on NextGen ID, check out the Fund's webpage and press release, and for a livelier idea of the kinds of work these units may involve, check out this recent Kentucky Teacher report on the common assignment work begun in 2013.

College impact even for students with weak scores

Can a large number of students with low admissions scores really complete four year degrees and reap the economic benefits of college completion? The New York Times is spotlighting two recent studies that suggest that, in fact, they can.

Using SAT data,  the studies tracked students who reached an 840 admission cutoff score for their states' universities, and also tracked those who narrowly missed the cutoff.   (Because the studies looked at outcomes over a long period, the original SAT scores used the 0-1600 scale in place when the participants were in high school.)

One study, based in Georgia, fund that roughly half the students who reached the cut  score earned bachelor's degrees within six years, compared to 17% of those who missed.  That's a clear indication that many students with marginal scores can, in fact, complete their degrees.

The other, focused on Florida students, found an average earnings difference of 20% between the two groups.  The Times qootes one of the authors of that study as saying, “If you give these students a shot, they’re ready to succeed,”

For those who make or influence policy, this sort of evidence can be a reminder to use test scores with caution and in combination with other evidence of academic capacity, commitment to work, and tenacity in confronting challenges.   Taken alone, the assessments can miss or underestimate quite a lot of human potential to persevere and succeed.

Scientific Practices: A Big Shift in Our New Standards

Kentucky's new science standards bring a giant shift toward science and engineering practices.  Eight  major practices are to be applied across physical science, life science, earth and space science, and engineering:

1. Asking questions (for science) and defining problems (for engineering)
2. Developing and using models
3. Planning and carrying out investigations
4. Analyzing and interpreting data
5. Using mathematics and computational thinking
6. Constructing explanations (for science) and designing solutions (for engineering)
7. Engaging in argument from evidence
8. Obtaining, evaluating, and communicating information

Those practices aren't an alternative to content knowledge: students will still need to read, write, study, and thing about scientific insights developed over centuries and those emerging now from current research.

Instead, the practices demand that the content knowledge move off of the textbook page (out of the lecture notes, beyond the PowerPoint slides) and into active thinking and work.  Each page of the standards begins with statements that "students who demonstrate understanding can" do specific things.  As illustrations, the middle school standards say call for students to be able to "undertake a design project to construct, test, and modify a device that either releases or absorbs thermal energy by chemical processes" and "analyze and interpret data to provide evidence for the effects of resource availability on organisms and populations of organisms in an ecosystem."

For student learning, that will require a big realignment to focus on puzzling through big issues and practical applications and building increasingly skilled use of each practice.  For example, with Practice 1 (asking questions), students may start with open-ended puzzling about phenomena they've observed, but then they'll need to learn to move smoothly into focusing their inquiries on testable questions. For Practice 3 (planning and carrying out investigations), those questions will need to be framed in terms of variables and controls.  The standards also provide progressions from grade to grade, so that students develop expertise over the years--but even the youngest are involved in regular, lively work to develop rich insight into the natural world.

In turn, for the community at large, this shift in science expectations creates a new puzzle: what kind of evidence will show us that students are developing those practices?  It seems hard, probably impossible, for machine-scored tests to give us real insight.  If we want to know how students are doing, we may indeed need to find innovative ways to check on this deeper learning.  To succeed, we may have to take on very active roles in defining problems (Practice 1), designing solutions (Practice 6), and making arguments from evidence (Practice 7).

Performance Tasks, Because Life is not Multiple Choice

Multiple-choice tests are easy to score, and there's a whole science built up around ensuring that results are fairly comparable from school to school and from year to year.  Those are good things.  And yet...

The work students will need to do as adults will be so different. They'll need to create their own solutions, analyzing the problem independently, hunting for their own information, thinking it through, and communicating their conclusions and reasoning effectively.  So I was glad to spot the new PerformanceTask.com, which is promising "articles, discussions, and resources" on some richer ways for students to demonstrate what they know and can do.

In the initial blog post, Jay McTighe offers a definition:

A performance task is any learning activity or assessment that asks students to perform to demonstrate their knowledge, understanding and proficiency. Performance tasks yield a tangible product and/or performance that serve as evidence of learning. Unlike a selected-response item (e.g., multiple-choice or matching) that asks students to select from given alternatives, a performance task presents a situation that calls for learners to apply their learning in context.

McTighe also offers four examples of performance tasks students might be asked to take on: one for siting and designing a public park, another for testing a company's claims about its cat litter, a third for choosing where a regional consultant should base her home and office, and the last on developing articles on the role of forensic investigators.

All of those tasks are harder than choosing from prepared answers.  Students have to work harder, and the scorers have to work harder, and public officials using the data have to think harder about the implications.  The trade-off is this: this kind of work offers evidence that really shows whether kids can really do the sorts of things we really want them doing.

Mind, I'm not saying we need to use this kind of task for statewide accountability.   They're expensive, and there's not a lot of public tolerance for the uncertainties that are included in the scoring.  We learned that in the 1990s when Kentucky tried to break new ground with this kind of activity.

Still,  there could be big benefits if we could use this kind of task as evidence for students, parents, teachers, and communities to consider.  We'd have a better idea of what students will know and be able to do as contributors to our shared future.   In those slightly looser uses, we should be open to richer ways of checking for learning and thinking through what's most important in teaching.

From that perspective, PerformanceTask.com is a welcome invitation to new discussion.

Will AMOs Ever Be Bigger Than One Point?

Question
Will Kentucky schools ever have AMOs (annual measurable objectives) that ask them to improve by more than a single point per year?

Short Answer
Under Kentucky's current accountability system, AMOs greater than 1.0 are effectively impossible.

More Detailed Answer
It is mathematically possible for Kentucky AMOs to be bigger than 1.0, but scores would have to change so dramatically that it is simply not a meaningful possibility.

For example, if the top half of elementary schools all raised their scores by 10 points and the bottom half all dropped their scores by 5 points, that would allow AMOs of 1.1 points.

A change like that, with elementary scores changing radically in both directions so that the strong and weak are pulled much further apart, is hugely improbable. Effectively, it's impossible.

Middle and high schools would need even more improbable-to-impossible changes, with even bigger growth in the difference between the upper and lower results.

Technical Answer 
(To Avoid Statistical Details, Skip the Fine Print Below)
In Kentucky, our AMOs are based on standard deviations. A standard deviation is a way of saying how results spread out around the mean.  If results all clump together close to the mean, the standard deviation is small. If they spread out a lot, the standard deviation is large.

Under changes voted on by the Kentucky Board of Education earlier this month, future AMOs will ask most schools to improve their next-generation learner scores by one-third of a standard deviation in five years, meaning one-fifteenth of a standard deviation a year.

That formula means a standard deviation of 15.8 or higher will be needed to allow a 1.1 AMO.

A web-based standard deviation calculator allowed some experiments about what scores would yield that 15.8.

Entering the 2014 next-generation learner results for our 720 elementary schools yielded a 9.2 standard deviation.  One-fifteenth of that would be about 0.6. With a result like that, the new minimum AMO of 1.0 will apply.

A next step was adding 5 points to the top 365 scores, subtracting 5 points from the bottom 355, and entering those results in the calculator.  That change was not big enough to yield a 15.8 standard deviation. 

Moving each group 6 points did not work either.

10 up and 5 down worked.

10 up and 4 down did not, showing 10 and 5 to be one of the smallest elementary changes yielding 15.8 as the standard deviation and 1.1 as an AMO.

For middle and high schools, the 2014 standard deviations were smaller than at the elementary level, so it would take even bigger changes to get to a 15.8 standard deviation and a 1.1 AMO for most schools.

Why did the experiments try scenarios with some schools moving up and others down?  Because, again, standard deviations get larger when results get further apart. Bigger AMOs can only happen when the distance between the top and bottom results gets larger.  

Bottom Line
Under Kentucky's current accountability system, AMOs greater than 1.0 are effectively impossible.  The scale of change that would be required just isn't going to happen.

--Posted by Susan Perkins Weston

Pending Changes to School Accountability

Last Wednesday, the Kentucky Board of Education took another step forward on revisions to our statewide accountability system.  There are still a few steps left in the "gauntlet" of steps needed to amend regulations, but the changes are now quite likely to be implemented.  Here's a report on the main shifts, backed up by a few notes to clarify elements that are unchanged.

Gap Results
Set novice reduction goals in reading and mathematics for each subgroup that has enough students to allow separate reporting

Report on the percent of those subgroup goals that each school meets
     ... and continue reporting proficiency results for the combined gap group

Growth Results
Report success in moving individual students to higher performance levels (apprentice, proficient or distinguished) and in keeping students at the proficient and distinguished levels
      ... and continue to report student growth percentile results

Combined Learner Results
For elementary schools, count achievement, gap and growth results equally
     ... and keep middle school weights of 30% achievement, 30% gap, 30% growth and 10% readiness
          ... and equal high school weights for achievement, gap, growth, readiness and graduation 

Annual Measurable Objectives
Specify that annual measurable objectives (AMOs) will be at least 1 point for needs improvement schools and at least half a point for proficient and distinguished schools

Set AMOs based just on the learner results (achievement, gap, growth, readiness and graduation) 
     ... and keep calculating overall scores combining learner results and program review results
          ... and using overall scores for schools' percentile rankings
               ... and to identify proficient, distinguished and needs improvement schools
                    ... and to identify priority schools
     ... and planning to add Professional Growth and Effectiveness System results to overall scores

Focus Schools
Identify focus schools based on graduation rates below 80% for two years 
     ... and drop focus based on 60% graduation rates 

Identify focus schools based on subgroup scores in the bottom 5% for that subject and group
    ... and drop focus based on subgroup scores in the third standard deviation
     ... and keep focus based on gap group scores in the bottom 10% of all gap group scores

These changes are shown in two documents approved during the April KBE meeting: the Statement of Consideration on 703 KAR 5:200, dealing next generation learner scores, and the Statement of Consideration on 703 KAR 5:225, on school and district accountability, recognition, support, and consequences.

--Posted by Susan Perkins Weston

Three quick notes for a busy morning...

1.  Student voices, teacher explanations, and learning success, are all part of this great CN2 report on Boyle County strategies!


2. "Exercise is ADHD Medication" at The Atlantic website starts from multiple studies on the brain benefits of physical activity and builds up to justified frustration with schools and communities that still aren't rebuilding students' opportunities to move and become ready to learn.

3. Current federal law mandates reading and math testing every year from grade 3 to grade 8, but recently filed legislation could give states more flexibility, requiring them to test either reading or math in each of those grades.  Representative Stephen Israel (NY) filed the bill with support from the American Federation of Teachers, and details are included in this EdWeek blogpost.

Will better math tests be good enough?

PARCC and Smarter Balanced are the two multistate consortia working on new assessments for the Common Core State Standards. After four years of design and pilot testing and field testing, they're nearly ready for prime-time use--meaning administration to large groups of students for use in statewide accountability systems.  Since Kentucky hasn't chosen to use either assessment yet, I've been noticing news about both, including a rising debate about the ways they approach mathematics.

The consortia argue that their new versions are definitely better than existing multiple choice math tests because of new on-line options.  As an EdWeek report recently summarized:

Unlike previous state assessments, those being developed by the two federally funded consortia will include complex, multipart word problems that students will answer on screen. While some of those questions will provide built-in tools that allow students to put points on a graph or draw lines on a ready-made picture, other questions will ask them to write their answers in narrative form, using a keyboard.

But others argue that the computer tools still aren't close enough to the real work of using math to solve problems.  In the same EdWeek account, David Foster of the Silicon Valley Mathematics Initiative shared his concerns:

"I'm a mathematician, and I never solve problems by merely sitting at the keyboard. I have to take out paper and pencil and sketch and doodle and tinker around and draw charts," he said. "Of course, I use spreadsheets all the time, but I don't even start a spreadsheet until I know what I want to put in the cells.

"All Smarter Balanced and PARCC are going to look at is the final explanation that is written down," he said, "and if there's a flaw in the logic, there's no way to award kids for the work they really did and thought about."

Mr. Foster added: "I've played with the platform, and it makes me sick. And I've done it with problems I've written."

I'm suspect they're both right.  PARCC and Smarter Balanced are offering some big steps forward in how students answer math prompts, and yet they are also far from inviting students to use math in ways that are close to real life applications on the job, in the home, or in civic life.

For the long-term education discussion, this debate turns yet another spotlight on an enduring puzzle: how can schools develop a balanced commitment to the skills that are easy to measure and the skills that matter at least as much but don't fit easily into standardized assessments?

Do note that Kentucky is not signed up to use either PARCC or Smarter Balanced.  Our current K-PREP assessments use methods that were in use years before we adopted Common Core, and any changes from that will occur when the Department of Education seeks new bids for our testing contracts.  The consortia will be eligible to submit proposals, in open competition with other companies that think they can provide the data we need.

--Posted by Susan Perkins Weston

Which AP tests do Kentucky students take and pass?

On Advanced Placement tests, scores of 3, 4, or 5 can qualify a student for college credit, placement in advanced courses, or both.  Monday, while posting on the Leaders and Laggards report,  I realized that the subjects where students earn those credits deserve closer attention.

So, below, two additional thoughts on AP test success in Kentucky.

First, a look at the major areas where 2013 public school students received successful scores, combining multiple tests in disciplinary clusters. The green shades identify science, math, and world languages, the subjects that Leaders and Laggards included in their economic competitiveness ratings.  The very small slice for world languages stands out as a weak result in the overall picture.

Second, a look at the top 12 tests where Kentucky students succeed, showing the number of students passing each test.  It isn't really a surprise to see the English tests at the top of this list, but it would definitely be good to see the science, math, and language numbers move up.

Source note: These numbers come from the page for "AP Program Participation and Performance Data 2013" at the College Board website.

--Posted by Susan Perkins Weston

Common Core and Kentucky: Fordham says we made a big step up!

In a single year, Kentucky mathematics standards moved from a D to an A-.  We did it by adopting Common Core.

For years, the Fordham Institute has provided the most consistent ratings of state standards, issuing letter grades for each subject accompanied by often biting reviews. In their early reports, before KDE's website became a complete source for key materials they sometimes relied out-of-date Kentucky documents, but in recent years, they've checked the right materials and given blunt assessments to what they read.  

In 2006, Fordham explained our D for mathematics this way:

The organization of Kentucky's standards is difficult to understand and often incoherent. In addition, the standards are often vaguely stated. Arithmetic is not identified as an elementary school priority and is developed poorly. The coverage of high school content is variable.

Have I mentioned that Fordham is famed for its blunt wording?

Now, however, the Fordham rating for Kentucky reflects the strength we have gained from working with other states:

The final version of the Common Core State Standards for math is exemplary in many ways. The expectations are generally well written and presented, and cover much mathematical content with both depth and rigor. But, though the content is generally sound, the standards are not particularly easy to read, and require careful attention on the part of the reader.  

The development of arithmetic in elementary school is a primary focus of these standards and that content is thoroughly covered. The often-difficult subject of fractions is developed rigorously, with clear and careful guidance. The high school content is often excellent, though the presentation is disjointed and mathematical coherence suffers. In addition, the geometry standards represent a significant departure from traditional axiomatic Euclidean geometry and no replacement foundation is established.  

Despite some weaknesses, the Common Core standards provide a solid framework for learning rigorous mathematics.

It's worth noting that Fordham did not think the Common Core is better than the prior standards in every state.  Using the same criteria for all ratings, the Institute found that:

Eleven states plus the District of Columbia have math standards in the "too close to call" category, meaning that, overall, they are at least as clear and rigorous as the Common Core standards.

Kentucky was not one of those states.  Here's Fordham's map showing those states and making a clear point: Kentucky made a substantial step up when we adopted Common Core:

Delivery targets: impact on overall scores

Meeting 2013 delivery targets will produce impressive growth in the "overall scores" that combines all of a school's data in a single accountability number on a 0-100 scale.

For example, meeting delivery targets will result in:

• 3.3 points improvement in the statewide elementary overall score.
• 4.0 points improvement in the statewide middle school score.
• 3.3 points improvement in the statewide high school score.

Improvement will be quicker for schools with results below the statewide level, and slower for those above. That's because delivery targets expect more improvement from those who have more to improve.

Because delivery targets are so ambitious, they can also have a big impact on schools' AMOs (short for annual measurable objectives).  AMOs are another part of our new approach to accountability, and they work by taking schools' overall score and asking for increases of:

• 1.0 points from schools in the bottom 70% for their level.
•  0.5 points for schools in the top 30% for their level.

Schools that make their delivery targets will exceed their AMOs with plenty of margin to spare!

Better yet, meeting delivery targets will allow schools to move rapidly toward overall scores of 100.
That's sweet news, because 100 is the overall score that will mean all Kentucky students are on track to graduate ready for college and career after consistently making expected growth.

Note to arithmetic lovers: my basic method here is to work out how much each subject's 2012 percent proficient will improve to hit that subject's 2013 delivery target; then multiply by 20% to show how that will change the achievement part of the overall overall score; and then work out how much that will change the overall score by multiplying by 30%, 28%, or 20% for elementary, middle or high schools.  I repeated the same approach for the gap group.  For middle school, I added in a readiness target weighted at 16%, and for high school I added in readiness and graduation each weighted at 20%. E-mail me if you want to see the whole set of calculations.

Delivery targets: halfway to 100 in five years

Here’s an excellent part of Kentucky’s new accountability system: delivery targets ask schools to move a set of indicators halfway to 100 in five years, which means a big step up every year.

For example, here’s what that will require statewide  for elementary students in each tested subject:

All schools and districts have delivery targets like these based on their own starting points. There are also matching targets for:

• The gap group that combines the student groups that have been historically underserved.
• Each of those student groups separately.
• For all students, gap group, and subgroups showing reading and mathematics combined. 
• Middle and high school readiness results.
• High school graduation rates.

 Want to see the delivery targets for a particular school or district? 

You can find almost all of them in the state’s excellent new on-line School Report Card portal by looking under delivery targets.  One exception is that graduation targets are under accountability, rather than delivery, in the report cards for each school and district.  Another exception is the gap group targets for individual subjects: at least so far, I haven’t found them in the individual report cards, but I did find them in the data sets section of the portal, in the delivery column’s gap file.  One more: I haven't found a middle school readiness delivery target listing yet at all, but the arithmetic makes that one easy to calculate, and high school targets are in the report cards and the data files.

A proficient high school example

Under Kentucky's new accountability system, schools are identified as proficient if their Overall Score is at or above the 70th percentile for schools at their level.  Now that first year results have been released, it's easier to illustrate what goes into the Overall Scores and the proficient classification  I'll use a high school right at that 70th percentile cut off to do just that.

To begin, each school now receives an Overall Score on a 0-100 scale.  For high schools, 100 will mean that all students are college and career ready, making typical or higher growth, and graduating on time.  Here's the actual example, showing the school's Overall Score of 58.0 and the five components that go into it.

The 64.5 for Achievement shown above is calculated from the students' performance levels in five subjects:

The 41.7 for Gap is also calculated from those five-subjects performance levels, using data only for students in groups that have historically been underserved:

The 59.5 for Growth reflects the students making typical growth in the two subjects tested more than once in high school. The 47.1 for Readiness looks at ACT and other tests of readiness for college, career or both, with half- point bonuses for students being ready for both options.  Here are the details on those two results:

Finally, the 77.3 for Graduation is simply the school's 77.3 percent averaged freshman graduation rate shown above.

In future years, the plan is to add program review results and professional growth and evaluation data to the Overall Score calculation. As that happens, the 70th percentile cut point will be adjusted. Once the full formula is in place, the state regulation calls for the 70th percentile to be recalculated every five years.

Added note: I've treated the school above simply as an example to help illustrate the main elements involved in being classified as a proficient school, using one of the three high schools with an Overall Score of 58.0.  Though my purpose is to clarify the process rather than focus on a particular school, I'll satisfy any reader curiosity and mention that charts above show data from Bracken County High.

The work begins anew

Kentucky has moved more quickly than any state in the country to define and assess student results that meet the demands of a knowledge-driven future.  This morning, in our newly released School Report Card, we meet the first data, and we see with new clarity what we will need to do to move all our children to those higher expectations.

The chart above shows where we stand in most succinct form, using the overall scores from  our new accountability system.  In that system, a score of 100 meaning that all students (including those in historically underserved groups) are on track to graduate from high school ready for college and career.  By that measure, we're a little more than half way there.

The elementary score combines three kinds of student results shown below.  The achievement number reflects results for all students in reading, mathematics, science, social studies, writing, and language mechanics.  The gap score indicates similar data for students in groups that are often "caught in the achievement gap," while the growth score based on the percent of students making expected growth from one year to the next in reading and mathematics.

The middle school score uses those three kinds of data and adds a college and career readiness indicator based on the Explore test taken by eighth graders.  Here's how those results break out this year.

The high school score reflects college and career readiness measured by ACT and other assessments, and adds in graduation data. Here's how those stack up.

The bottom line is that, this  morning, we can clearly define the work ahead of us. Kentucky has committed itself to raise all of these results to the high levels our children will need for full success in the economy and in their communities.  As you can see, that will be a mighty undertaking, and all of us--parents, teachers, citizens, elected leaders--will have major roles to play.

These results and further details--for our state, districts, and individual schools--is now easily accessible at the Department of Education's Open House.  

Aiming higher, scoring lower

Within a few months, we'll see the first scores from Kentucky's new assessment and accountability system, which means we'll be taking a frank look at where our students now stand compared to strong standards of readiness for college and career.

One good way to imagine what's coming is to think about one of those four-foot-high Fisher-Price basketball hoops.  Anyone who switches from one of those to a regulation court is going to score fewer goals the first season.

Kentucky testing is about to go through the same sort of shift, moving from an old definition of proficiency to a new one that fits better with what students will need in global competition.

While the Department still has some work to do to finalize the scoring system and calculate the statewide results, Commissioner Holiday is now sharing initial estimates that involve big drops.  Here's a comparison, reformatted from numbers shared by the Kentucky School Boards Association:

This is a hard thing, but a good thing: we're now aiming for what our children need, with an honest look at how much we must change to make those higher goals reality.