In three of four subjects, Kentucky's 2008 high school graduates who took the ACT came within one-tenth of a point of national average scores. They were eight-tenths of a point behind in mathematics, and two-tenths of a point behind in the composite score.
Do notice that a much higher proportion of graduates participated in ACT here than nationally. It is widely thought that if small proportions of a class take an optional college readiness test, the best-prepared students are likely to sign up and the scores are likely to be higher than a representative sample. As a state moves closer to all students participating, the less-prepared students are added and scores are likely to be lower. Kentucky sent most--though not all--of our students, and they still brought back English, reading, and science scores in line with the nation.
Susan, the ACT performance you show needs an awful lot of explanation.
ReplyDeleteFirst, these scores are for all tested graduates in each state, whether from public, private, or home school systems. Aside from the fact that public school-only scores in Kentucky have historically been lower than the all-student scores, there are other factors to consider about the data in your comparison.
For one thing, ACT reported that only 72 percent of Kentucky’s graduates took the ACT in 2008, down 5 points from the 2007 participation rate. That, as you mention, could inflate our 2008 scores a bit from earlier years. Of course, this participation drop is not a desirable trend.
Here is another important issue. Kentucky’s first cohort of 100-percent-tested graduates is not included in the 2008 data. They won’t show up in the ACT graduate report until the 2009 results are released, probably in August. However, Colorado, Illinois and Michigan have all conducted 100 percent ACT testing long enough for that to impact their 2008 results. Those three states’ scores are all lower than their pre-100 percent testing scores were, of course.
Furthermore, the ACT tested graduates in the three 100 percent testing states comprise nearly one out of four of the total number of graduates ACT tested in 2008. Thus, these states drag down the national average scores from what they would otherwise be.
So, the national average scores in your data have been lowered by testing changes elsewhere.
I ran a quick estimate of the impact of the 100 percent testing in these three states, and that shows the 2008 ACT composite would likely be 21.4 if those states had maintained their traditional levels of ACT participation.
To add some better perspective to this, based on my estimated corrected national ACT Composite score, Kentucky’s ACT composite in 2008 would be 0.5 points behind the national average once appropriate corrections are made. In 1991, 1992 and 1993, before KERA had much impact on our schools, the gap was 0.6 points. So, while we have made progress – assuming the ACT scoring has been maintained relatively constant over time – so have all the other states.
When the 2009 graduate ACT score report is released, Kentucky’s graduate ACT scores should fall, probably around 1.5 points or so. That is when our first 100 percent tested cohort shows up in the ACT graduates report. It will be inappropriate to simply compare those scores against the national average, and the public needs to understand why. Thanks to the changing landscape of 100 percent ACT testing, which is a good change that should benefit Kentucky, these scores simply cannot be presented without including a lot more detail.
By the way, we do have a few comparison ACT scores from the 2008 100 percent testing of 11th graders in a few other states now. Here’s how that more apples-to-apples comparison of composite scores looks:
Kentucky – 18.3
Colorado – 20.2
Michigan – 18.8
(Note – While Illinois does 100 percent testing, the results are not publicly available in standard ACT score formats).
I am hopeful that, as more Kentucky schools pay attention to the EPAS system (EXPLORE, PLAN and ACT), our true ACT performance will gain on other states. Colorado and Illinois have been doing this since 2001, and both states have seen about a half point or more rises in their ACT Composites since going to 100 percent testing. In fact, in 2008 while Illinois tested virtually all of its graduates (reported as 98%), its composite score for that group was 20.7, only 0.2 points behind Kentucky’s 20.9 score which only examined 72 percent of our graduates.
Richard,
ReplyDeleteCould you share a link to a good source of ACT public-school only data?
Thanks.
Susan
Susan,
ReplyDeleteState level public school only ACT data is very hard to find.
To my knowledge (and yes, I have asked ACT directly) there isn’t a lot of data on public school only ACT performance. The ACT only directly publicly releases the All-Student data as I mentioned in my earlier comment. ACT did report a national average public school data set in the past, but they quit releasing that years ago.
At one time the SchoolMatters.com site had some ACT scoring data, but that seems to have been removed when I checked for it today.
A few states, Colorado and Michigan, do have some limited 11th grade only ACT test results in their Web sites. I’ll soon put up links to those in a FreedomKentucky.org Wiki item I am preparing, but that’s about it – except for here in Kentucky.
For years, the Kentucky Office of Education Accountability has obtained a special data run from the ACT, Incorporated that shows the ACT participation numbers and the English, Math, Reading, Science, and Composite scores for each individual school’s graduating class. I have been able to access that through the help of State Senator Jack Westwood and have been analyzing this interesting data for many years. Unfortunately, the OEA has never put these files on line, however.
Also, following lots of “encouragement” from me and many others, the Kentucky Department of Education started to include some ACT data in each school’s report card a few years ago. Of course, you have to download over 230 individual report cards to find that information for just one year, and results for some students in special schools are not captured in these reports. Furthermore, the number of students who take the ACT in each school isn’t provided, which makes a lot of intelligent data analysis impossible. But, the report cards are on line in the KDE Web site. You can find the menu page to select a district and school at: http://applications.kde.state.ky.us/schoolReportCardArchive/
If anyone is interested, I have Excel files covering Kentucky’s public school ACT performance from 1993 to 2007. I am still waiting for the 2008 file. I can be contacted at innes@bipps.org.