the achievement gap, testing, and accountability

In recent months, the Colorado legislature has repeatedly debated reforming Colorado's standardized testing system. One issue that has frequently come up is the importance of preserving equity in education through testing. For instance, in an editorial last week, the Denver Post wrote:

"In all the furor surrounding testing that has erupted at the statehouse, the equity issue sometimes gets lost. Yet, it's immensely important to be able to tell whether kids in poverty and minority children are making progress and that their instruction is effective."

Equity in education is a goal everyone seems to agree on. The disagreement is how to achieve this. I believe that Colorado's current standardized testing system doesn't achieve equity. So we can't talk about "preserving equity through testing" because that doesn't actually exist in the first place.

Here are the issues I see with (in)equity in Colorado's standardized testing system:

1. Computer-based testing, especially at lower grade levels, naturally benefits students students with more computer experience, which are usually students from higher income brackets.

In Colorado, computer-based testing is encouraged beginning in fourth grade. This is problematic because kids don't necessarily have the computer skills in elementary school to type answers. This form of testing also exacerbates inequality because students who live in poverty are much less likely to use computers at home than students from a middle-class or wealthy backgrounds, and thus wealthier students and wealthier schools will score better on the tests.

2. The current science and social studies testing unfairly favors large schools.

As we discovered last fall, the high school science and social studies testing targets all of science and social studies. This includes subjects like economics, which small schools do not offer. Naturally, students who haven't taken these course will score lower on the standardized tests.

The task force recommended eliminating science and social studies testing for high school seniors in January; thus far, the state legislature has not taken action.

3. Some schools can afford to purchase test-oriented textbooks, while others cannot.

Colorado students have not fared well on the new standardized tests thus far. Pearson, which makes the PARCC and CMAS tests, also makes textbooks which are aligned to the tests. (I rip on Pearson a lot, but the other standardized testing companies all do the same thing.) This matters because, as Temple University professor and data journalist Meredith Broussard wrote last July:

"[S]tandardized tests are not based on general knowledge. As I learned in the course of my investigation, they are based on specific knowledge contained in specific sets of books: the textbooks created by the test makers. ... Pearson came under fire last year for using a passage on a standardized test that was taken verbatim from a Pearson textbook."

Even overlooking the obvious issue of whether Pearson textbooks are actually best for students' education, it's easy to see how this system contributes to further educational inequity.

Textbooks aren't cheap: some schools can't afford them whatsoever. Some schools can afford enough for classroom copies but not for students to take home; some schools can afford to give every student a personal tablet and e-book copies of the textbooks.

Yet all those students take the same standardized test.

4. The timing of testing (currently March and either April or May for most Colorado students) doesn't account for the achievement gap. 

Proponents of Colorado's standardized testing system have repeatedly advocated standardized testing as a measure of student growth from year-to-year.  However, studies have repeatedly shown that the income-based achievement gap in the United States primarily occurs over the summer.

Take the following (very oversimplified) scenario:

Students A and B both score "100" on a standardized test in May.

Student A is of high socioeconomic status.  Over the summer, he/she participates in a few summer camps, visits the library every week, or takes part in other summer learning programs. Over the summer, Student A improves 20 standardized testing points; if an exam was given on the first day of school, Student A would have scored 120. 

Student B doesn't have access or exposure to the same educational resources as Student A. Over the summer, Student B loses 30 standardized testing points; if an exam was given on the first day of school, Student B would have scored 70. 

The following May, Student A and Student B once again partake in standardized testing. Student A scores a 130; Student B scores a 110.

According to Colorado's current standardized testing system, Student A progressed more than Student B. However, when accounting for the summer learning achievement gap, Student B actually made miraculous strides, whereas Student A hardly improved.

Of course, in Colorado we don't have the data to determine how students gained or lost educational experience over the summer.  Consequently, while the achievement gap undoubtably affects Colorado students and their test scores, the current testing system doesn't measure how.

So what does this all mean?

I think the idea that standardized testing contributes to inequity can be difficult for some people because we'd like to think that measuring student growth is simple and that the data collected actually means something. Unfortunately, this logic is highly problematic.

Under Colorado's current system, the data collected via standardized testing is flawed. So when this data is then used to inform everything from teacher salaries to classroom standards to school comparisons, it leads to bad policymaking.

Some of the debate over testing in Colorado has been linked to the Common Core standards Colorado adopted in 2010. To me, these issues are only loosely related.  Holding Colorado students to high standards isn't controversial; the only question is how best to do it.

I'd be remiss if I didn't at least try to provide a solution, so the following are a few suggestions for Colorado's standardized testing system:

-Change the timing of testing. Minimize the amount of class time spent on testing. One option: hold 1-2 hours of math and 1-2 hours of literacy testing in August when students return to school, and another 1-2 hours in each subject in April or May. This tracks student progress, accounts for the achievement gap, and is still way less time than Colorado students currently spend testing.

(My middle school, a charter school, did this and I always found it way more useful than the state-administered standardized tests.)

-Making testing feedback more readily available. Right now, schools, parents, and students don't get testing feedback for 4-6 months. That's too long.

-If social studies testing is really desired, use an "exit exam" format. If Colorado has set standards for what students are supposed to learn in high school government, why not test them on those standards once they have completed their high school government class?

-Make learning materials equally available to all Colorado schools. This requires no explanation.

-Make paper-and-pencil exams the norm, at least in elementary school. Initially, schools were required to use computers; the state eventually relaxed this requirement. Nonetheless, 85% of schools still used the computer option. I'm doubtful that comparing computer-based scores and paper-based scores is perfectly accurate.

I think there is a tendency to characterize the movement against Colorado's standardized testing as anti-accountability.  To me, that couldn't be further from the truth.  Accountability in education is incredibly important, but it goes both ways.  We can't go on holding our teachers and schools and students "accountable" while we take for granted that testing works. My hope is that those who actually have the power to influence Colorado's standardized testing system will reach this conclusion, sooner rather than later.

-Jessica Piper

Like usual, if you have questions, feel free to ask in the comments or hit me up on twitter (@jpiper303)

What Senate Bill-223 gets wrong

Senate Bill 15-223, currently in the Colorado legislature, would legitimize the ability of parents and students to opt out of standardized testing by not punishing schools for low participation rates. But, as Nic Garcia of Colorado Chalkbeat wrote, most activists aren't happy about it.

Here's my take: Legalizing opt out recognizes our protest. That could be considered a good thing. But the fundamental issue with Senate Bill 15-223 is that it views opt out as an end, whereas we've always seen it as a means.

When we students protested last fall, it wasn't just to avoid taking a test. It was to stand up to a system that we believe wastes state money, preserves educational inequities, and hurts Colorado students.

Likewise, parents and students who are refusing state PARCC testing this spring hope that their refusal—their voice—will lead to actually policy change.

The goal of the opt out movement is not to get every student to opt out, it's to change the way Colorado administers and utilizes standardized testing.

Truthfully, opt out in itself isn't effective as an end. The students who opt out are mostly upper-middle class kids who would (for the most part) score well on the tests anyway. The movement also isn't spread out equally—some schools strongly discourage students from opting out by requiring a parent to sign their child out of class or refusing to administer alternative activities. This only exacerbates the socioeconomic gaps.

My concern is that the Colorado legislature views opt out as a method to placate angry parents and students. This leaves a broken standardized testing system intact.

-Jessica Piper

Like always, if you have any questions, ask me on Twitter (@jpiper303) or in the comments.

hi Pearson, please let people talk about PARCC

As we've written, PARCC testing is fully underway in Colorado and across the nation.  There's been a lot of talk about the merits of PARCC on social media and in the blogosphere.  It turns out, Pearson, the corporation which produces PARCC, is trying to keep that conversation to a minimum.

@michellemalkin PEARSON gets creepier by the minute. Check this out from one of our supers, PARCC testing underway NJ pic.twitter.com/Dnff2QQoWB
— Pierre (@pierreroderique) March 13, 2015

This isn't the only case we've heard of.  Pearson appears to want to keep information about PARCC testing off social media.  I have three huge issues with this:

1. It's flat out creepy that Pearson monitors students' social media posts.

2. Students' freedom of speech shouldn't be limited outside of the classroom, especially by a private corporation. If students want to use social media to express concerns about PARCC, they should feel free to do so.  As we've experienced, there is rarely a voice for students in the sphere of education policy; let's not eliminate the one place students can express themselves.

As we saw last fall, student dissent has the potential to have impact on public policymaking. We worry that, if students are discouraged from posting about PARCC on social media, their concerns about PARCC won't be heard.

3. The suppression of content is also rather ironic. According to Pearson, the PARCC testing is different from previous standardized testing because it is skill, not knowledge, based, and therefore forces students to think critically.  If social media posts really "undermine the integrity" of the test, there is no way that PARCC actually measures students' critical thinking skills.

We posted this last November about one student's CMAS experience.  We are all high school seniors and haven't taken the PARCC tests, but if there are anything students reading this who'd like to share their story, send an email to cmasprotest@gmail.com. We'd love to publish your story.

the issues with S.B. 215

Senate Bill 215, expected to be heard in committee this Thursday, does very little to address the issue of standardized testing in Colorado.

What the bill does do:

• makes testing in 11th/12th grade optional (except for the ACT). The decision to administer the PARCC and CMAS tests would be left up to districts.
• renews the state task force on standardized testing

There are a few issues with this:

1. Another task force seems redundant  The state spent $75,000 on the first task force; it's reasonable to assume a similar amount would be required if the task force was renewed. Yet even the supporters of the bill provide no justification for why another task force is necessary, especially given:

2. Senate Bill 215 doesn't actually listen to the recommendations of the first task force. From the task force report:

[F]indings from research studies and public input made it clear that Colorado’s current system of State and local assessments has created far too many demands on time, logistics, and finances that are impacting the teaching and learning process in schools and undermining public support for the assessment system as a whole.

Senate Bill 215  doesn't address over-testing in grades 3-10 whatsoever. The Task Force had recommended making Math and English CMAS testing optional at all levels; Senate Bill 215 doesn't do that.

The bill also doesn't provide a means for parents and students to opt-out of standardized testing without hurting their schools and districts. (This was another task force recommendation.)

This brings us to the question: what is the point in making another task force if the state hasn't even bothered to listen to the first one?

3. Senate Bill 215 doesn't address several other student, parent, and teacher concerns.

• When we protested last fall, we were concerned about the amount of money the state spends on testing.  Student protests since then have focused on the issue of continued privatization of our education. Under S.B. 215, the state would still pay corporations for these tests. 

• The bill doesn't address any concerns about data privacy, which a been an important issue for many families.

• The bill doesn't do anything to address the issue that Colorado's "accountability" system punishes low-performing schools for poor exam scores.

Senate Bill 215 doesn't do anything bad.  The problem is, it doesn't do anything much at all. 

Since our protests last November, we've been repeatedly assured by politicians and other officials they've heard us, that they will listen to the task force, that they recognize standardized testing is an issue. 

Senate Bill 215 doesn't show that. To the students who have been fighting for real change, it shows us that Colorado politicians would rather avoid substantive reform and kick the can down the road while a broken standardized testing system remains in place.

The State Board of Education has come around on CMAS

The State Board of Education, in a 4-3 vote last night, voted to reject the recommended scoring system on the CMAS tests that Colorado high school seniors took last fall. Per The Denver Post:

The move means that families, schools and districts awaiting the results of tests taken last fall will not get them -- at least soon -- and gives the state no way to use the tests in its accountability system, state officials say.

The board members who voted in favor of the motion cited "concerns that too many students would fall short of expectations, that students were tested on subjects [that] weren't taught, and also questioned the makeup and formation of the committee that set the so-called cut scores."

These are, of course, very similar to the concerns we students expressed about the tests last fall. For instance, we pointed out that economics was tested, despite not being a required course for Colorado high school students. From the Denver Post:

[Board member Steve] Durham said it's fundamentally unfair to test students on subjects they have not had the opportunity to learn. Most high schools do not teach economics, he said, and the proposal was to base more than a fourth of a social [studies] score based on that subject. He said it would lead to an unfair characterization that we really have a bunch of social studies illiterates in the state.

Essentially this move means that, for now, the state, districts, teachers, parents, and students won't use these scores.  We hope this is a step toward eliminating the CMAS test entirely.

an equation for PARCC: acceptance does not equal approval

The PARCC tests kicked off in Colorado (and around the nation) this past week. While there have been notable instances of opt-out (82% of students at Fairview High School opted not to take the test), the majority of Colorado students will sit through the PARCC exams this spring.

Some people believe that means the tests are working. The Denver Post reported this past weekend on the testing roll-out, including this line:

State officials offered a pencil-and-paper option to districts on all math tests and third-grade English PARCC tests. Only about 15 percent of Colorado students will use that option, which Hawley said indicates broad support for online tests.

As I wrote last week, paper-and-pencil math PARCC tests are infeasible for some schools, particularly at the higher levels, because of the calculator requirements.  Consequently, the decision to use computers doesn't inherently demonstrate approval.

My other concern about this statement is that the state might try to use low opt-out rates as evidence that the testing is widely accepted. This would be inaccurate, but not unprecedented. State officials repeatedly used low opt-out rates on the CSAP and TCAP tests as "proof" that students and parents approved of the test.

Of course, some students take the PARCC tests even if they don't believe it's a good idea. Some schools coerce students into taking the tests. We've heard from students who were informed the testing would be tied to graduation, or it would hurt their college chances if they chose to opt out. Other students also take the tests not because they approve but out of loyalty to their schools.

Furthermore, statements about the benefits of computer use might want to wait until the testing has completed; thus far, the technology has been far from perfect. The majority of students in Colorado (roughly 85%) are taking the tests on the computer. While many schools haven't had technology issues, some have. Per the Denver Post:

But when it really counted, students at Sheridan High trying to log into the testing system got one of those frustrating hourglasses that just keep spinning and spinning, Clough said.

Within an hour, five kids were able to log into the system, he said. Within two hours, maybe 50 percent of students were able to start. Most were able to get in within three hours, but in other instances the test would not properly function once students were able to log in and begin taking it, Clough said.

Stories like this exacerbate the many concerns that testing is taking to much time. And given these issues, to say that there is "broad support" for online tests is simply far-fetched.

-Jessica Piper. If you have questions, leave a comment below or ask me on twitter: @jpiper303.

We'll have more on PARCC testing as the month continues.

What is PARCC testing? We tried to find out...

Testing for the Partnership for Assessment of Readiness for College and Careers (PARCC) begins this week in Colorado (and many other states). While students who are taking the PARCC exams will soon be familiar with the tests, we wanted to learn a little more about who and where these tests come from and what they actually contain.

PARCC is produced by the Pearson Corporation, a British company which sells education materials in the United States. For a basic summary of how we feel about the Pearson Corporation, check out the open letter we wrote about the CMAS tests last year. Pearson also produces Common Core textbooks, which are supposed to better prepare students for the PARCC exams. 

According to the PARCC website:

As such, PARCC engages with educators throughout the entire development and implementation process in a variety of ways to ensure the assessment system reflects the best practices and meets the needs of teachers, and to ensure educators have ongoing opportunities to give feedback on and understand the Common Core and PARCC assessments before they are fully implemented.

We tried to figure out how educators (and other people) could provide feedback on the PARCC tests. So far, we haven't come across anything. Perhaps that's because, as students, we aren't given the same level of access. However, if PARCC is really a tool for community engagement with education, we hope they'll make is easier to give feedback.

So how are the PARCC tests made? Well, according to this PDF from the PARCC website produced in December 2014:

PARCC state educators and experts are highly selective! Classroom teachers and other local educators are involved at every step, including test design. More than 30 educators and other experts review each item. The process of developing test questions is unprecedented for its level of rigor and inclusiveness.

Unfortunately the number of exclamation marks outweighs the amount of concrete information in that paragraph.  We weren't able to find anything about who makes PARCC tests, but we were able to find sample exams. Pearson's website contains sample PARCC exams in Language and Math for grades 3-11. If you're interested in taking the tests yourself, they are available here. If you're not interested, we took several of them and came to a few conclusions.

PARCC's language testing follows the same general format at all levels.  Students read a passage, or watch a video, and then answer questions pertaining to it. Sample questions include:

(The above sample comes from a 9th grade Language exam. Students had previously read a Robert Oppenheimer speech.)

Something else about the Language exams that was new and different: they included video.  (For the paper exams, students read a transcript of the video.) While we understand that the PARCC exams want to be new and different, the utility of including video is questionable, especially as it seems to pose more technology issues. Furthermore, students who take the computer test will have a slightly different experience than students who take the paper-and-pencil test, so comparing scores will be more difficult.

One of the first things we noticed about the PARCC math exams is they do not go higher than Algebra II.  Thus, some high schoolers would be testing-down.  Although this isn't the end of the world, if the PARCC tests are going to be used to gauge what students have learned during the previous school year, it won't actually be effective.

Another concern we with the math problems was the manner of the fill-in questions.  The PARCC math tests use a fill-in-the-grid system for some problems.  This system is the same system as the ACT and SAT, so high schoolers would be familiar with it, but it seems it could be confusing for third graders:

(Above is a screenshot of the 3rd grade math instructions.)

While plenty of the PARCC math problems were standard, PARCC also has an affinity for word problems.  This isn't inherently a criticism--word problems can be engaging and useful--but the phrasing of some of the word problems seems it could be a cause for confusion. For example, take this question, which was on the third-grade mathematics test:

One final issue with the math test: technology.  While the PARCC tests are supposed to be computer-based, a lot of schools don't have the technology to support this, so a paper-and-pencil option is available. However, a TI-84 is required for both the Geometry and Algebra II tests.  TI-84s run around $100, and most students don't need them until pre-calculus. So schools either need to find a way to get every student these calculators, or paper-and-pencil isn't an option.

Those are just a few of the issues we've run into. Based on that, we have three basic ideas about PARCC:

1) We'd like to see more transparency surrounding the test creation process. Who are experts making these exams? How can educators, parents, and student give feedback?

2) How do we know that PARCC testing actually provides meaningful data? If test questions are confusing, if the technology or format poses problems for students, and if the tests aren't aligned with students' current level, what makes policymakers and Pearson's supposed "experts" believe the tests will give meaningful feedback.

3) To what extent did corporate lobbies play a role in the adoption of PARCC? Pearson makes money selling Common Core textbooks, supposedly to improve students' PARCC performance. Pearson also notably made the PARCC exam more difficult than the previous CSAP and TCAP standards Colorado used. Is this really about our education?

Editor's note: We'll have more on PARCC later this month as the testing continues. If you have questions about this article (or anything else on our blog), feel free to say so in the comments, send an email to cmasprotest@gmail.com, or ask Jessica on twitter: @jpiper303. If you'd like to reuse/repost any contents of the blog, feel free, but please give us credit. If you're a student and you'd like to write stuff, contact us in any of the above ways.

A few shout-outs to student-led anti-PARCC movements

PARCC testing begins in Colorado as well as 11 other states this week.  With growing criticism surrounding the tests, we want to acknowledge several of the student-led anti-PARCC movements going on at this time:

-the Colorado-based anti-test movement is planning a protest in Denver on March 7
-students are reportedly walking out in Hobbs, New Mexico as well as Albuquerque
-students in Bloomington and Normal Illinois have organized an anti-PARCC union