On Standardized Testing

My friend and colleague Jen Whetham sent me a note recently, along with a link to a segment from John Oliver’s HBO series, Last Week Tonight, with these comments:

“So my feelings about this piece are complex.  Clearly, John Oliver is brilliant—particularly the repetition of “here comes the monkey.”  And it’s poignant . . . the section about the testing booklets coming with instructions about what to do if a student vomits on them makes my heart hurt, as does the young woman who cries during the testimony of her experience.  And, of course, Nathan Hale’s [High school, in Seattle] juniors are mentioned, as is the search for graders on Craig’s List.

I feel like a lot of folks will watch this (and probably send it to me) and consider it the last word on SmarterBalanced and as further evidence to conflate their opposition to standardized testing in general (as well as the concerns about the overuse of testing . . . and testing driving instruction, as opposed to a more reciprocal/aligned relationship)—as opposed to using it as a springboard for a deeper, richer conversation…”

My first reaction is that it’s important to keep in mind that our longstanding American obsession with standardized testing may have accelerated under President Bush and the No Child Left Behind Act of 2001 but has been around since the early 1900’s. The obsession and resulting paranoia on the part of educators is an easy target for a comic, and of course he’s going mostly for laughs–that’s his job, after all. It’s all just critique by anecdote, and there are plenty of examples of silly behavior on the part of school administrators at all levels, or at least behavior that’s easy enough to portray as silly when it’s taken completely out of context. ​That said, there’s not really all that much I’d argue with in the substance underlying what Oliver presents. As a society we ARE too obsessed with tests, we put too much stock in the results, we DO often use the results in inappropriate ways, and we seem to think that we can measure our way to improvement, as if somehow an inevitably imprecise understanding of something as complex as student learning will magically lead to improved results. 

A few additional observations worth mentioning:

1) Besides the brief jokey references to the “opt out” business, the piece really had nothing to say about the new Common Core assessments, Smarter Balanced or PARCC, and nothing at all to do with the substance or format of either assessment; 

2) Oliver frames the argument at the beginning and the end with a reference to the poor performance–and persistent achievement gaps–by US students on TIMSS and PISA, which are (yes, you guessed it!) standardized tests, so apparently “international” standardized tests are inherently good and worth paying attention to, others inherently bad and terribly harmful to kids and other living things;

3) Not that I would necessarily expect a comic to do so, but as with most critiques like this, there were no alternatives proposed or answers given–if these particular assessments aren’t useful as a broad educational measuring stick, what should policymakers and the public use instead? 

4)  The deeper issue is that collectively as a society we seem to not trust teachers or teacher grades, at least in general, so these tests become an “objective” way of understanding what learning has occurred and how the system is doing. (In a recent hearing about the bill that references our transition courses as a senior-year alternative for students who don’t meet standard on the 11th grade Smarter Balanced, the League of Education Voters testified against the bill because the courses couldn’t possibly be as “rigorous” as the “objective” test results. (The fact that “objectivity” doesn’t really exist in any of these tests is a nuance completely lost on most people, sadly…)

5) The Leadership Conference on Civil and Human Rights recently released a statement opposing the anti-testing efforts springing up around the country, reflecting that in communities of color there’s a general understanding that the intent, at least, of these efforts is to address equity issues. I was particular struck by this point:

“When parents ‘opt out’ of tests—even when out of protest for legitimate concerns—they’re not only making a choice for their own child, they’re inadvertently making a choice to undermine efforts to improve schools for every child.”

 I’ve wondered if there’s any evidence of a connection between the parents having their students refuse to take these tests and those parents refusing to get their children vaccinated, and this quote suggests the underlying parallel: vaccination is as much about “herd immunity” as it is about individual vaccination, and opting out of the tests has a corrosive impact on the school community as a whole–it’s not just an individual choice, principled or not​. For many parents in affluent white communities, of course, education is a “zero sum game,” so any efforts to address equity means, or at least could mean, less resources for their own high-achieving kids). I’ve also gotten several inquiries from parents and schools in response to our efforts here to use the Smarter Balanced assessment suggesting that there’s a general perception that these assessments are “harder” than previous assessments and they worry that if their kid takes it it will affect college admissions somehow.  

It would be great if we could find a civil way to have the broader conversation about the common good that we should be having here instead of railing about the tests as if the issues around standardized testing were new, and portraying them to be much more consequential than the actual evidence suggests (see this posting from The Hechinger Report, for instance). But if we’re going to do that, let’s at least make sure we ground the conversation in evidence, not cherry-picked anecdotes, and let’s not leave it to satirists, even brilliant ones, to lead the debate…

 

Making the Smarter Balanced Placement Policy Work for Students and the System

Now that the Washington higher education system (community and technical colleges as well as the public baccalaureate institutions) has adopted policy agreements on using the Smarter Balanced assessment scores for 11th graders in course placement decisions, as we enter 2015 and move closer to the first full administration of the Smarter Balanced assessment this spring we are turning our attention to the critical logistical issues related to making the policy work.

For the community and technical college system, we assembled a work group of representatives from key stakeholder groups who will be involved in implementing the policy at the campus level, asking them to raise issues and questions from their various perspectives so that we could begin clarifying and making operational the particulars of the policy. (You can see the slides and notes from this meeting and provide input here.) We will be asking the work group representatives to consult with their various constituent groups beginning in their winter quarter meetings to continue the discussion, raising additional questions, developing answers where appropriate, and proposing solutions to process or logistical concerns where possible.

At the same time we will be connecting to our K-12 colleagues here in Washington as they finalize the work on Smarter Balanced score reporting format and process that will be critical to higher education’s ability to access and use the scores in the placement process. For the bulk of the students involved–juniors taking the 11th grade assessment and using it for placement when they enter college a year later–we have some time to figure out how things will work. The pressing immediate issue is for students who want to use the scores for placement into dual-credit courses (Running Start, College in the High School, …) during their senior year of high school (for this first cohort, that will be fall 2015). That group should be considerably smaller than the group seeking placement after graduation; given the timing and the scale, we may develop an interim “fix” for fall 2015 while we work on a more permanent solution for fall 2016. As the work moves forward we’ll provide updates here and/or on the work group page.

Responding to Ravitch

I got the following message recently from a Washington higher education faculty member in response to our reminder to folks about the opportunity to participate in the Smarter Balance online panel process in the fall:

Before my colleagues and I sign up for work on what’s clearly an important movement in education, I’m wondering if you can read and comment on the article I’m linking here, which indicates the lack of democratic process involved in writing the common cores standards, which seem to favor the corporations that sell testing and other testing-related materials to our schools. http://www.huffingtonpost.com/diane-ravitch/time-for-congress-to-inve_b_5473992.html .

Thanks so much for your help in this.  I had no idea that these standards had been formed in this way, and your response will certainly inform my decision on whether or not to become involved in this.

My response is below:

Happy to provide my perspective, though I should preface it by saying that of course participating in the Smarter Balanced online panel to provide input on the college readiness achievement level for the 11th grade assessment is completely optional; the note below is just an invitation to anyone who might be interested in getting involved in the work!

 

That said, and without wading too far into the current completely politicized and ideological “education war” happening across the country, I’ll say that I, too, had no idea the standards had been formed that way, mainly because it’s not true. You can see more about the development process for the Common Core State Standards (CCSS) here, but groups of teachers and higher education faculty ere deeply involved in creating and refining the standards, along with (and led by) resource experts and consultants. [One example: here’s a report from a national Validation Committee for the standards, including a list of the people involved.] Given the number of teachers and faculty in the country, I’m sure it was a very small percentage, but that doesn’t mean the process wasn’t democratic; it’s not practical to include huge numbers of people in design processes like that. There was, however, a very open “public comment” process to allow for broader input; whether people who are criticizing the process now were paying attention or offering input then is another story.

 

I would argue that the standards were no more or less democratically-defined than any of the existing state standards around the country, but that in any case, they should be examined and critiqued less on myths and half-truths about the process and more on the substance, e.g.,

  • do the standards reflect what really matters with respect to K-12 student learning in mathematics and English language arts?
  • do the standards capture what college faculty believe students need to know and be able to do to be “college-ready” for entry college-level courses in those disciplines?

 

Frankly, that would require far more time, careful thought, and expertise than most of the national pundits and blogosphere have or care to invest, so they argue about democratic processes, the role of foundations in education reform, and federal intrusion, none of which has much traction with the CCSS, in my opinion. There are legitimate concerns about the details of the standards in some areas, but on the whole the Washington faculty—both 2-year and 4-year—I’ve asked to review them as part of our Core to College work have been impressed with the nuance and thoroughness of what they’ve seen. [Here are a couple of national studies along the same lines, and with similar results, conducted by David Conley’s Educational Policy Improvement Center at the University of Oregon.] Based on a number of studies and polls, K-12 teachers are also generally supportive of the standards; while it varies somewhat by state, the CCSS are seen as a significant improvement over previous state K-12 standards (even in Washington, where the existing standards were considered pretty strong)—one reason why organizations like the NEA support the CCSS.

 

Some, including Ravitch, have argued that one of the big problems K-12 seems to have is this magical belief that better (“world-class”) standards will make a difference for students and student learning. I don’t know anyone involved in this work naïve enough to actually believe that improving the standards alone is sufficient; certainly the work in Washington is focused on shifting the whole system to support helping teachers and students address the standards. Standards are just standards, not curricula, and certainly not instructional support for teachers; to me the real challenge will be how well state K-12 systems engage in building capacity to tackle the learning outcomes and implicit pedagogical shifts defined in the CCSS. That’s a legitimate and very real concern, but not a critique of the Common Core.

 

Looking more specifically at the Diane Ravitch post you sent, it’s also worth considering a couple of different takes on her current arguments from John Merrow and David Brooks. Personally there are things I agree and disagree with in all 3 perspectives; the bottom line for me is that I see the issues as much much nuanced and “both/and” rather than as “right/wrong” or  “either/or” (my advantage is that unlike pundits, I’m not trying to attract attention, mouse clicks, or advertisers!) Basically her argument boils down to:

 

1)      poverty and profound inequity by race and class are serious problems in this country, often exacerbated rather than lessened, by our educational system;

2)      as a nation we are overly obsessed with standardized testing;

3)      foundations like Gates are trying to take over the education system.

 

I find the third point puzzling and preposterous, to be honest; funders like the Gates Foundation are certainly making investments in public education, but the amount provided is only a tiny percentage of the roughly 500 billion spent nationally on K-12 education. It’s true that money usually comes with expectations, but that’s far more true—with far more intrusion—for  state legislatures than for private foundations. The reality is that in many places foundation money is supporting innovations in ways impossible otherwise as state funding lessens; if that’s a disturbing trend, it needs to be addressed at a societal level in terms of tax structures and generating greater public willingness to support the common good, including public schools. But it’s not a legitimate critique of the Common Core.

 

On the first 2 points I happen to agree but like Brooks, I come to differenty conclusions than Ravitch does. First, I see the potential the Common Core has for helping to reduce inequity if taken seriously and implemented thoughtfully. At the same time, I believe we should do far more about poverty and inequity than we’re currently doing; to me it’s not an either/or choice. Second, also like Brooks, I see the negative effects of K-12 assessments as largely a function of ignorance, fear, and politics, not inherently the assessments themselves (though clearly there are plenty of bad tests out there—including, if one believes the recent national research in the area, Compass and Accuplacer). Given that it’s extremely unlikely that states, the federal government, or political leaders would ever eliminate testing altogether (and that they can play a useful role in the overall education system), I also believe we should push for high-quality assessments that move toward assessing the things that matter in learning rather than just the things that are easily assessed. I think the Smarter Balanced assessment system, while not perfect, is a step in that direction, and that’s why I support it.

 

The Smarter Balanced system, including summative and interim assessments along with an extensive Digital Library of classroom resources supporting teachers—will serve as a replacement for the current tests used for federal accountability purposes (grades 3-8 and high school). Whether we choose to support it or get involved in the work or not, the new assessment system is being implemented in the Washington K-12 public schools beginning in 2014-15, including all high school juniors, and to me that’s the other reason for studying it closely to see how the information we could get from the 11th grade assessment could be useful to us in the placement process for students coming to our system directly from high school.

Recruiting for Smarter Balanced On-line Process for Setting Achievement Levels

In addition to the in-person panel process I mentioned in my previous post, Smarter Balanced will be soliciting input on the achievement level performance expectations through an on-line “crowdsourcing” process, one open to as many as 250,000 people. This Online Panel, which is open to the public and allows for broad participation, creates a critical opportunity to collect input from a diverse group of participants and establish consistent measures of progress for the Smarter Balanced interim and summative assessments.  (For more details about the achievement level descriptors, see the Smarter Balanced web site.) The Consortium is recruiting participants only through their K-12 and higher education leads now but will open the process up to a broad range of organizations and associations beginning this summer. Our goal is to get strong involvement from Washington higher education faculty in this process as it will provide critical input to setting the college-readiness performance expectation for the Smarter Balanced assessments.

By participating in the online session, panelists will recommend an achievement level score that demonstrates how much students should know or be able to do in order to be proficient at the grade-level standards and to be on track for eventual college and career readiness. It is important to have as many participants as possible, from as many backgrounds as possible, provide these recommendations.

The process of setting Achievement Levels will rely on the professional judgment and experience of educators and administrators in order to develop performance standards that are rigorous, fair, and accurate. The recommendations will be based on Achievement Level Descriptors written and approved by Smarter Balanced Governing States last year, the claims and targets defined for each content area and grade level, as well as educators’ expertise in their content area and experience with students. Acceptance of recommended scores is subject to existing approval processes within individual states.

The Online Panel to recommend Achievement Levels will occur during a specific 2-day window (within the October 6–17, 2014, time period) you will specify when you register. Unlike the 3-day process for the in-person panels, this process will focus only on the Level 3 performance score, and will take panelists no more than 3 hours to complete the orientation process, review test questions, and recommend a score for Achievement Level 3.

Interested participants can register here.

On the registration site, you will be asked to choose the grade and content area (English language arts/literacy or mathematics) and grade in which you want to participate. You will provide an email address, role, and demographic information. You will then be asked to verify your email and select a 2-day window for participation.

Registrations must be submitted online by September 19, 2014.

For more information, see this FAQ document:

Online Panel FAQs

If you have any questions feel free to email (bmoore@sbctc.edu) or call (360-704-4346) Bill Moore.

Recruiting for Smarter Balanced In-Person Achievement Level-Setting Process

We are looking to recruit participants for the Smarter Balanced Assessment Consortium In-Person Panel charged with recommending Achievement Levels for Grade 11, setting common expectations for student readiness/proficiency on the assessment. Participants in the Panel will make recommendations about the minimum test scores they believe to be necessary for a student to be considered performing at each of the defined Achievement Levels on the Smarter Balanced assessments. (For more details about the achievement levels, see the Smarter Balanced web site.)

The process of setting Achievement Levels will rely on the expertise and professional judgment of participants in order to develop performance standards that are rigorous, fair and accurate. The recommendations will be based on Achievement Level Descriptors written and approved by Smarter Balanced Governing States last year (with extensive input from higher education faculty), the claims and targets defined for each content area and grade level, as well as educators’ expertise in their content area and experience with students in their classrooms.

Participants in this panel will not set the final operational scores. A “Vertical Articulation Committee” will make recommendations to the chief school officers in Smarter Balanced Governing States using a variety of information sources. These sources will include the recommendations of the In-Person Panel, recommendations from an Online Panel which will engage in a similar – but less comprehensive – process as the In-Person Panel, and external assessment information. Acceptance of recommended scores is subject to existing approval processes within individual states.

The In-Person Panel will take place October 13–19, 2014, at a location to be determined, with the first three days of this panel (October 13–15) will be devoted to establishing Achievement Level recommendations for Grade 11. The Grade 11 recommendations will then be considered as teams develop Achievement Level recommendations for earlier grades.

Washington has been asked to nominate 10-12 higher education participants (5-6 each for math and English), of which 6 (3 per discipline) will be selected for the panel.  Please let me know by Friday, May 16 if you are interested in being considered as one of our nominees; if you are included on the nominee list we submit, I will then send you a link to an online registration form to complete by Friday, May 23.  Smarter Balanced will then select higher education representatives for the final panel from among the nominations submitted. The distribution of participants will represent a balance of states, content areas, institution types, and other relevant factors. You will be notified by Smarter Balanced about panel selections in June.

Smarter Balanced will pay for travel, lodging, meals, and other allowable expenses for all participants. An honorarium may be provided as applicable.

If you have any questions feel free to email me or call me (360-704-4346 office, 360-528-1809 cell); take care!

Bill Moore, State Board for Community & Technical Colleges

Director, Core to College Alignment

Smarter Balanced Agreement Background & Overview Session

I just led a webinar session providing some background information for the Common State Standards and the Smarter Balanced assessment system, including some details about the performance expectation-setting process to take place next fall, followed by a description of the proposed higher education agreement for using the 11th grade Smarter Balanced assessment in the placement process and a rationale for that proposal. I’ve included the slide deck below; to view the recording, click the link below:

View the Blackboard Collaborate recording

May 6 Webinar SB_CCSS

Smarter Balanced Practice Tests

Just a quick reminder that anyone interested in getting a very clear sense of the Smarter Balanced grade 11 assessment, both in terms of item focus and format as well as the assessment interface students will be using,  should really spend some time with the practice test available through the Smarter Balanced web site. Just go to this link, log on using the default “guest” settings and selecting Grade 11 (math or English), then follow the instructions from there!

Also, if you want to see specific details about how many and what kind of items will be on the assessments (grades 3 through 8 and 11), here are the test blueprints for both math and English:

Mathematics Blueprint Summative_2014-04-23_Draft

ELA-Literacy Blueprint Summative_2014-04-23_Draft

Common Core Field Tests in Progress

Both Smarter Balanced and PARCC, the two consortia developing Common Core-aligned assessment systems to be available to state K-12 systems as an alternative for their federal accountability testing requirements (grades 3 through 8 and high school), are doing massive field testing of their assessments this spring, with a goal of identifying and addressing any glitches or problems in the on-the-ground testing processes across their respective states. So far–and Smarter Balanced is around 2.5 million tests administered and counting–things have been going pretty well; for a brief overview see this recent Education Week article.

Common Core Instructional Resources

Recent news items from Education Week (courtesy of Education First Core to College News Blast e-newsletters) about progress on developing, vetting, and refining useful instructional resources on the Common Core State Standards:

  • The National Education Association and the for-profit firm BetterLesson have unveiled a jointly designed, $7 million free platform with more than 3,000 lessons aligned to the Common Core State Standards-a move that comes concurrent with NEA’s strongest endorsement yet for the standards. The new platform breaks each lesson into its component pieces, with narratives and artifacts from the master teachers explaining how they introduce a topic, arrange practice activities, and troubleshoot. The lessons they produce for the site will be free for all K-12 teachers in the United States, and are searchable by topic or by the standard they target.
  • Using $1.6 million in grants from the Helmsley Charitable Trust, the PARCC and Smarter Balanced assessment consortia will work with the American Federation of Teachers and the National Education Association to give teachers a central role in designing instructional resources for the common standards and tests. The two new grants-$830,000 to PARCC and $785,000 to Smarter Balanced-will bring teachers together, through their two national unions, to work face-to-face with consortium designers to create instructional and diagnostic resources for the Common Core, such as sample lessons, that will be housed in each group’s online library. The grants will also go toward building networks of teachers who are experts in the two assessment systems and can serve as trainers for their colleagues in the states.
  • Several new tools have been designed to help teachers and leaders gauge alignment to the Common Core State Standards. EQuIP (Educators Evaluating Quality Instructional Products),originated with the work of three states-Massachusetts, New York and Rhode Island-that designed a set of criteria to use in judging lessons and units for alignment with the standards. Other tools or processes designed to evaluate instructional materials for Common Core alignment include free portals by the two teachers’ unions where teachers can post and comment on lessons, and a set of free online tools by Student Achievement Partners that can be used to judge the fidelity of instructional materials to the standards.For-profit groups-like the Austin, Texas-based Learning List, which uses panels of judges to size up instructional materials-are also wading into the alignment-evaluation business, and the Business Roundtable is talking with partners about creating a group to do “Consumer Reports-type reviews” of Common Core materials. Finally, a group of experts led by Maria M. Klawe, the president of Harvey Mudd College, is creating a nonprofit to review the most widely used Common Core math materials.

College Readiness/Transition Course Update

A small design team of math and English faculty and high school teachers met in late January 2014 to generate some preliminary ideas about the 12th grade college readiness courses. On April 15th we are hosting a meeting at South Seattle’s Georgetown campus for the broader group of math and English faculty and high school teachers who met there last fall to begin this work. The goal for the April 15th gathering will be for the larger group to review, refine and adopt proposals for these courses so that the more intensive work of adapting and developing instructional resource materials for these courses can happen this summer.

At the same time we are recruiting school districts–preferably districts with staff involved in the development of the courses–interested in working with a college partner and pilot-testing these courses during the 2014-15 academic year. This summer the teachers for these pilots would be involved in the development of the courses and receive professional development, both in the summer and during the academic year.