The New York State Education Department (NYSED) had delayed release of test scores and analysis from the 2008-2009 school report cards, due in February, but that did not inhibit the discussion of district assessments at the Mamaroneck school board study session on March 9. (Keep checking the district website for links to the NYSED report cards.)
Annie Ward, assistant superintendent of curriculum and instruction, and Mike Kollmer, director of technology, used data from other instruments to show how the district is increasingly using test scores to both gauge and guide the effectiveness of instruction.
District Students Outperform County and State
Although the district is precluded from discussing the specifics of the NYSED reports until they are publicly released later this month, Mr. Kollmer said the headlines are good. Mamaroneck students turned in a “strong performance” on state standardized tests for English Language Arts (ELA) and Mathematics relative to students across the county and the state.
In addition, against the backdrop of the district’s continuing focus on progress of minority students, Ms. Ward reported subgroups of minorities in Mamaroneck are outpacing subgroups at both the county and state level. Further, the gap between minority and white student scores, while still apparent, has narrowed significantly in the past four years.
In reviewing the limited data available, “it is important to contextualize,” said Ms. Ward. Mamaroneck ELA passing rates (above) seem to peak in 7th grade and dip at 8th (the red bar). However, Mamaroneck students outperformed county (green bar) and state averages (yellow bar) by larger margins in the 8th grade than in other grades.
Ms. Ward and Mr. Kollmer cautioned that each year, a different group of students receives a different test that is scored on a different scale, so comparisons are difficult. Tests may be harder or easier than the previous year. Checking local scores against county and state performance can give a more accurate picture.
Beyond Proficiency: Outperforming High Norms on a Writing Test
In any event, Ms Ward explained, the district typically outperforms the county and state, so it is important that “we not let that be our ceiling.”
That is part of why the district chooses to administer the Educational Records Bureau Writing Assessment Program (ERB WrAP), which allows comparisons with students from high-performing suburban and independent schools. Here, the bar is set higher than with the mandatory state tests.
Ms. Ward described the ERB as “the Zagat guide to writing,” because the test “pinpoints the strengths and relative deficiencies” of students overall and on each of five different subtests that measure organization, support, sentence structure, word choice and mechanics. And unlike the ELA, the ERB striates scores. A student who gets a 3 on the ELA is proficient according to New York standards, but the ERB shows specifically where he or she needs work.
Ms. Ward explained that district students performed well on the more rigorous ERB. Of particular interest is a comparison of the scores of different grades. In fifth grade, district students “are right on par with the suburban norming group,” she said, which is good, since this group sets a high bar. But by 7th and 8th grade, “we move away from the pack,” Ms. Ward noted, with students who are in the middle of the district pack exceeding the performance of the suburban group. In 8th grade, a student who places in the 50th percentile in Mamaroneck does better than 77% of the suburban students.
The District Routinely Distributes Data Reports to Teachers and Administrators
The district has been working to develop “a local district data warehouse,” explained Mr. Kollmer, a task made easier with the implementation of the Eschooldata system several years ago. Guidance counselors and administrators can now easily use the computerized system to view all past data about a particular student, including grades, standardized tests and other assessments.
The district also looks carefully at the collective data for each standardized test, examining it on a question-by-question level.
After the “item level” review, teachers are given easy-to-read reports with a limited amount of highly relevant data on a particular question or area of focus. “It’s sort of like draping the patient for surgery,” explained Ms. Ward. Otherwise, it is easy to get lost.
In the past, said Ms. Ward, there was more of a “smile and file” approach to hard copy test results.
Now, principals, department chairs and instructional coaches work with teachers at department and team meetings to review reports and use them to improve instruction. The reports help assess whether curriculum is effective and also highlight effective teaching strategies. A teacher whose students perform particularly well in a given area might be asked to share her teaching tips with other teachers, for example.
This type of analysis is not as easy at the high school, because the Regents do not provide as much detail as the standardized assessments done in grades 3-8.
“We’ve come a long way in increasing the district appetite for data,” Ms. Ward noted. Although teachers were initially uncomfortable in looking at data from their individual classes, they’re getting used to it, and even embracing it. Fifth grade teachers at Mamaroneck Avenue School, for example, asked to compare results of their students’ tests in 5th grade to those the following year, in order to determine the “value added” by their teaching.
Data Allow District to Study The Impact of Instructional Interventions
Looking at data allows administrators to target students for extra help. Thirty-five Mamaroneck students did not pass the 2009 ELA as 8th graders. This year, all are receiving academic intervention services (AIS) as 9th graders at the high school with progress monitoring. Twenty-eight of these students (80%) passed English second quarter, sixteen of them (46%) with grades of 75 or higher.
Pulling together all available data will allow high school administrators to consider whether particular supports or teachers are making a difference with these at-risk students.
In the coming year, the district will work to track the impact of instructional interventions and develop an AIS module in Eschooldata.
Data Allow District to Study Student Acceleration
Examining acceleration trajectories has been an area of focus for the math department chairs at the the middle and high schools.
227 students in the current senior class were accelerated in math as freshmen, which in theory gave them time to later take calculus. But what happened next?
District research shows 77 % of that group has continued in accelerated math, with 174 seniors (approximately half the class) currently taking some form of calculus. This suggests that placement decisions were largely appropriate and that “keeping the door open” to accelerated classes worked for the majority of students.
However there are additional concern about placement in honors classes. Passing a Regents exam, a relatively low bar, is not necessarily a good predictor of future success in an honors math class. For example, students who took the new Regents Algebra exam in June received a passing score (65 or above) by answering only slightly more than a third of the questions correctly (34.5%).
Eighty-eight 9th graders were placed in Honors Geometry this year. At the end of the second quarter, 67% achieved grades above 85; 33% did not. The district will be using this information in conjunction with Regents scores to see if the tests can help predict success at the honors level.
District Embraces New Assessments
As reported in the fall in the context of the school goals, the district is adding two new assessments this year at the elementary level. Phonemic awareness, or the ability of a student to recognize the distinct sounds in spoken language, is being assessed in kindergarten. The Words Their Way spelling inventory is being administered in grades 1 and 2.
In addition, district goals call for more consistent use of running records, which provide a written tally of a child’s oral reading performance.
Administrators and instructional coaches have worked in the past year to develop these elementary assessments and train teachers to use them to tailor instruction to the needs of their students. Collecting data about how the changes impact student success will help to determine what types of gains may reasonably be expected.
Using Data to Set Appropriate Targets
“Measurable goals are the next frontier,” said Ms. Ward.
Rather than setting arbitrary targets that may not make sense in a classroom setting, however, she will be working with administrators, department chairs, instructional coaches and teachers to examine data from “landmark classrooms” in order to try to develop “ambitious but attainable targets for the following year.”
In addition, Ms. Ward expects to see more data used to assess the goals that teachers set each year in connection with the professional (APPR) review process.
Even collecting data can have a positive impact, Ms. Ward said. “Once you start measuring something, you are thinking about it.”