Teachers Prepare To Evaluate Each Other

Pat Brittingham is preparing to take on a big task: Grading six of her peers on job evaluations with newly raised stakes.

Brittingham, who works at King/Robinson Magnet School, is one of 18 teachers who last week completed an 18-hour training to prepare them to observe and score their peers as part of New Haven’s job evaluation system.

The training is part of a larger effort underway in New Haven to ​“calibrate” its teacher evaluation system, so that teachers receive consistent quality of feedback, and consistent scores, no matter who is grading them.

It’s also part of an effort to include more teachers in the evaluation process.

Teachers are now graded on a five-point scale based on two factors: goals teachers set for how their kids will improve on tests; and their ​“instructional practice and professional values,” as measured by classroom observations. The evaluation system, which launched in the 2010-11 school year, was one of the first in the nation to tie student performance to job evaluations. It is being nationally watched, mostly for the way in which the teachers union has collaborated with management.

Four years after the evaluation system rolled out, the level of consistency is ​“OK at best,” said teachers union President Dave Cicarella. Some evaluators ​“have a great handle on it,” but others don’t, he said.

“Most evaluators do a great job — not all do, and that’s the issue,” agreed schools Superintendent Garth Harries.

Now the district is addressing that problem by hiring a local firm called Revision Learning to train and re-train everyone who evaluates teachers. The training is one of the first programs paid for by a $53 million Teacher Incentive Fund (TIF) grant the city received last year from the Obama Administration to improve the way New Haven rewards, recognizes and develops educators.

The training comes as teachers have accepted a new labor contract that begins to attach consequences to the evaluation system: Starting next fall, New Haven will withhold raises from all teachers who are rated ​“needs improvement” or ​“developing,” the lowest two scores, until they attend up to 10 hours of extra professional development.

“Huge Responsibility”

Teachers who are evaluating their peers gathered last week in the library of Truman School for a third, six-hour training session with Patrick Flynn. Flynn has been involved with New Haven’s new teacher evaluation system since its inception: first as an employee of Area Cooperative Educational Services (ACES), and now as CEO and founder of Revision Learning. His firm has a $200,000 contract with New Haven public schools to run these trainings, support several individual schools, and send ​“validators” to observe and evaluate teachers who have received high or low scores.

Eighteen teachers and one new assistant principal spent their entire workday Friday honing their evaluation skills with Flynn. Most had never evaluated teachers before. Some, like Brittingham (at right in photo, with Michele Ricci), had.

Brittingham, a magnet resource coordinator at King/Robinson, became an evaluator three years ago at her principal’s request. She said she received no extra training before she began grading her peers: ​“only the experience of being observed.” She had relevant experience — she had already been observing and giving feedback to teachers who were implementing the school’s International Baccalaureate theme—but she had never conducted formal evaluations.

“It’s a huge responsibility,” she said. ​“We’re colleagues.”

Part of the reason teachers got so little training was due to a rushed timeline. Teachers approved a landmark contract in October 2009 making way for the new evaluations. Then a team of six teachers and six administrators hashed out the details. The new evaluations launched in the fall of 2010, with consequences: Teachers who were flagged as ​“needs improvement” in November could lose their jobs at the end of the year if they did not improve.

That first summer, only principals received training in how to perform the evaluations, according to Cicarella. That’s because they have 12-month jobs. Assistant principals, who work 10 months per year, reported for work just one week before teachers. There wasn’t much time for training.

Brittingham said in her second year, she received more training: four two-hour sessions.

This year, as she prepares to grade six of her peers, she’s getting the most extensive training yet. She and fellow teachers are taking an 18-hour course calls a ​“calibration” process. They spent the first day learning the rubric by which New Haven teachers are graded. In the final two sessions, they practiced observing and grading sample lessons on video.

On Friday, eyes were on a woman named Tess Kaji, who taught a 10th grade math lesson videotaped and broadcast by the Baker Evaluation Research Consulting Group. 

Kaji had never taught math before, according to BERC. She led kids through a lesson on how to measure the height of an object using the tangent function from trigonometry. Kaji took her kids through SOH-CAH-TOA, the catchy memory technique for remembering the definitions of sine, cosine and tangent. Then she prepared to lead them outside to put their math skills into action.

Teachers watched her on a projector screen, rubrics on hand.

Then they discussed how the teacher would fare on New Haven’s evaluation rubric. 

Did she ​“offer students multiple methods to approach material and to demonstrate learning?”

Yes, decided Billy Gibson, an English-as-a-second-language teacher at Columbus Family Academy (pictured at the top of this story).

Indeed, Gibson’s group gave Kaji high marks on the test, including on whether she ​“engages and includes all students in classroom activities.”

“I didn’t see anyone off task,” said Mia Edmonds-Duff, a math coach at Columbus school.

In can be easy to get a feeling for how a teacher performed, but it’s a challenge to provide evidence to support those observations, said Flynn.

Flynn said his firm is also working with a group of assistant principals and principals who performed evaluations last year to help them improve their evaluative skills. Revision Learning reviewed the written feedback they gave teachers last year and talked about how to make it more useful.

Flynn (pictured) said his firm also seeks to ​“calibrate” those evaluators’ scores to the new cohort he worked with Friday. He has all of them watch a video and grade the teacher. Then he compares the groups of New Haven educators to each other, as well as to outside ​“validators” who are using the district’s rubric, too.

Revision Learning took over as the official ​“validators” of New Haven teachers’ scores beginning in the 2011-12 school year, Flynn said. (ACES did that task the previous year.) The validators double-check high and low scores. They conduct classroom observations of any teacher flagged as ​“needs improvement” or ​“exemplary” in the fall. Revision Learning also works with 17 other school districts in Connecticut to train evaluators in how to grade teachers. That’s because of a new state law requiring schools to start linking job evaluations to student performance. As part of the new law, the state called for districts to train its evaluators in how to do so in a fair manner, Flynn said.

Watching a video is not the same as being with a teacher in person, Flynn acknowledged. The evaluator has less information, because you can’t see what’s on the classroom walls or get a feeling for the room, and you can’t discuss what led up to the lesson. And when an evaluator is grading one of her own teachers, she has to put aside any personal biases she may have based on their relationship.

Flynn said besides training their observational eyes, the hardest thing for new evaluators is to ​“be able to critique a teacher in a critical way that challenges them, and includes support.” This happens in writing as well as in person, in face-to-face meetings like this one. 

New Haven’s evaluation system has more wiggle room — than evaluation programs in cities like New York and Chicago: New Haven does not use an algorithm to measure student progress. New Haven allows teachers to set their own goals for student learning, based in part on standardized tests. Teachers can revise those goals, in cooperation with their supervisors, at a mandatory mid-year conference. Student learning comprises roughly one half of teachers’ grades, though it gets weighed more or less depending on the results.

Harries contends that the goal of the evaluations is not to push teachers out, but to help them develop. In its ideal form, the evaluation creates a new relationship between a teacher and their evaluator, and a new forum in which teachers can discuss how to improve their craft.

“The most important thing is the quality of the feedback,” said Superintendent Harries. ​“Having good feedback can be awkward and challenging,” he said, but when conversations go well, ​“they can be really valuable.”

The quality of feedback varies, said Cicarella. Some evaluators give ​“targeted” and helpful feedback. ​“Others, what they give is not really going to help the teacher.”

Cicarella said the process by which teachers and their evaluators agree on their student-learning goals stands to be improved.

“Goal-setting hasn’t really gone that well,” Cicarella said. Some teachers feel their principals impose goals on them; others don’t know that they can adjust their goals mid-year.

Another problem, Cicarella said, has been an ​“impossible case load.” After the first year, some principals reported to the school board that they were overwhelmed with the work associated with observing teachers, filing paperwork, and meeting with teachers regarding their goals.

Now New Haven is expanding the ranks of its evaluators to include more teachers. Teachers don’t have to reach a certain score on the teacher evaluation to sign up as evaluators, Cicarella said, but they tend to all score on the top two levels of the five-point scale.

New evaluators included teachers who have their intermediate administrative certificate (092) as well as those who don’t. New Haven allows classroom teachers to evaluate their peers even if they don’t have that certification. Most are asked by their principals to sign up. While a few are classroom teachers, most have jobs that already include some supervising, such as literacy and math coaches.

Some teachers prefer having a colleague grade them instead of an administrator who may have been out of the classroom for 10 years.

Cicarella said to ensure fairness, the system has a double opt-out feature: If a teacher is assigned another teacher as his or her evaluator, both parties can opt out for personal reasons.

“We’re being creative about who does the evaluating,” said Harries, ​“so that we can try to create the time in the day to get meaningful feedback and evaluation.” He said in an ideal world, each evaluator would be grading eight to 10 teachers.

There are now 137 evaluators for 1,640 teachers, a ratio of about 1 to 12, according to Michele Sherban-Kline, the district’s point-person for teacher evaluation. Of the 137, 100 are administrators, 23 are teachers who evaluated their peers last year, and 14 are teachers who are becoming new evaluators.

Flynn said Friday’s group of new trainees will reconvene in January to talk about how to do another part of the evaluation process: Sitting down with a teacher for a mid-year conference and discussing how their practice is going.

Cicarella professed confidence in Flynn’s training sessions.

“Patrick Flynn does a great job,” he said. Once the new evaluators have had significant training, and then one year of experience observing and evaluating teachers, he said, ​“then we will be in good shape.”