TBO.com: Tampa Bay Online, The Tampa Tribune and The Tampa Times - breaking news and weather.
Monday, Sep 01, 2014
Education

How does the VAM formula measure teachers?


Published:

TAMPA – It is calculated by plugging numbers from standardized test scores, class size, and more into a tricky math equation. Attendance is considered. So is age, whether students have disabilities, and how long they’ve been English-speakers.

This so-called value-added model, or VAM, generates a score designed to show the effect a teacher has on student learning.

It is so complex it will make your head spin.

Hundreds of thousands of teachers’ value-added scores were released to the public for the first time last week, the result of a newspaper’s public records lawsuit against the Florida Department of Education.

Objections to the release were raised by teachers, school district officials and union representatives across the state, on the grounds that the numbers by themselves misrepresent a teacher’s worth. Some described the state’s system as flawed at best.

“There are certain things that lend themselves easily to mathematical interpretation,” said Stephanie Baxter-Jenkins, executive director of the Hillsborough Classroom Teachers Association. “I don’t think teaching or how kids learn are things that are easily captured in a mathematical formula.”

Florida’s education system is undergoing a lot of changes – next school year, sweeping new standards will be fully in place along with a new exam to replace the Florida Comprehensive Assessment Test.

But it doesn’t look like VAM is going anywhere anytime soon.

❖ ❖ ❖

In 2011, Florida passed legislation that requires including student achievement data in teacher evaluations.

The value-added model used today is how that data is spun into a score.

VAM was researched and recommended by the state’s Student Growth Implementation Committee — a group of teachers, principals, parents, union representatives, superintendents, school board members, district administrators and post-secondary faculty members.

While parents are warned not to consider a teacher’s score as the only measure of his or her worth, the numbers still pack a punch – they make up 40 to 50 percent of teachers’ overall evaluations.

So how does VAM work? That’s a question that takes 25 pages of math, text, charts and graphs to explain.

The white paper detailing the formula assigns teachers a score based on how their students performed on standardized tests compared with how they were predicted to perform.

Used in the lengthy equation are the scores of students in fourth through 10th grades on the math and reading portions of the FCAT, as well as the ninth-grade state end-of-course Algebra 1 test.

Essentially, if a teacher’s value-added score is higher than zero, the average performance of the teacher’s students exceeded expectations. Those expectations are set based on the performance of similar students across the state. A negative score means the average student scores fell below expectations.

The prediction for each student is based on a statistical model that takes a number of factors into account. Those factors include two prior years of student achievement scores, the number of subject-relevant courses in which the student is enrolled, attendance, the number of times the student has switched schools, and class size. The state says the model “levels the playing field” by accounting for differences in the proficiency and characteristics among students.

The formula considers whether the student has disabilities, or is categorized as an English language learner or gifted.

In addition to individual teacher scores, the state last week released schoolwide and districtwide aggregate scores in reading, math, Algebra 1, and all three combined.

Of the Hillsborough schools that received a VAM score, about half were on the positive side of the equation.

Hillsborough County School Board member Doretha Edgecomb serves on the state committee that chose the model. She said the method is still in the “embryonic stages.”

“We have to make sure it sends the message we want about our teachers,” Edgecomb said. “They’re too valuable to be demoralized by information that does not really give the total picture of their hard work and effort.”

❖ ❖ ❖

Others among the most populous states in the nation also use the value-added model, including Ohio and California.

The effectiveness of a VAM model depends on which student test is used to calculate a teacher’s score, said Joan Herman, a senior research scientist at the National Center for Research on Evaluation, Standards and Student Testing at the University of California in Los Angeles.

“It sound so obvious that teachers ought to be accountable for their kids’ learning, but figuring out how to do that is complicated,” Herman said. “Using FCAT scores may be OK for teachers who teach courses that most contribute to FCAT performance, but what about the music teacher?”

Not every teacher in Florida gets a VAM score. Beginning next school year, school districts will be required by state law to choose other test results to measure those who teach subjects other than the ones subject to the FCAT.

Hillsborough County already calculates teacher scores with its own district exams — a process that is working well, district spokesman Stephen Hegarty said.

The score makes up 40 percent of a teacher’s evaluation. The other 60 percent comes from principal and peer observations. District teachers’ VAM scores are derived from student performance on end-of-course assessments in subjects they teach.

Art teachers are measured on their students’ achievement on art tests. Social studies teachers are measured on their students’ growth in that subject.

The district created its own end-of-course assessments and uses them along with student scores on the college-entrance SAT to help grade teachers. Factors considered include whether a student is over-age, gifted, an English language learner or has special needs.

“The state has been doing end-of-course exams for a couple of years,” Hegarty said. “We have been doing EOCs for a couple of decades, which means our teachers have an assessment that actually measures what they’re doing. We have something for all of the different disciplines for our teachers. Your scores are derived from students you actually taught in an area you actually taught.”

Baxter-Jenkins, the local union leader, said the district’s method is more reliable than the state’s. She said the test scores release by the state cause more harm than good.

“These are people we are entrusting our children to all day every day,” she said. “Yet, we are trying to publicly shame them with completely inaccurate information.”

Department spokeswoman Cheryl Etters said the data used to calculate VAM scores were submitted by school districts, so teachers who suspect inaccuracies in their scores should contact their district.

❖ ❖ ❖

The VAM scores are available in detail at the website of the newspaper that won their release, the Florida Times-Union in Jacksonville. Access requires an online subscription.

The education department did not post the results online and made them available to news organizations only through password-protected accounts.

Among the teachers who didn’t receive a value-added score from the state is Reza Razavi, who teaches AP economics and honors philosophy at Wharton High School in New Tampa. He did receive a score from the Hillsborough school district.

To him, the scores released by the state “don’t represent anything.”

“These scores create doubts,” he said. “Unfortunately, it’s just throwing numbers out without explaining the details to parents. Parents look at it and think, ‘This teacher is not so good,’ because they don’t understand where these numbers are coming from.”

As Florida ushers in the new Common Core State Standards, which supporters describe as a rigorous set of annual educational goals in language arts and math, the state is also selecting a new test aligned with those goals. Students are expected to take the test for the first time next school year.

A new test could improve Florida’s VAM process, according to Herman, UCLA researcher.

“The value added measure is only as good as the test that is used to compute it,” Herman said. “If you think FCAT captures everything that is important for students to know and be able to do, you might think it’s reasonable to use that measure as the only measure of whether teachers are contributing to student learning.

“If you think FCAT is missing some important skills or knowledge, you might think it has some real deficits.”

ekourkounis@tampatrib.com

(813)259-7999

Twitter: @ErinKTBO

Subscribe to The Tampa Tribune

Comments