More Kids' Grades Ride on a Portfolio of Work

In a basement room of the red-brick Mary Lyon Elementary School in Boston, Deborah Rooney is teaching first-graders to be archaeologists.

But instead of digging for fossils, Ariel, Larry, Marcus, and Ms. Rooney's 12 other students are learning to archive their art projects, math worksheets, and writing samples to create a portfolio of their best academic efforts.

"I don't keep number grades," Ms. Rooney says. So how do I assess this early in the year? I have to look at their portfolios. It's a huge help."

Student portfolios have been popular for a century or more. But the old standby has been whittled into a state-of-the-art tool for "authentic assessment" - grading based on students' in-class work rather than on standardized tests.

The idea caught on in the early 1990s as one way to assess children without the bias and artificial pressures some say are created by standardized testing.

But Vermont, Kentucky, Michigan, and New Mexico are taking it further, using portfolio assessment - along with testing - as a central means of judging children's achievement. Massachusetts and New York also have large portfolio-assessment pilot programs. Iowa, Florida, and Texas and others have a smattering of schools and districts involved.

"We needed a different kind of test - one that didn't just measure minimal standards," says Tom Bisson, spokesman for the Vermont Department of Education. "We wanted an assessment system that measures how well we can do, not how little. Portfolio assessment shows students that this is where you are - and this is the level you need to get to."

But there have been disappointments. As the nation's leading portfolio assessment pioneer, Vermont was first to implement the system statewide. And there were once high hopes that portfolios would be the central, if not sole, means of assessing students. Yet the system's unexpectedly high cost in time and dollars, and public pressure to measure student and school performance against national standards, led to standardized testing's return last year - in addition to portfolios.

Vermont's portfolio system has demonstrated its effectiveness in measuring student achievement, Mr. Bisson says. But not all are convinced.

Soft and fuzzy

"This is just one more of these soft assessments left over from the 1970s," says Peter Berger, an English teacher at the Weathersfield Middle School in Ascutney, Vt., a town of about 3,000. He opposes the system's "pseudo-objective scoring," charging that it provides no accurate measure of student performance since teachers' judgments are subjective.

E.D. Hirsch, founder of the Core Knowledge Foundation in Charlottesville, Va., concurs. "Portfolios are fine for the classroom, absurdly unreliable for high-stakes testing," he says.

In Vermont, for example, writing is measured in five "dimensions" - purpose; details; organization; voice; and mechanics, grammar, and usage. Each category is given one of four scores: extensively, frequently, sometimes, or rarely. The scores become a four-point scale - 1 to 4.

The problem is that one teacher may say a student's writing has "explicit" details and so would rate it "extensively." But another teacher may see fewer details and give it a lower "frequently" rating.

The Washington-based Rand Institute on Education and Training concluded portfolio assessment was a useful classroom teaching tool many teachers and students liked. But as an assessment tool, it is too subjective, it said.

A 1994 Rand study of Vermont's first two years was blunt: "The Vermont portfolio program has been largely unsuccessful ... in meeting its goal of providing high-quality data about student performance."

Now with nearly five years of experience, Vermont's portfolio assessment has been refined, scoring reliability improved, and costs reduced, according to Bisson. Rand has not been back to verify this.

One open critic of standardized testing has praised Vermont's program. In a survey of state assessment systems released this summer, the Cambridge-based National Center for Fair and Open Testing, said "only one state, Vermont, reached the top level and is close to having a model [assessment] system."

Growth industry

Other states are also getting in on the act. Massachusetts has a pilot program, with Boston as a focal point. Last week Boston officials announced that the city's public schools will be adopting a portfolio program modeled after Vermont's - but with electronic portfolios for every student courtesy of an $875,000 grant by IBM.

Education observers see growth in portfolio assessment, but in a more gradual and localized way than previously envisioned by its boosters.

"I don't see more states moving to [portfolio assessment] as a statewide testing tool," says Ed Roeber, director of student assessment programs for the Council of Chief State School Officers, a lobby group that represents state superintendents in Washington. "I do see states doing more localized programs and teacher training."

Theodore Sizer, an education professor emeritus at Brown University in Providence, R.I., lauds Vermont's efforts and says college admissions officers would be much better off looking at a CD-ROM student portfolio than standardized test scores.

"I admire the people who look at the kids' real work and try to make some judgment - not only child by child but across children," he says. "The effort in Vermont was filled with wonderful examples."

But he also warns against the notion that many hold of portfolio assessment, that teachers can be trained to all give the same rating to the same work through a "grid system" like the one Vermont uses. "I get nervous when people say there's only one standard of good expository prose," he says.

Portfolio Software Is Big Business

Here are millions to be made in the hunt for the perfect testing technique - especially if it happens to be the technology hungry portfolio-assessment regime.

With so many states looking for the most effective way to gauge how students measure up to new state-wide academic standards, Sriram Bhamidipati is confident he will capture converts.

As product manager for Forte Systems Inc., in Troy, Mich., he handles a software program called Authentic Convergence Assessment that can track a portfolio of academic work from first grade through high school graduation.

"It makes it easy to produce a CD-ROM of a student's work customized to requirements of the college he wants to go to - or to an employer he or she wants to work for," he says. "Or it could produce something for grandmother that looks like a family album."

The cost to license the software is $40 to $50 per student. The small company has already hit the big time with a multimillion-dollar contract to supply the Rochester, N.Y., public schools.

A much larger player in the hunt to supply portfolio-assessment technology is International Business Machines. IBM is in the process of handing out 11 grants for portfolio systems worth $10 million nationwide.

"We are expecting exponential growth this year and next," Mr. Bhamidipati says. "We went to a trade show for school superintendents and their jaws just dropped open."