Data methodology

Across all the education programmes it managed, E3 held to one principle: decisions should follow the evidence. The Initiative put that principle to work through a deliberate data practice.

A dedicated practice

The Initiative established quantitative, data centred reporting from the beginning, and in 2007 it brought in a full time data specialist, whose brief was to look across the whole portfolio for patterns rather than to examine each school on its own.

A large amount of information was gathered directly from schools at the outset. As schools released their data reports through the year, the Initiative ran comparative and trend analyses. That allowed it to look across the network and to set results against three reference points:

• Oregon benchmarks
• Oregon schools serving similar student populations
• Other schools within the same community

Working within limits

Four limitations shaped what the data could show, and the Initiative named them openly.

1. Schools opened on a staggered schedule, so they were at different stages of development.
2. Grade levels were phased in over time, which complicated year to year comparisons.
3. On some shared campuses, figures could not be broken out for each individual school.
4. State and district figures reached the Initiative late, since gathering and reconciling them takes time.

Going deeper

To move past those limits, the Initiative arranged access to student level records through partner agencies. With that finer grain of detail, the analysis could go deeper and the computations could be trusted more. ECONorthwest, an economics firm based in Oregon, then ran regression analysis. It asked how students in Initiative schools performed next to students elsewhere with similar backgrounds and circumstances.