Person at a computer

Analyze pilot data to determine the extent to which the ed-tech product solved your need, and whether to scale up implementation. Consider the goal, size, and complexity of the pilot when deciding how to analyze data. Districts can make the strongest claims and clearest decisions by comparing outcomes for pilot participants with those for teachers and students who did not participate.  

Comparing pre- and post-pilot data

“By creating our own templates in Excel, we can get the fields that we know we need, which align with how we measure outcomes. …[When we] pilot a program, we can send the company our template and say, ‘You need to figure out how to align whatever your metrics are with these.’” —Eian Harm, West Ada School District

Read more at "Gather Data to Measure Success" >

Tools & Resources

For definitions of our taxonomies, see our About section.

Tool

LearnPlatform Research-based Rubric for Grading Ed-Tech From LearnPlatform

LearnPlatform’s peer-reviewed ed-tech grading rubric provides structure for educator feedback and insights on products with context.

Tool

SUNY Oswego Pre-Post Analysis Tool From SUNY Oswego

An Excel spreadsheet with formulas and charts to help you analyze students’ pre- and post-test scores.

Resource

Ed Tech RCE Coach: Analysis Appendices From Mathematica Policy Research, Inc

An example of a way to analyze and present data from an ed-tech pilot prepared by Mathematica for the U.S. Department of Education.

Resource

Ed Tech RCE Coach: Learn Who Used Your Technology From Mathematica Policy Research, Inc

This document contains information about how to effectively use student data generated by an ed-tech program.

Resource

Ed Tech RCE Coach: Matching From Mathematica Policy Research, Inc

This document gives an overview of matched comparison design, which allows you to compare results from students who used an ed-tech tool and students who did not.

Resource

Ed Tech RCE Coach: Prepare Your Data for Analysis From Mathematica Policy Research, Inc

This worksheet will help you organize data so you can analyze the results of the pilot.

Resource

Pilot to Purchase Project Report

Results from a study of six school districts piloting ed-tech tools, including a look at how they made purchasing decisions.

Resource

SDP Toolkit for Effective Data Use From Harvard CEPR

A five-part toolkit to guide you through the collection and analysis of student achievement data.

Tips for Success!

A list of tips
  • If possible, compare results to a control group
  • Consider multiple types of data in your analysis
  • Ensure claims are supported by the data