laptop

Analyze pilot data to determine the extent to which the edtech product solved your need, and whether to scale up implementation. Consider the goal, size, and complexity of the pilot when deciding how to analyze data.

Districts can make the strongest claims and clearest decisions by comparing outcomes for pilot participants with those for teachers and students who did not participate.  

Comparing pre- and post-pilot data

“By creating our own templates in Excel, we can get the fields that we know we need, which align with how we measure outcomes. …[When we] pilot a program, we can send the company our template and say, ‘You need to figure out how to align whatever your metrics are with these.’” —Eian Harm, West Ada School District

Read more at "Gather Data to Measure Success" >

Tips for Success!

A list of tips
  • If possible, compare results to a control group
  • Consider multiple types of data in your analysis
  • Ensure claims are supported by the data

Tools & Resources

For definitions of our taxonomies, see our About section.

Resource
Tool

Lea(R)n IMPACT Analysis Rapid Cycle EdTech Evaluation From Lea(R)n

Informed by more than 100 product evaluations across diverse education environments, IMPACT Analysis is used to rapidly analyze individual edtech product utilization, pricing and student achievement data in seconds to inform instructional, operational and budget decisions.

Tool

LearnPlatform for Administrators From Lea(R)n

Always free, LearnPlatform for Administrators provides access to the largest database of edtech products, including teacher grades, broken down by categories including feedback, usage, impact, and price to help them support the academic freedom of their educators while upholding resource allocation responsibilities for support, PD and implementation needs. LearnPlatform for Administrators also includes real-time organization-wide edtech usage information via easy-to-access and understand dashboards.

Tool

LearnPlatform Free EdTech Monthly Analytics Reports From Lea(R)n

LearnPlatform provides free monthly edtech analytics reports to administrators so they can quickly and easily gain a comprehensive view of what edtech is actually being used across their schools and district.

Resource

Ed Tech RCE Coach: Analysis Appendices From Mathematica Policy Research, Inc

An example of a way to analyze and present data from an ed-tech pilot prepared by Mathematica for the U.S. Department of Education.

Resource

Ed Tech RCE Coach: Learn Who Used Your Technology From Mathematica Policy Research, Inc

This document contains information about how to effectively use student data generated by an edtech program.

Resource

ED Tech RCE Coach: Matching From Mathematica Policy Research, Inc

This document gives an overview of matched comparison design, which allows you to compare results from students who used an ed tech tool and students who did not.

Resource

Ed Tech RCE Coach: Prepare Your Data for Analysis From Mathematica Policy Research, Inc

This worksheet will help you organize data so you can analyze the results of the pilot.

Resource

Glossary: Research and Analysis Terminology From Lea(R)n

Curated glossary of research terminology to assist educators and administrators with defining edtech measurement and efficacy needs, as well as deciphering data sets, analytics, reports and other academic studies and materials.

Resource
Tool

Lea(R)n IMPACT Analysis Rapid Cycle EdTech Evaluation From Lea(R)n

Informed by more than 100 product evaluations across diverse education environments, IMPACT Analysis is used to rapidly analyze individual edtech product utilization, pricing and student achievement data in seconds to inform instructional, operational and budget decisions.

Resource

LearnPlatform Resource Center From Lea(R)n

Library of free resources with which educators and administrators can support momentum to find, buy, manage and measure educational technology easily and effectively. Resources include FAQs, videos, etc.

Resource

Pilot to Purchase Project Report

Results from a study of six school districts piloting edtech tools, including a look at how they made purchasing decisions.

Resource

SDP Toolkit for Effective Data Use From Harvard CEPR

A five-part toolkit to guide you through the collection and analysis of student achievement data.