Analyze & Decide - Edtech Pilot Framework
Edtech Pilot Framework

Analyze & Decide

Framework Steps

laptop

Analyze pilot data to determine the extent to which the edtech product solved your need, and whether to scale up implementation. Consider the goal, size, and complexity of the pilot when deciding how to analyze data.

Districts can make the strongest claims and clearest decisions by comparing outcomes for pilot participants with those for teachers and students who did not participate.

Comparing pre- and post-pilot data

“By creating our own templates in Excel, we can get the fields that we know we need, which align with how we measure outcomes. …[When we] pilot a program, we can send the company our template and say, ‘You need to figure out how to align whatever your metrics are with these.’” —Eian Harm, West Ada School District

Tips for Success!

A list of tips
  • If possible, compare results to a control group
  • Consider multiple types of data in your analysis
  • Ensure claims are supported by the data

Tools & Resources

For definitions of our taxonomies, see our About section.

Lea(R)n IMPACT Analysis Rapid Cycle EdTech Evaluation 
From Lea(R)n

Informed by more than 100 product evaluations across diverse education environments, IMPACT Analysis is used to rapidly analyze individual edtech product utilization, pricing and student achievement data in seconds to inform instructional, operational and budget decisions.

LearnPlatform for Administrators 
From Lea(R)n

Always free, LearnPlatform for Administrators provides access to the largest database of edtech products, including teacher grades, broken down by categories including feedback, usage, impact, and price to help them support the academic freedom of their educators while upholding resource allocation responsibilities for support, PD and implementation needs. LearnPlatform for Administrators also includes real-time organization-wide edtech usage information via easy-to-access and understand dashboards.

LearnPlatform Free EdTech Monthly Analytics Reports 
From Lea(R)n

LearnPlatform provides free monthly edtech analytics reports to administrators so they can quickly and easily gain a comprehensive view of what edtech is actually being used across their schools and district.

LearnPlatform Starter 
From Lea(R)n

Always free, LearnPlatform for Administrators provides access to the largest database of edtech products, including teacher grades, broken down by categories including feedback, usage, impact, and price to help them support the academic freedom of their educators while upholding resource allocation responsibilities for support, PD and implementation needs. LearnPlatform for Administrators also includes real-time organization-wide edtech usage information via easy-to-access and understand dashboards.

Personalize Learning @ Scale Self-Assessment 
From Lea(R)n

Quick (10-minute) self-assessment that district administrators can take to gauge and compare their edtech management status and gather aggregated from like districts, as well as receive a custom report with actionable steps for their organization to personalize learning at scale.

Ed Tech RCE Coach: Analysis Appendices 
From Mathematica Policy Research, Inc

An example of a way to analyze and present data from an ed-tech pilot prepared by Mathematica for the U.S. Department of Education.

Ed Tech RCE Coach: Learn Who Used Your Technology 
From Mathematica Policy Research, Inc

This document contains information about how to effectively use student data generated by an edtech program.

ED Tech RCE Coach: Matching 
From Mathematica Policy Research, Inc

This document gives an overview of matched comparison design, which allows you to compare results from students who used an ed tech tool and students who did not.

Ed Tech RCE Coach: Prepare Your Data for Analysis 
From Mathematica Policy Research, Inc

This worksheet will help you organize data so you can analyze the results of the pilot.

Edtech Usage Trends Report 
From Lea(R)n

Review usage trends, aggregate data and factors that may be influencing edtech use to fidelity in the US with Lea(R)n’s 2017 EdTech Usage Trends Report.

Glossary: Research and Analysis Terminology 
From Lea(R)n

Curated glossary of research terminology to assist educators and administrators with defining edtech measurement and efficacy needs, as well as deciphering data sets, analytics, reports and other academic studies and materials.

Glossary: Research and Analysis Terminology 
From Lea(R)n

Curated glossary of research terminology to assist educators and administrators with defining edtech measurement and efficacy needs, as well as deciphering data sets, analytics, reports and other academic studies and materials.

How Do We Interpret Developers’ Claims? 
From Ed Tech RCE Coach

This case study reviews strategies to thoughtfully interpret developers’ impact claims when analyzing and deciding.

How to Make Sense of Mixed Results? 
From Ed Tech RCE Coach

This case study considers a decision making process at the conclusion of a pilot with mixed results.

Lea(R)n IMPACT Analysis Rapid Cycle EdTech Evaluation 
From Lea(R)n

Informed by more than 100 product evaluations across diverse education environments, IMPACT Analysis is used to rapidly analyze individual edtech product utilization, pricing and student achievement data in seconds to inform instructional, operational and budget decisions.

Continue to Step 7

Improving learning with the power of technology

Let's connect and we'll send you the latest from Digital Promise.

Sign Up For Updates! Sign Up For Updates

Sign up for updates!

×