A new project by Instruction Partners evaluates AI learning tools, finding both promise and pitfalls. The study includes 16 in-depth reviews of individual products, assessing their impact on student learning. This initiative is particularly timely as the number of AI tools marketed to schools has surged, creating a pressing need for systematic assessments of their instructional quality.
With a growing number of AI tools marketed to schools, this initiative provides one of the first systematic assessments of their instructional quality. Emily Freitag, co-founder and CEO of Instruction Partners, emphasizes the need for “subtle decision-making” when integrating AI into education. She highlights that while some tools offer real value, others pose significant risks, showing the importance of careful evaluation.
A Mixed Bag of AI Tools
The review covers 20 tools, ranging from general-purpose chatbots like ChatGPT to education-focused platforms such as Google Classroom. Researchers conducted extensive interviews with teachers, students, district leaders, and product developers, and observed the tools in action across 16 school systems. This full approach ensures a subtle understanding of how these tools function in real-world educational settings.
While some tools demonstrate potential, others are criticized for being “isolating” and “boring”, which could negatively impact student engagement and learning. The findings show the necessity of thoughtful integration, ensuring that AI tools complement rather than hinder the educational process.
General-Purpose Chatbots Pose Risks
General-purpose chatbots, including ChatGPT, are identified as the most significant risk. Teachers report that students often use these tools to bypass effortful thinking, a critical component of the learning process. The study warns that such tools can “significantly compromise learning” by reducing the cognitive effort required to engage with material.
For example, students were observed asking chatbots questions, then going to Google, asking Google the question, and putting the answer in the AI tool. This behavior not only undermines the learning process but also supports a reliance on external tools rather than developing critical thinking and problem-solving skills. The study highlights the need for educators to guide students in using these tools responsibly.
Instructional Tools Show Promise, But Aren’t Perfect
Tools designed for specific instructional purposes, such as Amira Learning for reading practice, are viewed more favorably. These tools can offer practical feedback and assist teachers in delivering tailored lessons. For instance, in reading and English/language arts, AI tools can provide actionable feedback on student writing faster than an individual teacher could on their own, or diagnose which phonics skills students have mastered and which they still struggle with.
Related Post: Charter Schools Turn Teacher Hiring Into People Business
In math, the review team praised tools that could identify students’ misconceptions, or collect and analyze many students’ explanations of their mathematical reasoning. However, concerns remain about AI-generated feedback being inconsistent and impersonal, which can diminish its effectiveness. Additionally, some tools that rewrite text at a lower reading level may inadvertently reduce the rigor of the material, potentially limiting student growth.
As AI in education continues to evolve, the role of evaluators and regulators becomes increasingly critical. Currently, there is limited consensus on how to assess the instructional quality of AI tools, yet schools are rapidly adopting them. This gap highlights the need for standardized evaluation frameworks that can keep pace with technological advancements.
Instruction Partners aims to provide a balanced perspective, offering detailed analyses rather than simplistic ratings. Their profiles highlight the advantages and disadvantages of each tool, enabling district leaders to make informed decisions based on their specific needs and contexts. This approach encourages a more subtle understanding of AI tools’ potential and limitations.
The organization plans to expand its review, collaborating with organizations like Stanford University to examine additional tools and evidence-of-impact studies. This ongoing effort reflects the dynamic nature of AI in education, where evaluation methods must continually adapt to address new challenges and opportunities.
Instruction Partners and EdReports play a key role in equipping educators and leaders with the information needed to make informed choices about AI integration in classrooms. Their work is essential as the field works through the complexities of AI adoption, ensuring that these tools enhance rather than detract from student learning.
The analysis shows the importance of critical thinking in the learning process, warning against tools that discourage this essential skill. It also highlights the need for AI tools to be designed in ways that support, rather than replace, meaningful teacher-student interactions.
Education-focused AI tools, while promising, require careful evaluation to ensure they enhance, rather than hinder, student learning. This includes considering how these tools align with educational goals, support teacher effectiveness, and promote student engagement.
