Microsoft has new AI privacy rules for schools

Microsoft agreed to a set of safety and privacy principles for AI in schools a week after two major school systems announced a ban on student-facing AI.

In a new agreement with the American Federation of Teachers (AFT), the second-largest teachers union in the US, and its New York City affiliate the United Federation of Teachers (UFT), Microsoft committed to ten principles that can be contractually enforced by school districts that adopt them. The terms include pledging not to train AI models on student or educator data, limiting the amount of data Microsoft collects in the first place and disclosing to families how its tools work in plain language, prohibiting AI companions, and requiring human review for “high-risk” decisions.

The agreement comes after New York City and Los Angeles both implemented one-year bans on AI tools for many students, giving the districts time to consider the proper use and guardrails. As parents and some educators have resisted the use of AI and screens more generally in schools, Microsoft likely recognizes that similar decisions could be coming. Beginning in November, school districts can opt to add the terms to new or existing agreements without having to renegotiate their entire contracts.

The AFT, for its part, has been critical of tech in schools, but also open to working with tech companies to prepare for the current AI wave. In a May speech, AFT President Randi Weingarten called for a ban on screens before third grade as well as “student-facing AI” before middle school. But the group also announced a $23 million AI training hub for educators funded by Anthropic, Microsoft, and OpenAI.

In a statement about the new privacy agreement with Microsoft, Weingarten said the terms were “iron-clad” and necessary “because no one else, including the federal government, has stepped up to do the real work.” She added, “We can get angrier and angrier, or we can act decisively; anything less than legally enforceable provisions is simply a wish list.”