What is Khan Academy’s approach to responsible AI development?

As a nonprofit educational organization, Khan Academy is committed to explore how AI can shape the future of learning. We believe AI has the potential to transform education in meaningful ways, but we also recognize the serious risks it can pose.

That’s why we’ve developed a set of guidelines and safeguards to ensure our AI development prioritizes the needs of students and teachers. Our goal is to use AI responsibly in educational settings and to ensure its benefits are shared equitably across society. As we continue learning about AI and the technology evolves, so will our guidelines.

1. We educate people about the risks and we are transparent about known issues.

We provide clear communication about the risks and limitations of AI both before and after users gain access to our tools. We also offer easy ways for users to report any issues they encounter.

Participants must read and accept the known and potential unknown risks and limitations of AI, including the fact that:

• AI can be incorrect or misleading
   
• It may generate inappropriate or harmful content
   
• It can make factual and mathematical errors
   

To help the public better understand AI, we’ve also launched a free course called AI for Education, which covers:

• How large language models (LLMs) work
   
• How LLMs apply to education
   
• What AI is (and isn’t) good at
   
• Key questions educators and learners should be asking about AI
   

2. We use proven practices from respected organizations to help guide our risk evaluation and mitigation efforts.

We’ve studied and adapted frameworks from the National Institute of Standards and Technology (NIST) and the Institute for Ethical AI in Education to evaluate and mitigate AI risks specific to Khan Academy.

We acknowledge that AI is not always accurate and not entirely safe, and that it's not possible to eliminate all risks at this time.
That’s why we work diligently to identify risks and put mitigation measures in place. We mitigate risk by using technical approaches such as: 

• Fine-tuning the AI to improve accuracy
• Prompt engineering to guide and narrow the focus of the AI. This allows us to train and tailor the AI for a learning setting.
   
• Monitoring and moderating of participant interactions to detect and respond to inappropriate content and apply appropriate community controls (such as removing access)
• Limiting daily usage, since longer sessions are more likely to result in poor AI behavior
• Red teaming to intentionally testing the AI to uncover vulnerabilities or flaws
• Collecting and consistently reviewing in-product feedback from users to better understand what’s working and where improvements are needed

We also make it clear that AI-generated content may include errors (even in subjects like math) and that inappropriate content is possible.

These efforts are designed to help build stronger, safer, and more trustworthy AI tools over time.

3. At this time, we only grant access to our AI applications through khanmigo.ai or through a Khan Academy Districts partnership.
   
   To access our AI-powered tools:

• Individual users must be at least 18 years old and register through khanmigo.ai.
• Adults who register can grant Khanmigo access to any child accounts linked to their Khan Academy parent account.
• Users under 18 can only use Khanmigo if:
  • They are part of a Khan Academy Districts partnership, or
  • They have been granted access by a parent or guardian through a linked child account.
  • Their teacher has assigned them a writing coach essay assignment.

Note: Outside of these scenarios, users under 18 are not eligible to use Khanmigo.

Every child who has access to our AI-powered tools receives clear communication that their chat history and activity are visible to their parent or guardian, and, if applicable, their teacher and school administrator. We also use moderation technology to detect interactions that may be inappropriate, harmful, or unsafe. When the moderation system is triggered, it sends an automatic email alert to an adult connected to the student’s account.

4. We embrace and encourage a culture where ethics and responsible development are embedded in our workflows and mindsets.

Individuals and teams are asked to identify ethical considerations and evaluate risks at the outset of every project. Our decision-making is guided by risk evaluation. We prioritize risk mitigation, embrace transparency, and continuously reflect on the impact of our work.

We have a detailed monitoring and evaluation plan in place during this testing period. We will learn, iterate, and improve.

Looking ahead:

AI is a rapidly developing field with tremendous potential. We are enthusiastic about AI's ability to benefit education, and we recognize that we have a lot to learn. Our ultimate goal is to harness the power of AI to accelerate learning. We will evaluate how AI works and share our learnings with the world. We expect to adapt our plans along the way.