Long & Magerko's 17-Competency Framework
Diana Long and Brian Magerko's 2020 paper in AI and Ethics, "What Is AI Literacy? Competencies and Design Considerations," synthesized research into a comprehensive 17-competency AI literacy framework.
The competencies span four domains: (1) Basic AI Fundamentals—understanding what AI is, how it differs from human intelligence, recognizing AI applications in daily life; (2) How AI Works—understanding data collection and preprocessing, model training processes, supervised vs. unsupervised learning, feature importance, algorithmic decision-making; (3) AI Limitations and Risks—understanding bias sources (training data bias, historical bias, representation bias), recognizing when AI confidently makes wrong predictions, understanding failure modes (adversarial examples, domain shift), appreciating computational resource demands and environmental impact; (4) Societal Impact—understanding AI's role in labor displacement, surveillance, and inequality, recognizing responsible AI principles, understanding regulatory landscape (GDPR, AI Act), appreciating interdisciplinary considerations.
Notably, Long & Magerko distinguished AI literacy from machine learning technical expertise: one can understand AI systems critically without building them. The framework proved influential in K-12 curriculum development; states adopting Long & Magerko competencies report improved student conceptual understanding compared to tool-focused curricula (Georgia Tech 2021 evaluation study).
Ng, Temiyasathit, & Lim's Know-What Framework
Andrew Ng, Sahit Temiyasathit, and Wey Lim's 2021 AI for Everyone course and associated paper structured AI literacy into four progressive competency levels: (1) Know-Why—understanding business problems that AI can address, recognizing which problems are suitable for AI solutions (prediction, classification, generation) versus unsuitable (requiring real-time interaction, physical manipulation without precedent), understanding AI's strategic value; (2) Know-What—understanding AI workflow (problem definition → data collection → model development → deployment → monitoring), recognizing AI terms (training, testing, overfitting, accuracy metrics), understanding data requirements and annotation; (3) Know-How—developing practical skill (writing code, tuning models, evaluating results), applying frameworks to real problems, debugging AI systems; (4) Know-Do—independent problem-solving, continuous learning as AI advances, contributing to AI ethics discussions in organizational contexts. This progression emphasizes that know-what (what is a neural network?)
without know-why (why would we use one here?) produces inefficient learning. Ng's framework proved widely adopted in corporate AI literacy programs; Microsoft, Google, and Facebook all integrated variations in employee training (Microsoft 2022 training assessment reports).
The model acknowledges that most professionals need know-why and know-what, fewer need know-how, and only specialists need deep know-do. This hierarchy guided resource allocation in educational programs.
UNESCO AI Competency Framework (2022)
UNESCO's 2022 "Recommendation on AI and Education" established competency frameworks addressing distinct stakeholder groups: students, teachers, and policymakers. The student framework identifies eight competency areas: (1) Understanding AI fundamentals and terminology; (2) Data literacy—understanding data quality, bias, privacy; (3) Algorithmic literacy—understanding how algorithms make decisions, recognizing algorithmic bias; (4) Critical AI thinking—evaluating AI claims, understanding limitations and failure modes; (5) Responsible AI awareness—understanding ethical implications, privacy concerns, societal impact; (6) AI and work—preparing for AI-transformed labor markets; (7) AI creativity—using AI tools creatively (prompting, system design); (8) Collaboration with AI—developing human-AI team capabilities.
For teachers, UNESCO emphasized need for adult AI literacy before pedagogical capability: most teachers lack technical background; frameworks prioritize teacher understanding sufficient to guide student learning rather than requiring teaching-level technical expertise. The teacher framework emphasizes critical evaluation, ethical implications, and bias detection.
For policymakers, UNESCO emphasized domain-specific competency: health policymakers need understanding of AI applications in health (diagnosis systems, drug discovery) and associated risks; education policymakers need understanding of algorithmic student tracking, bias in automated grading. UNESCO's 2023 follow-up ("Generative AI and Education") updated frameworks following ChatGPT emergence, adding competencies: prompt engineering, understanding generative AI limitations (hallucinations, training data recency), evaluating generative AI outputs for accuracy/bias.
OECD AI Principles Framework (2019)
The Organization for Economic Cooperation and Development's 2019 "Recommendation on Artificial Intelligence" articulated five foundational principles for responsible AI: (1) Inclusive growth, sustainable development, well-being—AI should benefit society, not concentrate advantages; (2) Human-centered values and fairness—AI systems should respect human rights, democratic values, non-discrimination; (3) Transparency and explainability—AI decision-making should be understandable to stakeholders and subject to appropriate disclosure; (4) Robustness, security, safety—AI systems should be resilient, secure against adversarial attack, safe across deployment contexts; (5) Accountability—AI developers and deployers should be accountable for system behavior and impacts. OECD's framework did not prescribe technical competencies but established principle-based governance requiring stakeholder understanding: developers need technical literacy to implement transparency; policymakers need understanding to enforce accountability; citizens need sufficient literacy to evaluate AI's societal effects.
The OECD principles became foundational for EU AI Act (2024), US Executive Order on AI (2023), and national AI strategies globally; countries adopting these principles required AI literacy across professional sectors to implement compliance (Fjeld et al. 2020 Harvard Berkman Klein Center analysis of 28 national AI strategies).
Algorithmic Awareness and Behavioral Research
Cotter & Reisdorf's 2020 paper in New Media & Society, "Algorithmic Literacy and the Responsibilities of Platforms," examined what lay people understand about algorithmic curation. Qualitative research (interviews with 50 social media users) revealed fundamental misconceptions: 60% believed algorithms are neutral mathematical processes without values; 40% believed algorithms are manually curated (a person chose what appears in their feed); 30% believed their data is not recorded/used.
Quantitative assessment showed even high-self-confidence internet users (70th percentile self-assessed competence) scored 40% accuracy on multiple-choice questions about social media algorithms. Exposure to basic algorithmic literacy training (20-minute video explaining recommendation systems) improved accuracy to 65%, but knowledge retention at 2-week follow-up declined to 48%, suggesting conceptual difficulty or motivation factors limit sustained learning.
The research implies substantial AI literacy gaps even among heavy technology users, with implications for informed consent (users cannot meaningfully consent to data practices they don't understand) and democratic participation (algorithmic curation's effects on information exposure depend on user awareness). Susser et al.
2019 paper on "Technology as a Tool for Understanding and Shaping Behavior" proposes that meaningful digital citizenship requires minimum algorithmic literacy threshold; falling below threshold, users are effectively manipulable through algorithmic targeting.
Competency Assessment Challenges
Assessing AI literacy proves methodologically challenging. Simple knowledge tests ("What is machine learning?" ) measure recall without understanding. Competency-based assessments ("Identify sources of bias in this dataset") require domain expertise in evaluation and allow subjective grading.
Performance assessments (build a classifier, explain its decision-making) demand substantial time and technical infrastructure. Meta-analysis of 50+ AI literacy assessment instruments (published 2018-2023, Ng et al.
2024 meta-review in AI and Education Research) found only 12% of instruments demonstrated validity beyond face validity; most instruments lack test-retest reliability, construct validity evidence, or cross-cultural validation. This represents critical research gap: schools and organizations implementing AI literacy curricula cannot reliably measure competency gains.
Development of validated AI literacy measures represents urgent need for the field, especially given increasing importance of AI competency for economic participation and informed citizenship.