KHKnowledge Hub

AI Security, Stage by Stage

A newbie-to-expert curriculum on securing AI systems in twelve stages: threat models, injection, RAG, agents, MCP, privacy, supply chain, adversarial ML, guardrails, red teaming, and detection & governance — each with a runnable lab.

13 pages · 7.2 h reading

  1. 1

    Start here: the curriculum, order and time budget

    Twelve standalone guides and labs from threat modeling to detection and governance, with the suggested order, a time budget, a fast path, and the capstone lab that wires Stages 10–12 together.

    6 min read · September 30, 2026

  2. 2

    Threat modeling foundations

    A newbie-to-expert guide to trust boundaries, STRIDE, the OWASP Top 10 for LLM Applications and MITRE ATLAS, ending with a worked threat model of a RAG chatbot.

    35 min read · October 1, 2026

  3. 3

    Prompt injection and jailbreaks

    Direct vs indirect injection, the instruction hierarchy, a full attack taxonomy, layered defenses, and a runnable lab where a poisoned webpage hijacks an agent — until you move the boundary.

    28 min read · October 3, 2026

  4. 4

    Insecure output handling

    How model output becomes XSS, SQL injection, SSRF, command execution or an unauthorized action — and how sink-specific controls, strict contracts and authorization stop it.

    33 min read · October 5, 2026

  5. 5

    RAG and vector database security

    Embedding and document poisoning, tenant isolation, ACL-aware retrieval, caches and revocation — with a lab where a planted document leaks another user's data.

    32 min read · October 7, 2026

  6. 6

    Agent and tool security

    Least privilege, the confused deputy, sandboxing and scoped credentials — with a lab where an agent with shell and email tools leaks data until a permission layer blocks the chain.

    40 min read · October 9, 2026

  7. 7

    MCP security

    Tool poisoning, rug pulls, tool shadowing, OAuth scopes and server authentication — with a lab where a malicious MCP-style server exfiltrates a fictional secret.

    37 min read · October 11, 2026

  8. 8

    Data leakage and privacy

    PII exposure, training-data extraction, membership inference and DLP, from first principles to a measured local lab.

    34 min read · October 13, 2026

  9. 9

    Model supply chain

    Serialization risks, safetensors, signing and provenance, AI-BOMs, lookalike names, and an admission pipeline that refuses untrusted model artifacts.

    38 min read · October 15, 2026

  10. 10

    Adversarial ML

    Evasion, data poisoning, backdoors and model extraction, with a lab that backdoors a classifier using 1% poisoned data, detects it three ways and repairs it.

    43 min read · October 17, 2026

  11. 11

    Guardrails and runtime enforcement

    Input/output classifiers, policy engines, deterministic checks and human-in-the-loop, with a local lab that puts a policy gateway in front of an agent and blocks risky actions before they execute.

    37 min read · October 19, 2026

  12. 12

    Red teaming and evals

    garak, PyRIT and promptfoo, attack success rate with honest denominators, regression suites, and a local lab whose CI gate fails the build when a new jailbreak lands.

    30 min read · October 20, 2026

  13. 13

    Detection, incident response and governance

    Audit ledgers, detections with stated assumptions, an incident response playbook for a compromised agent, a scored tabletop, and the mapping to NIST AI RMF, ISO/IEC 42001 and the EU AI Act.

    40 min read · October 20, 2026