--- title: "Foundations of Responsible AI" --- # Chapter 2: Foundations of Responsible AI ## Why it matters Evaluation (Chapter 1) tells you how a model behaves. Responsible AI tells you how it *should* behave, and how to build an organization that delivers that. This chapter defines responsible AI, introduces its dimensions, shows that responsibility spans the whole design-build-operate lifecycle, and gives a concrete method for assessing the risk of an AI application. ## What is responsible AI? ```{admonition} Definition :class: tip **Responsible AI** is the practice of designing, developing, and using AI technology with the goal of maximizing benefits and minimizing risks and unintended harms. ``` In practice, responsible AI is defined through a core set of **dimensions** that an organization assesses and updates over time as the technology evolves. Three points are easy to miss: - Responsible AI is an **organizational structure, a set of principles, and a practice**, not a subdomain of AI you can delegate to one team. - The dimensions **depend on the organization** and its responsible-AI maturity. - **New dimensions can emerge** as new scientific evidence appears. Why so much focus on it? Because building and maintaining customer trust is a top priority, and generative AI introduces new failure modes: hallucinations and inaccuracies, instructions that leak private information, biased or hateful text, and unlicensed or unlawful content. A regulatory landscape is actively developing to understand and mitigate these risks. ## The responsible AI dimensions AWS frames responsible AI around eight dimensions. Chapter 3 explores each in depth; here is the map: ```{list-table} :header-rows: 1 :widths: 26 74 * - Dimension - In one line * - **Controllability** - Mechanisms to monitor and steer AI system behavior. * - **Privacy & Security** - Appropriately obtaining, using, and protecting data and models. * - **Safety** - Preventing harmful system output and misuse. * - **Fairness** - Considering impacts on different groups of stakeholders. * - **Veracity & Robustness** - Achieving correct outputs, even with unexpected or adversarial inputs. * - **Explainability** - Understanding and evaluating outputs generated by an AI system. * - **Transparency** - Enabling stakeholders to make informed choices about engaging with the system. * - **Governance** - Incorporating best practices across the AI supply chain, providers and deployers alike. ``` ## Responsibility spans the whole lifecycle No matter which part of the lifecycle you work on, **design**, **build**, or **operate**, you should always consider responsible AI. **Design.** Discuss the use case with diverse stakeholders, evaluate whether AI actually adds value, and conduct a thorough risk assessment of the proposed use case. **Build.** Verify training data is safe, relevant, and representative; consider legal requirements (licensing, privacy, consent); use metrics with confidence intervals to evaluate outcomes; and apply safeguards, value alignment, and (where appropriate) model disgorgement to mitigate risk. **Operate.** Give end users a way to inquire about outputs for high-risk use cases and be transparent about limitations; check for **model drift** as the world changes; and ensure the model is used as intended (a model trained on US data should be used for the US context). ## Assessing the risk of an AI application Risk assessment is a structured, three-step process: 1. **Define** the use case and the relevant stakeholders. 2. **Identify** harmful events and evaluate both **inherent** and **residual** risk. 3. **Summarize** risk levels across all dimensions and conclude findings. ### Quantifying risk: likelihood and severity Following the NIST AI Risk Management Framework, risk is quantified along two axes: - **Likelihood**: how probable an event is. - **Severity**: the magnitude of its consequences. Each is scored on a scale. Likelihood runs from *Highly unlikely* (less than once per decade) through *Possible* to *Frequent* (more than 100 times a year). Severity, for a given dimension, runs from *Very low* to *Extreme*. For the veracity dimension, for instance, *Very low* severity is negligible hallucination, while *Extreme* is persuasive, dangerous output causing irreversible real-world harm. Combining the two in a matrix yields an overall rating: ```{list-table} Likelihood x Severity (illustrative) :header-rows: 1 :widths: 20 16 16 16 16 16 * - Likelihood \\ Severity - Very low - Low - Moderate - Major - Extreme * - **Frequent** - Low - Medium - High - Critical - Critical * - **Possible** - Very Low - Low - Medium - High - Critical * - **Highly unlikely** - Very Low - Very Low - Very Low - Low - High ``` ```{admonition} Worked example: a medical triage assistant :class: note For a symptom-checking assistant, a veracity failure (a confident but wrong suggestion) is *Possible* in likelihood and *Major-to-Extreme* in severity, landing it at **High** or **Critical** risk. That rating tells you to add human oversight, strong disclaimers, and tight guardrails before launch, the techniques of Chapter 4. ``` ## The NIST AI Risk Management Framework 1.0 The likelihood-and-severity approach above comes from the **NIST AI Risk Management Framework (AI RMF 1.0)**, a voluntary framework published by the U.S. National Institute of Standards and Technology in January 2023 to help organizations manage the risks of AI systems. It is worth knowing in its own right because it has become a common reference point for responsible-AI governance. ### Characteristics of trustworthy AI The framework defines AI risk as a function of the **likelihood** of an event and the **magnitude (severity) of its impact**, and it organizes "trustworthiness" into seven characteristics. They map closely onto this module's dimensions: ```{list-table} :header-rows: 1 :widths: 42 58 * - NIST trustworthiness characteristic - Related dimension in this book * - Valid and reliable - Veracity and robustness ({doc}`03-dimensions-of-responsible-ai`) * - Safe - Safety * - Secure and resilient - Privacy and security; robustness * - Accountable and transparent - Transparency; governance * - Explainable and interpretable - Explainability * - Privacy-enhanced - Privacy and security * - Fair, with harmful bias managed - Fairness ``` ### The four core functions The AI RMF organizes practice into four functions, which align with the design-build-operate lifecycle above: ```{list-table} :header-rows: 1 :widths: 20 80 * - Function - What it covers * - **Govern** - A culture of risk management: policies, accountability, roles, and oversight that cut across the other three functions. * - **Map** - Establish context and identify risks: the use case, stakeholders, and where harms could arise (the "define and identify" steps above). * - **Measure** - Analyze, assess, and track risks using quantitative and qualitative methods (the evaluation of {doc}`01-evaluating-llms` and the likelihood x severity rating). * - **Manage** - Prioritize and act on risks: allocate resources, apply mitigations (guardrails, oversight), and monitor over time. ``` ```{admonition} Why it matters here :class: note Using the AI RMF as scaffolding means the responsible-AI work in this module, evaluation, the dimensions, and the security-and-safety techniques, lines up with a recognized national framework, which is exactly what auditors, funders, and regulators increasingly expect. Consult the official framework () for the authoritative text; details and companion profiles are periodically updated. ``` ## In the news Responsible AI has moved from voluntary principle to emerging regulation, with frameworks such as the EU AI Act and the NIST AI Risk Management Framework shaping how organizations classify and govern AI by risk level. The dimensions in this chapter map closely onto these regimes, which is why treating responsible AI as a governance practice, rather than a feature, increasingly aligns with legal obligation, not just good intentions. ## Key takeaways - **Responsible AI** maximizes benefit and minimizes harm, defined through evolving **dimensions** and practiced organization-wide. - It spans the **design-build-operate** lifecycle; everyone is responsible. - **Risk assessment** is define -> identify and evaluate -> summarize, with risk quantified as **likelihood x severity** per the NIST framework. - **Governance** ties the practice together across teams. Next, we examine each responsible-AI dimension in detail.