AAT-24: AI & Autonomous Technologies Development Practices

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Artificial Intelligence and Autonomous Technology (AAT) domain capabilities are ad hoc and inconsistent. Capability criteria associated with this control may include: ▪ Policies, standards & procedures associated with AAT domain capabilities provide limited coverage due to the depth and breadth of the existing documentation. ▪ AAT-related processes are expected to follow the organization's existing processes (e.g., incident response, asset management, change control and risk assessments). ▪ No formal Governance, Risk & Compliance (GRC) team exists to provide AAT oversight, where the Chief Information Officer (CIO), or similar function, governs technology decisions what is acceptable for AAT within the organization.

Artificial Intelligence and Autonomous Technology (AAT) capabilities are requirements-driven, but are not standardized across the entity (e.g., local/regional level consistency). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are documented and maintained by process owners. ▪ IT and/or cybersecurity personnel work with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address applicable statutory, regulatory and/or contractual requirements for Technology Assets, Applications, Services and/or Data (TAASD). ▪ Artificial Intelligence (AI)-related controls are primarily administrative and preventative in nature (e.g., policies, standards and procedures & guidelines). ▪ Asset management may be a defined function (e.g., team or department) or assigned as an additional duty to existing IT and/or cybersecurity personnel. ▪ AAT is regarded as a technology and governed by the entity's existing IT governance practices. ▪ No formal Governance, Risk & Compliance (GRC) team exists to provide oversight of AAT-related activities. GRC functions are assigned to existing IT and/or cybersecurity personnel.

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ Measures exist to ensure AAT are designed and developed to: (1) Achieve an appropriate level of accuracy, robustness and cybersecurity; (2) Perform consistently in those respects throughout the AAT system's lifecycle; and (3) Be effectively overseen by competent individuals.

Artificial Intelligence and Autonomous Technology (AAT) capabilities, in addition to being standardized across the entity and centrally managed to ensure consistency across Technology Assets, Applications, Services and/or Data (TAASD), efforts are metrics driven to provide sufficient insight for decision makers to predict optimal performance, ensure continued operations and/or identify areas for improvement. Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Applicable SCR-CMM Level 3 (Well Defined) capabilities are implemented and operational. ▪ Metrics reporting includes quantitative analysis of Key Performance Indicators (KPIs). ▪ Metrics reporting includes quantitative analysis of Key Risk Indicators (KRIs). ▪ Scope of metrics, KPIs and KRIs covers organization-wide cybersecurity and data protection controls, including functions performed by third-parties. ▪ Organizational leadership maintains a formal process to objectively review and respond to metrics, KPIs and KRIs (e.g., monthly or quarterly review). ▪ Based on metrics analysis, process improvement recommendations are submitted for review and are handled in accordance with change control processes. ▪ Business and technical stakeholders are involved in reviewing and approving proposed changes to evolve capabilities.

Artificial Intelligence and Autonomous Technology (AAT) capabilities are "world class" efforts the leverage predictive analysis (e.g., machine learning and AI) to enable continuously improving capabilities. Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Applicable SCR-CMM Level 3 (Well Defined) capabilities are implemented and operational. ▪ Based on predictive analysis, process improvements are implemented according to “continuous improvement” practices that affect process changes. ▪ Stakeholders make time-sensitive decisions to support operational efficiency, which may include automated remediation actions via Artificial Intelligence (AI) or Machine Learning (ML) technologies.