IAO-15: Control Assurance Automation (CAA) Automation
Mechanisms exist to utilize Control Assurance Automation (CAA) engineering practices, pipelines and tooling to automate applicable Control Validation Testing (CVT) through: (1) Machine-executable tests; and (2) Automated evidence collection and evaluation.
Control Question: Does the organization utilize Control Assurance Automation (CAA) engineering practices, pipelines and tooling to automate applicable Control Validation Testing (CVT) through: (1) Machine-executable tests; and (2) Automated evidence collection and evaluation?
General (2)
| Framework | Mapping Values |
|---|---|
| NIST 800-53 R5.2 | SI-06(02) |
| NIST 800-82 R3 | SI-06(02) |
APAC (1)
| Framework | Mapping Values |
|---|---|
| APAC Malaysia RMiT 2025 | 10.6 10.14 |
Capability Maturity Model
Level 0 — Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 — Performed Informally
Information Assurance (IAO) domain capabilities are ad hoc and inconsistent. Capability criteria associated with this control may include: ▪ Policies, standards & procedures associated with IAO domain capabilities provide limited coverage due to the depth and breadth of the existing documentation. ▪ Pre-production security testing-related activities are decentralized (e.g., a localized/regionalized function) and uses non-standardized methods to implement secure, resilient and compliant practices. ▪ IT and/or cybersecurity personnel implement and maintain an informal process to conduct limited control testing of High Value Assets (HVAs) to meet specific statutory, regulatory and/or contractual requirements for pre-production cybersecurity and data protection control testing.
Level 2 — Planned & Tracked
Information Assurance (IAO) capabilities are requirements-driven, but are not standardized across the entity (e.g., local/regional level consistency). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with IAO domain capabilities are formally documented and centrally-managed by the entity. ▪ Standardized Operating Procedures (SOP) associated with IAO domain capabilities are documented and maintained by process owners. ▪ IT and/or cybersecurity personnel work with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with IAO domain capabilities to address applicable statutory, regulatory and/or contractual requirements for Technology Assets, Applications, Services and/or Data (TAASD). ▪ Information Assurance (IA)-related controls are primarily administrative and preventative in nature (e.g., policies, standards and procedures & guidelines). ▪ IA management may be a defined function (e.g., team or department) or assigned as an additional duty to existing IT and/or cybersecurity personnel.
Level 3 — Well Defined
Information Assurance (IAO) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with IAO domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with IAO domain capabilities are well-documented and kept current by process owners. ▪ An information assurance team, or similar function, is appropriately staffed and supported to implement and maintain IAO domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of information assurance operations (e.g., assessment scheduling software and risk assessment software). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with IAO domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
Level 4 — Quantitatively Controlled
Information Assurance (IAO) capabilities, in addition to being standardized across the entity and centrally managed to ensure consistency across Technology Assets, Applications, Services and/or Data (TAASD), efforts are metrics driven to provide sufficient insight for decision makers to predict optimal performance, ensure continued operations and/or identify areas for improvement. Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Applicable SCR-CMM Level 3 (Well Defined) capabilities are implemented and operational. ▪ Metrics reporting includes quantitative analysis of Key Performance Indicators (KPIs). ▪ Metrics reporting includes quantitative analysis of Key Risk Indicators (KRIs). ▪ Scope of metrics, KPIs and KRIs covers organization-wide cybersecurity and data protection controls, including functions performed by third-parties. ▪ Organizational leadership maintains a formal process to objectively review and respond to metrics, KPIs and KRIs (e.g., monthly or quarterly review). ▪ Based on metrics analysis, process improvement recommendations are submitted for review and are handled in accordance with change control processes. ▪ Business and technical stakeholders are involved in reviewing and approving proposed changes to evolve capabilities.
Level 5 — Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
Assessment Objectives
- IAO-15_A01 the Control Validation Testing (CVT) activities that are applicable for automation through Control Assurance Automation (CAA) are identified and documented, including the controls and Technology Assets, Applications and/or Services (TAAS) in scope.
- IAO-15_A02 Control Assurance Automation (CAA) engineering practices, pipelines and tooling are used to execute machine-executable tests that validate the implementation and operation of in-scope controls.
- IAO-15_A03 Control Assurance Automation (CAA) pipelines and tooling automatically collect evidence from in-scope Technology Assets, Applications and/or Services (TAAS) and evaluate it against defined control expectations.
- IAO-15_A04 evidence and test results produced by Control Assurance Automation (CAA) are timestamped, attributable to the source Technology Assets, Applications and/or Services (TAAS) and protected from unauthorized modification.
- IAO-15_A05 machine-executable tests are version controlled and are reviewed and updated when the controls, Technology Assets, Applications and/or Services (TAAS) or evaluation criteria they test change.
- IAO-15_A06 failed or inconclusive Control Assurance Automation (CAA) test results are reviewed by assigned personnel and tracked to resolution.
Evidence Requirements
- E-CPL-11 Control Assurance Automation (CAA) Governance
-
Documented evidence that Control Assurance Automation (CAA) pipelines, tooling and automated evidence collection are governed under the organization's technology governance and risk oversight.
Compliance
Technology Recommendations
Micro/Small
- Manual Controls Validation Testing (CVT)
- Cloud provider security posture reports (e.g., Microsoft Secure Score, AWS Security Hub)
Small
- Cloud Security Posture Management (CSPM) built-in checks
- Compliance automation platform (e.g., Vanta, Drata, Secureframe)
Medium
- Compliance automation platform (e.g., Vanta, Drata, Secureframe, Hyperproof)
- CSPM (e.g., Wiz, Prisma Cloud, Microsoft Defender for Cloud)
- Automated configuration compliance scanning (e.g., CIS-CAT, Nessus)
Large
- Continuous Controls Monitoring (CCM) integrated with GRC
- Policy-as-code testing (e.g., Open Policy Agent, InSpec, Checkov)
- CSPM/CNAPP with compliance frameworks
Enterprise
- Enterprise Continuous Controls Monitoring (e.g., Hyperproof, MetricStream, ServiceNow IRM, Cyturus)
- Policy-as-code and automated CVT pipelines
- CNAPP and CSPM feeding automated evidence
- Integrated assurance reporting to executive management