ISO 42001: 2023 - A.10.4 Customers and AI Systems

ISO 42001 Control Description

The organisation shall identify and address the needs and expectations of customers in relation to AI systems whose outputs or behaviours affect them, ensuring that customers receive appropriate information about AI systems, that their expectations regarding responsible AI use are considered in the organisation's governance arrangements, and that mechanisms are provided through which customers can raise concerns and seek redress.

Control Objective

To ensure that the organisation appropriately considers and responds to the interests of customers in its AI governance arrangements, provides customers with the transparency information they need to understand how AI systems are being used in relation to them, and maintains accessible channels through which customers can communicate their concerns, expectations, and requirements regarding AI system use.

Purpose

Where an organisation deploys AI systems whose outputs affect customers — whether through automated decision-making, personalised services, recommendation systems, risk assessments, or other AI-mediated interactions — customers have a legitimate interest in understanding how those systems operate, how their data is used, and what recourse is available when they experience adverse outcomes. The failure to adequately engage with customer needs in this context can erode trust, create regulatory exposure, and result in real harm to the individuals most directly affected by AI system outputs.

Customer engagement in the context of AI governance is both an ethical and a practical imperative. Ethically, customers have a right to meaningful transparency about AI systems that affect them and to accessible channels through which they can exercise their interests. Practically, customer feedback about AI system behaviour represents a valuable source of intelligence that can inform improvements to system design, operation, and governance.

This control recognises that the organisation's relationship with its customers is not a peripheral concern in AI governance but is central to achieving outcomes that are genuinely responsible. It also recognises that customer expectations and requirements regarding AI may be shaped by applicable regulations — such as data protection legislation governing automated decision-making — and that the organisation's customer engagement arrangements must be sufficient to meet both its voluntary commitments and its legal obligations.

Guidance on Implementation

Identifying Customer AI Interactions

The organisation shall identify the AI systems whose outputs or behaviours have material interactions with customers, including systems that make or inform decisions about customers, systems that personalise content or services for customers, systems that assess customer eligibility, creditworthiness, or risk, and systems that handle customer communications or service interactions. For each identified interaction, the organisation shall understand the nature of the interaction, the potential impacts on customers, and the information and safeguards that customers are entitled to expect.

Understanding Customer Needs and Expectations

The organisation shall take active steps to understand the needs and expectations of customers in relation to AI system use, including expectations regarding transparency and explainability, expectations regarding the fairness and non-discrimination of AI system outputs, requirements for human review of AI-informed decisions, and the importance customers attach to data privacy in AI system contexts.

Customer needs and expectations shall be considered in the design of AI system governance arrangements and in the definition of responsible use objectives. Where customer expectations exceed the organisation's baseline commitments, the organisation shall assess whether those expectations should be addressed through enhanced governance measures. Where regulatory requirements establish minimum entitlements for customers, the organisation shall ensure that its arrangements meet or exceed those requirements.

Customer Transparency and Communication

The organisation shall provide customers with clear, accessible information about AI systems that affect them. Transparency information shall address the role that AI systems play in decisions or interactions that affect the customer; the types of data used by the AI system in relation to the customer; the significant factors that the AI system takes into account; the limitations and known risks of the AI system relevant to the customer's interaction; and the customer's rights in relation to AI-informed decisions, including any right to request human review.

Transparency information shall be communicated through channels and in formats that are appropriate to the customer relationship and to the nature of the AI interaction. The organisation shall avoid presenting AI transparency information in a manner that is overly technical or difficult for affected customers to understand.

Customer Feedback and Complaints Channels

The organisation shall maintain accessible mechanisms through which customers can provide feedback about AI system behaviour, raise concerns about AI-informed decisions that have affected them, and request review of decisions in which AI has played a role. These mechanisms shall be clearly communicated to customers and shall be operated in a timely manner proportionate to the nature and impact of the concern raised.

Where customers raise concerns that indicate a potential deficiency in AI system performance, a fairness issue, or an adverse impact, these concerns shall be assessed through the organisation's feedback and incident management processes. The outcomes of customer concern handling shall be communicated to the customer concerned and, where appropriate, shall inform improvements to AI system governance.

Managing Customer Contractual Expectations

Where the organisation provides AI-related services or products to business customers, the organisation shall consider whether formal contractual provisions are appropriate to address the customer's requirements regarding responsible AI use, the performance standards to which the AI system will be held, incident notification obligations, and the organisation's obligations in the event that the AI system fails to operate as intended. Contractual provisions shall be consistent with the organisation's responsible AI standards and shall not purport to transfer responsibilities to the customer that appropriately rest with the organisation.

Integration with Regulatory Obligations

Customer engagement arrangements under this control shall be reviewed for consistency with applicable regulatory obligations. Where legislation governing automated decision-making, data protection, or sector-specific AI use creates specific customer entitlements — such as rights to explanation, rights to challenge automated decisions, or rights to human review — the organisation's customer arrangements shall be designed to ensure that these entitlements can be exercised effectively.

Related Controls

• A.8.4 – Feedback and Improvement: Customer feedback mechanisms established under this control shall be integrated with the broader feedback and improvement process, ensuring that customer concerns and experiences are systematically assessed and acted upon.
• A.9.2 – Processes for Responsible Use of AI Systems: Responsible use processes shall reflect customer requirements and expectations, ensuring that the manner in which AI systems are used aligns with the commitments communicated to customers.
• A.10.2 – Allocating Responsibilities for AI Systems: Responsibility allocations shall address the respective obligations of the organisation and any third parties involved in customer-facing AI system use.
• A.10.3 – Supplier Requirements for AI Systems: Where AI systems used in customer interactions involve third-party components, supplier requirements shall address the standards of responsible AI that the organisation has committed to its customers.
• A.5.4 – Human Oversight of AI Systems: Human oversight arrangements shall reflect the oversight entitlements of customers, particularly in contexts where customers have a right to request human review of AI-informed decisions.