The development of Artificial Intelligence (AI) systems is a fascinating process that is manifesting itself in many areas, bringing increased efficiency and productivity as well as improved customer service. However, the field faces a number of challenges, such as ethical issues, transparency and trust, cybersecurity, and concerns about employment impacts. Therefore, organisations should support initiatives related to the creation or use of AI systems from the outset.
ISO/IEC 42001 standard
At the end of 2023, a new standard ISO/IEC 42001:2023 Information Technology – Artificial Intelligence – Management System was issued for all organizations that provide or use products or services based on artificial intelligence.
The standard contains Chapters 4 to 10 specifying requirements for the creation, implementation, maintenance and continuous improvement of an artificial intelligence management system (AIMS). Among other things, it also contains a normative Annex A – Reference Objectives for Actions and Measures to be used by organisations to manage risk and achieve AI objectives.
Gap analysis of ISO/IEC 42001 compliance and its benefits
By assessing compliance with the above requirements, an organisation can evaluate whether and to what extent it is compliant, where there are gaps (non-conformities) and where there are opportunities for improvement. It can also determine the effectiveness or maturity of existing practices. All of this is collectively referred to as a gap analysis or gap analysis with the following benefits.
- Identification and understanding of the current status and maturity of the management system and control measures in relation to relevant requirements
- Optimisation of planning for the implementation of the management system and control measures
- Evaluation of progress in the development of the management system and control measures
- Effective use of outputs in the risk management process
Differential analysis procedure
The process of difference analysis consists of 5 subsequent steps.
- Determination of context and scope
- Identification and assessment of the current situation
- Definition of the target state
- Specification of the steps to achieve the target state
- Consolidation of information and final report
The purpose of setting the context is to define the objectives and scope of the difference analysis.
The purpose of the identification and assessment of the current status is to gather information and document the current status of the management system or control measures and to assess their level of maturity. The determination of the level of maturity of the process or control measures shall be made using a quantitative hierarchical scale. In practice, the scale is usually expressed as a 0-5 scale, with the higher the number, the higher the maturity level. It is important that the maturity assessment provides objective evidence of compliance at the appropriate level.
The purpose of defining a target state is to briefly describe the desired target state with respect to the relevant requirements and to set its target level of maturity.
The purpose of specifying the steps to reach the target state is to identify the gaps between the current and target state and to determine the actions needed to bridge the gaps and move to the target state.
The purpose of the consolidation is to review and confirm the results of the difference analysis and to draw conclusions from it. The results and conclusions of the gap analysis should form part of the final report. It is good practice to present the results of the gap analysis graphically, which gives an indication at a glance of where the biggest gaps are and where attention should be focused.
Maturity level
Determining the maturity level of processes or measures is a good practice to help organisations understand their current level of performance. There are several models, such as the Capability Maturity Model Integration (CMMI), which defines 5 maturity levels:
- Initial: Processes and measures are undefined and chaotic.
- Managed: processes and measures are planned, implemented and controlled.
- Defined: Processes and measures are described and standardized.
- Quantitatively Managed: processes and measures are measured and analyzed.
- Optimizing: Processes and measures are continuously improved.
Example of an entry in a difference analysis
| ID | Topic | Controls | Brief description of the status | Maturity | Key change factor | Priority |
|---|---|---|---|---|---|---|
| A.2.2 | AI Policy | The organisation must document a policy for the development or use of AI systems. | The organization has approved and communicated the general IA policy as per clause 5.2 of the standard. However, the AI policy does not address impacts on relevant stakeholders or topic-specific aspects (AI asset management, AI systems impact assessment, AI systems development, etc.). | 1 | Document, approve and communicate topic-specific AI policies. | 1 (disagreement) |