In response to UNESCO's open consultation on the Draft Guidelines for the Use of AI Systems in Courts and Tribunals, I had the privilege of contributing my thoughts on the role AI could play in shaping the future of judicial systems. As AI technologies increasingly influence key sectors, including law and justice, it is imperative to ensure their application is guided by ethical principles that uphold fairness, transparency, and respect for human rights.

The draft guidelines, rooted in the UNESCO Recommendation on the Ethics of AI, aim to create a framework for integrating AI in a way that strengthens the rule of law, enhances access to justice, and safeguards judicial integrity. See my responses below which focus on these key areas and provide suggestions on how AI can be responsibly utilized within courts and tribunals, fostering trust and accountability. It’s important to note that the numbering in this article does not correspond directly with the survey, as my responses are organized around the relevant themes and questions.

SECTION 1 Principles for the use of AI systems in courts and tribunals

1. Are the principles for the development, procurement and deployment of AI systems to be followed by organizations of the justice sector adequate? 

• Yes 

2. Are the principles for the use of AI systems to be followed by judicial operators adequate?

• No

Rationale –

While the guidelines provide a solid foundation for AI adoption in the judiciary, they could be further strengthened by offering more concrete guidance on specific use cases. For instance, explicit protocols for handling cases where AI-generated outputs directly influence judicial decisions would enhance clarity and ensure ethical practices.

Also, it would be beneficial for the guidelines to explicitly state the consequences for violating these principles. This would ensure accountability and reinforce the importance of adhering to the established standards, thereby safeguarding the integrity of judicial processes and the ethical use of AI systems.

3. Should we consider adding or deleting some principles?

• Yes

Rationale –

Consider adding principles related to ethical review and compliance - This principle is will ensure that AI systems exploited in courts and tribunals undergo regular ethical reviews and that compliance with ethical standards is continuously monitored. This would help to address the fast-paced advancements in AI technology and the ethical challenges they may present.

Moreover, public engagement and transparency is key to build trust and emphasise transparency with the public regarding the use of AI in judicial processes. This could include public reporting on AI system performance, impact assessments, and steps taken to mitigate risks, which would foster public trust.

Finally, stipulating clear consequences for non-compliance by the judiciary would explicitly state the consequences for failing to adhere to the guidelines. This would enhance accountability and ensure that violations are met with appropriate corrective actions.

4. Should there be specific principles for generative AI systems?

• Yes

Rationale –

Yes, there should be specific principles for generative AI systems due to their unique risks and challenges. These systems can produce convincing but potentially inaccurate or biased outputs, necessitating tailored guidelines for their use in judicial contexts. The Rationales are outlined below. 

The use of generative AI systems require clear guidelines on transparency and accountability in crafting legal documents, judicial decisions, and providing legal analysis such as labelling AI-generated content and ensuring it can be audited and traced back to its source. 

In addition, generative AI systems can unintentionally infringe on intellectual property rights or privacy laws, making it essential to have specific principles that address these legal and ethical issues.

Moreover, there is a risk that judicial operators might over-rely on generative AI outputs, leading to diminished human oversight and critical thinking. Specific principles can help mitigate this risk by emphasising the importance of human judgment in all AI-assisted judicial decisions. In addition, any output generated by AI systems in judicial contexts must be thoroughly verified by a human operator before it is used in any official capacity. This includes cross-checking facts, legal precedents, and ensuring the AI's output aligns with established legal standards.

SECTION 2 Specific guidance for organizations that are part of the judicary

5. Are there any other rules or standards that organizations should follow with regard to the development, procurement and deployment of AI systems?

• Yes

Rationale –

When developing, procuring, and deploying AI systems within the judiciary, it is crucial to adhere to additional rules and standards that address the unique responsibilities and sensitivities of the judicial context. Ethical AI development and design should be a priority, with a strong emphasis on mitigating biases in algorithms by using diverse and representative datasets. Establishing ethical AI principles that align with judicial values and legal obligations can ensure that these systems support the integrity of judicial processes.

In the procurement phase, judiciary systems must enforce vendor accountability by ensuring that third-party providers comply with stringent legal, ethical, and technical standards. This can be achieved through rigorous contract terms and regular compliance checks, including requiring third-party audits of AI systems to validate their reliability, fairness, and adherence to legal standards before procurement.

Judiciary AI systems should be designed with a human-centred approach, focusing on enhancing, rather than replacing, human judgment. This includes maintaining a human oversight, where AI assists but does not substitute human judgement, particularly in critical legal determinations. Also, data security and privacy are critical in the judicial context. All data handled by AI systems must be encrypted both at rest and in transit, and a data minimisation approach should be adopted to reduce the risk of breaches by collecting only the necessary data for the system’s operation.

Transparency and accountability are essential in maintaining public trust in the judiciary. This requires clear documentation of AI systems, including development processes, data sources, and decision-making logic, to facilitate audits and ensure transparency. Transparent communication with all stakeholders about the capabilities, limitations, and potential risks of AI systems is equally important to ensure informed usage within the judiciary.

Ethical governance should be reinforced by establishing an AI ethics committee within the judiciary to oversee AI development and deployment, ensuring adherence to ethical guidelines and addressing any dilemmas that arise. Regular ethical impact assessments should be conducted to evaluate how AI systems affect judicial processes and the broader society. Furthermore, stakeholder engagement is vital; judiciary systems should actively involve the legal community, civil society, and other relevant stakeholders to gather diverse perspectives on the AI systems being implemented.

Finally, contingency planning is critical in the judiciary context. Judiciary systems must develop robust disaster recovery plans to address potential AI system failures or errors, ensuring that there are clear protocols for human intervention and corrective actions. Building resilience into AI systems to adapt to unexpected challenges, such as changes in legal frameworks or societal norms, is also necessary to ensure reliability of these technologies in the administration of justice.

6. Should there be any specific rules or standards for generative AI systems?

• Yes

Rationale –

Yes, there should be specific rules and standards for generative AI systems within the judiciary due to their unique capabilities and the potential risks they pose. Generative AI systems, particularly those based on large language models, can generate highly convincing outputs, but these outputs may sometimes contain inaccuracies, biases, or even fabricated information. Given the critical nature of judicial decisions, it is essential to establish clear guidelines to ensure these tools are used responsibly and ethically.

Firstly, there should be stringent verification and validation requirements. Any output generated by a generative AI system must be thoroughly reviewed and validated by a human operator before it is used in any official judicial capacity. This includes cross-checking facts, legal precedents, and ensuring that the AI’s output is consistent with established legal standards. Such a measure is crucial to prevent errors or misleading information from influencing judicial outcomes.

Transparency and disclosure are also vital. Judicial operators should be required to disclose when generative AI has been used in drafting documents or making decisions as stated in the guidelines. This transparency helps maintain public trust and ensures that all stakeholders understand the role AI played in the judicial process. AI-generated content should be clearly labeled, and the processes used to create it should be documented and accessible for audit and review.

Bias and fairness assessments should be conducted regularly. Generative AI systems need to be evaluated periodically to identify and mitigate any biases in their outputs. Given that these systems learn from large datasets, which may contain biased information, continuous monitoring is necessary to ensure that the AI does not perpetuate or exacerbate discrimination within the judicial process.

Additionally, there should be restrictions on the use of generative AI in decision-making. Generative AI should not be used to make final or binding legal decisions. Instead, its role should be limited to supporting human decision-making by providing drafts or suggestions that are always subject to human review and modification. This ensures that the ultimate responsibility for judicial decisions remains with human judges and legal professionals.

Finally, ethical compliance is essential. The use of generative AI in the judiciary must align with strict ethical standards, including respect for privacy, confidentiality, and intellectual property rights. Any AI-generated content must be handled with the same level of care as human-generated content, ensuring that it adheres to the legal and ethical norms of the judiciary.

Overall, specific rules and standards for generative AI systems in the judiciary are necessary to address the unique risks these technologies pose.

SECTION 3 Specific guidance for individual members of the judicary

7. Are there any other rules or standards that individual members of the judiciary should follow with regard to the use of AI systems?

• Yes

Rationale –

Individual members of the judiciary should adhere to additional rules and standards when using AI systems to ensure their responsible and ethical integration into judicial processes. Firstly, it is crucial for judicial members to engage in continuous education and training on AI technologies to stay updated on their capabilities, limitations, and developments. This includes developing the skills necessary to critically evaluate AI-generated outputs, ensuring that reliance on AI does not overshadow traditional legal reasoning and human judgment. While AI can assist in decision-making, ultimate responsibility must always remain with the judicial officers, who should use AI as a tool to complement, not replace, human analysis.

Ethical use and integrity are also important. Judicial members must ensure that their use of AI aligns with the highest ethical standards, maintaining fairness, transparency, and impartiality in all judicial processes. Confidentiality is another critical concern, particularly when interacting with AI systems developed by third-party vendors. Ensuring that AI systems do not expose sensitive information to unauthorised parties is essential in maintaining trust in the judicial system. Transparency in decision-making is equally important; judicial members should document and disclose when and how AI was used in their processes, making sure that all relevant parties are aware of AI’s role in the outcome.

To prevent over reliance on AI, judicial members should balance its use with traditional legal analysis, carefully reviewing AI outputs and cross-verifying them with reliable sources. AI should never be accepted at face value without independent verification. Moreover, there should be a strong focus on accountability, where judicial members are responsible for any decisions influenced by AI and must correct any errors or biases introduced by these systems. 

Regular bias checks are necessary to ensure that AI systems do not perpetuate discrimination or unfairness, especially concerning protected characteristics such as race, gender, migration status, and socioeconomic status. Judicial members should promptly report any biases or flaws they identify in AI systems and refrain from using those systems until the issues are resolved. Also, ethical impact assessments should be conducted regularly to evaluate how the use of AI affects access to justice, fairness, and public trust in the judiciary. Feedback loops should be established so that judicial members can contribute to the improvement of AI systems based on their experiences.

Lastly, it is important for judicial members to follow established protocols and guidelines regarding AI use, reporting any incidents or malfunctions promptly to ensure that corrective actions can be taken. 

8. Should there be specific rules or standards for the use of generative AI systems by individuals? 

• Yes

Rationale –

Yes, the same principles mentioned earlier are relevant here, including transparency, fairness, human oversight, data protection, training and awareness, restricted use in sensitive contexts, bias and fairness, transparency and disclosure, accuracy and reliability, and safety. These principles are essential to ensure the responsible, ethical, and effective use of generative AI systems by individuals, particularly in professional and sensitive environments such as the judiciary.

9. How would you use these guidelines?

To effectively use these guidelines, I would integrate them into my daily work with AI systems. I will prioritise fairness, regularly checking AI outputs for biases and taking corrective actions where  needed. Maintaining human oversight is crucial, so I will use AI to support, not replace, my judgment, and always verify AI-generated content before using it officially. I will protect data by implementing strong security measures and comply with privacy regulations. I will continuously engage in training to understand AI's capabilities and limitations, ensuring I use it responsibly. In sensitive contexts, I will restrict AI use and ensure rigorous human review of any outputs. Transparency, accuracy, and safety will guide my approach, ensuring that AI-generated content is reliable and ethically sound.