Artificial intelligence bias auditing – current approaches, challenges and lessons from practice

What the paper says

Purpose This study aims to explore current approaches, challenges and practical lessons in auditing artificial intelligence (AI) systems for bias, focusing on legal compliance audits in the USA and the European Union (EU). This emphasizes the need for standardized methodologies to ensure trustworthy AI systems that align with ethical and regulatory expectations. Design/methodology/approach A qualitative analysis compared bias audit practices, including US bias audit report summaries under New York City’s Local Law 144 and conformity assessments (CAs) required by the EU AI Act. Data was gathered from publicly available reports and compliance guidelines to identify key challenges and lessons. Findings The findings revealed that AI systems are susceptible to various biases stemming from data, algorithms and human oversight. Although valuable, legal compliance audits lack standardization, leading to inconsistent reporting practices. The EU’s risk-based CA approach offers a comprehensive framework; however, its effectiveness depends on developing practical standards and consistent application. Research limitations/implications This study is limited by the early implementation stage of regulatory frameworks, particularly the EU AI Act, and restricted access to comprehensive audit reports. A geographic focus on US and EU jurisdictions may limit the generalizability of the findings. Data availability constraints and the lack of standardized reporting frameworks affect the comparative analysis. Future research should focus on longitudinal studies of audit effectiveness, the development of standardized methodologies for intersectional bias assessment and the investigation of automated audit tools that can adapt to emerging AI technologies while maintaining practical feasibility across different organizational contexts. Practical implications This research underscores the necessity of adopting socio-technical perspectives and standardized methodologies in AI auditing. It provides actionable insights for firms, regulators and auditors into implementing robust governance and risk assessment practices to mitigate AI biases. Social implications Effective AI bias auditing practices ensure algorithmic fairness and prevent discriminatory outcomes in critical domains like employment, health care and financial services. The findings emphasize the need for enhanced stakeholder engagement and community representation in audit processes. Implementing robust auditing frameworks can help close socioeconomic gaps by identifying and mitigating biases disproportionately affecting marginalized groups. This research contributes to developing equitable AI systems that respect diversity and promote social justice while maintaining technological advancement. Originality/value This study contributes to the discourse on AI governance by comparing two regulatory approaches, bias audits and CAs and offers practical lessons from current implementation. It highlights the critical role of standardization in advancing trustworthy and ethical AI systems in the finance and accounting contexts.