Introduction
The Australian Taxation Office (ATO) is increasingly leveraging artificial intelligence (AI) to assess non‐compliance risks and conduct tax audits. A recent audit report provides an in‐depth examination of the ATO’s AI governance arrangements—from design through deployment and monitoring. While progress has been made, the report finds that current frameworks are only “partly effective.” This analysis explores how these findings may impact the use case of deploying AI to detect non-compliance in taxpayer audits.
Governance Frameworks in Context of Tax Audits
The report highlights that the ATO has established dedicated structures, such as an AI Governance team and a Data and Analytics Governance Committee. However, gaps in clearly defined enterprise‐wide roles and responsibilities remain. In the context of tax audits, such fragmented oversight can lead to inconsistent application of AI tools, potentially affecting the reliability of non‐compliance risk assessments. Enhanced organisational clarity is essential to ensure that AI-driven models are appropriately governed and that their outputs—used to flag potential audit targets—are both accurate and fair.
Design, Development, and Deployment of AI Models
Design
While the ATO typically identifies relevant business problems and documents stakeholder input, the design process lacks comprehensive documentation. In the context of tax audits, insufficient design rigor—particularly around risk management and ethical, privacy, and legal considerations—could result in models that do not fully capture the nuances of taxpayer behavior. This gap may lead to false positives or negatives in assessing non-compliance risks.
Development
The report notes that although standardised modelling reports (AP455s) and version control (via GitLab) are in use, practices vary in assessing data suitability, bias, and reproducibility. For an AI system aimed at detecting non-compliance, inconsistent practices may compromise model accuracy and fairness. For example, if bias is not adequately mitigated, certain taxpayer groups might be unfairly targeted.
Deployment
The ATO follows its IT change enablement policy during model deployment, yet the lack of clearly defined performance benchmarks and insufficient pre-deployment testing raise concerns about reliability. In the tax audit context, this could mean that models deployed to assess non-compliance risks may not perform consistently over time, potentially affecting both revenue outcomes and public trust.
Risk Management and Ethical Considerations
A core finding of the audit is that key risks—such as the misuse of data and the propagation of bias—are currently managed only partially effectively. For tax audits, these risks are critical. Incomplete ethics assessments (with many AI models lacking full data ethics reviews) may lead to scenarios where AI-driven risk assessments are neither fair nor transparent. This, in turn, could undermine the legitimacy of audit decisions and result in unwarranted scrutiny of compliant taxpayers.
Monitoring, Evaluation, and Reporting
The audit report stresses the need for robust monitoring and evaluation post-deployment. For non-compliance risk assessment, ongoing performance tracking is vital. Without structured evaluation, issues like model drift or degradation in predictive accuracy may go unnoticed, leading to unreliable audit targeting. Enhanced monitoring ensures that the models continue to produce actionable insights that accurately reflect taxpayer behavior.
Implications for AI-Driven Tax Audits
The findings have significant implications for the ATO’s use of AI in assessing non-compliance risks:
- Accuracy and Reliability: Gaps in design, development, and deployment can impact the precision of risk assessments. Inaccurate models may result in either overlooking high-risk cases or flagging compliant taxpayers unnecessarily.
- Fairness and Bias: Inadequate ethical assessments can lead to biased outcomes, potentially skewing audit focus and eroding public trust.
- Transparency and Accountability: Weak governance and poor record keeping reduce transparency, making it difficult for stakeholders to understand how audit decisions are derived.
- Continuous Improvement: Without robust monitoring and evaluation, the ATO may struggle to update or recalibrate its AI systems in response to changing taxpayer behavior or emerging risks.
Collectively, these challenges could affect the overall effectiveness of AI in targeting non-compliance, thus impacting the ATO’s ability to carry out fair and efficient tax audits.
Key Recommendations and the Way Forward
The audit outlines seven key recommendations that, if implemented, could mitigate these risks and enhance the ATO’s AI use case in tax audits:
- Align Implementation Arrangements: Integrate AI strategies with enterprise-wide requirements to support consistency.
- Clarify Roles and Responsibilities: Establish clear accountabilities for AI governance to ensure reliable application in risk assessments.
- Review and Update Risk Management: Enhance controls, particularly for data misuse, to better support non-compliance detection.
- Enhance Ethical Frameworks: Ensure AI systems are reproducible and auditable, aligning them with the data ethics framework.
- Develop Standardised Policies: Create comprehensive policies for the entire AI lifecycle—from design to deployment—to support accurate risk assessments.
- Establish Performance Measurement: Implement robust evaluation mechanisms to continuously monitor model performance in detecting non-compliance.
- Improve Information Management: Strengthen record keeping to support transparency and accountability in audit decision-making.
Conclusion
The ATO’s journey toward an effective AI governance framework has important ramifications for its core use case: assessing non‐compliance risks to conduct tax audits. Addressing the identified gaps—in risk management, ethical assessments, and performance monitoring—is critical to ensuring that AI tools deliver accurate, fair, and transparent outcomes. Strengthening these areas will not only improve audit effectiveness but also enhance public trust in the taxation system.
See how AuditCover uses AI and request a demo today.