Auditing Artificial Intelligence
Business is constantly evolving. As part of evolution, businesses are turning to AI to improve and perfect their operations. AI is being used in a variety of ways to improve efficiency, save time and decrease costs. AI is constantly advancing and becoming a resource for companies in all industries. Auditors play a key role in ensuring the security, compliance, effectiveness and privacy of AI when used by organizations. During this course, auditors will learn the skills needed to effectively audit AI. Topics such as the fundamentals of AI, challenges and risks introduced by AI, and the key steps auditors should take to audit AI will be addressed. Students will learn about AI governance, AI management, AI policies, and data audits including the assessment of Large Language Models (LLM). By developing the skills necessary to audit AI, students will be able to assist their organizations to ensure the AI strategy being pursued is addressed in a secure, and effective manner.
CPE:
14
Program Level:
Intermediate
Delivery Method:
Live
Cost :
$675 for this 2 day course
Course Contents/Objectives
I. Introduction
• AI Background
• AI Terms
• Understand machine learning models, AI models, and Gen AI models
• Data sources for AI
• Deployment strategies
• Feature engineering
• Types of AI available
• LLMOps Framework
• AI Lifecycle
• AI Bias
• Adversarial Training
• Trustworthy and Responsible AI Principles
II. Understanding AI Prompts/Training
• Building prompts
• Common AI Prompt Mistakes
• Optimizing LLMs
III. Managing risk related to AI
• AI versus Traditional Software Risks
• Positive and negative implications of AI
• Risk management techniques
• AI standards and policies
IV. Challenges for AI Risk Management
• Understanding an organization’s AI vision and strategy
• Risk measurement
• Risk tolerance
• Risk prioritization
• Harmful bias
• Unintended/unpredictable consequences
• Security concerns
• Understanding the attack surface
• Third party risks
V. Auditor’s role in Evaluating Efficiency and Bias in AI
• Addressing bias in AI
• Data quality
• AI KPIs
• Efficiency
• Effectiveness
• Business impact
• Technical performance
• Fairness/compliance
• Harmful bias identification
• Monitoring and Auditing
VI. AI Security
• OWASP Top 10 for LLM Applications
• Prompt Injection
• Sensitive info disclosure
• Supply chain
• Data and Model poisoning
• Output handling
• Excessive agency
• System prompt leakage
• Vector and embedding weaknesses
• Misinformation
• Unbounded consumption
• OWASP API Security Top 10
• AI Vulnerability repositories
• AI policies
• AI committee
• Ethical AI practices
• Data protection controls
• Validation and Training
• Access controls
• Audit security checklists for LLM applications and GenAI
VII. AI Auditing and Risk Management Frameworks
• Govern, Map, Measure, Manage
• NIST AI Risk Management Playbook
• Controls for mitigating AI risk
• Well-architected Machine Learning Design Principles
• Well-architected frameworks
VIII. Conclusion
Laptop Required
Students are required to have a laptop in order to complete the hands-on exercises. The laptop should meet the following specifications for the student to get the most from the exercises:
• 16 GB RAM or higher
• 25 GB available hard drive space
• Windows 10 professional or later
• Administrator privileges including the capability to install and run tools, as well as disable anti-virus
• VMWare Player should be installed
Target Audience
• Internal Auditors
• IT Specialist Auditors
• IT Auditors
• IT Audit Managers
• Information System Auditors
• Information Technology Auditors
• Information Security Officers
• Developers
Prerequisites
Participants should be familiar with Internet technologies and commonly used Internet security controls. No other advance preparation is required.
What People Say About Us
We help you see the world differently, discover opportunities you may never have imagined and achieve results that bridge what is with what can be.
Kendall
Ashley
Joel