Adversarial AI & Machine Learning

Adversarial attacks are typically carried out with the intention of causing a malfunction in a machine learning model. This can involve feeding the model inaccurate or misleading data during its training phase or introducing maliciously crafted data to deceive an already trained model.

The main goal of these attacks is to undermine the reliability of the model and compromise the accuracy of its decision-making process. In some cases, adversarial attacks are even aimed at extracting sensitive training data or gaining insights into the inner workings of a model.

As researchers and developers in this field, it is crucial for us to stay vigilant against such threats and continuously enhance our defence against adversarial attacks. By understanding and mitigating these risks, we can ensure that AI and machine learning models remain trustworthy and reliable tools in various domains.

• Face identification system evasion by bypassing the AI based facial recognition system
• Email protection system subversion by building a copy-cat email protection ML model
• A universal bypass string that evades detection by AI malware detector when appended to a malicious file

These are just a few examples of a wide variety of internal and external threats that AI and ML models can be exposed to.

Adversarial Attack Landscape

• Evasion
  Evasion is the most common attack on the machine learning model performed during inference. It refers to designing an input, which seems normal for a human but is wrongly classified by ML models.
• Poisoning
  Poisoning attacks change classification boundary using adverse data during training.
• Extraction
  The goal of this attack is to know the exact model or even a model’s hyperparameters. This information can be useful for attacks like evasion in the black-box environment.
• Inference
  Attribute inference (guessing a type of data) and membership inference (data examples) are vital not only due to privacy issues but also as an exploratory phase for evasion attacks.

Attacks against Generative AI, Classification and Regression Models

• Generative AI models can be vulnerable to carefully crafted inputs designed to deceive the model into producing incorrect or unexpected outputs.
• Model poisoning: Adversaries can attempt to poison the training data used for generative AI models influencing the model’s behavior and cause it to generate harmful or biased outputs.
• Model inversion attacks: Adversaries can perform model inversion attacks to extract sensitive information from the generative AI model. By providing inputs and analyzing the generated outputs, they can try to infer private or confidential information used during the model’s training.

Ensuring the robustness of generative AI models against adversarial attacks is a significant challenge. Developing effective defense mechanisms, such as adversarial training or input sanitization techniques, is crucial to mitigate the impact of adversarial attacks.

Five steps to AI Model Security & Compliance

• Detailed model baseline metrics
• Model metrics under adversarial attack simulation
• Automatic retrained models for enhanced robustness
• Centralized model hub for governance with daily model reports
• Cyber security alerts for AI attacks on AI inference end points