Salah GHAMIZI

Salah GHAMIZI

Postdoctoral Researcher in the team of Pr. Le Traon

Selected publications

GAT: Guided Adversarial Training with Pareto-optimal Auxiliary Tasks

Published in ICML2023, 2023

While leveraging additional training data is well established to improve adversarial robustness, it incurs the unavoidable cost of data collection and the heavy computation to train models. To mitigate the costs, we propose Guided Adversarial Training (GAT), a novel adversarial training technique that exploits auxiliary tasks under a limited set of training data. Our approach extends single-task models into multi-task models during the min-max optimization of adversarial training, and drives the loss optimization with a regularization of the gradient curvature across multiple tasks. GAT leverages two types of auxiliary tasks: self-supervised tasks, where the labels are generated automatically, and domain-knowledge tasks, here human experts provide additional labels. Experimentally, GAT increases the robust AUC of CheXpert medical imaging dataset from 50% to 83% and On CIFAR-10, GAT outperforms eight state-of-the-art adversarial training and achieves 56.21% robust accuracy with Resnet-50. Overall, we demonstrate that guided multi-task learning is an actionable and promising avenue to push further the boundaries of model robustness.

Recommended citation: Salah Ghamizi, Jingfeng Zhang, Maxime Cordy et al. "GAT: Guided Adversarial Training with Pareto-optimal Auxiliary Tasks" arXiv preprint arXiv:2302.02907 (2023). https://arxiv.org/pdf/2302.02907

How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attacks

Published in ACL2023, 2023

Natural Language Processing (NLP) models based on Machine Learning (ML) are susceptible to adversarial attacks – malicious algorithms that imperceptibly modify input text to force models into making incorrect predictions. However, evaluations of these attacks ignore the property of imperceptibility or study it under limited settings. This entails that adversarial perturbations would not pass any human quality gate and do not represent real threats to human-checked NLP systems. To bypass this limitation and enable proper assessment (and later, improvement) of NLP model robustness, we have surveyed 378 human participants about the perceptibility of text adversarial examples produced by state-of-the-art methods. Our results underline that existing text attacks are impractical in real-world scenarios where humans are involved. This contrasts with previous smaller-scale human studies, which reported overly optimistic conclusions regarding attack success. Through our work, we hope to position human perceptibility as a first-class success criterion for text attacks, and provide guidance for research to build effective attack algorithms and, in turn, design appropriate defence mechanisms.

Recommended citation: Salijona Dyrmishi and Salah Ghamizi and Maxime Cordy "How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attacks" arXiv preprint arXiv:2305.15587 (2023). https://arxiv.org/pdf/2305.15587

On The Empirical Effectiveness of Unrealistic Adversarial Hardening Against Realistic Adversarial Attacks

Published in S&P2023, 2023

While the literature on security attacks and defense of Machine Learning (ML) systems mostly focuses on unrealistic adversarial examples, recent research has raised concern about the under-explored field of realistic adversarial attacks and their implications on the robustness of real-world systems. Our paper paves the way for a better understanding of adversarial robustness against realistic attacks and makes two major contributions. First, we conduct a study on three real-world use cases (text classification, botnet detection, malware detection)) and five datasets in order to evaluate whether unrealistic adversarial examples can be used to protect models against realistic examples. Our results reveal discrepancies across the use cases, where unrealistic examples can either be as effective as the realistic ones or may offer only limited improvement. Second, to explain these results, we analyze the latent representation of the adversarial examples generated with realistic and unrealistic attacks. We shed light on the patterns that discriminate which unrealistic examples can be used for effective hardening. We release our code, datasets and models to support future research in exploring how to reduce the gap between unrealistic and realistic adversarial attacks.

Recommended citation: Salijona Dyrmishi, Salah Ghamizi and al. "On The Empirical Effectiveness of Unrealistic Adversarial Hardening Against Realistic Adversarial Attacks arXiv:2202.03277 (2022). https://arxiv.org/pdf/2202.03277

A Unified Framework for Adversarial Attack and Defense in Constrained Feature Space

Published in IJCAI2022, 2022

Vulnerability to adversarial attacks is a well-known weakness of Deep Neural networks. While most of the studies focus on single-task neural networks with computer vision datasets, very little research has considered complex multi-task models that are common in real applications. In this paper, we evaluate the design choices that impact the robustness of multi-task deep learning networks. We provide evidence that blindly adding auxiliary tasks, or weighing the tasks provides a false sense of robustness. Thereby, we tone down the claim made by previous research and study the different factors which may affect robustness. In particular, we show that the choice of the task to incorporate in the loss function are important factors that can be leveraged to yield more robust models.

Recommended citation: Simonetto, Thibault, Ghamizi, Salah, et al. "A Unified Framework for Adversarial Attack and Defense in Constrained Feature Space" arXiv preprint arXiv:2112.01156 (2021). https://arxiv.org/pdf/2110.15053

Adversarial Robustness in Multi-Task Learning: Promises and Illusions

Published in AAAI2022, 2022

Vulnerability to adversarial attacks is a well-known weakness of Deep Neural networks. While most of the studies focus on single-task neural networks with computer vision datasets, very little research has considered complex multi-task models that are common in real applications. In this paper, we evaluate the design choices that impact the robustness of multi-task deep learning networks. We provide evidence that blindly adding auxiliary tasks, or weighing the tasks provides a false sense of robustness. Thereby, we tone down the claim made by previous research and study the different factors which may affect robustness. In particular, we show that the choice of the task to incorporate in the loss function are important factors that can be leveraged to yield more robust models.

Recommended citation: Ghamizi, Salah, et al. "Adversarial Robustness in Multi-Task Learning: Promises and Illusions." arXiv preprint arXiv:2110.15053 (2021) https://arxiv.org/pdf/2110.15053

Evasion Attack STeganography: Turning Vulnerability Of Machine Learning To Adversarial Attacks Into A Real-world Application

Published in ICCV - AROW, 2021

Evasion Attacks have been commonly seen as a weakness of Deep Neural Networks. In this paper, we flip the paradigm and envision this vulnerability as a useful application. We propose EAST, a new steganography and watermarking technique based on multi-label targeted evasion attacks. The key idea of EAST is to encode data as the labels of the image that the evasion attacks produce. Our results confirm that our embedding is elusive; it not only passes unnoticed by humans, steganalysis methods , and machine-learning detectors. In addition, our embedding is resilient to soft and aggressive image tampering (87% recovery rate under jpeg compression). EAST out-performs existing deep-learning-based steganography approaches with images that are 70% denser and 73% more robust and supports multiple datasets and architectures.

Recommended citation: Ghamizi, S., Cordy, M., Papadakis, M., & Traon, Y.L. (2021). Evasion Attack STeganography: Turning Vulnerability Of Machine Learning To Adversarial Attacks Into A Real-world Application. Proceedings / IEEE International Conference on Computer Vision. IEEE International Conference on Computer Vision.

A Hybrid Predictive Model for Mitigating Health and Economic Factors during a Pandemic

Published in ERCIM News, 2021

We have developed a machine learning (ML) driven approach, intended to function as an instrumental backup to the economic recovery strategy and ensure granular mitigation of the pandemic’s effects. Our approach is complemented by human-centric modelling of the impacted ecosystem, including social, economic and health aspects. This model-based approach aims to correct the potential lack of data; fine-tuning the ML results and providing better user control. Ultimately, we aim to deliver a decision-making tool that helps find the right balance between health protection and economic recovery.

Recommended citation: Veiber, L., Ghamizi S. and Sottet J. (2021). “A Hybrid Predictive Model for Mitigating Health and Economic Factors during a Pandemic.” ERCIM News 2021 (2021).

Search-based adversarial testing and improvement of constrained credit scoring systems

Published in ESEC/SIGSOFT FSE, 2020

Credit scoring systems are critical FinTech applications that concern the analysis of the creditworthiness of a person or organization. While decisions were previously based on human expertise, they are now increasingly relying on data analysis and machine learning. In this paper, we assess the ability of state-of-the-art adversarial machine learning to craft attacks on a real-world credit scoring system.

Recommended citation: Ghamizi, S., Cordy, M., Gubri, M., Papadakis, M., Boystov, A., Traon, Y.L., & Goujon, A. (2020). Search-based adversarial testing and improvement of constrained credit scoring systems. Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering.

Data-driven Simulation and Optimization for Covid-19 Exit Strategies

Published in KDD *BEST PAPER*, 2020

The rapid spread of the Coronavirus SARS-2 is a major challenge that led almost all governments worldwide to take drastic measures to respond to the tragedy. Chief among those measures is the massive lockdown of entire countries and cities, which beyond its global economic impact has created some deep social and psychological tensions within populations. While the adopted mitigation measures (including the lockdown) have generally proven useful, policymakers are now facing a critical question: how and when to lift the mitigation measures? A carefully-planned exit strategy is indeed necessary to recover from the pandemic without risking a new outbreak. Classically, exit strategies rely on mathematical modeling to predict the effect of public health interventions. Such models are unfortunately known to be sensitive to some key parameters, which are usually set based on rules-of-thumb. In this paper, we propose to augment epidemiological forecasting with actual data-driven models that will learn to fine-tune predictions for different contexts (e.g., per country). We have therefore built a pandemic simulation and forecasting toolkit that combines a deep learning estimation of the epidemiological parameters of the disease in order to predict the cases and deaths, and a genetic algorithm component searching for optimal trade-offs/policies between constraints and objectives set by decision-makers. Replaying pandemic evolution in various countries, we experimentally show that our approach yields predictions with much lower error rates than pure epidemiological models in 75% of the cases and achieves a 95% R² score when the learning is transferred and tested on unseen countries. When used for forecasting, this approach provides actionable insights into the impact of individual measures and strategies.

Recommended citation: Ghamizi, S., Rwemalika, R., Veiber, L., Cordy, M., Bissyandé, T.F., Papadakis, M., Klein, J., & Traon, Y.L. (2020). Data-driven Simulation and Optimization for Covid-19 Exit Strategies. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.

FeatureNET: Diversity-Driven Generation of Deep Learning Models

Published in ICSE Tool Track, 2020

We present FeatureNET, an open-source Neural Architecture Search (NAS) tool 1 that generates diverse sets of Deep Learning (DL) models. FeatureNET relies on a meta-model of deep neural networks, consisting of generic configurable entities. Then, it uses tools developed in the context of software product lines to generate diverse (maximize the differences between the generated) DL models. The models are translated to Keras and can be integrated into typical machine learning pipelines. FeatureNET allows researchers to generate seamlessly a large variety of models. Thereby, it helps choosing appropriate DL models and performing experiments with diverse models (mitigating potential threats to validity). As a NAS method, FeatureNET successfully generates models performing equally well with handcrafted models.

Recommended citation: Ghamizi, S., Cordy, M., Papadakis, M., & Traon, Y.L. (2020). FeatureNET: Diversity-Driven Generation of Deep Learning Models. 2020 IEEE/ACM 42nd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 41-44.

Automated Search for Configurations of Convolutional Neural Network Architectures

Published in SPLC, 2019

Convolutional Neural Networks (CNNs) are intensively used to solve a wide variety of complex problems. Although powerful, such systems require manual configuration and tuning. To this end, we view CNNs as configurable systems and propose an end-to-end framework that allows the configuration, evaluation and automated search for CNN architectures. Therefore, our contribution is threefold. First, we model the variability of CNN architectures with a Feature Model (FM) that generalizes over existing architectures. Each valid configuration of the FM corresponds to a valid CNN model that can be built and trained. Second, we implement, on top of Tensorflow, an automated procedure to deploy, train and evaluate the performance of a configured model. Third, we propose a method to search for configurations and demonstrate that it leads to good CNN models. We evaluate our method by applying it on image classification tasks (MNIST, CIFAR-10) and show that, with limited amount of computation and training, our method can identify high-performing architectures (with high accuracy). We also demonstrate that we outperform existing state-of-the-art architectures handcrafted by ML researchers. Our FM and framework have been released to support replication and future research.

Recommended citation: Ghamizi, S., Cordy, M., Papadakis, M., & Traon, Y.L. (2019). Automated Search for Configurations of Convolutional Neural Network Architectures. Proceedings of the 23rd International Systems and Software Product Line Conference - Volume A.