State of the Field
Recent research in security is increasingly focused on addressing vulnerabilities in machine learning models, particularly those used in software security and large language models (LLMs). Investigations into data leakage have revealed that common training practices can inflate the perceived effectiveness of secret detection models, highlighting the need for more robust evaluation methods. Meanwhile, as traditional CAPTCHAs fail against advanced GUI agents, new frameworks are being developed to exploit the cognitive gap between humans and machines, ensuring more effective defenses. The rise of federated learning has introduced unique security challenges, such as backdoor attacks that exploit specific neural network layers, necessitating layer-aware detection strategies. Additionally, the proliferation of prompt injection attacks against LLMs has prompted systematic reviews to categorize and enhance mitigation strategies. Collectively, these efforts indicate a shift toward more nuanced and proactive security measures, essential for safeguarding applications in an era of rapidly evolving AI capabilities.
Papers
1–8 of 8From Data Leak to Secret Misses: The Impact of Data Leakage on Secret Detection Models
Machine learning models are increasingly used for software security tasks. These models are commonly trained and evaluated on large Internet-derived datasets, which often contain duplicated or highly ...
Next-Gen CAPTCHAs: Leveraging the Cognitive Gap for Scalable and Diverse GUI-Agent Defense
The rapid evolution of GUI-enabled agents has rendered traditional CAPTCHAs obsolete. While previous benchmarks like OpenCaptchaWorld established a baseline for evaluating multimodal agents, recent ad...
Exploiting Layer-Specific Vulnerabilities to Backdoor Attack in Federated Learning
Federated learning (FL) enables distributed model training across edge devices while preserving data locality. This decentralized approach has emerged as a promising solution for collaborative learnin...
Image-based Prompt Injection: Hijacking Multimodal LLMs through Visually Embedded Adversarial Instructions
Multimodal Large Language Models (MLLMs) integrate vision and text to power applications, but this integration introduces new vulnerabilities. We study Image-based Prompt Injection (IPI), a black-box ...
Exposing the Systematic Vulnerability of Open-Weight Models to Prefill Attacks
As the capabilities of large language models continue to advance, so does their potential for misuse. While closed-source models typically rely on external defenses, open-weight models must primarily ...
A Systematic Literature Review on LLM Defenses Against Prompt Injection and Jailbreaking: Expanding NIST Taxonomy
The rapid advancement and widespread adoption of generative artificial intelligence (GenAI) and large language models (LLMs) has been accompanied by the emergence of new security vulnerabilities and c...
Mitigating the OWASP Top 10 For Large Language Models Applications using Intelligent Agents
Large Language Models (LLMs) have emerged as a transformative and disruptive technology, enabling a wide range of applications in natural language processing, machine translation, and beyond. However,...
Scores Know Bobs Voice: Speaker Impersonation Attack
Advances in deep learning have enabled the widespread deployment of speaker recognition systems (SRSs), yet they remain vulnerable to score-based impersonation attacks. Existing attacks that operate d...