Yi Liu

Yi Liu

* denotes equal contributions. ^ denotes corresponding author(s).

LLM Security & Jailbreaking

  • LLM-VA: Resolving the Jailbreak-Overrefusal Trade-off via Vector Alignment Haonan Zhang, Dongxia Wang, Yi Liu, Kexin Chen, Wenhai Wang. ACL 2026.

  • Sticking to the Mean: Detecting Sticky Tokens in Text Embedding Models Kexin Chen, Dongxia Wang, Yi Liu, Haonan Zhang, Wenhai Wang. ACL 2025.

  • Libra-Leaderboard: Towards Responsible AI through a Balanced Leaderboard of Safety and Capability Haonan Li, Xudong Han, Zenan Zhai, Honglin Mu, Hao Wang, Zhenxuan Zhang, Yilin Geng, Shom Lin, Renxi Wang, Artem Shelmanov, Xiangyu Qi, Yuxia Wang, Donghai Hong, Youliang Yuan, Meng Chen, Haoqin Tu, Fajri Koto, Tatsuki Kuribayashi, Cong Zeng, Rishabh Bhardwaj, Bingchen Zhao, Yawen Duan, Yi Liu, Emad A. Alghamdi, Yaodong Yang, Yinpeng Dong, Soujanya Poria, Pengfei Liu, Zhengzhong Liu, Xuguang Ren, Eduard Hovy, Iryna Gurevych, Preslav Nakov, Monojit Choudhury, Timothy Baldwin. NAACL 2025.

  • Detecting LLM Fact-Conflicting Hallucinations Enhanced by Temporal-Logic-Based Reasoning Ningke Li, Yahui Song, Kailong Wang, Yuekang Li, Ling Shi, Yi Liu, Haoyu Wang. arXiv preprint arXiv:2502.13416, 2025.

  • Breaking the Loop: Detecting and Mitigating Denial-of-Service Vulnerabilities in Large Language Models Junzhe Yu, Yi Liu, Huijia Sun, Ling Shi, Yuqi Chen. arXiv preprint arXiv:2503.00416, 2025.

  • ORFuzz: Fuzzing the “Other Side” of LLM Safety — Testing Over-Refusal Haonan Zhang, Dongxia Wang, Yi Liu, Kexin Chen, Jiashui Wang, Xinlei Ying, Long Liu, Wenhai Wang. arXiv preprint arXiv:2508.11222, 2025.

  • A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models Zihao Xu, Yi Liu, Gelei Deng, Yuekang Li, Stjepan Picek. ACL 2024 Findings.

  • Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues Zhiyuan Chang, Mingyang Li, Yi Liu, Junjie Wang, Qing Wang, Yang Liu. ACL 2024 Findings.

  • Pandora: Jailbreak GPTs by Retrieval Augmented Generation Poisoning Gelei Deng*, Yi Liu*, Kailong Wang, Yuekang Li, Tianwei Zhang, Yang Liu. AISCC 2024.

  • Drowzee: Metamorphic Testing for Fact-Conflicting Hallucination Detection in Large Language Models Ningke Li, Yuekang Li, Yi Liu, Ling Shi, Kailong Wang, Haoyu Wang. OOPSLA 2024.

  • Efficient Detection of Toxic Prompts in Large Language Models Yi Liu*, Junzhe Yu*, Huijia Sun, Ling Shi, Gelei Deng, Yuqi Chen, Yang Liu. ASE 2024.

  • Glitch Tokens in Large Language Models: Categorization Taxonomy and Effective Detection Yuxi Li*, Yi Liu*, Gelei Deng, Ying Zhang, Wenjia Song, Ling Shi, Kailong Wang, Yuekang Li, Yang Liu, Haoyu Wang. FSE 2024.

  • Automatic Code Summarization via ChatGPT: How Far Are We? Weisong Sun, Chunrong Fang, Yudu You, Yun Miao, Yi Liu, Yuekang Li, Gelei Deng, Shenghan Huang, Yuchen Chen, Quanjun Zhang, Hanwei Qian, Yang Liu, Zhenyu Chen. ICSE 2024.

  • Lockpicking LLMs: A Logit-Based Jailbreak Using Token-Level Manipulation Yuxi Li, Yi Liu, Yuekang Li, Ling Shi, Gelei Deng, Shengquan Chen, Kailong Wang. arXiv preprint arXiv:2405.13068, 2024.

  • A Cross-Language Investigation into Jailbreak Attacks in Large Language Models Jie Li, Yi Liu, Chongyang Liu, Ling Shi, Xiaoning Ren, Yaowen Zheng, Yang Liu, Yinxing Xue. arXiv preprint arXiv:2401.16765, 2024.

  • Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attacks Kexin Chen, Yi Liu, Dongxia Wang, Jiaying Chen, Wenhai Wang. arXiv preprint arXiv:2408.09326, 2024.

  • Self and Cross-Model Distillation for LLMs: Effective Methods for Refusal Pattern Alignment Jie Li, Yi Liu, Chongyang Liu, Xiaoning Ren, Ling Shi, Weisong Sun, Yinxing Xue. arXiv preprint arXiv:2406.11285, 2024.

  • Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study Yi Liu*, Gelei Deng*, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, Kailong Wang, Yang Liu. arXiv preprint arXiv:2305.13860, 2023.

  • Prompt Injection Attack against LLM-Integrated Applications Yi Liu*, Gelei Deng*, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, Leo Yu Zhang, Yang Liu. arXiv preprint arXiv:2306.05499, 2023.

  • MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots Gelei Deng*, Yi Liu*, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, Yang Liu. NDSS 2024.

AI Agent & Software Supply Chain Security

  • Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems Yubin Qu, Yi Liu, Tongcheng Geng, Gelei Deng, Yuekang Li, Leo Yu Zhang, Ying Zhang, Lei Ma. arXiv preprint arXiv:2604.03081, 2026.

  • Credential Leakage in LLM Agent Skills: A Large-Scale Empirical Study Zhihao Chen, Ying Zhang, Yi Liu, Gelei Deng, Yuekang Li, Yanjun Zhang, Jianting Ning, Leo Yu Zhang, Lei Ma, Zhiqiang Li. arXiv preprint arXiv:2604.03070, 2026.

  • What Makes a Good LLM Agent for Real-world Penetration Testing? Gelei Deng, Yi Liu, Yuekang Li, Ruozhao Yang, Xiaofei Xie, Jie Zhang, Han Qiu, Tianwei Zhang. arXiv preprint arXiv:2602.17622, 2026.

  • Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale Yi Liu, Weizhe Wang, Ruitao Feng, Yao Zhang, Guangquan Xu, Gelei Deng, Yuekang Li, Leo Zhang. arXiv preprint arXiv:2601.10338, 2026.

  • Malicious Agent Skills in the Wild: A Large-Scale Security Empirical Study Yi Liu, Zhihao Chen, Yanjun Zhang, Gelei Deng, Yuekang Li, Jianting Ning, Ying Zhang, Leo Yu Zhang. arXiv preprint arXiv:2602.06547, 2026.

  • Oedipus: LLM-Enhanced Reasoning CAPTCHA Solver Gelei Deng, Haoran Ou, Yi Liu, Jie Zhang, Tianwei Zhang, Yang Liu. CCS 2025.

  • A Rusty Link in the AI Supply Chain: Detecting Evil Configurations in Model Repositories Ziqi Ding, Qiang Fu, Junchen Ding, Gelei Deng, Yi Liu, Yuekang Li. IEEE S&P Workshop 2025.

  • SAVANT: Vulnerability Detection in Application Dependencies through Semantic-Guided Reachability Analysis Wang Lingxiang, Quanzhi Fu, Wenjia Song, Gelei Deng, Yi Liu, Dan Williams, Ying Zhang. arXiv preprint arXiv:2506.17798, 2025.

  • ai.txt: A Domain-Specific Language for Guiding AI Interactions with the Internet Yuekang Li, Wei Song, Bangshuo Zhu, Dong Gong, Yi Liu, Gelei Deng, Chunyang Chen, Lei Ma, Jun Sun, Toby Walsh, Jingling Xue. arXiv preprint arXiv:2505.07834, 2025.

  • PentestGPT: Evaluating and Harnessing Large Language Models for Automated Penetration Testing Gelei Deng, Yi Liu*, Víctor Mayoral-Vilches, Peng Liu, Yuekang Li, Yuan Xu, Tianwei Zhang, Yang Liu. USENIX Security 2024.

  • NAUTILUS: Automated RESTful API Vulnerability Detection Gelei Deng, Zhiyi Zhang, Yuekang Li, Yi Liu, Tianwei Zhang, Yang Liu, Guo Yu, Dongjin Wang. USENIX Security 2023.

  • ExploitFlow: Cyber Security Exploitation Routes for Game Theory and AI Research in Robotics Víctor Mayoral-Vilches, Gelei Deng, Yi Liu, Martin Pinzger, Stefan Rass. arXiv preprint arXiv:2308.02152, 2023.

Software Testing & Analysis

  • STEAMROLLER: A Multi-Agent System for Inclusive Automatic Speech Recognition for People who Stutter Ziqi Xu, Yi Liu, Yuekang Li, Ling Shi, Kailong Wang, Yongxin Zhao. arXiv preprint arXiv:2601.10223, 2026.

  • MiniScope: Automated UI Exploration and Privacy Inconsistency Detection of MiniApps via Two-phase Iterative Hybrid Analysis Siyuan Wang, Yuekang Li, Kailong Wang, Yi Liu, Hui Li, Yang Liu, Haoyu Wang. TOSEM 2025.

  • SPOLRE: Semantic Preserving Object Layout Reconstruction for Image Captioning System Testing Yi Liu, Guanyu Wang, Xiaojun Zheng, Gelei Deng, Kailong Wang, Yang Liu, Haoyu Wang. TOSEM 2025.

  • Medusa: Unveil Memory Exhaustion DoS Vulnerabilities in Protocol Implementations Zhengjie Du, Yuekang Li, Yaowen Zheng, Xiaohan Zhang, Cen Zhang, Yi Liu, Sheikh Mahbub Habib, Xiao Li, Limin Wang. WWW 2024.

  • Semantics-aware Human-Computer Interaction Software Testing Yi Liu. Ph.D. Thesis, Nanyang Technological University, 2024.

  • Automatic Code Summarization via ChatGPT: How Far Are We? Weisong Sun, Chunrong Fang, Yudu You, Yun Miao, Yi Liu, Yuekang Li, Gelei Deng, Shenghan Huang, Yuchen Chen, Quanjun Zhang, Hanwei Qian, Yang Liu, Zhenyu Chen. ICSE 2024.

  • ASTER: Automatic Speech Recognition System Accessibility Testing for Stutterers Yi Liu, Yuekang Li, Gelei Deng, Felix Juefei-Xu, Yao Du, Cen Zhang, Chengwei Liu, Yeting Li, Lei Ma, Yang Liu. ASE 2023.

  • Effective ReDoS Detection by Principled Vulnerability Modeling and Exploit Generation Xinyi Wang, Cen Zhang, Yeting Li, Zhiwe Xu, Shuailin Huang, Yi Liu, Yican Yao, Xiao Yang, Yanyan Zou, Yang Liu, Wei Huo. IEEE S&P 2023.

  • PumpChannel: An Efficient and Secure Communication Channel for Trusted Execution Environment on ARM-FPGA Embedded SoC Jingquan Ge, Yuekang Li, Yi Liu, Yaowen Zheng, Yang Liu, Lida Zhao. DATE 2023.

  • Morest: Model-based RESTful API Testing with Execution Feedback Yi Liu, Yuekang Li, Gelei Deng, Yang Liu, Ruiyuan Wan, Runchao Wu, Dandan Ji, Shiheng Xu, Minli Bao. ICSE 2022.

  • RESTInfer: Automated Inferring Parameter Constraints from Natural Language RESTful API Descriptions Yi Liu. FSE 2022 (SRC).

  • RESTCluster: Automated Crash Clustering for RESTful API Yi Liu. ASE 2022 (SRC).

  • Morest: Industry Practice of Automatic RESTful API Testing Yi Liu, Yuekang Li, Yang Liu, Ruiyuan Wan, Runchao Wu, Qingkun Liu. ASE 2022 (Industry Showcase).

  • Automatic Web Testing Using Curiosity-Driven Reinforcement Learning Yan Zheng*, Yi Liu*, Xiaofei Xie, Yepang Liu, Lei Ma, Jianye Hao, Yang Liu. ICSE 2021.

  • Industry Practice of JavaScript Dynamic Analysis on WeChat Mini-Programs Yi Liu, Jinhui Xie, Jianbo Yang, Shi Guo, Yuetang Deng, Shuqing Li, Yechang Wu, Yepang Liu. ASE 2020.

  • An Exploratory Study of Bugs in Extended Reality Applications on the Web Shuqing Li, Yechang Wu, Yi Liu, Dongxia Wang, Ming Wen, Yida Tao, Yulei Sui, Yepang Liu. ISSRE 2020.

  • JSOptimizer: An Extensible Framework for JavaScript Program Optimization Yi Liu. ICSE 2019 (SRC).

Multimodal AI & Other

  • TombRaider: Entering the Vault of History to Jailbreak Large Language Models Junchen Ding, Jiahao Zhang, Yi Liu, Ziqi Ding, Gelei Deng, Yuekang Li. EMNLP 2025.

  • Mission: Impossible — Image-Based Geolocation with Large Vision Language Models Yi Liu, Gelei Deng, Junchen Ding, Yuekang Li, Tianwei Zhang, Weisong Sun, Yaowen Zheng, Jingquan Ge, Yang Liu. PETS 2025.

  • IllusionCAPTCHA: A CAPTCHA Based on Visual Illusion Ziqi Ding, Gelei Deng, Yi Liu, Junchen Ding, Jishan Chen, Yulei Sui, Yuekang Li. WWW 2025.

  • A Methodology for Replicating Historical Exploits on EVM-Compatible Blockchains Zhenhao Chen, Philip Kemper, Yi Liu, Jake Gorzny, Daniel Siqueira, Yuekang Li, Damiano Pellegrino. IEEE/ACM WETSEB 2025.

  • Groot: Adversarial Testing for Generative Text-to-Image Models with Tree-Based Semantic Transformation Yi Liu, Guowei Yang, Gelei Deng, Feiyue Chen, Yuqi Chen, Ling Shi, Tianwei Zhang, Yang Liu. arXiv preprint arXiv:2402.12100, 2024.

  • MetMap: Metamorphic Testing for Detecting False Vector Matching Problems in LLM Augmented Generation Guanyu Wang, Yuekang Li, Yi Liu, Gelei Deng, Tianlin Li, Guangjun Xu, Yang Liu, Haoyu Wang, Kailong Wang. AIware 2024.

  • Digger: Detecting Copyright Content Mis-usage in Large Language Model Training Haodong Li, Gelei Deng, Yi Liu, Kailong Wang, Yuekang Li, Tianwei Zhang, Yang Liu, Guoai Xu, Guosheng Xu, Haoyu Wang. arXiv preprint arXiv:2401.00676, 2024.