

Code Unmasked: Advancing Vulnerability Detection in Decompiled x86 Binaries Using Instruction-Tuned Language Models
Project ID:
P7_25B
Principal Investigator:
Dr. Houman Homayoun
Email:
Co-Principal Investigator(s):
Dr. Setareh Rafatirad
Topic Areas:
Edge Security
Abstract
In today’s security landscape, the analysis of binary-only software—particularly x86 applications and executables—has become a critical priority for identifying hidden vulnerabilities in systems with no source code access. Traditional reverse engineering tools provide limited insight into decompiled code, especially after compiler optimizations erase meaningful structure [10]. Meanwhile, Large Language Models (LLMs), which excel at understanding high-level source code, face significant challenges when applied to decompiled binaries, where syntax and semantics are severely distorted [4, 6]. This proposal introduces X-VulnBench, a curated dataset and experimental framework designed to elevate LLM performance for security analysis of decompiled x86 binaries. Our work will center around training and evaluating open-source LLMs to perform four key tasks: vulnerability detection, vulnerability classification, functional description generation, and function signature prediction—all using decompiled x86 assembly outputs. By instruction-tuning models on X-VulnBench, we aim to bridge the semantic chasm between clean source code and stripped-down binary representations [6, 14]. The project will produce fine-tuned model checkpoints, a high-quality open dataset, and an extensive evaluation suite to set a new benchmark for LLM-assisted reverse engineering. While the current work focuses on x86, future phases will extend to cross-architecture learning and robustness across optimization levels.