, Siena
This project investigates how attackers can now use large language models (LLMs) and AI agents to autonomously create phishing infrastructure, such as domain registration, DNS configuration, and hosting personalized spoofed websites. While earlier research has explored how LLMs can generate persuasive phishing emails, our study shifts the focus to the back-end automation of the phishing lifecycle. We evaluate how modern frontier and open-source models—including Chinese models like DeepSeek and Western counterparts such as Claude Sonnet and GPT-4o—perform when tasked with registering phishing domains, configuring DNS records, deploying landing pages, and harvesting credentials. The tests will be conducted with and without human intervention. We measure success through metrics like task completion rate, cost and time requirements, and the amount of human intervention required. By demonstrating how easy and low-cost it has become to scale phishing infrastructure with AI, this work underscores the growing threat of AI-powered cybercrime and highlights the urgent need for regulatory, technical, and policy countermeasures.
While much attention has been given to how large language models (LLMs) can craft convincing phishing emails, less focus has been placed on how these models can automate the underlying infrastructure of phishing campaigns. This talk presents new research demonstrating how modern AI agents—powered by both frontier and open-source models such as GPT-4o, Claude Sonnet, and DeepSeek—can autonomously register domains, configure DNS records, deploy spoofed landing pages, and harvest credentials, often with minimal human oversight.
We systematically evaluate these capabilities across a range of agentic tasks, measuring success by task completion rate, time and cost efficiency, level of human intervention required, and evasion of registrar and DNS-level defenses. By comparing fully autonomous runs with human-in-the-loop processes, we offer a detailed look at where automation currently excels—and where it still encounters friction.
Our findings suggest that phishing infrastructure, once a manual and resource-intensive process, is becoming increasingly scalable and accessible through AI. We conclude with key implications for defenders, including updated technical countermeasures, coordination strategies with registrars and hosting providers, and policy recommendations to address the growing misuse potential of advanced language models. We believe this talk will resonate with the BSides community as it highlights the often overlooked (but essential) backend components that enable phishing attacks.
Dr. Fred Heiding is a research fellow at the Harvard Kennedy School’s Belfer Center. His work focuses on computer security at the intersection of technical capabilities, business implications, and policy remediations. Fred is a member of the World Economic Forum's Cybercrime Center, a teaching fellow for the Generative AI course at Harvard Business School, and the National and International Security course at the Harvard Kennedy School. Fred has been invited to brief the US House and Senate staff in DC on the rising dangers of AI-powered cyberattacks, and he leads the cybersecurity division of the Harvard AI Safety Student Team (HAISST). His work has been presented at leading conferences, including Black Hat, Defcon, and BSides, and leading academic journals like IEEE Access and professional journals like Harvard Business Review and Politico Cyber. He has assisted in the discovery of more than 45 critical computer vulnerabilities (CVEs). In early 2022, Fred got media attention for hacking the King of Sweden and the Swedish European Commissioner.
Simon is a AI security researcher who has worked on AI-powered phishing and removing safety guardrails from AI-models. He is interested in researching how AI agents could pose global catastrophic risk through cyberattacks.