RSAC Former NSA cyber-boss Rob Joyce thinks today’s artificial intelligence is dangerously close to becoming a top-tier vulnerability exploit developer.
“At RSAC last year, I told people: ‘Don’t worry about the zero-day AI armageddon,’ but I am increasingly worried that AI is going to be a good bug finder this year, [and] an exploit developer in the near future,” the retired Director of the NSA’s Cybersecurity Directorate told The Register during an interview this week at the RSA Conference in San Francisco.
How near is the near future? Either this year or next, predicted Joyce, who now serves as an advisor to Sandfly Security, a supplier of intrusion detection tools for Linux systems.
“All the frontier models have got very good at coding,” Joyce noted. “In fact, OpenAI models are out-competing humans in many of the code competitions.”
Case in point: The Hack The Box capture-the-flag contest earlier this month during which AI-powered entrants performed at about the same speed as pure-human teams, and nearly matched humans in tests of problem-solving ability.
By the end of the contest, the top AI team captured 19 of 20 flags, placing 20th out of 403 teams with 15900 points; most of the AI teams captured 19 flags in fact.
It doesn’t matter if you’re a defender or an attacker, those who use AI will outperform those who don’t
“I don’t worry about the big red easy button where you get somebody who’s a script kiddie that knows nothing going ahead and attacking,” Joyce told The Register. “But what it will do is it will take and automate the things that the good attackers need to do, and allow them to do more, faster, and at scale.”
99 reasons not to click: AI supercharges phishing campaigns
Joyce also feels that LLMs will help miscreants and spies – even those for whom English is not their first language – to create believable and effective phishing campaigns.
“Now you can make a culturally relevant, accurate activity that get you to phish,” Joyce said, noting that AI also helps scale creation of these malicious emails. “I watched one campaign where each and every email sent was individualized,” he said. “At that point, some of the current technologies that are looking for a lot of similar features across many emails just don’t work.”
Sandfly Security founder and CEO Craig Rowland said he’s seen fake invoices being sent to companies’ accounts payable departments that include a full email thread to make the phish look more authentic. “People acting like ‘We need to pay this now’, and even including AI-generated PDFs that look official.”
Playing defense
AI can also help defenders. Roland said one his human staff engineers reverse engineered a piece of eBPF code – a job that took about half a day. “The AI system took about 30 seconds,” Rowland said.
Joyce had one condition for the interview: No questions about the Trump administration nor NSA operations. But he indulged us with one query about what he would say if the annual NSA’s State of the Hack session at RSAC had not been pulled and if Joyce had been a speaker as was the case in previous years.
The former NSA cyber chief said he’d describe “one of the more interesting hacks” he saw this year during which a ransomware gang used valid, stolen credentials to access a company’s desktop — but the computer had endpoint detection products installed.
“They realized they couldn’t deploy their ransomware malware, so they pivoted inside the network,” he said. That effort found a small, Linux-based video camera, and the crooks successfully deployed the ransomware on that device. “And it mounted the hard drives around the enterprise, and brought all that data up to the video camera, encrypted it, and put them in a state where they were now ransomwared.” Joyce recalled, describing it as a “fascinating pivot to an unmonitored, undefended part of the network.”
Plus: “I can’t imagine how hot that damn little camera got trying to encrypt all the data in this company,” he noted. “But it worked, right?” ®
