New Phishing Attacks Bypass Machine Learning Security: Experts Talk Solutions

Phishing — the malicious action of attempting to trick users into downloading malware or giving away sensitive data — is still, to this day, the top vector of attack.

While cybercriminals continue to run wild, increasing the rate and damage of cyber incidents, phishing detection cybersecurity solutions driven by artificial intelligence (AI) and machine learning (ML) have become the norm among organizations looking to protect themselves.

However, cybercriminals are raising the stakes, creating new ways to bypass these security guardrails, leaving organizations once again exposed to phishing.

Techopedia sat with experts in the security community to talk about how attackers can override ML phishing security technologies and what solutions are at hand to fortify defenses.

Table of Contents Table of Contents

1. Key Takeaways
2. Email Security Solutions Aren't Stopping Threats
3. Researchers Discover New Phishing Threat in the Wild
4. Email and Messages ML Security Controls
5. How Widespread Are Conversational Overflow Attacks?
6. Red Flags To Look Out For
7. How Developers and Security Teams Can Adapt and Enhance ML Security Controls
8. The Bottom Line
9. References

Show Full Guide

Table of Contents

1. Key Takeaways
2. Email Security Solutions Aren't Stopping Threats
3. Researchers Discover New Phishing Threat in the Wild
Show Full Guide
4. Email and Messages ML Security Controls
5. How Widespread Are Conversational Overflow Attacks?
6. Red Flags To Look Out For
7. How Developers and Security Teams Can Adapt and Enhance ML Security Controls
8. The Bottom Line
9. References

Email Security Solutions Aren’t Stopping Threats

The 2024 Annual State of Email Security report of Cofense found that phishing attacks that bypass secure email gateways (SEGs) increased by 104.5% in 2023.

Secure email gateway (SEG) are email security solutions that combine signature analysis and ML to identify and block malicious emails before they reach recipients’ inboxes. Cofense said that, in 2023, SEGs struggled to keep up with the evolving and sophisticated nature of today’s phishing campaigns. The report concluded that security solutions are ineffective, adding: “In other words, your email security solutions aren’t stopping the threats you think they are.”

Researchers Discover New Phishing Threat in the Wild

On March 19, SlashNext threat researchers uncovered a dangerous new type of cyberattack in the wild that uses cloaked emails to trick ML tools into accepting malicious payload. The malicious payload in the email then penetrates enterprise networks to execute credential thefts and other harmful types of data harvesting.

Stephen Kowski, Field CTO, SlashNext, walked Techopedia through the specifics of this attack technique and explained how it tricks ML security controls to drop malicious payloads.

“Conversation Overflow attacks are email phishing attacks that contain two parts — the first part is designed for the intended victim to see and interpret as a need to take action (e.g. click on a link, enter credentials, other typical phishing lures).

“Below this portion of the email message, the threat actor hits the return key numerous times, so there is significant blank space separating the top of the message from the second, hidden part of the message.

“This second part contains hidden text intended to read like a legitimate message that could conceivably be part of an ordinary email exchange. This second part of the message is not visible to the intended victim but is visible to the ML security controls.”

Kowski explained that ML controls work differently from other types of security controls. These ML controls look for “known good” behavior to allow or block messages from entering inboxes — as opposed to looking for “known bad” behavior.

“By including this benign portion of an email conversation that does not include any signs of a phishing message, the threat actors are tricking ML tools into allowing the message to enter inboxes.”

Email and Messages ML Security Controls

In the past year, numerous scientific studies have analyzed the performance of AI and ML as potential tools to deter phishing attacks. ML security controls have proven to be more effective than simple blacklisting strategies. These models can adapt and learn and are automated, freeing up security resources and time.

However, studies like Improved Phishing Attack Detection with Machine Learning: A Comprehensive Evaluation of Classifiers and Features conclude that the performances of ML phishing security control are directly affected by the choice of features (key data which the ML model uses) and classifiers — the algorithm that automatically orders or categorizes data into one or more of a set of classes.

Jan Miller, CTO of Threat Analysis at OPSWAT, a cybersecurity platform working with the public and private sector to secure critical infrastructure, told Techopedia that the tech has advanced and is today essential for the detection and neutralization of threats in email communication.

“These measures include dynamic analysis, where suspicious files are tested in secure environments, and similarity searches through databases, which look for known malicious patterns in files,” Miller said.

“Nevertheless, attackers continually adapt, using methods such as obfuscation and polymorphism to disguise malware and trick ML models.”

Developing, training, fine-tuning, and customizing ML algorithms is a complex task.

“They require customization with specific customer data to effectively counter sophisticated threats like spear phishing, which involves crafting numerous deceptive email variations to breach defenses,” Miller said.

“It is crucial that these algorithms are perpetually updated with diverse and recent customer data to identify and adapt to new tactics, thereby avoiding the exploitation of vulnerabilities that could be shared across various models.”

Miller explained that attackers can also engage in additional vectors to undermine ML defenses, such as injecting synthetic data to confuse learning processes, exploiting algorithmic weaknesses, manipulating feedback loops, tampering with hyperparameter settings, and transferring exploits to similar systems.

“These techniques reveal the necessity for ongoing vigilance and development in ML security to combat the continually evolving threat landscape.”

How Widespread Are Conversational Overflow Attacks?

Because phishing attacks that manipulate and deceive ML security controls are relatively new, there are no statistics or reports revealing how widespread they are. However, Techopedia asked Kowski of SlashNext — the company that detected this type of attack in the wild — about his thoughts on this issue.

“Currently, these attacks are not widespread, but we have observed repeated instances of them in recent months. Right now, attackers are essentially beta-testing this technique and making adjustments as needed to hone a successful attack archetype that will consistently and reliably bypass security controls that rely on ML.

“Once they (cybercriminals) are satisfied with their technique, that is when we expect to see Conversation Overflow attacks en masse.”

Miller from OPSWAT said that adversarial attacks on ML systems have become more common and are a growing concern, especially for industries that rely heavily on AI, like finance, healthcare, and defense.

“These industries are attractive targets because they use ML for high-stakes decision-making. Attacks can exploit the entire AI/ML life cycle, from initial model training with poisoned data to evasion techniques against deployed models.”

Miller recognized the field of adversarial AI as one “still maturing”. “These threats have already prompted the development of specialized frameworks, like the Adversarial ML Threat Matrix, to help security analysts identify and mitigate attacks,” Miller said.

“However, most organizations are still catching up with the necessary tools and strategies to secure their ML systems. The challenge is compounded by the use of publicly available datasets and models, which can be manipulated to include hidden vulnerabilities.”

Red Flags To Look Out For

The IBM 2023 Cost of Data Breach study found that organizations that heavily utilize artificial intelligence (AI) and automation in their security have a significant advantage. These organizations experience a double benefit: reduced costs and faster response times in the face of data breaches.

The IBM study revealed that extensive use of AI and automation has saved organizations nearly $1.8 million per organization in data breach costs.

Additionally, these organizations were able to identify and contain breaches over 100 days faster on average. This faster response time applies to a variety of security threats, including identity theft, hacking attempts, phishing attacks, and malware infections.

While automation, AI, and ML play a critical role in the future of cybersecurity, humans in the loop are necessary to identify red flags that automation tools fail to detect, do digital forensics, and update algorithms and databases to respond to the ever-evolving cybercriminal tactics.

Kowski spoke about the challenges that security teams face when trying to identify red flags in phishing attacks.

“Conversation Overflow attempts are hard to spot because many folks won’t keep scrolling once they see a large empty area of an email. You want to look for superfluous conversations that look like they could be real — relevant, but they are positioned specifically where folks are least likely to look.”

Miller added that other issues to look for to adapt ML security controls include:

• Unexpected or unsolicited emails with urgent or alarming content
• Messages requesting sensitive information, like passwords or financial details
• Poor grammar and spelling errors in emails that are supposedly from professional organizations
• Emails with mismatched URLs, where the text of the link doesn’t match the actual URL

How Developers and Security Teams Can Adapt and Enhance ML Security Controls

In their recent blog post, SlashNext concluded that criminals are “morphing their attack techniques” as AI security emerges to respond to threats. SlashNext researchers expressed concern that an entirely new toolkit is being refined by cybercriminal groups. SlashNext experts are scanning the Dark Web in search of evidence of these new toolkits.

When questioned about what developers and security teams can do, Kowski of SlashNext offered valuable insight.

“Security teams must ensure they have in place ML technology that can see all levels of email content — both visible and invisible to the user.”

“ML technology adapts and learns and helps inform security professionals what is required to keep evolving security protections to stay one step ahead of malicious actors,” Kowski said.

“It is important to understand that constant vigilance is required — there is no finish line when it comes to cybersecurity because attackers are constantly evolving their techniques and looking for new ways in.”

Adapting and Enhancing ML Algorithms to Counter Attacks

To adapt ML algorithms Kowski recommends companies to move beyond specific words and statistical heuristics associated with the early days of natural language understanding.

“Companies have to put multiple AI layers of detection in place and ultimately understand what good communication looks like. For example, adding Generative AI layers can help in this area”.

Miller from OPSWAT added that security teams, developers, and companies should also use ensemble learning techniques where multiple models contribute to decision-making, improving overall accuracy.

“Applying anomaly detection mechanisms to identify deviations from standard communication patterns and incorporating feedback loops that allow systems to learn from missed attacks and user-reported phishing attempts are beneficial.”

The Bottom Line

Phishing attacks have evolved to bypass the latest and most modern automated machine learning defenses. Attacks that trick ML security controls prove that hackers will create new techniques to answer to the deployment of new automated tech, and AI cybersecurity tools.

The solution not only lies in a layered defense but also in understanding how these new attacks work to enhance ML security controls. With nine out of ten attacks starting out as a phishing attack, the need to act on this new information should not be underestimated.