How AI Digs into the Dark Web to Hunt Cyber Threats

From new malware on sale, data breaches, offers of ransomware services and phishing kits — the most visible elements of the global cybercriminal world are hidden on the dark web.

To stay one step ahead of attackers, we find law enforcement, governments, and companies monitoring the dark web in search of signals among the vast noise.

But finding anything at all on the dark web is easier said than done. Dark web monitoring services are not new, but now there is a new player in town — artificial intelligence.

Techopedia sat with AI dark web monitoring experts to understand how the new technology works, what are its challenges and limitations, and how organizations can deploy these solutions to take security to the next level.

Table of Contents Table of Contents

1. Key Takeaways
2. A Dark Web Market Worth $1.7Bn By 2030
3. Criminals Use of AI Dark Web Tech is Not Ruled Out
4. Traditional Dark Web Monitoring Vs. AI-Driven
5. 'The Dark Web is Not Such a Dark Place'
6. The Bottom Line
7. References

Show Full Guide

Table of Contents

1. Key Takeaways
2. A Dark Web Market Worth $1.7Bn By 2030
3. Criminals Use of AI Dark Web Tech is Not Ruled Out
Show Full Guide
4. Traditional Dark Web Monitoring Vs. AI-Driven
5. 'The Dark Web is Not Such a Dark Place'
6. The Bottom Line
7. References

A Dark Web Market Worth $1.7Bn By 2030

The global market for dark web threat intelligence is surging. A report from Data Intelligence estimated that the sector will reach $1.7 billion by 2030.

Driven by the rising recurrence of digital threats, companies like ZeroFox, CrowdStrike, Digital Shadows, Flare, IBMX-force, and other leading players are investing in innovation and AI to build the next generation of dark web monitoring technologies.

As the dark web market develops, Flare, a threat exposure management company, took the AI lead, recently rolling out its new AI-dark web solution — Threat Flow. The company claims that Threat Flow is the first transparent generative AI application that delivers timely, relevant, and trustworthy reports of threat actor activity on the dark web.

Techopedia spoke with Mark MacDonald, Senior Product Marketing Manager at Flare, to understand what type of AI and machine learning models they integrated into Threat Flow.

“No single language model can accomplish all tasks effectively,” MacDonald said. “Using AI in a way that builds value for customers involves employing different tools for different highly specific tasks.

“We use natural language processing (NLP) for tracking threat actors across forums and marketplaces, large language models (LLMs) for identifying high-value posts that may benefit specific organizations, and traditional machine learning for scoring specific events.”

The best approach applies each ‘type’ of AI for a specific task it’s well suited for.

Threat Flow classifies every single dark web forum thread across the most high-value dark web forums. Then the language models summarize the threats and allow easy pivoting through various forum conversations, enabling customers to seamlessly identify high-value dark web intelligence.

Criminals Use of AI Dark Web Tech is Not Ruled Out

Flare’s MacDonald told Techopedia that it is unlikely that cybercriminals will gain anything from reverse engineering their technology. However, MacDonald did recognize the possibility of threat actors leveraging AI to better detect “undercover” law enforcement agents and ban them from the platform

Dane Sherrets, Solutions Architect at HackerOne spoke to Techopedia about malicious uses of AI in the dark web.

“Cybercriminals can incorporate generative AI into tools that act similar to web crawlers and related technologies, which simplifies how criminals find credential dumps and other information that helps them plan or carry out cyberattacks.”

Sherrets explained that AI on the dark web works because of the vast amounts of data that need to be analyzed. Shifting manually through these mountains of data, which are structured differently than in the rest of the web, is difficult.

“AI can help with both of these problems at scale”

Shawn Waldman, CEO and Founder of Secure Cyber, a cybersecurity consulting firm that manages detection and response services for different industries, told us:

“AI enhances dark web monitoring through speed and efficiency. AI can sustain longer life cycles and take on a larger monitoring load than traditional methods.

“Additionally, AI not only performs deeper dives but also correlates data from various searches, potentially leading to better-quality alerts.”

However, Shawa warned that there is only so much that an AI can learn.

“While AI is powerful, it has its limits, so having human expertise is imperative.”

Shawn said that human involvement is especially important for what’s called “ground truth,” which is the ability to read and interpret every new data on the dark web or data that requires contextual understanding.

Sherrets from HackerOne agreed and said that human expertise is still vital, as automation can pick up vulnerabilities that have already been disclosed, as well as other intelligence threat signals.

“But the security researcher community plays a huge role in finding bugs that are new and novel,” Sherrets said.

MacDonald from Flare also addressed the important role that human expertise plays in AI-powered dark web monitoring solutions.

“AI acts as an enabler to humans, helping them identify relevant information faster, find context more quickly, and prioritize the right events.”

“In the end, humans have to decide what to do with that information and to combine it with other broader context related to their business and other information they are consuming.”

Traditional Dark Web Monitoring Vs. AI-Driven

Traditionally dark web monitoring involves analysts using keywords to identify relevant posts across vast dark web databases. But with AI, analysts can narrow down the search to specific events or elements that match customers’ interests, MacDonald explained.

Additionally, AI can find what others would not.

For instance, Threat Flow adds a layer of language models to the dark web collection to allow what the company calls ‘in-context semantic search‘. Thanks to this enhancement, the AI identifies highly relevant results that traditional keyword searching could never find.

“Imagine a threat actor is speaking Russian and referencing a major company but misspelling the company,” MacDonald said. “There are two things that would defeat a keyword search here, the language and the spelling”.

Flare’s AI dark web monitoring does not scrape through unindexed onion addresses, but MacDonald said it does not have to.

“Yes, the dark web consists of many unindexed websites…but criminal and threat actor activity has typically consolidated under a relatively small amount of known forums and platforms that are subject to the same network effects as legitimate internet platforms.”

‘The Dark Web is Not Such a Dark Place’

Michael Nizich, Ph.D., Director of the Entrepreneurship & Technology Innovation Center and Cyber Defense Education at the New York Institute of Technology (NYIT) told Techopedia that the dark web is data, just like everything else in our digital world.

“Modern AI solutions can quickly identify anomalies that are happening in real-time from dark web monitoring that would take humans or even current desktop computers years to process.”

Nizich explained that accurate identification of anomalies (e.g. multiple login attempts from an unfamiliar location using a known MAC address from an attacker) can automatically trigger actions that can stop the attacker (or attacker) in their tracks and keep the person’s information safe.

“The dark web is not as dark as it seems.”

Nizich compared that dark web to an amusement park that nobody can see or know is there in their town. However, this park still has access points. “Once inside the park, many of the usual attractions work as normal,” Nizich said. “Dark web monitoring uses these access points to monitor the traffic and the contents of packets flowing through the networks (inside the park), and AI applied to these processes just makes the monitoring more effective.”

The Bottom Line

For decades, gangs and bad actors have enjoyed privileges that derive from the dark web’s anonymous, obfuscated nature. While AI dark web technologies are still in their infancy and their capabilities are limited, the advances being made in this field could shake the foundations of the cybercriminal world.

Traditional dark web monitoring has played a vital role. Now, with AI dark web monitoring, organizations can identify patterns and analyze vast amounts of data while generating customized threat signal reports.

The battle continues…