The Dark Side of GenAI: How GenAI Is Powering the Next Wave of Scams?

For the past couple of years, the world has been fascinated by GenAI and how it has been making life easier for individuals and businesses alike.

But there’s a dark side to GenAI that very few people talk about.

While the technology is simplifying our lives, it's also making things easier for fraudsters, scammers, and opportunists.

How?

Generative AI has emerged as a powerful tool for fraudsters enabling the creation of hyper-realistic deepfakes, synthetic identities, and AI-powered phishing scams.

How effective is GenAI in committing fraud?

Researchers at Cornell University set out to see if they could create an autonomous AI scam agent.
Well, they not only created it successfully but they were overwhelmed by how well it worked.
It stole credentials 60% of the time, in under two minutes, for less than a dollar per attempt.

That figure is scary as hell.

How does GenAI make it easy for scammers to commit fraud?

A report shared by IBM reveals that GenAI shortens phishing prep time dramatically from 16 hours to just 5 minutes. 

Think of it this way: GenAI is making large-scale attacks faster, cheaper, and more accessible than ever.

It’s concerning for individuals—and even more so for digital businesses.

GenAI-driven fraud is alarming for every business operating online. In just the first quarter of this year, businesses reported over $200 million in financial losses due to deepfake-related fraud.

Biggest GenAI fraud attempts:

• Hong Kong CFO Deepfake Scam – January 2024 – $25 million loss

• Arup (UK) Executive Deepfake Case – February 2024 – £20 million (~$25 million) loss

• UAE Voice Clone Bank Transfer Fraud – End of 2021 – $35–53 million loss

• UK CEO Zoom Call Deepfake Scam – March 2023 – $243,000 loss

• Chinese Multinational AI Voice Scam – July 2023 – $622,000 loss

Which industry is the prime target of GenAI fraud?

Financial Institutions are the most vulnerable with GenAI fraud as there has been a massive 700% increase in deepfake incidents annually. 

How can financial institutions (or digital institutions) can fight GenAI fraud?

To fight GenAI fraud, financial institutions have to take a multilayered proactive approach. 

• An approach that’s beyond visual confirmation. 

• An approach that proactively judges on the very first session whether a user (and their device) is trustworthy or not. 

• An approach that combines device signals by pointing out the level of riskiness of every device based on the fraud tools available like VPNs, emulators, cloners, etc. 

The only fraud detection technique offering these capabilities is Device Intelligence—the most effective way to fight GenAI fraud and protect digital businesses from evolving threats.