How AI gets misused: scams, cyberattacks, influence and weapons risks, and what the evidence shows

In early 2024, an employee at the engineering firm Arup joined a video call with people he believed were his company's chief financial officer and several colleagues. Every one of them was a deepfake, a fake video and voice generated by AI. By the time the fraud was discovered, he had sent about HK$200 million (roughly US$25 million) across 15 transactions⁶. Arup later confirmed "that fake voices and images were used"⁶.

Stories like this are why AI misuse makes headlines. But headlines mix very different risks together, from phone scams to state hacking to weapons, and rarely say how common any of them are. This article sorts the main kinds of misuse, sets out what the best recent evidence shows about each, and gives you a practical way to tell a real shift from a scary anecdote.

What "misuse" means, and how it differs from other AI risks

The most authoritative overview of the field is the International AI Safety Report 2026, written by over 100 experts and chaired by Montreal's Yoshua Bengio. It sorts the risks of general-purpose AI into three buckets: "malicious use, malfunctions, and systemic risks"¹.

• Malicious use is when a person deliberately uses AI to cause harm: scams, fake images, cyberattacks, propaganda, or help with weapons. This is the subject of this article.
• Malfunctions are when the AI itself behaves in ways nobody intended, from confident errors to the misalignment problems covered in our article on AI alignment.
• Systemic risks come from AI being used everywhere at once, such as effects on jobs or on people's independence of thought¹.

The lines blur in practice, and researchers have catalogued far more risks than three buckets suggest. The MIT AI Risk Repository, a peer-reviewed database published in Patterns, gathered 1,725 distinct risks from 74 existing frameworks. One of its more useful findings is that "human decisions cause nearly as many AI risks (38%) as the AI systems themselves (42%)"². Misuse sits squarely on the human side.

The key question: how much does AI actually add?

The single most useful idea for thinking about misuse is marginal risk, sometimes called uplift. The question is not "could someone use AI to do something bad?" (almost always yes) but "how much more harm can they do with AI than they could already do without it?"

A team of 25 researchers, including Sayash Kapoor, Rishi Bommasani and Arvind Narayanan, built a framework around this question in 2024, to analyse "their marginal risk" of misuse for openly released AI models. They found it helps "explain why the marginal risk is low in some cases" and "clarifies disagreements about misuse risks"³. Anthropic's 2026 threat reporting uses a similar lens, defining uplift as "how much more harm was caused with AI versus without AI," measured through "speed, scale, and depth"⁸.

A simple analogy: a powerful search engine "could" help a burglar find a floor plan, but so could a library. What matters is whether the new tool removes a real bottleneck. For some kinds of misuse, AI mostly makes existing crime cheaper and faster. For a few, it may remove barriers that used to keep dangerous capabilities out of reach. The evidence below is organised with that distinction in mind.

What the evidence shows, area by area

Scams, fraud and fake intimate images

This is where AI misuse touches the most ordinary people, and where the evidence of real harm is strongest. The International AI Safety Report states plainly that "AI systems are being misused to generate content for scams, fraud, blackmail, and non-consensual intimate imagery"¹.

The best North American numbers come from police and consumer agencies, with important caveats. The FBI's Internet Crime Complaint Center (IC3) received 1,008,597 complaints in 2025, with reported losses of $20.877 billion, up 26% from 2024. Of those, 22,364 complaints carried an "AI Related" tag, with adjusted losses of $893,346,472⁴. That AI figure needs careful reading. It counts complaints where the information reported mentioned AI, not losses proven to be caused by AI. It depends on victims recognising and reporting AI involvement, which is hard when a fake is convincing. It is best read as a floor, not a total.

In Canada, the Canadian Anti-Fraud Centre recorded at least C$704 million in fraud losses in 2025, and the Competition Bureau estimates that "fewer than one in 10 frauds are reported"⁵. The acting commissioner of competition warned that "AI has armed fraudsters with powerful tools to create highly convincing impersonations"⁵. That figure covers all fraud, not only AI-assisted fraud, so it shows the scale of the problem AI is feeding into rather than AI's specific share.

Governments are starting to respond. The US TAKE IT DOWN Act, signed in May 2025, makes it a crime to knowingly publish intimate images, including AI-generated "digital forgeries", without consent. Since May 19, 2026, covered platforms must remove such images "within 48 hours of receiving a valid removal request", with enforcement by the Federal Trade Commission¹².

The honest summary is the report's own: "Although the occurrence of such harms is well-documented, systematic data on their prevalence and severity remains limited"¹.

Cyberattacks

Here the picture changed sharply in the past year. The International AI Safety Report notes that "AI systems can discover software vulnerabilities and write malicious code," citing a competition in which "an AI agent identified 77% of the vulnerabilities present in real software"¹. It also states that "criminal groups and state-associated attackers are actively using general-purpose AI in their operations"¹.

The clearest documented case came in November 2025. Anthropic reported that a group it assessed "with high confidence was a Chinese state-sponsored group" had used its Claude Code tool to attempt intrusions into "roughly thirty global targets," succeeding "in a small number of cases." The attackers used AI "to perform 80-90% of the campaign," with humans stepping in at perhaps "4-6 critical decision points"⁷. Anthropic's September 2026 threat report, covering December 2025 to August 2026, goes further: "A majority of the operations described in this report were enabled by AI via direct execution or orchestration," under a section titled "Sophisticated attacks no longer require sophisticated attackers"⁸.

In Canada, the Canadian Centre for Cyber Security had already warned in its 2025–2026 national assessment that profit-driven cybercriminals "are using artificial intelligence to enhance their capabilities"¹⁰.

Two caveats matter. First, these reports come from AI companies describing misuse of their own products. They show what is possible and happening, not how common it is across the internet. Anthropic itself says its cases "aren't typical misuse, but rather examples of the most notable and novel threat activity"⁸. Second, AI still makes mistakes in attacks. In the 2025 campaign, Claude "occasionally hallucinated credentials or claimed to have extracted secret information that was in fact publicly-available," which Anthropic called "an obstacle to fully autonomous cyberattacks"⁷. And the same tools help defenders. The International AI Safety Report concludes that "whether attackers or defenders will benefit more from AI assistance remains uncertain"¹.

Influence and manipulation

Many people expected AI to flood elections with propaganda. The evidence so far is more measured. The International AI Safety Report finds that "in experimental settings, AI-generated content can be as effective as human-written content at changing people's beliefs," but that "real-world use of AI for manipulation is documented but not yet widespread, though it may increase as capabilities improve"¹.

OpenAI's February 2026 threat report, marking two years of public reporting, describes how threat actors "typically use AI in combination with other, more traditional tools such as websites and social media accounts," and often "use different AI models at various points in their operational workflow"⁹. In other words, AI is usually one tool in a larger operation. It makes content cheaper to produce, but getting real people to see and believe that content remains the hard part.

Biological and chemical weapons

This is the area with the lowest frequency and the highest potential severity, and the one experts watch most closely. The concern is not that AI can make a weapon, but that it might supply the specialist knowledge that has historically been a barrier.

The evidence here is about capability, not incidents. In a 2025 benchmark of hard, practical virology questions, OpenAI's o3 model scored 43.8%, compared with an average of 22.1% for expert virologists answering questions in their own specialties, and outperformed 94% of those experts¹¹. A benchmark score is not the same as real-world help to an attacker, but results like this are why labs have tightened controls. The International AI Safety Report notes that in 2025, "multiple developers released new models with additional safeguards after they could not exclude the possibility that these models could assist novices in developing such weapons"¹.

The key uncertainty is whether knowledge is still the main bottleneck. As the report puts it, "it remains difficult to assess the degree to which material barriers continue to constrain actors seeking to obtain them"¹. Obtaining materials, equipment and hands-on skill remains hard, and nobody has good public data on how much AI changes that.

Limits of the evidence, and open questions

It helps to be clear about what this body of evidence can and cannot tell you.

Most data comes from the companies themselves. Threat reports from AI companies such as Anthropic and OpenAI are the richest source on real misuse, but each sees only its own platform. Anthropic notes it has "visibility into Claude usage" only, though it believes its case "probably reflects consistent patterns of behavior across frontier AI models"⁷. Independent, cross-platform measurement is still thin.

Crime statistics undercount. Police data relies on victims reporting, and on victims knowing AI was involved. With fewer than one in ten frauds reported in Canada⁵, and the FBI's AI tag depending on what victims mention⁴, the true scale is almost certainly larger, but by how much is unknown.

Safeguards help but leak. The International AI Safety Report notes that "users can still sometimes obtain harmful outputs by rephrasing requests or breaking them into smaller steps." It also points out that openly released models "cannot be recalled once released, their safeguards are easier to remove, and actors can use them outside of monitored environments"¹. Whether that tradeoff is worth the benefits of openness is a live debate in its own right.

The open question is direction, not existence. Misuse is real and documented. What is uncertain is how fast the marginal uplift grows as AI agents become more capable, and whether defensive uses keep pace.

What this means for you

You can act on this evidence this week without any technical background.

1. Agree on a verification step for money and passwords. The Arup case worked because a video call felt like proof⁶. For any urgent request to move money or share credentials, confirm through a separate channel you already trust, such as calling a known number. Families can agree on a code word for emergencies.
2. Treat realism as worthless evidence. A convincing voice, face or writing style no longer proves who you're dealing with. Judge requests by whether they make sense, not by how real they look.
3. Report fraud, even small amounts. In Canada, report to the Canadian Anti-Fraud Centre. In the US, report to the FBI's IC3. Underreporting is one reason the data is so thin⁴⁵. If you mention that AI seemed to be involved, it helps the statistics too.
4. Know your takedown rights. If someone shares a fake intimate image of you, US platforms covered by the TAKE IT DOWN Act must remove it within 48 hours of a valid request¹².
5. Ask the marginal-risk question. When you read a headline about AI misuse, ask: Could this already be done without AI? Is this a real incident or a test? Who measured it, and what can they see? Those three questions sort most stories quickly³⁸.

Sources

1. Bengio, Y., et al. (2026, February 3). International AI Safety Report 2026. DSIT 2026/001. internationalaisafetyreport.org/publication/international-ai-safety-report-2026
2. Slattery, P., Saeri, A. K., Grundy, E. A. C., Graham, J., Noetel, M., Uuk, R., Dao, J., Pour, S., Casper, S., & Thompson, N. (2026). The AI risk repository: A meta-review, database, and taxonomy of risks from artificial intelligence. Patterns, 101517. arXiv:2408.12622 (v3, May 5, 2026). arxiv.org/abs/2408.12622
3. Kapoor, S., Bommasani, R., Klyman, K., Longpre, S., Ramaswami, A., Cihon, P., et al., Liang, P., & Narayanan, A. (2024). On the societal impact of open foundation models. arXiv:2403.07918. arxiv.org/abs/2403.07918
4. Federal Bureau of Investigation, Internet Crime Complaint Center. (2026). 2025 Internet Crime Report. ic3.gov, 2025 Internet Crime Report (PDF)
5. Got, J. (2026, March 6). Canadians lost $704 million to fraud in 2025. Advisor.ca (reporting Canadian Anti-Fraud Centre data and a Competition Bureau release). advisor.ca/industry-news/industry/canadians-lost-704-million-to-fraud-in-2025
6. Magramo, K. (2024, May 16). British engineering giant Arup revealed as $25 million deepfake scam victim. CNN. edition.cnn.com/2024/05/16/tech/arup-deepfake-scam-loss-hong-kong-intl-hnk
7. Anthropic. (2025, November 13). Disrupting the first reported AI-orchestrated cyber espionage campaign. anthropic.com/news/disrupting-AI-espionage
8. Anthropic. (2026, September). Detecting and countering misuse of AI: September 2026. anthropic.com/threat-intelligence-report-september-2026
9. OpenAI. (2026, February 25). Disrupting malicious uses of AI. openai.com/index/disrupting-malicious-ai-uses
10. Canadian Centre for Cyber Security. (2024). National Cyber Threat Assessment 2025–2026. Communications Security Establishment Canada. cyber.gc.ca/en/guidance/national-cyber-threat-assessment-2025-2026
11. Götting, J., Medeiros, P., Sanders, J. G., Li, N., Phan, L., Elabd, K., Justen, L., Hendrycks, D., & Donoughe, S. (2025). Virology Capabilities Test (VCT): A multimodal virology Q&A benchmark. arXiv:2504.16137. arxiv.org/abs/2504.16137
12. WilmerHale. (2026, June 15). The TAKE IT DOWN Act goes live. Client alert. wilmerhale.com/en/insights/client-alerts/20260615-the-take-it-down-act-goes-live