AI: the new frontier of cyber threats

Our experts have shared their thoughts on the key topics and trends that will define the years ahead in cyber security. These insights aim to inspire action and preparation as we confront a world of rapid disruption and boundless potential. Whether you're steering a business, shaping policy, or enhancing personal security, the future is here-and it's mov ing fast.

AI: the new frontier of cyber threats - Chris Anley, Chief Scientist

AI investment is likely to drop off as the benefits and limitations of current AI techniques become more measurable. For example, while Large Language Models (LLMs) are valuable and provide clear improvements in productivity, they are inherently prone to "hallucinations" - false or misleading outputs - which reduces their commercial appeal. These hallucinations are a result of the statistical nature of LLMs, and despite partial mitigation techniques - human feedback, Retrieval Augmented Generation, and careful prompting - the problem may be a fundamental characteristic of these models as they are currently designed.

The rise in AI-enabled crime also poses a significant threat. The US Federal Trade Commission and UK banks have issued warnings about voice cloning fraud, with criminals using cloned voices of family members to deceive victims. Similarly, phishing emails, automated calls, and text message scams have become more convincing, a trend expected to accelerate as generative AI becomes more accessible.

Deepfake images and videos are also becoming increasingly realistic; at the beginning of 2024, they were already of sufficiently high quality to fool most people and were used in some high value fraud cases. During 2025, improvements in quality are likely to continue to the point where generated images and video are of such high quality that they are almost impossible for people and automated detectors to spot. We can anticipate their continued misuse for fraud, and their deployment in political and celebrity-related manipulations.

As new forms of business emerge, their growing success and maturity can often be gauged by the extent to which lawyers are involved. Litigation is already in progress in the field of AI; we are already seeing - and can expect to see increases in - intellectual property issues around unlicensed use of training data, performers and creators suing in relation to their image rights, lawsuits in relation to AI data breaches, lawsuits and regulatory action relating to improper use of personal information and many more cases of AI-related litigation.