A few years into the era of large language models being deployed by both attackers and defenders, it’s worth taking stock of what’s actually changed in the phishing threat landscape. The headlines have been breathless. The on-the-ground reality is more textured.

This is a defensive perspective from someone who has spent the last several years watching how SOC teams, identity teams, and end-user-facing security functions cope with the shifts. The conclusions below are deliberately practical. Anyone telling you AI has either “transformed phishing” or “left phishing unchanged” is selling something.

What’s genuinely different

A few things have changed in ways that matter.

Quality of lure content. The grammatical errors and obvious linguistic tells that used to flag a phishing email to the trained eye are mostly gone. Modern attackers can produce well-written, contextually appropriate, audience-targeted content at scale. This affects the floor of the threat, not the ceiling. The most dangerous targeted attacks were already well-written by skilled operators a decade ago. What’s changed is that the long tail of less sophisticated phishing is now indistinguishable from the better attacks of five years ago.

Personalisation at scale. Open-source intelligence about a target — their employer, recent activity, the people they work with, their public communications — can now be pulled and synthesised into a phishing approach quickly enough that customised attacks against entire organisations become economically viable. This used to be reserved for high-value targets. Now mid-tier targets get treatments that previously only executives saw.

Voice and video impersonation. The capability to clone a recognisable voice from a short audio sample has been well-publicised. Less talked about is how the cost has dropped. It now takes minutes of work and pennies of compute to produce convincing voice messages purporting to be from a colleague or family member. Video is harder but the bar is falling. Australian institutions have already seen several high-profile incidents where voice cloning has been used in business email compromise variants.

Conversational depth. A particularly worrying shift is the use of AI to maintain back-and-forth conversations with targets that are convincing enough to walk them through credential capture or wire transfer authorisation. Where attackers used to fire and forget, they’re now patient, willing to maintain plausible exchanges over days, because the cost of doing so is low.

What’s not actually new

It’s important to acknowledge what hasn’t changed, because some of the narrative around AI phishing obscures the steady-state threats that still cause the bulk of damage.

The fundamental social engineering vectors. Urgency, authority, fear, helpfulness, reciprocity — the levers attackers pull are the same ones they’ve always pulled. The packaging is better. The underlying psychology is identical.

The infrastructure of attack. Domain registration patterns, sending infrastructure, mail authentication abuses, redirector chains. Most of this looks much like it did three years ago. The detection and disruption mechanisms that work against the infrastructure side of phishing still work.

The defensive fundamentals. Phishing-resistant MFA (FIDO2, passkeys properly configured), conditional access, hardened authentication flows, and well-designed reporting channels are still the controls that move the needle. None of them have been obsoleted by AI-generated content.

The user-level signals. Many of the original training points about phishing — be suspicious of unexpected urgency, verify out of band, never click a link in a message and enter credentials, treat attachments cautiously — remain valid. The execution is harder because the lures are better, but the underlying advice is unchanged.

Where defenders are responding well

A few patterns I’ve seen at organisations that have shifted their posture well in response.

Investing in phishing-resistant authentication. The organisations with broad rollout of passkeys or FIDO2 hardware tokens have substantially de-risked the credential theft pathway. When the credentials themselves can’t be replayed against your environment, much of the phishing economic model breaks. The ACSC’s guidance on phishing-resistant MFA is the public starting point and worth reading.

Tuning email security to the new threat shape. Pure content-based filtering is less effective when content is human-quality. The better defences combine sender reputation, behavioural baselining, attachment sandboxing, and link rewriting. This shifts the detection burden away from “did this look weird” toward “did this come from where it claims to come from, did the recipient and the sender have a prior relationship, does this fit their pattern.”

Building reporting and triage into the workflow. Organisations with mature reporting buttons and rapid SOC triage of reported phishing benefit from a thousand eyes effect. Users still detect a meaningful proportion of attacks. The question is whether their reports land somewhere that does anything useful with them within the window where the report matters.

Tabletop exercises that include AI-augmented scenarios. Voice cloning of an executive demanding an urgent wire transfer; sustained conversational social engineering of an IT help desk; deepfake video conference invitations. The organisations that have walked their incident response through these scenarios are markedly better prepared when something like them lands.

The Australian Signals Directorate’s Annual Cyber Threat Report covers the broader threat environment and is useful context.

Where defenders are stuck

The harder problems are the ones where the technology hasn’t given defenders an equivalent answer to what it’s given attackers.

Deepfake detection at scale. The current detection capability for voice and video deepfakes lags the generation capability and probably will for the foreseeable future. The right defensive posture is to assume voice and video can be spoofed and design out-of-band verification into any consequential workflow, rather than relying on detection.

Conversational phishing across channels. Phishing that moves from email to SMS to phone call, sometimes spanning days, is hard for traditional email security tools to even see, let alone defeat. Cross-channel correlation is the next frontier and most organisations don’t have the integration to do it well.

Social media and supply-chain identity attacks. The number of attacks that originate from compromised LinkedIn accounts, freelance platforms, or third-party communication channels has grown. The visibility for the defender into these channels is limited, and the targets often perceive these channels as more trustworthy than direct email.

Where outside help is sensible

For organisations that don’t have deep internal security engineering capability, the right move is often to partner with specialists for specific components. Detection engineering, identity hardening, security awareness programmes that go beyond compliance theatre. Several Australian firms work well in this space, and the AI-augmented threat shift has created demand for partners who can think clearly about both the technology and the human dynamics — the team at Team400 AI consultancy and a handful of specialists in the identity and detection space are credible. The pattern of success is consistent: the engagement is scoped around an outcome (reduce successful phishing to X events per quarter, get to Y% passkey coverage in six months), not around a technology purchase. The technology follows the outcome.

The summary view

AI has made phishing better-written, more scalable, more personalised, and increasingly multi-channel. It has not changed the underlying social engineering dynamics, the infrastructure of attack, or the fundamental defensive principles. Organisations that have invested in phishing-resistant authentication, sensible email security, fast reporting and triage, and realistic incident response readiness are coping. Organisations relying on user training to catch better-written lures are not. The threat is not going to stop evolving and neither can the defence. The work is permanent.