ChatGPT creates phisher’s paradise by serving wrong URLs

The team found that the AI would produce the correct web address just 66 percent of the time. 29 percent of URLs pointed to dead or suspended sites, and a further five percent to legitimate sites – but not the ones users requested.

Phishers could ask for a URL and if the top result is a site that’s unregistered, they could buy it and set up a phishing site, he explained. “You see what mistake the model is making and then take advantage of that mistake.”

The problem is that the AI is looking for words and associations, not evaluating things like URLs or a site’s reputation.

2 Likes

AI-generated URL suggestions… What could possibly go wrong?..

1 Like

This could be fairly easily mitigated to some degree? since their training data has a date cutoff they could just automatically filter responses that contain domains that are registered after such date.