Now I’m not saying that the example here is not worrying, but nobody seems to consider people who actually use webpages that use punycode.
As someone whose browsing activity does involve visiting some Japanese sites, I was not aware that Japanese sites actually used punycodes in their URLs. I can’t immediately think of any. I assumed this practice was similar for other languages, because no one ever brings them up in these discussions in my experience.
Mojeek doesn’t display them at all because it only indexes languages that use Latin characters, but they might re-evaluate that when they start indexing, say, Japanese webpages.
Firefox displays both the encoded characters and the punycode when you go into about:config and force the option on:
I think this UI is bad because you can easily miss the URL at the bottom (especially if you have autocomplete on), but it demonstrates that you can display both. For example, if a search engine detects that the domain uses a punycode, highlight it in some way to the user and display both of them. Educating the average user about punycodes in five seconds is a tall ask, but trying something is better than doing nothing at all. Unwitting Keepass users might have had a chance if the UI was better.
I think you’re right about not ignoring this altogether if there are actually legitimate websites out there using punycodes. I just wasn’t aware it was something used by anyone other than scammers (which is my bad for my ignorance, but where does one even find these examples?). The Mastodon instance you mentioned demonstrates that punycodes are in legitimate use.
I still think Google with their size could do a much better job at this stuff, and it’s especially bad that they’re kind of enabling it in the first place with how they sell advertising space to literal scammers.
I have seen far too many reports of malvertising on Google and personally witnessed malvertising on YouTube to trust Google with my security, which is why…
Maybe search engines are just not the holy grail that saves us all from malicious websites.
I think you’re completely right.
I think many of us need to grok at some point that web search is a service that you probably want to pay for, the same as you pay for privacy-respecting email, cloud storage etc. and not just falls from the sky.
Brave Search is about the only independent option you can pay for at the moment. According to the Kagi forum, they display punycodes. I pay for Kagi, but it’s only partially independent (mostly dependent on Google).
Mojeek aligns very strongly with many of my values, so I’d love to pay for a Mojeek that serves me Japanese results one day.
Unfortunately, Kagi was disqualified from PG precisely because it is paid, so I’m not sure a paid search engine will ever be recommended under the current guidelines.