No matter what I search, the direct site does not come up, even searching direct titles of articles returns nothing.
Pure conjecture, the WSJ may have rules for index bots to control themselves when visiting the site. Or it’s a Brave Search index bug.
I did get a summary of the OpenAI potential price drop, which was June 10’s news and continued into today. Brave Search did summarize the WSJ story, But didn’t link to it.
Web search is so hard these days to find precise content.
I guess posting on Brave’s forum may be more likely to fix it.
Can anyone that understands robots.txt see if it’s because of what they put in it? Cuz if it’s wsj’s choice I won’t bring it up with Brave. https://www.wsj.com/robots.txt
(post deleted by author)
The robots.txt should allow Brave, as they allow Googlebot and Brave follows rules specified for Googlebot to ensure they are able to index the same content as Google.