This chatbot wants to be your newsfeed. Is it the answer to fake news?
The artificial intelligence-powered chatbots reached a consensus: The biggest news story of the day on Tuesday is a temporary halt on oil sanctions as part of a preliminary deal between the U.S. and Iran.
That’s according to prompts generated by some of the leading AI tools, including OpenAI’s ChatGPT, Google’s Gemini and Microsoft’s Copilot. But only one of the chatbots that Straight Arrow put to the test was built specifically with news consumers in mind.
It’s called NewsGuard AI, a chatbot released Tuesday by a for-profit media literacy company that evaluates the reliability of thousands of online news sources. Unlike general interest chatbots like ChatGPT, NewsGuard AI was built with a mission: to provide accurate responses about topics in the news.
In an interview with Straight Arrow on Tuesday, NewsGuard executives said its chatbot only cites reputable sources — and that cited publishers will get 50 percent of the profits.
“What we’re doing is the exact opposite of what the LLMs are doing,” Steven Brill, NewsGuard’s co-CEO and co-editor-in-chief, told Straight Arrow, referring to large language model AI programs. “We’re paying for content, we’re only using content from reliable publishers that we have been in the process of vetting for the last eight years, we’re citing all the sources and we’re linking back to all the sources.”
NewsGuard’s new chatbot was released at a moment of profound challenges for traditional media publishers, many of which have gone to war against Big Tech over AI tools that gobble up their proprietary content — and advertising revenue — and regurgitate that information to readers as their own. As a growing share of news consumers turn to AI-generated responses for information about current events, research indicates chatbots routinely serve up misinformation.
Leading publishers have taken a hostile stance on chatbots, and have filed lawsuits accusing Big Tech companies of copyright infringement and of rigging the online advertising market. Among them is The New York Times, which has taken aim at OpenAI and Microsoft on allegations the companies use the newspaper’s published work to train its AI tools without permission.
It’s up to news publishers to band together to fight back against AI companies that have failed “to ensure the public has access to trustworthy news,” Times publisher A.G. Sulzberger said in a speech earlier this month.
“Their hijacking of the public square is made possible by the original sin that animates their AI products — a brazen theft of intellectual property that has occurred at an unprecedented scale,” Sulzberger said in a speech at the AN-IFRA World News Media Congress in France.
Is NewsGuard AI any better — for news consumers or for the industry?
How does NewsGuard AI work?
When Straight Arrow gave NewsGuard AI a test drive Tuesday morning and asked the tool to highlight the day’s top news story, its output resembled the responses from other leading chatbots.
But its citations were more robust.
With 10 citations total, NewsGuard AI cited stories by The New York Times, CBS News, NPR and Democracy Now!, among others.
Gemini cited two sources, CBS News and the London-based think tank Chatham House.
Copilot offered just one source: Just Security, a publication by the New York University law school. The Copilot citation pointed to a roundup of the day’s top stories from various publishers.
When NewsGuard launched its AI feature Tuesday, the company said its tool was the first chatbot designed specifically to serve “reliable responses” drawn from “rigorously vetted” sources. Specifically, the chatbot aggregates content from some 12,000 sources, including news websites, state and local governments, think tanks and universities.
Of the 36,000 news sources that NewsGuard has rated since 2018, reliable sources are those that don’t “publish false or egregiously misleading information,” corrects errors, avoids “deceptive headlines” and is transparent about conflicts of interest, among other factors.
According to the NewsGuard executives, the chatbot aggregates information from 6,000 news websites, each of which has been given the opportunity to opt out. Only two outlets — neither based in the U.S. — asked to be excluded.
NewsGuard said publishers should expect quarterly checks, with the first one coming soon.
Cordon Crovitz, NewsGuard’s other co-CEO and co-editor-in-chief, said the company hopes the chatbot will generate $400 million in annual revenue — ”with $200 million going to publishers.”
“It could be more, it could be less,” Brill added, “but no matter what it is, it’s 100% profit [for the publishers] because they don’t have any costs associated with receiving that money.”
The chatbot is currently free for users, but, NewsGuard hopes to eventually charge $6 per month. Several national and local news outlets — among them, The Atlantic, the New York-based nonprofit the Investigative Post and the libertarian Reason Magazine — are already on board as “co-marketing partners.” Some of the partners will “receive half of all revenue from subscribers they introduce” to the chatbot, the company said in a media release.
While The Associated Press is among the reliable sources included in the tool, NewsGuard executives said, government-owned and operated outlets like Russia Today don’t make the cut. The company declined to provide Straight Arrow with a list of the news outlets the company includes in its tool. Straight Arrow, which received a 100% reliability rating from NewsGuard, is included in the company’s chatbot.
“If we gave you the spreadsheet of the 6,000,” Brill said, “we’d basically be giving away all of our proprietary intellectual property that we’ve gathered over the last eight years.”
Should people use chatbots for news?
A growing share of news consumers are turning to chatbots for information about current events — only to be confronted by a mixed track record for reliability.
About 10% of people use chatbots weekly for news, according to a global survey published last week by the Reuters Institute for the Study of Journalism. The results represent a 3-percentage-point increase from last year. Just 1% of respondents said AI chatbots are their main source of news.
“While this is still a fairly small minority of the population,” researchers concluded, the increase “indicates that AI is beginning to play a more meaningful role in news consumption alongside established pathways.”
NewsGuard expects the numbers to only go up from there.
“The types of consumers we are targeting are discerning news consumers who already pay for online content,” Brill said. The chatbot, he said, will allow those news consumers to discover stories from sources they don’t read regularly.
And while a growing share of consumers are turning to chatbots for information, that doesn’t mean the answers they’re receiving in return are accurate. Oftentimes, they aren’t. Recent research on Google’s AI Summaries, which is powered by the company’s Gemini chatbot, found the feature serves users inaccurate information in about 10% of searches.
Chatbots are particularly inaccurate when providing responses to “the very users who could most benefit from them,” according to a recent report by the Massachusetts Institute of Technology’s Center for Constructive Communication. Among them are users with low levels of English proficiency or formal education.
NewsGuard research released last year found that leading chatbots routinely serve up misinformation. In an audit of the 10 largest models, the company found the chatbots “spread false claims when prompted with questions about controversial news topics 35 percent of the time.”
“They’re either trained on or access all of the propaganda, healthcare hoax sites, conspiracy sites on the internet, ” Crovitz told Straight Arrow. That’s the precise problem NewsGuard AI strives to address.
“The way we thought we could solve this problem was to rely instead only on reliable sources,” Crovitz said.
In its quest to categorize news outlets on a 100-point scale based on their reliability, NewsGuard’s work hasn’t been without controversy. The company has faced allegations of bias, both by the Trump administration and conservative media outlets.
In February, NewsGuard sued the Federal Trade Commission on allegations the agency was using its regulatory authority in retaliation for its low ratings for conservative outlets. Meanwhile, in a report last week, conservative outlet The Washington Free Beacon accused the company of bias for providing lower reliability scores to rightwing U.S. outlets like Newsmax and The Federalist than state-owned “Chinese propaganda” outlets like China Daily. All of the sites receive low reliability ratings.
NewsGuard maintains its ratings system is nonpartisan and based on “transparent, apolitical basic journalistic criteria.” Company executives disputed the Free Beacon’s conclusions.
“They were quibbling over the low-rated sites, all of which are on every one of our exclusion lists,” Crovitz said. “Every foreign malign influence disinformation site from China, Russia and Iran is on every one of our exclusion lists. They’re not included, of course, in NewsGuard AI.”
Round out your reading
- Not red or blue: America’s politically homeless middle.
- Peter Thiel’s ‘Dialog’ network was super-secret. A data leak changed that.
- The novel legal strategy that Taylor Swift and Matthew McConaughey are using to fight AI.
- Illinois balances budget with new $200 million social media tax that tracks in-state users.
- When Trump serves up ‘Just the News,’ it comes with a side of bias.
Ella Rae Greene, Editor In Chief
