How AI and data boost journalism sustainability
Big databases help news organizations predict who might subscribe or donate
You’re reading the My News Biz newsletter. My goal is to help digital media entrepreneurs find viable business models. This edition originally appeared May 27, 2021, before my health sabbatical. It has been updated and shortened.
A book on artificial intelligence caught my attention recently because its findings have so much relevance for news media and journalists. The book makes robots seem more like a friend and less of a threat to producers and writers.
I’m talking about Prediction Machines: The Simple Economics of Artificial Intelligence by three economists, Ajay Agrawal, Joshua Gans, and Avi Goldfarb. (I will deal with ChatGPT and other amazing new AI tools in a later newsletter.)
The relevance for news media is that prediction machines (algorithms) developed by Google and Facebook ravaged the business model of news organizations, but could also rescue them.
The tech platforms’ strategic advantage is that they know more about publishers’ users — their preferences, online behavior, personal and professional relationship — than the publishers themselves, which often collected little more than a subscriber’s name, address, and telephone number.
The tech platforms used machine learning to refine their algorithms and target ads relevant to the interests and habits of consumers, thus generating more sales for advertisers at a lower cost than the publishers. Of course advertisers fled to the platforms.
For publishers to compete, they have to find ways to acquire enough data at an affordable cost to develop their own algorithms (prediction machines), and not just for advertising. As they transition to user-generated revenue, they need to be able to predict which online users are most likely to subscribe and target them with appealing offers.
Not enough business intelligence
A study of 130 artificial intelligence projects by news organizations found that nearly half of them were focused on “augmenting reporting capacity” and about a fourth on reducing costs. Only 12% focused on optimizing revenue.
Among other potential applications of AI, the report, funded by the Knight Foundation, suggested using shared data to develop dynamic paywalls and subscriber prediction algorithms.
In machine learning, the algorithm keeps refining its predictions the more data it collects. Most publications don’t have these kinds of richly detailed databases nor the technology staff to develop sophisticated algorithms. But Google does, and has been using what they have learned to coach publishers. (More on their work with news media here: These metrics predict who will pay for news.)
Let Sophi decide
Canada’s Globe & Mail offers an excellent example of how this can work if a news organization has enough vision and financial resources to take advantage of AI. In an interview with the Press-Gazette, Publisher and CEO Phillip Crawley described how the Globe & Mail accelerated the growth of digital subscribers to 170,00 through a software program they developed. It’s called Sophi.
A bit of background. In 2012 the Globe & Mail established a metered paywall, which allowed users to view a certain number of articles a month before requiring them to pay for a subscription. But the executives weren’t entirely satisfied, so they brought in some tech experts to develop a program to drive up digital subscriptions.
The result was Sophi, a computer program which decides whether articles should be free to access or put behind the paywall based on article content and reader information.
Crawley said Sophi is largely responsible for the rapid subscription growth. The program learns from user behavior how to decide which articles to place in which positions on which pages in order to maximize engagement, loyalty, and propensity to subscribe. Each day, Sophi adds more data on user behavior to refine its predictions–machine learning.
The program reviews all content every 10 minutes to determine which articles merit greater promotion on the website or on social media, Crawley told the Press-Gazette. Now the G&M is selling the software to other media.
The villains become saviors
It is somewhat paradoxical that Google, a destroyer of publishers’ business model, has been helping them in a significant way. It offers a low-cost service for small publishers called Newspack, which harnesses machine learning to, in its own words, combine “the best of what the industry has learned about publishing and revenue generation”–machine learning, in other words.
Probably no one knows more than Google about internet users’ tastes and buying habits. The Institute for Nonprofit News, working with the Google News Initiative, has developed tools for raising money from events and sponsorships. Another Google News Initiative partner is Lion Publishers, an association of 275 local news entrepreneurs in the U.S and Canada. The platform sponsored an eight-week bootcamp for 16 digital news startups in Canada.
Increasing probability
Like some doomsayers, Journalists typically see artificial intelligence as destroying jobs in the profession by having machines produce routine stories in finance, sports, weather, and other fields that have large structured databases. But actually, AI might be helping to save quality journalism.
In economic terms, the news has little economic value. Within moments, everyone in the world knows what President Biden or the Federal Reserve did, and why would anyone pay to get access to that?
As the authors of Prediction Machines demonstrate, algorithms can use data to make educated guesses using millions of data points on thousands of variables far faster and better than any human marketing director ever could. And they can improve the probability of accuracy the more data they have.
How could this newsletter serve you better? This survey will take less than 1 minute, and it will let you tell me what you need and what problems you face. I promise to keep your response confidential and anonymous. Click here for the survey.