Using Machine Learning to Spotlight Quality Journalism on the Web

• How will publishers use Deepnews.ai practically?
A batch of stories will be submitted to the platform and they will be scored on a scale of 1 to 5 based on their journalistic quality (see definition below). This process will be performed automatically and in real time.

• How does Deepnews.ai define quality?
We define quality narrowly; in its simplest terms, we look for value-added journalism. This means coverage built on a genuine journalistic approach: depth of reporting, expertise, investigation, analysis, ethical processes, and resources deployed by the newsroom.

 

• How does Deepnews.ai work?  
The platform is based on a combination of two models.

— The first model involves two sets of “signals” to assess the quality of journalistic work: Quantifiable Signals and Subjective Signals. Quantifiable Signals are collected automatically. These signals include the structure and patterns of the HTML page, advertising density, use of visual elements, bylines, word count, readability of the text, information density (number of quotes and named entities). Subjective Signals are based on criteria used by editors (and intuitively by readers) to assess the quality of a story: writing style, thoroughness, balance & fairness, timeliness, etc. (This set will be used only in the building phase of the model).

— The second model is based on deep learning techniques, like "text-embedding" in which texts from large volumes of data (millions of articles) are converted into numerical values to be fed into a neural network. This neural net returns probabilities of scoring. ​

 

• Will the scores of individual articles be visible to readers?
No. Usage of the score will be reserved for publishers and news distributors; primarily, the score is to be read by other machines. Deepnews.ai has no business ranking journalists, even if their long-term reputation is a signal taken into account by Deepnews.ai algorithms. We believe ranking individual journalists would be problematic for many reasons, including the fact that reporters operate under a wide array of instructions and demands that vary from editor to editor and publication to publication. Most journalists would be happy to produce nothing but value-added reporting; however, the realities of the news business mean some journalists can only devote a fraction of their time to the pursuit of truly value-added reporting.

 

•Will you make the components of the models public?
Partially. While we might publicly share some details of the engineering process, it is essential that we prevent any gaming of the algorithm. Several vital parts will, therefore, remain under wraps. Once our system is up and running, everyone will be able to test the platform for free. We expect only the processing of large volumes of articles will require the payment of a fee.

 

• Why rely on A.I. and not on human assessment?
Humans don’t scale. News aggregators process about 100 million new pieces of information per day, half of them in English. Artificial intelligence is the only way to process such a large stream. In the “Signals” model, none of the indicators, considered separately, say much about the quality of a story; only their combination does. Assigning the proper weight to each signal can’t be done in a deterministic way, but it is a perfect job for a neural network. This is why the model is based on A.I.

 

• Does it mean that Deepnews.ai will totally exclude human judgment?
No. Human assessment will play a critical role in the training of the algorithms. A dedicated interface, dubbed Human Scoring Interface, will be deployed soon to allow the evaluation of stories by the public. (In the first phase, the interface will be promoted in a circle of news professionals —journalists, editors, students, academics, etc.). The goal is to hand-score as many articles as possible. This stream of articles will be used as the "evaluation set" against which the machine learning algorithm will be trained and also as a "validation set" to confirm (or invalidate) the score given by the deep learning model.

 

• Does Deepnews.ai address issues such as accuracy and fake news?
Indirectly, yes. From what we have observed, fake news has a different structure from legitimate journalism. We are certain that it can be measured; for instance, we found some distinctive patterns in the sentence structure from different news sources. Also, by relying on the reputation of publishers and authors, and by taking into account their performance over time, Deepnews.ai should be able to downgrade suspicious content items.

 

• Who will be the primary customers of Deepnews.ai?
News publishers (whatever their size), news distributors, and advertising industry players who are interested in being associated with high-quality editorial environments.

 

• What is the revenue model?
News publishers, news distributors, and advertising industry players will pay a fee to access the platform. Pricing will be set according to the volume of articles processed on the platform.