• [Formerly Known as News Quality Scoring Project]

    Using Machine Learning to Spotlight Quality Journalism on the Web

  • The Project

    Restoring the economic value of quality journalism

    Goals

    • The news media’s economic landscape is devastated. Over the last fifteen years, newsrooms across the Western world have lost half their workforce. Profits have evaporated, resources for reporting have dwindled, and investment in technology has been desperately low. Deepnews.ai wants to make a decisive contribution to the sustainability the information ecosystem in two ways:

     

    • On the economic side

    Currently, there is no correlation between the cost of producing great content and its economic value. A story that required months of work and cost hundreds of thousands of dollars to report and edit carries the same unitary value (a few dollars per thousand page views) as a gossipy article about the Kardashians.
    But times are changing. In the digital ad business, indicators are blinking red: CPMs, click-through rates, and viewability are on a steady downward decline. We believe that inevitably, advertisers and marketers will seek refuge in havens of high-quality content—as long as they can rely on a credible indicator of quality. Deepnews.ai aims to provide this gauge.

     

    • On the editorial side

    The ability to assess the quality of news will open up a vast array of opportunities for new products and services. ➜

    Applications

    • Recommendation engines: Deepnews.ai will surface stories based on quality, which will increase the number of articles read per visit. (Currently, visitors to many news sites read less than two articles per visit).

     

    • Personalization: We believe a reader's profile should not be limited to consumptions analytics but should reflect his or her editorial preferences. That's why Deepnews.ai is working on the concept of a Quality Tag (Qtag™) able to connect stories' metadata with a reader's affinity.

     

    • Curation: Publishers will be able to use Deepnews.ai to enter the curation sector, which is currently left to players like Google and Apple. By providing technology that can automatically surface the best stories from trusted websites (even small ones), Deepnews.ai can help publishers expand their footprint.

     

    • New editorial products: Powered by Deepnews.ai technology, publishers may offer new premium subscription products that give readers access to valuable, in-depth pieces of journalism.

     

    • Advertising: Deepnews.ai’s scoring system will interface with ad servers to assess the value of a story and price and serve ads accordingly. The higher a story’s quality score, the pricier the ad space adjacent to it can be. This adjustment will substantially raise the revenue per page.

  • FAQ

    • How will publishers use Deepnews.ai practically?
    A batch of stories will be submitted to the platform and they will be scored on a scale of 1 to 5 based on their journalistic quality (see definition below). This process will be performed automatically and in real time.


    • How does Deepnews.ai define quality?
    We define quality narrowly; in its simplest terms, we look for value-added journalism. This means coverage built on a genuine journalistic approach: depth of reporting, expertise, investigation, analysis, ethical processes, and resources deployed by the newsroom.

     

    • How does Deepnews.ai work?  
    The platform is based on a combination of two models.

    — The first model involves two sets of “signals” to assess the quality of journalistic work: Quantifiable Signals and Subjective Signals. Quantifiable Signals are collected automatically. These signals include the structure and patterns of the HTML page, advertising density, use of visual elements, bylines, word count, readability of the text, information density (number of quotes and named entities). Subjective Signals are based on criteria used by editors (and intuitively by readers) to assess the quality of a story: writing style, thoroughness, balance & fairness, timeliness, etc. (This set will be used only in the building phase of the model).

    — The second model is based on deep learning techniques, like "text-embedding" in which texts from large volumes of data (millions of articles) are converted into numerical values to be fed into a neural network. This neural net returns probabilities of scoring. ​

     

    • Will the scores of individual articles be visible to readers?
    No. Usage of the score will be reserved for publishers and news distributors; primarily, the score is to be read by other machines. Deepnews.ai has no business ranking journalists, even if their long-term reputation is a signal taken into account by Deepnews.ai algorithms. We believe ranking individual journalists would be problematic for many reasons, including the fact that reporters operate under a wide array of instructions and demands that vary from editor to editor and publication to publication. Most journalists would be happy to produce nothing but value-added reporting; however, the realities of the news business mean some journalists can only devote a fraction of their time to the pursuit of truly value-added reporting.

     

    •Will you make the components of the models public?
    Partially. While we might publicly share some details of the engineering process, it is essential that we prevent any gaming of the algorithm. Several vital parts will, therefore, remain under wraps. Once our system is up and running, everyone will be able to test the platform for free. We expect only the processing of large volumes of articles will require the payment of a fee.

     

    • Why rely on A.I. and not on human assessment?
    Humans don’t scale. News aggregators process about 100 million new pieces of information per day, half of them in English. Artificial intelligence is the only way to process such a large stream. In the “Signals” model, none of the indicators, considered separately, say much about the quality of a story; only their combination does. Assigning the proper weight to each signal can’t be done in a deterministic way, but it is a perfect job for a neural network. This is why the model is based on A.I.

     

    • Does it mean that Deepnews.ai will totally exclude human judgment?
    No. Human assessment will play a critical role in the training of the algorithms. A dedicated interface, dubbed Human Scoring Interface, will be deployed soon to allow the evaluation of stories by the public. (In the first phase, the interface will be promoted in a circle of news professionals —journalists, editors, students, academics, etc.). The goal is to hand-score as many articles as possible. This stream of articles will be used as the "evaluation set" against which the machine learning algorithm will be trained and also as a "validation set" to confirm (or invalidate) the score given by the deep learning model.

     

    • Does Deepnews.ai address issues such as accuracy and fake news?
    Indirectly, yes. From what we have observed, fake news has a different structure from legitimate journalism. We are certain that it can be measured; for instance, we found some distinctive patterns in the sentence structure from different news sources. Also, by relying on the reputation of publishers and authors, and by taking into account their performance over time, Deepnews.ai should be able to downgrade suspicious content items.

     

    • Who will be the primary customers of Deepnews.ai?
    News publishers (whatever their size), news distributors, and advertising industry players who are interested in being associated with high-quality editorial environments.

     

    • What is the revenue model?
    News publishers, news distributors, and advertising industry players will pay a fee to access the platform. Pricing will be set according to the volume of articles processed on the platform.

  • Participate in the project

    Deepnews.ai is a collaborative work. Get involved. Have your say.

    Get updates, be part of various testings

    No spamming, promised.

  • News

    Articles about the project and misinformation

    Frederic Filloux / Monday Note

    Frederic Filloux / Monday Note

    Frederic Filloux / Monday Note

    Frederic Filloux / Monday Note

  • Your opinion matters to us

    Let us know what you think