• [Formerly Known as News Quality Scoring Project]

    Using Machine Learning to Spotlight Quality Journalism on the Web

  • The Project

    Restoring the economic value of quality journalism


    • The news media’s economic landscape is devastated. Over the last fifteen years, newsrooms across the Western world have lost half their workforce. Profits have evaporated, resources for reporting have dwindled, and investment in technology has been desperately low. Deepnews.ai wants to make a decisive contribution to the sustainability the information ecosystem in two ways:


    • On the economic side

    Currently, there is no correlation between the cost of producing great content and its economic value. A story that required months of work and cost hundreds of thousands of dollars to report and edit carries the same unitary value (a few dollars per thousand page views) as a gossipy article about the Kardashians.
    But times are changing. In the digital ad business, indicators are blinking red: CPMs, click-through rates, and viewability are on a steady downward decline. We believe that inevitably, advertisers and marketers will seek refuge in havens of high-quality content—as long as they can rely on a credible indicator of quality. Deepnews.ai aims to provide this gauge.


    • On the editorial side

    The ability to assess the quality of news will open up a vast array of opportunities for new products and services. ➜


    • Recommendation engines: Deepnews.ai will surface stories based on quality, which will increase the number of articles read per visit. (Currently, visitors to many news sites read less than two articles per visit).


    • Personalization: We believe a reader's profile should not be limited to consumptions analytics but should reflect his or her editorial preferences. That's why Deepnews.ai is working on the concept of a Quality Tag (Qtag™) able to connect stories' metadata with a reader's affinity.


    • Curation: Publishers will be able to use Deepnews.ai to enter the curation sector, which is currently left to players like Google and Apple. By providing technology that can automatically surface the best stories from trusted websites (even small ones), Deepnews.ai can help publishers expand their footprint.


    • New editorial products: Powered by Deepnews.ai technology, publishers may offer new premium subscription products that give readers access to valuable, in-depth pieces of journalism.


    • Advertising: Deepnews.ai’s scoring system will interface with ad servers to assess the value of a story and price and serve ads accordingly. The higher a story’s quality score, the pricier the ad space adjacent to it can be. This adjustment will substantially raise the revenue per page.

  • FAQ

    • How does Deepnews.ai define quality?
    We define quality narrowly; in its simplest terms, we look for value-added journalism. This means coverage built on a genuine journalistic approach: depth of reporting, expertise, investigation, angle, ethical processes, and resources deployed by the newsroom.


    • How does Deepnews.ai work?
    Under the hood of the Deepnews system is a
    convolutional neural network. This type of deep learning model is usually employed for image recognition. (Read this good explanation by a UCLA student, or, if you are math freak, watch this series of lectures from Stanford). The Deepnews.ai model is made with 360 filters and 22~25 million parameters or weights (depending on the version) which is roughly speaking, the grid created by the model to look at a story. Our Deepnews Scoring Model detects and classifies the features, creates links between them, organizes a library of weights for each of them and infers probabilities on their interactions, mutual influence, and meanings. To put these interactions in concrete terms, our model has the equivalent of about 70,000 pages of text. We built and tested 55 versions of the model for more than 1300 hours on Google Cloud, during which hundreds of thousands of articles have been crunched.


    • How accurate is the Deepnews Scoring Model?
    The model is right about 80~85 percent of the time. Which is not bad for a deep learning system. A sophisticated analysis of news articles is a challenging task for artificial intelligence. While perfectly clean datasets of images, for instance, will yield nearly 100 percent accuracy, deep-learning has a difficult time dealing with fuzzy and subjective material such as news.

    Sometimes, our scoring model goes awry, granting a mundane story an outstanding score, or trashing a Pulitzer-worthy piece. Countering these aberrations is obviously difficult. But it gets better after each iteration.


    • Does Deepnews.ai totally exclude human judgment?
    No. We frequently calibrate the model against human testers by submitting different batches of articles to English-speaking journalism students. To warrant some statistical reliability, three of them score one article; simultaneously, the model does the same. So far, the deviation between humans and machine ranges from 0.5 to 0.8 points, depending on the type of article.


    • Does Deepnews.ai address issues such as accuracy and fake news?
    Indirectly, yes. From what we have observed, fake news has a different structure from legitimate journalism. We are certain that it can be measured; for instance, we found some distinctive patterns in the sentence structure from different news sources. Also, by relying on the reputation of publishers and authors, and by taking into account their performance over time, Deepnews.ai should be able to downgrade suspicious content items.

  • Participate in the project

    Deepnews.ai is a collaborative work. Get involved. Have your say.

    Get updates, be part of various testings

    No spamming, promised.

  • News

    Articles about the project and misinformation

    Frederic Filloux / Monday Note

    Frederic Filloux / Monday Note

    Frederic Filloux / Monday Note

    Frederic Filloux / Monday Note

  • Your opinion matters to us

    Let us know what you think