College student used GPT-3 to create self-help blog, but what about quality?

Editor’s note: This is one of the occasional posts we do speaking to someone with something to say on topics that we find interesting.If you are seeing this and are not yet signed up to Deepnews, click the button bellow to start receiving our blog posts every week, and a Digest of quality news on an important subject every Friday

By Christopher Brennan

Would you take productivity advice from a robot writer? The truth is that it can write much more than you can.

Following on my blog post a few weeks ago on GPT-3 answering users’ questions on Reddit, there are still a lot of questions about how else the text generation technology from Open AI is being used and what it means for the future of articles on the internet.

One similar story is from Liam Porr, a senior at University of California Berkeley. While the Reddit bot (using Philosopher AI) would output short answers of up to six paragraphs, Porr took it even further. He input a headline and an introductory paragraph as a prompt and then had GPT-3 go to work, producing articles for a Substack blog like “Boldness and creativity trump intelligence” and “The reading habits of successful people” that don’t seem at all out of place among the large numbers of inspirational, self-help pseudo-think pieces on blogging platforms. 

“I was mostly just trying to play to GPT-3’s strengths in terms of the content. I know that its strength is in writing pretty language, and not in rigorous logic,” he told Deepnews in a recent interview.

“So, if I were to try to write something about some philosophical concept, it’d be really hard to keep it on topic, and for it to take a single thesis and develop that thesis, it’s not very good at that. So I figured I’d be best off with a large category that wouldn’t allow it to repeat itself and I think the self-help just kind of fit the bill.”

Porr, who did not have access to GPT-3 but worked with a PhD student at Berkeley who did, was successful with his first attempt, an article titled “Feeling unproductive? Maybe you should stop overthinking” that reached the top of Hacker News. While some commenters there suspected that the post was AI-generated, other members of the community down-voted those comments, supporting the idea that the post was human.

The student eventually used his increasingly popular blog (“Adolos” as in Dolos the Greek god of deception) to unveil his trick to the world late this summer in a post titled “What I would do with GPT-3 if I had no ethics.”

There are differences between Porr’s experiment and the Reddit bot however. One is that Porr was posting all the content under his real name and taking responsibility for it. The other is that he admits to having more human involvement in the content itself. He told Deepnews that for each self-help blog post he generated eight outputs and then selected the best one, making sure to edit out any “egregious errors” such as attributing a Bob Dylan quote to Ralph Waldo Emerson.

Beyond the oft-cited worry of using GPT-3 to ooze information into every corner of the internet, this more involved approach also brings to mind other less nefarious uses of AI-generated text, such as automated copywriting for marketing. Instead of having humans generate text that can be A/B tested to find the most engaging combination, the entire process could be done automatically, which means one of the real efficiencies of GPT-3 is easing the step of having something, anything, written that can then be optimized and tweaked by an editor.

“It’s really good for that first part where you have a blank page, and you need some inspiration and you need somewhere to start. GPT-3 would be really good for getting you off your feet. I think that for a lot of writers like starting from the blank page is the most difficult part,” Porr said.

But the ease of creating content, whether journalism, fake news or commercial, means that there could be a lot more of it. One of the interesting questions for Deepnews is of course the question of quality, and what happens when there is a bunch of low quality writing.

Will people be satisfied to float around in a flood of text from human and robot sources, or will they want something more value-added? Current metrics based on engagement may not be enough to differentiate what’s worth reading from fluff.

“I think what’s gonna end up happening is, once it becomes pervasive enough, the threshold for high quality writing is going to be raised basically,” Porr said.

“So, whatever GPT-3’s quality output is, people’s threshold is going to be above that. So, in that, in that sense, we’re going to have an abundance of content and we’re going to be really pushed for quality, which is where an application will really shine.”