Headline writing is something of an art form. But how many editors and writers wonder if the words they’ve selected for a story will be strong enough to really grab the attention of their readers?

Late last year, Chartbeat, a data analytics company, issued a report on what makes a powerful headline, one that will not only bring readers to a page, but encourage them to stick around and read more content than the article that first drew their attention.

The goal of Chartbeat’s report was to help publishers determine what users are engaging with and want to read, said Chris Breaux, a data scientist and team lead with Chartbeat.

“One of the most powerful things about headline testing is it allows you to try out different ideas for the same article,” pitting multiple headlines on the same copy against each other to determine which is more effective for driving traffic, Breaux said. “When you’re running a real test … stripping away the compounding variables, you allow headline editors to see what it really is that’s causing a headline to be successful. It gives them a way to validate that quantitatively.”

Using data collected from running more than 10,000 A/B tests run on Chartbeat’s platform from more than 100 different publishers who gave the organization permission to anonymously collect their readers’ practices, Chartbeat conducted an exploratory analysis “trying to find correlations between what the prominence of this addition is and how many trials you might need to determine succinctly that one headline is better than another, and how that relates to the number of headlines you’re testing,” he said.

The second part of the trial involved using metrics to determine what is really driving the increased click rate. “What you might find is that it might be high, but it might be high because those headlines are more likely to be used in prominent positions," he said. "By narrowing that scope to comparing the performance of headline ‘A’ in a given spot versus headline ‘B,’ you’re really stripping away some of the compounders again and getting at the essence of what made that headline work.”

Breaux said the biggest takeaway from Charbeat’s research, at least for him, was “the types of headlines we looked at didn’t turn out to be that predictive of future performance. We looked at 12 or so different headline types and only one had statistically significant performance.”

That difference maker? The use of demonstrative adjectives. Use of words like “this, these, that, those” can “introduce some kind of intrigue to what you’re saying. Instead of saying something outright, you might be able to allude to it,” piquing a reader’s curiosity and drawing him or her in without giving away too much.

— Amber Healy

On this week's It's All Journalism podcast, Producer Michael O'Connell talks to Chris Breaux, a data scientist and team lead at Chartbeat, about using A/B testing to determine which headline is the most successful at generating page views. Breaux talks about a recent study that compared the results of 10,000 headline tests run by 100 of Chartbeat's customers. Is a question headline the best way to garner clicks or will a short, declarative statement more draw eyes to your content? View similar data-based studies on Chartbeat's blog.