They say in SEO you need to use headings. Those can be H1, H2, or even H3 tags. But do they really impact your rankings? Sure, a lot of CMS systems put headings on each of your web pages by default. They do this with the title of the page (or blog post) and sometimes to sections within a page. But again, the real question is, do they help with rankings? I decided to run a fun experiment to find out if they really help.
How the experiment worked
Similar to past experiments I ran, I reached out to a portion of my email list to ask if they would like to participate. Just like how I did with the one on blog comment links and this one on link building.
4,104 of you responded wanting to participate. But unlike previous experiments, we only ran this one on websites that generated at least 100,000 visitors a month from organic search.
We picked larger sites because you can easily tell if a change had an impact on traffic. With smaller sites, external factors can more easily skew results, especially if a site only gets 100 visitors a month. One simple thing like a PR push could cause double the visitors in that case.
We also removed sites with seasonality and sites that weren’t at least 3 years old. Again, we just wanted to decrease anything skewing the results.
For example, with young sites, they tend to grow faster in organic traffic versus established sites… even when they do less SEO work because they are starting from a smaller base.
In the end, 61 sites met our requirements. It wasn’t a big number, but each site on average has 426 pages.
Now with a traditional A/B test, you would show 50% of your visitors one version and the other half a different version. But when it comes to SEO, you have to make a change and once Google indexes the change you have to compare the results to the previous 30 days.
So, with each site, we ran numerous tests at the same time to see the impact of headings. With each site, we took their web pages and split them up in 4 groups:
- Control group – we left these pages unmodified. Whether they used headings or not, we wanted to see what happened to their organic traffic over time as it would give us another baseline to compare the results.
- Headings – with this group, we used H1 tags for the title of the page, H2 tags for the subsections of the page, and even H3 and H4 tags if the subsections had subsections.
- Using normal <p> text – with all of the pages in this group, we made sure they were not using headings. In addition to that, we made sure all of the font sizes were the same size.
- Using normal <p> text and adjusting font sizes – with this group, we didn’t use headings. Instead, we made sure different parts of the text were in different font sizes. For example, the title of the page was the largest font size.
Before we dive into the results, the last thing to note is the experiment ran for 90 days. Even though we were comparing results of the pages we made the changes to using data from 30 days prior and 30 days after, keep in mind Google has to index the change, so you have to account for that as well.
Control group
The control group saw an increase in traffic of 2.89%.
As I mentioned above, no changes were made to the control group. But it shows that they naturally grew in their rankings and search traffic over time. This wasn’t much of a shocker either as 2.89% isn’t a large jump.
Headings
Now when I saw the results of the group that was using headings, the results were pretty much what I expected…
As you can see from the graph above, the before and after results weren’t much of a change when you compare it to the control group. Instead of a 2.89% gain, they had a 2.72% gain. Keep in mind some of the pages in the control group were naturally using headings and some weren’t. Again, in that group, we made no changes.
But now as we dive into the next two experiments, you’ll see that the data gets interesting.
Using normal <p> text
What was interesting about this group is that no headings were used. And on top of that, we made sure all of the font sizes on these pages were exactly the same size.
What we saw was a decrease in traffic of 3.53%. That doesn’t seem like a big swing, but when you compare it to the control group that’s a difference of 6.42%. Now I wanted to see if the drop in traffic was due to the use of headings or usability. Because you have to keep in mind that when you make all of the text on the page the same size it impacts usability as well. It makes the page less readable. And we saw that as the average time on page dropped by 12%. As for the bounce rate, we didn’t see much of a change.
Using normal <p> text and adjusting font sizes
This group didn’t use any headings but they did use different font sizes on the page to keep the pages usable (readable).
The graph shows that this group saw an increase in traffic of 2.85%. Although headings may not be the biggest SEO factor, it does seem usability is When font sizes on a page are larger, it helps tell users and potentially search engines what part of a page and even which keywords are more important.
Conclusion
When you compare all 4 groups, the control had the largest gains. But it was insignificant, and you have to keep in mind that a lot of the pages in the control group also use headings. That group just had no changes.
From what the data shows, it doesn’t look like headings have a big impact on rankings.
Maybe if I ran the experiment longer the data would have shown otherwise, but my hunch tells me the data would be similar.
One thing we didn’t try was removing headings from all pages of a site or adding headings to all pages of a site that didn’t have any in the first place. If I were to re-run the experiment I would add in these 2 tests.
From what the data shows, Google does care about usability. Having different font sizes on a page helps tell the reader which elements are more important than others. It also makes the page easier to read.
Whether you make certain elements or words on the page stand out through large font sizes or headings, it’s clear that it is a good practice.
Now if I were you, I would still use headings because it can be useful for accessibility software that helps users navigate a page. Plus, it can potentially help with other search engines like Bing.
Plus with SEO, you aren’t going to see massive gains from one single tactic like you used to be able to. It’s about doing every little thing right. That’s why I recommend you run your site through this audit and fix every error.
So, do you use headings on your site?