HubPages is an open publishing platform, meaning that we allow Hubs on a wide range of topics to be published. People new to the site can also enjoy the gratification of being able to share their Hubs with others and receive feedback almost immediately after publishing.
For the past few years, the HubPages staff (yep, all of us) has been rating a sampling of Hubs every month in order to get a sense of quality changes over time. Using a specially-designed admin-only Hopper that’s similar to the one available to Hubbers, we rate Hubs on a scale of 1-10 (from abysmal to mind-blowing; I’ve shared some of our internal guidelines at the bottom of this post). The Hubs are a mixture of old and new Hubs, to weed out any rating drift (i.e. any tendency to rate Hubs more critically or more generously over time). The rating systems gives us the opportunity to benchmark our efforts to raise the quality bar and see how Hubs and traffic respond.
The graph below is a trendline showing the ratings of Hubs that received traffic during the months on the x-axis, weights them with the traffic they receive, and averages them (i.e. rated Hubs with more traffic the month will influence the average more than those with less traffic during that month):
Quality was relatively high during the first two months, when HubPages was in private beta and only open to family and friends of the founders and a select group of first users. Quality then dipped when HubPages was open to the public, and when we got a lot of adult content and adult traffic. When we took down adult content in July 2007, it accounted for 10% of our Hubs but about a third of our traffic!
We also really didn’t begin to moderate until 2007, and didn’t have a scalable way of doing so until later in that year. A lot of Hubs that were lower in quality than we wanted were published, and got traffic, until we started enforcing higher standards. Standards have continued to rise, and traffic to higher-quality Hubs has also risen remarkably: the traffic-weighted average rating is almost double that of the trough in 2006-2007, and even 30% more than when we were in private beta.
Another interesting graph looks at the quality ratings of Hubs published by month of publication. The trendline in green shows the rating of Hubs that were ever published in the month on the x-axis; the blue trendline shows only the rating of Hubs that are still published. The difference is attributed to Hubs that were unpublished, primarily by our moderation team:
We can glean some interesting insights from this:
- Average Hub quality has never been higher, even better than when HubPages was in private beta, limited to people we trusted.
- The moderation team has really improved the quality of Hubs still published. Maddie Ruud came on as our first full-time moderator in early 2008, and has been “kicking ass and taking usernames” ever since. Norah Casey joined us in early 2010 with moderation experience from working at Justin.tv. She has recently been managing our team of moderators, which tripled in size as we enforced tougher standards this year in response to Panda.
- The gap between what’s ever been published and what’s still published has narrowed, primarily due to the fact that increasingly sophisticated publishing technology can prevent publishing of some types of Hubs that don’t meet our standards. Our engineering team continues to work on things that prevent Hubs that don’t meet our minimum quality standards from even being published.
As HubPages recovers from Panda, this is the first time that we’ve seen a significant shift in traffic patterns towards higher-quality. August 2011 was the first month where the average weighted Hub rating for those rated Hubs receiving traffic was above 7. We think that in the future, effort, authenticity and quality will continue to be rewarded when it comes to useful, interesting content on HubPages.
———————–
Sample of ratings guidelines:
HubPages Blog 
Now would I love to get my hands on my ‘internal’ ratings LOL
Very interesting.
I’m hoping that my current traffic losses with Google are temporary – but I’ll do what I do regardless.
I only wish I’d got into Hubpages earlier.
Wow, I am really impressed at the professional manner that HubPages uses to evaluate Hubs. And I appreciate this information being available. I often have wondered about these things (and others:). This makes me feel much better about Flagging low quality or personal Hubs. I have a better understanding as to the impact on every writer and HubPages in general. thank you, thank you.
More of these revelations, please!
I’ve been writing on HP for two years now and never experienced the drop in traffic that I experienced right after this past labor day. Still not sure what caused it because my traffic numbers were increasing after the switch to subdomains. I’m wondering if Google fiddled with the algorithm again and that caused the drop. I’ve read elsewhere that subdomains are no longer a factor in the algorithm. It’s very frustrating when you strive to publish quality content and Google ignores it.
Patrice: What you’re experiencing is not totally unusual. There is a group of Hubbers who are seeing their traffic plunge and soar. I would check your Hubs to make sure they’re top-notch and updated; if you’ve already done that, sit tight and hope that Google will do its job of recognizing the quality of your Hubs. 🙂
stayalivemoma: We don’t review all Hubs, and don’t categorize ratings by author. This exercise is just for us to track overall quality trends among those Hubs published, and to see if Google is getting better at sending traffic to truly high-quality Hubs. We will continue to share actionable insights with Hubbers in the Learning Center, though.
Kangaroo Jase: I agree with you, and it appears that Google is getting better at directing traffic to quality (as we define it). Hopefully that will create a good feedback loop, where Hubbers will see traffic flowing to their best Hubs, and will want to replicate that success with more high-quality Hubs. 🙂
And to everyone else who commented: thank you for reading and for your feedback!
This is great information and I am happy that my work is being measured. Any chance that we could receive feedback?
This is very interesting Jason. Thanks for the update.
Very interesting numbers Jason, someone has really spent their time on this with really good statistical analysis. What I will find interesting is where this ends up with, in the future.
Personally, I see the standards and the bar being incrementally and constantly raised. Standards will improve over a period of time.
This will mean that give a time in the future and even though HubPages is already a great platform for many reasons, there will be a standard of quality for producing hubs that will appeal to existing and new to come Hubbers (and of course our readers), yet those who look to use this site for bad quality and quick links for their affiliate accounts and adsense, will be penalized.
Some of these low quality items will slip through the system, yet quality will win out more often than not.
I think there are some that havent seen anything yet
This was an interesting article and I think it was good information to share with Hubbers as it gives some specific things that are being looked at in the ratings and an overall picture of how your random spot ratings are showing improvement over time. Thanks for sharing this feedback with us.
Thanks for your input, Jason. I admit, I still have some things to do with my hubs like add meta descriptions on about half of them, but if they were doing well before without the descriptions, I don’t understand why the sudden drop. That would be something HubPages could put its analytical efforts toward figuring out.