Tag Archives: Quality Assessment Process

Boot Camp: A Five Featured Hub Barrier to Fight Spam

Heads up! This week, we are implementing a new quality-related policy change.

Until you have published five simultaneously-Featured Hubs, new Hubs that do not pass the Quality Assessment Process (QAP) will be unpublished rather than simply being not Featured.

The period before one has published five Featured Hubs will be known as Boot Camp, as Hubbers who have yet to reach the five Featured Hub milestone have to engage in some training and hard work before passing.

While this policy is designed primarily to help us deal more efficiently with spam, we hope it will provide new Hubbers with a gentle notice of HubPages’ high quality standards and to help them produce the sort content that is likely to succeed.

More on the Five Featured Hub Minimum to Fight Spam

Until you have published five Featured Hubs and passed Boot Camp, Hubs that would not otherwise be Featured will be moved back to an unpublished state after going through the QAP.

Additional details on how this works:

  • Before you have met the five Featured Hub minimum, Hubs you publish that pass our Quality Assessment Process will still be published and Featured on the site. Only Hubs that do not pass the QAP will be transitioned back to draft mode.
  • Once you have five Featured Hubs and have passed Boot Camp, Hubs you publish that do not pass the Quality Assessment Process will no longer be blocked from publication (though until you improve their quality, they will not be made available to search engines or be shown on related Hubs and Topic Pages).
  • Hubs that were unpublished because of quality reasons before you met the five Featured Hub threshold will remain unpublished until you change them and re-publish them (we recommend making significant improvements or else they’ll just not be Featured, which is somewhat of a waste).
  • Should you reach a point at which you have five Featured Hubs but then dip down to a point at which you only have four Featured Hubs, you will still be able to publish Hubs that aren’t Featured (in other words, you only have to prove yourself once).

This new policy will keep spammers from progressing on HubPages by preventing their work (which very rarely passes the minimum quality threshold required to be Featured) from being published.

A New Accolade for High Quality Hubbers

In addition to using Boot Camp to hinder spammers, we are using it to reward awesome content creators on HubPages who really do want to improve their skills and make a name for themselves online. We are doing this via the introduction of a new Accolade that highlights the number of Featured Hubs you have.

Our Featured Hub Accolade, shaped like a bullhorn (because we like to amplify Featured Hubs by making them available to search engines and featuring them throughout our site) is HubPages’ newest badge indicating a Hubber’s ability to create useful, valuable, and high quality content.

You will first earn the Accolade as soon as you reach the five Featured Hub minimum. The Accolade’s number will continue to climb as you create and publish more Featured Hubs. As we care ten times as much about the great articles people write, we’re far more interested in seeing how many Featured Hubs authors have versus their total number of Hubs. For this reason we find the Featured Hub Accolade to be a helpful reference point (not to mention a good motivator). We hope you find it to be rewarding, informative, and motivating, too!

Updates to Featured Hub Grace Periods and Quality Boundaries

Next week, we will be rolling out to updates to Featured Hubs and the Quality Assessment Process:

  1. We will be giving Featured Hubs a longer grace period based on Hubber Score
  2. We will be raising the quality bar applied to newly-published Hubs

 

About the Longer Grace Period

As it stands, Featured Hubs have between 30 to 60 days to prove that they have a heartbeat. We define a Hub with a heartbeat as one that has demonstrated sufficient interest from readers and is getting some search traffic (e.g. a small handful of search hits per week, around a dozen per month, or a little over 50-90 per year, depending on Hubber Score).

Starting next week, Featured Hubs will be given a grace period between 30 days to one year based on Hubber Score. To make Hubber Score a more accurate representation of the overall quality of one’s Hubs, we have adjusted the manner in which it is calculated to factor in human ratings collected through the Quality Assessment Process.

Because new Hubbers tend to have more volatile Hubber Scores, their grace period will be artificially inflated until these scores normalize, so rest assured that promising new Hubbers will not be penalized for having fresh Hubber Scores.

Should a Hub lapse out of Featured status after the grace period has passed (no matter how long it may be), all one has to do is edit and improve upon the Hub (ideally looking at present search competition and tweaking the title and text of the Hub to make it more search-friendly) to have it once again become Featured.

About the Higher Quality Bar

Because we will be giving most Hubs longer grace periods, we are also raising the minimum quality ratings required for a Hub to earn Featured status. This means that fewer Hubs will make the cut, but this does not mean that your Hubs need to suffer. If you make sure that your Hubs meet the substance, organization, and grammar and mechanics criteria of an 8+ Hub on our rating scale (shown below), and are clearly above the criteria of a 6 Hub, you should be fine.

Note: A collective score of eight on all levels is NOT the minimum requirement for a Hub to be Featured; the minimum quality score a Hub needs to become Featured is actually much lower, but given our margin of error with human ratings, we want to make it clear that if you meet the criteria of an eight, you can be very confident that your Hub will be Featured.

Should you publish a Hub to find that it is immediately not Featured (a clear indication that it did not get sufficiently high quality ratings), you are confident that your Hub’s substance, organization, and grammar/mechanics meet the descriptions we provide for Hubs of an 8+ rating, please file a buggy quality assessment report and we will look into the problem.

We hope this adjustment to our Featured Hubs system helps to give your Hubs a better shot at success and also improve your traffic (by improving HubPages’ overall reputation through even higher standards). For more information about the update and what qualifies as search traffic, please visit our FAQ.

New Developments in Our Emphasis on Quality

Our long-term goal is to make HubPages the go-to place to write in-depth, media-rich pages on topics you’re passionate about.

We are not alone in our emphasis on quality and engagement. Hubbers recommend that we improve the overall quality of Hubs we present to the world. Search engines, not to mention sites like Pinterest, Reddit, Facebook and Twitter, also actively seek to improve the quality of content people discover through their services. We all want quality, we have the same goals, and we believe that those who focus on quality will see great success in the long run.

How does HubPages highlight good content?

One of the ways we maintain a reputation for quality content is by only showcasing great long-form, media-rich Hubs on Topic Pages, related Hubs, and making them available to search engines. We announced the first part of this effort a few months ago as Featured Hubs. Our initial version of Featured Hubs does a good job with basic, broad evaluations of quality, but it is mostly driven by assessments of reader engagement, which by no means offers a complete picture of a Hub’s quality.

How does HubPages decide what is quality content?

Identifying quality content is no easy task. Computer algorithms, though useful in identifying very poor content, are not capable of the depth of understanding needed to reliably assess the quality of “good” content. Human beings can truly understand the content but have widely varying assessments of its quality. While researching this problem, it became clear to us that even highly-qualified and knowledgeable people can disagree fairly drastically on the quality of an article for very valid reasons.

In an attempt to address these challenges, HubPages is evaluating the effectiveness of several potential approaches to solving this problem. We refer to this effort as our Quality Assessment Process (QAP), which addresses three primary factors:

1. Automated computer-based quality ratings through intelligent learning algorithms

HubPages employs some of the brightest minds in artificial intelligence and machine learning to help develop and refine computer algorithms to reliably identify and rate the quality of content. These algorithms constantly monitor their own success and perform adjustments to a multitude of variables to optimize their own performance. Additionally, whenever available, these algorithms ingest ratings from human raters to identify successful patterns and provide suggestions for improvements to the automated algorithms.

2. Hubbers assessing content quality via the Hub Hopper

A few months back we made some changes to a feature we call “The Hub Hopper” to allow the data gathered there to inform our assessment of a Hub’s quality. This combines a feature that Hubbers frequently use to discover new content with an easy means of providing actionable feedback on the quality of the content in question.

While Hub Hopper ratings are very helpful, the current HubPages volume is not sufficient to assess the quality of the huge quantity of content published on HubPages on a daily basis, which is why we have reached to external sources for help.

3. Paid quality assessments via Mechanical Turk

Our most recent addition to the Quality Assessment Process is the inclusion of paid quality assessments. We currently acquire these assessments from Amazon’s Mechanical Turk. We are garnering Hub ratings from Mechanical Turk because ratings through the app are available at a scale capable of handling the volume of content published on HubPages. In order to mitigate the inherent variability of these assessments, raters must pass a qualification process and go through ongoing assessment. In addition, learning algorithms are applied to compensate for each individual rater’s specific biases. As long as sufficient volume is met, the result is a surprisingly reliable quality assessment.

We welcome Hubbers to use this app (and receive compensation for Hopping). If you are interested, check out HubPages’ HITs on Mechanical Turk and consider creating an account.

So, what’s changing?

 

At this point, the following things happen when you publish a Hub:

  1. Ads are shown on your Hub and its status is Pending for approximately 24 hours as it is assessed to see whether it meets our publishing standards
  2. So long as the Hub meets our basic publishing standards, it is Featured
  3. Should your Hub not see continued engagement from readers over time, it might lapse out of a Featured state, but editing it can bring it back to a Featured state within 24 hours

 

Starting early next week, when you publish a Hub, the flow will be a little different:

  1. When you publish a Hub, its status is Pending as it undergoes the Quality Assessment Process, which should take approximately 24 hours; this process includes a quality assessment in addition to a check as to whether the Hub meets our basic publishing standards
  2. After a Hub undergoes the Quality Assessment Process, ads will be added to the page; Hubs that meet certain Quality Assessment Process criteria will also be Featured
  3. After being edited, Hubs go through the Quality Assessment Process once more, hence each time a Hub is edited, it is given a new chance to be Featured

 

Put simply:
Hubs are now being assessed by real people (including Hubbers such as yourself), and their evaluations will contribute toward your Hubs’ chances of being Featured
Ads will not be shown on new Hubs until they are no longer Pending (this should prevent Featured Hubs from seeing delayed coverage in search engine results)

We are excited about the direction in which we are going and hope these changes will help to make you a stronger, better, more successful Hubber.