Our long-term goal is to make HubPages the go-to place to write in-depth, media-rich pages on topics you’re passionate about.
We are not alone in our emphasis on quality and engagement. Hubbers recommend that we improve the overall quality of Hubs we present to the world. Search engines, not to mention sites like Pinterest, Reddit, Facebook and Twitter, also actively seek to improve the quality of content people discover through their services. We all want quality, we have the same goals, and we believe that those who focus on quality will see great success in the long run.
How does HubPages highlight good content?
One of the ways we maintain a reputation for quality content is by only showcasing great long-form, media-rich Hubs on Topic Pages, related Hubs, and making them available to search engines. We announced the first part of this effort a few months ago as Featured Hubs. Our initial version of Featured Hubs does a good job with basic, broad evaluations of quality, but it is mostly driven by assessments of reader engagement, which by no means offers a complete picture of a Hub’s quality.
How does HubPages decide what is quality content?
Identifying quality content is no easy task. Computer algorithms, though useful in identifying very poor content, are not capable of the depth of understanding needed to reliably assess the quality of “good” content. Human beings can truly understand the content but have widely varying assessments of its quality. While researching this problem, it became clear to us that even highly-qualified and knowledgeable people can disagree fairly drastically on the quality of an article for very valid reasons.
In an attempt to address these challenges, HubPages is evaluating the effectiveness of several potential approaches to solving this problem. We refer to this effort as our Quality Assessment Process (QAP), which addresses three primary factors:
1. Automated computer-based quality ratings through intelligent learning algorithms
HubPages employs some of the brightest minds in artificial intelligence and machine learning to help develop and refine computer algorithms to reliably identify and rate the quality of content. These algorithms constantly monitor their own success and perform adjustments to a multitude of variables to optimize their own performance. Additionally, whenever available, these algorithms ingest ratings from human raters to identify successful patterns and provide suggestions for improvements to the automated algorithms.
2. Hubbers assessing content quality via the Hub Hopper
A few months back we made some changes to a feature we call “The Hub Hopper” to allow the data gathered there to inform our assessment of a Hub’s quality. This combines a feature that Hubbers frequently use to discover new content with an easy means of providing actionable feedback on the quality of the content in question.
While Hub Hopper ratings are very helpful, the current HubPages volume is not sufficient to assess the quality of the huge quantity of content published on HubPages on a daily basis, which is why we have reached to external sources for help.
3. Paid quality assessments via Mechanical Turk
Our most recent addition to the Quality Assessment Process is the inclusion of paid quality assessments. We currently acquire these assessments from Amazon’s Mechanical Turk. We are garnering Hub ratings from Mechanical Turk because ratings through the app are available at a scale capable of handling the volume of content published on HubPages. In order to mitigate the inherent variability of these assessments, raters must pass a qualification process and go through ongoing assessment. In addition, learning algorithms are applied to compensate for each individual rater’s specific biases. As long as sufficient volume is met, the result is a surprisingly reliable quality assessment.
We welcome Hubbers to use this app (and receive compensation for Hopping). If you are interested, check out HubPages’ HITs on Mechanical Turk and consider creating an account.
So, what’s changing?
At this point, the following things happen when you publish a Hub:
- Ads are shown on your Hub and its status is Pending for approximately 24 hours as it is assessed to see whether it meets our publishing standards
- So long as the Hub meets our basic publishing standards, it is Featured
- Should your Hub not see continued engagement from readers over time, it might lapse out of a Featured state, but editing it can bring it back to a Featured state within 24 hours
Starting early next week, when you publish a Hub, the flow will be a little different:
- When you publish a Hub, its status is Pending as it undergoes the Quality Assessment Process, which should take approximately 24 hours; this process includes a quality assessment in addition to a check as to whether the Hub meets our basic publishing standards
- After a Hub undergoes the Quality Assessment Process, ads will be added to the page; Hubs that meet certain Quality Assessment Process criteria will also be Featured
- After being edited, Hubs go through the Quality Assessment Process once more, hence each time a Hub is edited, it is given a new chance to be Featured
Put simply:
Hubs are now being assessed by real people (including Hubbers such as yourself), and their evaluations will contribute toward your Hubs’ chances of being Featured
Ads will not be shown on new Hubs until they are no longer Pending (this should prevent Featured Hubs from seeing delayed coverage in search engine results)
We are excited about the direction in which we are going and hope these changes will help to make you a stronger, better, more successful Hubber.
HubPages Blog 
This is wonderful! But I had trouble with MTurk providing me two of the same paid practice/trial HITs on Friday night. I wrote to HP over the weekend and have not heard a reply yet. I’d love to be able to be a part of this! Thanks, Rachel
Just signed up for Mechanical Turk, waiting to here back. What a great idea! Once again HubPages makes good choices that are in the community’s best interest. Thank you for keeping us moving in the right direction!! 😀
Respectful HubHugs~
What qualifications do the “outsourced evaluators” at Mechanical Turk have to show before they are allowed to assess our work? Are they professionals at this or merely anxious to earn a few bucks?
SSSSS
Can you define what this means ( 4, 6, 8 ?)
“Hubs that meet certain Quality Assessment Process criteria will also be Featured”
I’m glad to see that you’re refining this process and although many are complaining, I think you’re doing the right thing here. Keeping quality up will help to restore HP and make things better for all of us here.
Yikes- I’m sorry about that Rachel! I’m glad you’re up for participating. I’m not personally familiar with the process of getting started on MTurk, so might I suggest emailing the MTurk team, too? The problem might be on their end.
K9keystrokes, thanks also for signing up!! It’s awesome to have your support.
Randy, please see this FAQ entry: http://hubpages.com/faq/#hopper-credibility
John, hopefully this FAQ entry answers your question: http://hubpages.com/faq/#featured-criteria (not really sure what you mean by (4, 6, 8)…
Sondra, thanks for those kind words!
Anything I publish is already under moderation until approved by an administrator.
In order for my newly published hubs to become “live” for the search engines I’ve been waiting 24-48 hours for approval by an administrator…
PLUS an additional 24 hrs for the hub to be free and clear out of pending.
How will this emphasis of “quality” affect the hubs I submit for publication?
It seems reasonable that the administrators would not allow my hubs to be published if they didn’t meet quality standards.
Okay. So tell us who the “Gold Panel” consists of so we may peruse their writing to be sure they are qualified to judge ours. Certainly they would not wish be anonymous if they are sure of their own abilities. Thanks for the info.
SSSSS
added to note website and;
In my particular case, I NEVER edit a hub because it puts the particular hub back through the “waiting for approval” by an administrator.
During this waiting period, a healthy hub is missing out on potential page views from search engines.
I would absolutely love to improve upon/update/clean-up some of my first hubs BUT don’t want to miss-out on the consistent page views that I’ve been getting.
Now it sounds like this is going to be the situation for everyone; a waiting period for edited hubs. I wonder if people are going to be hesitant with updating healthy hubs.
Sorry, Rachel! I’ve been getting so many emails from turkers that it can be hard to keep up with sometimes, but as discussed over email, I believe we’ve fixed the problem for you.
I have no problem with this. Will this help take care of problems such as numerous misspellings and typo’s? I recently saw a HOTD with 9 typo’s in the first paragraph. I was so disgusted that I quit reading it. Other readers have the same reaction to this type of thing. NOT quality.
Are you saying that every time we edit an already featured hub that it will goe through this process before it’s published again?? I truly hope not. Many of us update some of our our hubs frequently. Or are you talking about new hubs that are published and need editing?? Please clarify. Thank you.
Zoelle, the extra level of review your Hubs undergo is independent from the QAP. The difference this new develop provides is that your Hubs will also be evaluated by human raters in addition to our algorithms. The QAP is not all that similar to the moderation review process.
The Gold Panel consists of engineers, moderators, and HubPages founders.
Becky, it probably will, though grammar is not the only factor considered in the hopper.
Rebekah, every time a Hub is edited, it will go through the QAP, but so long as you’re only updating/improving a Hub, editing it will only do good. 🙂
That’s what i was afraid of, Simone.
SSSSS
When I tried to sign up for Amazon through HP, it said I couldn’t because I’m a resident of Illinois. MTurk says your payment will go right to your Amazon account so does that mean I can make a independent Amazon account set up with MTurk or does that mean that I can’t participate in this?
I do not like having my hubs go into pending for 24 hours every time I improve them. I also do not like my hubs being in pending status at all. I consistently have high quality hubs and keep a high hubber score. why do I have to continue being graded like a child before you deign to let me have ads and show my good quality work?
You should let people alone if they do not have problems with the first dozen hubs, unless they start having problems. Writers are leaving in droves because they do not appreciate being treated like ten year olds in school. You need to start treating the adult professional writers like adult professional writers.
Always interesting to watch evolution! Hopefully this will keep things moving a little smoother and help all of us. Onward and editing!
HoneyBB, I’m not really sure. I have never personally used MTurk before and am not sure what their registration processes are like. I would contact their support team or read through their FAQ to find out.
Becky, Featured Hubs that you edit are still Featured while they are Pending. The QAP is not a penalty, it is our means of giving Hubs a chance of being Featured.
Thanks Tillsontitan!!
The Mechanical turk sounds good. My issue with huppage input is that I’m not into networking so much, and my experience of forums for the past 18 years on the web is that many people vote for someone just because they like the person, not because of the quality of the work. I think I would prefer impartial…
That said, I’m really happy that this is happening, although I don’t like the 24 hour wait.
Hi Tess! We definitely have taken account for potential for biased ratings, and have corrected for them. You can read more about what we have done to ensure that ratings are fair here: http://hubpages.com/faq/#hopper-credibility
Will the fees used to pay evaluators at Mechanical Turk be reflected in our Hubpages earnings?
I appears that the algo and human scores are fed into the hubscore, which is the defacto rating for each hub. Hubscore also includes a traffic factor that changes in time, plus other engagement measures etc. So is Hubscore effectively the HP score for the page? The ability for authors to see the scores for their pages is a major omission from the system. There is no feedback whatsoever, except whether the page is blanked or not. You have said you are working on this feedback (when?), but is hubscore a defacto score?
Quality when published hubs are no longer than one paragraph, with all kinds of errors, no paragraph breaks.
Yet those of us who produce quality have to sit and sit and sit til hub is published,
Our earnings stink.
I can see HP losing hubbers, quality producing hubbers. Oh, well.
I am glad to see that the new changes will should prevent the delay in featured hubs being found by search engines because this has been a major concern for me.
While this will help new hubs, is there anything that can be done for hubs that are already in limbo?
I have 2 hubs still not indexed after 2 and 4 weeks respectively. The one that is 4 weeks old was a Hub of the Day and so has been shared several times, and its stats show it has been crawled by Google. Yet it is definitely not indexed.
I have tried manually indexing through Webmaster, but it made no difference.
Good question tammyswallow! MTurk earnings will be entirely separate from HubPages-related earnings. They will be paid through that system.
John Anderson, while we will ultimately have the QAP assessment factor into HubScore, we have yet to actually do it. We have no set date regarding the time at which additional feedback will be rolling out, but it is the next feature that we will introduce. 🙂
Melovy, those Hubs that are Featured but have yet to be indexed by Google will eventually be indexed, but the ball is in Google’s court- there does not seem to be anything else we can do to help the process along once a Hub has been crawled. Sorry about that!!
I am really happy about this. I really hope that this process wil help to weed out the spun content and garbage from the site.
I only have one more concern and that is: The filters for copied content within the site needs to be updated as I still find hubs that are copied from other authors on the site.
I fully support the comment made by Betty Katz on November 8. In my opinion, she speaks for a lot of HP contributors. Please stop the ridiculously long wait before publishing our hubs. In the past two weeks I have diverted my writings to Helium because my submissions are instantly approved and I start earning right away.
We are not kids. We are writers. We know what we are doing and writing. The wait time is killing us. You guys have made the quality rating process so complicated and so time consuming. Because you have focused so much on grading and rating contributions, you seem to have become myopic.
Back out for a bit, look at the big picture, see the forest from the trees. Put yourselves in our shoes. We are your contributors. I joined Helium and HP at about the same time, and so far I have earned $22 at Helium and only about $2 at HP.
I just finished a major work about skin care, but do you think I will submit it to HP and suffer like the rest over the waiting period? No way, I’m submitting my piece to Helium. If there is something wrong at all with my work, the fact checker would call my attention to it. But I don’t have to wait at all to get my work published.
We want to earn from our writings, too, so wake up people, time to shake off that tunnel vision before you lose your best writers. Right now, I’m just content to comment and answer questions. I will just rely on my already published hubs and not contribute anything new, until all this waiting is over. Please, please, put yourselves in our shoes.
Correction re my previous comment, I meant Becky Katz, not Betty. I apologize for the error.
Well.. I usually never find time to write or ask questions because I am so busy writing very “LONG” Hubs, and have all of them that are already publishes, although I plan to write now more compact and shorter Hubs, with the same quality as the lengthy ones I already have, I still struggle with traffic.
These past months I have editing and revamping all my Hubs, and my Hub scores went up a bit. I do not really have good traffic, and that means I earn a paltry income. That is not a problem for me because when I joined HP, I knew there was potential for one to earn something, but the main reason for me was to write very serious and involved Hubs which I have hoped will have “staying power” for the unforeseeable future
I think I like the idea of QAP, and at the same time, since I am a neophyte in marketing my Hubs on the Web, I know that the length of my Hubs is daunting-I also put too much time per Hub-I would like to know how to increase it. Now, I do get traffic and some few comments on this or that Hub, but getting a serious flow of traffic is what I cannot manage nor see my Hubs get-like in a very big way. Anyway, I am still working on the old Hubs, and hope that whatever it is I am saying, will get an answer as to how one can increase traffic. One more thing, I only got two ‘pendings” on my Hubs since I joined, and I fixed them(thus my re-editing and recreating and adding to my old Hubs, and all of them are ‘featured’. If only I knew how to generate traffic, and by the way, I would not change the length of my Hubs, and will keep on writing some in the future of same or longer lengths. Maybe the 500 word or a little more so might do the trick, too. I really am not sure, but will try. I hope I get a response, and am still reworking and updating and editing my Hubs, and the score is not bad, but would like to up it from the lower seventies, to the lower eighties, for all my Hubs; and getting traffic will, for now be the only thing I do not know how to do well… Any tips, after my lengthy diatribe above? Thanks and it is good we have some Quality Control measures being taken, for me it is a challenge I would like to take on..
Thanks Cardisa! We’re definitely working on that. Our filters that monitor copied content sometimes take time to work.
Thanks for your feedback, Richie Mogwai. Keep in mind that that 24 hour Pending period is just 24 hours, and sometimes shorter. That period enables us to keep things like spam and spun content from being published in the first place. If we left out those particular controls, we could probably shorten the wait, but our overall goal is to improve the experience of everyone on the site, writers and readers alike, and reducing the amount of spam and spun content on our site makes a big difference.
Ixwa, we actually think that in-depth, media-rich Hubs have the best shot at doing well, so while it never hurts to make your Hubs easier to skip around and more scannable, we don’t recommend making them all shorter. The more knowledge you share, the better! 😀
Pending is just a temporary period; it is nothing to ‘fix’.
And again, Hubs over 800 words on our site tend to do significantly better than those just over 500 words. Keep that in mind!
Hope that helps!
The key, W. Wilmoth, is to adapt one’s academic articles so that they address common (related) searches people run online. I bet if you do some keyword research, add more detail to your academic articles on the visual front (e.g. videos, images, polls, etc.), and alter your title to address the good keywords you find, you can do quite well!
Here’s our guide to keyword research for reference: http://hubpages.com/learningcenter/google-keyword-tool