Google Updates Quality Rater Guidelines for YMYL & Offensive Results
- Updated on: 2018-10-29
- Read original article here
You are here: Home / Google / Google Updates Quality Rater Guidelines for YMYL & Offensive Results
Google Updates Quality Rater Guidelines for YMYL & Offensive Results
May 30, 2017
Google has made another update to the Google Quality Rater Guidelines, coming only a couple of months after the last release, which saw major changes made to target fake news, science denail and clickbait , among other things.
This updates sees Google bring added clarification to specific areas of the guidelines and other minor changes. The biggest changes are to news sites with Your Money Your Life, as well as further clarifications as to when a page or site should get the Upsetting-Offensive flag, which was introduced with the last update to specifically target fake news as well as other sites publishing fake information.
Here’s the full breakdown of what changed.
Contents
This section is primarily for the raters themselves about how ratings are done.
Releasing Tasks
This was removed from the beginning of the rater guidelines, but Google expanded this section and included it in the final section, which is detailed below.
Page Quality Rating Guideline
This section primarily talks about overall page quality, and there was one significant change made for news sites with regards to YMYL.
Your Money or Your Life (YMYL) Pages
Google makes a point of stating that not all news articles are considered to be YMYL pages. Previously, they did not make it clear that not all news would fall under that category.
In the “News articles or public/official information pages important for having an informed citizenry”, Google added “Keep in mind that not all news articles are necessarily considered YMYL.”
So this means that some types of news sites will not be held up to the much higher standard of YMYL, which was not clear in the last version. This is good because some types of news sites really don’t impact Your Money or Your Life, yet they would also be held to this higher standard.
To clarify, news sites that would somehow potentially impact the future happiness, health, or financial stability of users ARE still being held to YMYL. If your news site does not fall under that, it would not.
Needs Met Rating Guideline
This section has a variety of changes made, primarily to Fails to Meet and porn queries.
Examples of Fails to Meet (FailsM) Result Blocks
This is an interesting addition. Google has added an additional example of an inaccurate featured snippet. This shows they want their raters to be aware that featured snippets are not always right, and that raters need to consider that and rate them with Fails to Meet appropriately.
Here is the added example, which is a much more obvious featured snippet than the other example used in the guidelines:
Needs Met Rating for Possible Porn Intent Queries
Google has made some changes to this section. Most notably, they removed additional keywords as types of searches that could have possible porn intent but also non porn intent. The removed keywords are girls, gay, thong, spanking.
They also make it clear that they want raters to rate based on the query being non-porn in intent. Here they also changed their example keyword from bikini to breast.
Here is what the new section looks like:
Some queries have both non-porn and porn interpretations. For example, the following English (US) queries have both a non-porn and an erotic or porn interpretation: [breast], [sex]. We will call these queries “possible porn intent” queries.
For “possible porn intent” queries, please rate as if the non-porn interpretation were dominant, even though some or many users may be looking for porn. For example, please rate the English (US) query [breast] assuming a dominant health or anatomy information intent.
For the examples in this section, they removed the “pictures of girls” example.
Needs Met Rating for Clear Porn Intent Queries
They removed a bit in here. The italicized part was removed:
For very clear porn queries where no other intent is possible, assign a rating to the porn landing page based on how helpful it is for the user. Even though there is porn intent, the page should still be assigned a Porn flag.
Not clear why specifically this was removed.
Reporting Illegal Images
Here, Google has changed multiple references of “reporting to your vendor” to “reporting to your employer/company” throughout this section.
This is likely in response to the fact raters must now be employees of the companies doing the ratings, and not mere contractors, a change Google began requiring for the companies that are responsible for hiring those to do the ratings.
Using the Upsetting-Offensive Tag
Google has brought some additional clarification to how raters should interpret if a result deserves the upsetting-offensive tag or not. Google has added this section:
Important: Please think about the purpose of the page and how the content is presented when determining whether to assign the Upsetting-Offensive flag. There are many results around the web that focus on sensitive topics such as child abuse, violence, or racism; however, you should assign the Upsetting-Offensive flag based on the purpose, type, and/or presentation of the content on the page —not because the topic itself is sensitive or potentially upsetting.
For example, a result with content that encourages or graphically depicts child abuse should be flagged as Upsetting-Offensive . However, an accurate informational page about child abuse (such as child abuse statistics, prevention, etc.) should not be flagged, even though child abuse itself is a sensitive topic that users may find upsetting.
This serves more of a reminder about how simply because a topic is a sensitive one, does not mean it automatically deserves the upsetting-offensive tag, but rather it should be based upon factors such as page purpose. For example, even though reading statistics about child support is upsetting, it is still quality information and should not be flagged as upsetting-offensive. On the other hand, a page on how to abuse a child would be considered upsetting-offensive.
This is very new to the quality rater guidelines and was brought up into response for queries like “Did the Holocaust happen” as well as the increase in fake news sites and articles. But the clarification seems to imply that perhaps some raters were having trouble distinguishing if a result really should have the upsetting-offensive tag, or if some raters were flagging because the general topic was upsetting to them.
Needs Met Rating for Upsetting-Offensive Tolerant Queries
Again, more clarification has been brought to a page which is considered upsetting-offensive and the needs met scale. Google is reminding that even if the needs of the query was met, it doesn’t make an upsetting-offensive queries less upsetting-offensive.
Remember that you should assign the Upsetting-Offensive flag to all upsetting/offensive results, even if they satisfy the user intent.
For example, someone could search for “prove the Holocaust is fake”, and even if the query was technically met with a page that IS upsetting-offensive, doesn’t mean it loses that designation simply because the query matched the result. Remember, Google is trying to prevent these types of results from surfacing for various queries.
Needs Met Task Page Screenshot
Curiously, Google has removed this screenshot from the guide. This screenshot was one of the few we have that gives insight into the interface the raters use when it comes to rating search results and webpages.
Using the Evaluation Platform
This section is specific to the raters doing the actual evaluations, but there is often some interesting information found within this section for SEOs and site owners.
Simplified Needs Met Tasks
The big change here is Google has increased the size of the screenshots showing the backend – they were so tiny before they were impossible to make out.
They also changed the wording which also includes a new reference to “contextual results”.
Some Needs Met rating tasks may not require ratings for all blocks. In these blocks, the Needs Met and EAT rating scales will be greyed out and annotated with the text “No Rating Required”.
The greyed-out results (sometimes referred to as “contextual results”) are shown to help you understand the query and inform your ratings. You can click on them when you feel like they will be helpful in determining your ratings.
It is not clear why Google felt the need to refer to these as contextual results however.
Understanding the User Location on the Task Page
Google has updating the wording with regards to the maps feature that sometimes sees a map included with some results, so searchers can evaluate if the query and results match the location.
While most query-based rating tasks have a user location listed below the query, a map will often be provided as well. This map can help clarify where the user was physically located when the query was issued in a more precise way.
Notes about Using the Needs Met Rating Interface
This section was completely removed, but it served the actual raters from a technical point of view.
Pre-Identified Duplicates
They included guidelines for raters about how the system determines duplicates, and removed some clarifying points:
● Results annotated by “ Dupe of… ” may be given different ratings/flags/comments.
● Results annotated by “ Same as… ” may not be given different ratings/flags/comments. Their Needs Met and EAT ratings, flags, and comments will be automatically transferred to each other.
● You cannot uncheck dupes that have been automatically detected and preidentified.
Releasing Tasks
As noted above, this section was removed from the start of the guidelines. Google also added a new screenshot showing the reasons for released tasks. Of note is the new “I did not sign up for adult content” addition.
More Notes about Using the Needs Met Rating Interface
Most of this section has new parts, but they are directly related to the backend technical stuff for the raters. Anything that could impact SEOs in this section has been included above.
Final Thoughts
This update is not nearly as big as the last one published earlier this year, which saw significant changes made to fake news, science denial and clickbait. This particular update seems to be more targeted to the raters workflow with some more detailed clarifications where there was previously some questions about the ratings. If this is your first time reading the guidelines, my previous update has many more changes within it.
The full guidelines have been published by Google here .
