How does Google know if you're being a naughty blogger?

[Nords]

I'm confused over a technical blogging question.  Every time I ask it among personal-finance bloggers I get admonished that it's "against the rules" and "Google will punish you".  Yet I'm still trying to figure out (1) how Google will tell you're breaking the rules and (2) how you know you're being punished. 

So maybe I need to ask the question of a broader audience.  I'll try to keep the vocabulary non-technical so that, even if you're not a blogger, you can help me figure out how Google could enforce a law that's on their books.

Please refrain from quoting the rules at me.  I know the rules.  What I don't understand is how violations are detected and what punishments are handed out.  Please refrain from quoting the ethics, too-- I already understand the philosophy and I want to learn more about the law enforcement. 

Everybody "knows" that Google doesn't want bloggers manipulating the Google ranking systems.  One of those systems is PageRank, and one of the inputs to PR is the "quality" of your links to & from other blogs.  If I link to the MMM blog, or if it links back to my blog, then those are high-quality links which will hypothetically raise my PR.  It might even raise MMM's PR.  But if I link to a spammy site then it could hypothetically lower my PR. 

You have to be careful who you swap PR with-- or presumably you'll get a reputation. [Insert high-school dating metaphor here.] I have inadvertently linked to a site that linked to a site on Google's malware list, and I received an e-mail warning that I was at risk of also being added to Google's malware list.  So I know that Google takes action when they feel it's justified.

One way to avoid "passing" your awesome PR to other sites is to format your links so that Google's algorithms don't count them.  The most common way to do this is to use a "NOFOLLOW" tag in your link.  Human readers can still click through to the site you're linking to, but Google's crawlers will ignore NOFOLLOW links. 

This leads to a chicken-vs-egg paradox.  If everyone uses NOFOLLOW in their links, then Google's PR algorithm can't use links to determine a blog's PR.  (It probably uses other factors as well.)  Yet if people use links without NOFOLLOW, then some unscrupulous bloggers will attempt to manipulate PR with their links. 

The current situation is that most bloggers link without NOFOLLOW.  (I've never used them.)  As near as I can tell, sites who do not use NOFOLLOW are still getting rising PRs.  For example, over the last two years my PR has risen to 3 (out of 10).  My problem is that I cannot tell whether my PR would be higher if I used NOFOLLOW tags... unless I change every link to include a NOFOLLOW tag and wait a few months for Google's algorithms to catch up.  If there's another way to test this question, I'd love to hear it. 

I have not read of any blogger experiences where they were rewarded by Google for using NOFOLLOW tags.  Google does not explicitly reward you for using NOFOLLOW-- they just tell you that it's the "right thing to do".  If Google promised me a PR6 for using NOFOLLOW in my links then you know I'd add it before lunch.  But they don't say one way or the other. 

Now the question becomes how Google determines whether these links (without NOFOLLOW) are legitimate (me quoting MMM's blog) or whether they're manipulating PR (me swapping links with a bunch of other bloggers to build our PR).

I could sell links on my blog to advertisers.  This means that the advertiser either (1) pays you to format their link around some text in one of your existing posts or (2) pays you to put their post on your blog, with the understanding that their post has text which links to their product.  As far as I'm aware, Google cannot usually detect that you've sold links.  Sure, you can be stupid about it by writing a post that's bloated with links, or by adding a link that's totally out of context to the rest of the paragraph.  However a good blogger is presumably capable of using their chosen language to insert a link that matches the context-- indeed the linked site would even improve the information in the post.

PR is one of the factors that bloggers and advertisers use to set our rates.  (This is separate from Google's AdSense rates.)  We bloggers all have a bunch of different metrics that we use to show how our blog clearly rules the Internet and thus is worth thousands of dollars per link, but PR happens to be a widely accepted standard (even though it can be manipulated).  Anyone can use a metrics website to pull Google's data to determine the PR of anyone else's blog, but there's no way to tell how legitimately that blogger earned their PR. 

I could tell when I've sold a link because the balance in my checking account would go up.  However I cannot tell how Google "knows" that I've sold that link, and I certainly can't tell whether I'm being "punished" by Google for selling the link.  Google doesn't send me e-mails or say anything on their site about the enforcement-- it's an empty threat.  If there's a way that Google will "know" that I've sold a link, then prove it-- show me where I've sold a link on my blog, or show me that I've never sold a link on my blog. 

Again, plenty of bloggers will quote the rulebook at you, and everyone knows an urban legend of someone's cousin's friend's co-worker who was publicly flogged in Times Square by Google for selling links.  But I have never heard from anyone who was told by Google that they're being punished for selling links, and I've never heard from anyone who could tell that Google even noticed. 

So... aside from my own high ethical standards, what keeps me from selling links?  How does Google detect the problem (aside from blatant incompetence) and how do they mete out punishment?

[kudy]

Hi Nords, I work in the search industry, and it is understandable how the water is muddy when trying to figure this stuff out - it's never clear. Part of why it's muddy is Google has traditionally been very bad at communicating with webmasters when they have a problem with how they are running a site. Fortunately, they have been communicating much more efficiently in the past few years through the webmaster tools platform - I get messages regularly through webmaster tools regarding problems on websites, and if you don't have it set up, that's the best step you can take toward staying in communication with Google. They would then have the ability to send you a message informing you that your site is breaking guidelines.

Most people in the industry listen closely to what the head of Google's web spam team, Matt Cutts, has to say, because he drops hints and tries to be transparent when he's allowed. They are trying to protect their proprietary algorithms as well as avoid being gamed, so they can't be too clear about *how* they detect web spam.

I believe there is an algorithmic portion - some crawler out there that looks for things that raise red flags - a sidebar full of "follow" links to unrelated websites might look a lot like a list of paid links (and then it might raise a red flag that is manually reviewed by a member of the web spam team). I assume you've read Google page detailing what they consider "Link Schemes" - these are the things they will try and detect using software algorithms. They also rely on people "reporting" spam, and from my perspective, it seems like a lot of what is found by robots and is reported manually is followed up on manually by the web spam team.

Of course, I can't speak for Google, but I can speak to my experience, and I would *never* suggest that a client worry about "follow" vs. "no follow" links - I would recommend that they use links naturally and liberally in their content, as they would normally. I publish content and encourage clients to publish content containing a variety of links to related websites without worrying about the details - I'd say leave them all "follow" and don't worry about it. If my client is selling links, I would advise them to add a nofollow tag to those links only. I don't believe Google has said anywhere that they encourage or reward the use of nofollow links on everything, just on those links that could be considered unrelated to the content. Are you being asked to sell links *without* using a nofollow tag? If so, I think the chances are good that you could get away with it, and you may be informed if they detect anything through webmaster tools.

I could probably write more coherently about this in the evening, as my lunch break is ending!

[gooki]

Just do it.

What will raise a red flag is if the person/company buying your links, also buys links on every other related blog.

[Nords]

Are you being asked to sell links *without* using a nofollow tag? If so, I think the chances are good that you could get away with it, and you may be informed if they detect anything through webmaster tools.

That's my confirmation bias!

I'm surprised more bloggers aren't writing about it, but perhaps they're afraid of Google's backlash.  Yet we bloggers are supposed to diversify our income streams, and this seems like a high-revenue/low-risk way to do it.

I was surprised at all the finger-shaking admonishments I got, especially from people who were very good at quoting the laws without considering whether they're actually enforced.  I agree that selling links in a stupid manner will attract the wrong attention, but I think companies buy links in direct proportion to their size.  Nobody would be surprised to see a ton of links out there for Quicken or TurboTax, with many fewer links for financial software products from smaller companies.

And while I'm on a rant, it ticks me off that Google's AdSense terms of service doesn't allow advertising that AdSense revenue is donated to charity.  Google is rightfully concerned that bloggers will mislead readers into clicking for charity, or just flat-out lie about where the money goes.  However I'd be happy to document my AdSense charitable contributions and maybe even pay a little money for the privilege of being confirmed by a reputable entity.  Google seems to be passing up an easy way to support "Don't Be Evil", and frankly they're missing an opportunity to create a revenue stream of their own. [/rant]