The whole idea of this post might seem crazy. After all, don’t you want search engines to find your site? That’s why you optimize it for searchand why you create an XML sitemap to improve your chances of ranking in search.
That said, there are a number of reasons why you might want to stopGoogle from indexing your WordPress site:
- Brand new WordPress installations: It’s not likely that a completely barren and un-optimized site will be crawled and indexed by search engines the second after you’ve installed WordPress. However, you still won’t want to run the risk of someone stumbling upon your incomplete live domain while it’s a work in progress.
- Development sites: Dev sites, testing sites, staging sites—you set these up so that team members or clients could test out the new (or rebranded) site in real time. You obviously don’t want anyone else stumbling upon a site that’s obviously not ready to be shared with the world.
- Private sites: If you’ve created something like a company intranet or a private blog (or simply have certain pages you don’t want seen by others), you might also have good reason to want to stop Google from indexing your WordPress site.
- Non-essential pages: It probably isn’t a huge deal if someone should stumble across your “Thank You” page or some other non-essential that isn’t worth indexing, but know that there are ways to block these as well.
So, once you’ve decided that you want to stop search engines from indexing your site, how do you let them know to stay away? Here are six options to consider, based on what your end goal is.