Knowing how to submit websites and individual pages to search engines is an essential skill for SEO professionals and webmasters alike.
Whether you’re building a new website or simply adding new content, knowing the ins and outs of indexation is key.
What You’ll Need Before Submitting
First, you’ll need access to edit your website.
Some people may refute this and claim backend web access is not necessary to submit a website to search engines. Well, they’re right.
However, there are some cases where you’ll need access to a website’s backend.
Situations Where You’ll Need Backend Access
- The website doesn’t have a sitemap.
- The website doesn’t have a robot.txt file.
- The website doesn’t have Tag Manager or a way to verify Google Search Console/Bing Webmaster Tools access.
If your client or IT team doesn’t allow you to have access to their backend, or your CMS has certain limitations, see if you’re able to obtain FTP access. This will come in handy later in this article.
Get Access to Google Search Console & Bing Webmaster Tools
If you really want to maximize your organic traffic potential, make sure to submit your website to as many relevant search engines as possible. This may seem pretty obvious, but a little reminder is always nice to have.
So, what will we need?
Most search engines have their own set of webmaster tools to help us manage our web presence. However, the big two you really need are:
- Google Search Console
- Bing Webmaster Tools
Setting up Google Search Console
Before you can submit your website to Google, you’ll need to set up a Search Console account and verify website ownership. You use your Gmail account for this.
If you manage multiple website domains, you will be able to manage all of them from the same account.
Once your account is set up, make sure to follow Google’s guidelines on verifying your website property. There will be prompts that provide you with multiple options to verify your website. If your account is the same as your Analytics account, your website will be auto-verified.
Setting up Bing Webmaster Tools
The first step here is to set up an account. Bing makes this easy by allowing you to sync your existing email accounts to quickly create a Bing Webmaster Tools profile.
Once you’re logged in to Bing’s Webmaster Tools, you’re ready for the next step of submitting your website.
How to Submit an Entire Website
1. Create a Sitemap Index with Categorized Sitemaps
When managing the indexation for an entire website, it’s important to know how to manage it at scale. Having an optimized sitemap can help make this process much easier for webmasters, and most importantly search engines.
When I refer to a sitemap here, I’m referring to an XML file, not an HTML sitemap.
Depending on the nature of your site, it may make sense to create multiple sitemaps to help silo your content into relevant categories. You can even create an image sitemap to help boost your image optimization strategy.
You can use a sitemap index as a root, and link to each sitemap from there.
If you use the Yoast plugin, all of this can be done automatically.
If you aren’t on WordPress, there are many other tools that can help you create your own sitemap. Screaming Frog is my go-to for situations where there’s no automatic sitemap tool.
If you’re more advanced, you can even get fancy with Python to help reorganize XML sitemaps.
Uploading XML Sitemap Through FTP
If you don’t have access to your website’s backend, having FTP access can really save you here.
If this is your first time accessing the backend of your website, this can be tricky. Once you’re connected to your FTP, follow these steps to upload your XML sitemap.
- Search for your public_html directory.
- Open your public_html directory.
- Upload your sitemaps to that directory.
Easy!
Now you need to test your site to make sure it has been uploaded correctly.
To test, simply copy your file name and add it to the end of your website URL. For example:
https://www.example.com/sitemap.xml
or
https://www.example.com/sitemap_index.xml
2. Optimize Your Robots.txt
So you’ve created and optimized your sitemap. What does a robots.txt file have to do with this?
Well, there are few simple steps to make sure that search engines are able to crawl and index your website.
Add a Link to Your Sitemap
You can actually add a link to your sitemap file in your robot.txt file. This helps search engines quickly locate your sitemap and may improve your crawl rates.
Double Check Disallow Directives
If you’re launching a new website, make sure your website DOES NOT say this:
User Agent: *
Disallow: /
That simple two lines of text will block all major search engines from crawling your website. You’d be surprised how often web developers don’t check for this when launching websites.
3. Submit Sitemap to Google & Bing
Remember at the beginning of this article when we set up Search Console and Bing Webmaster Tools? Well, that’s about to come in handy.
Both of these platforms allow us to submit sitemap links. This is a quick way to tell search engines which pages you want them to crawl.
You can submit individual sitemaps, but it may be a bit quicker to just submit your sitemap index.
How to Submit Individual Pages
Did you know that both Google and Bing allow you to submit individual pages?
That’s right! But don’t get too excited.
While some pages can get indexed quickly, sometimes search engines may take longer to index your submitted pages.
Google Search Console – URL Inspection Tool
Google’s new Search Console platform has one of my favorite new tools, the URL Inspection Tool. This is a fairly comprehensive tool that allows webmasters to get instant feedback about how Google perceives certain aspects of a webpage.
One of the best features of this tool is the ability to request indexing. Sound familiar?
That’s because the URL Inspection Tool has replaced the Fetch as Google tool from the old Search Console.
I’ve personally seen instant indexation using this tool, but other SEO professionals have reported slower results.
Bing Webmaster Tools – Submit URLs
In a recent announcement, Bing said that they’re allowing webmasters to submit up to 10,000 URLs per day.
By using the Submit URLs tool in Bing Webmaster tools, you’re helping Bing save crawling resources. Makes sense, but it goes against how search engines typically work.
If Bing is openly encouraging webmasters to submit their URLs for indexation, then why not add this to your marketing checklist?
Bonus Tips!
We covered some direct methods for getting your links indexed, but there are also some indirect methods for getting links indexed.
For those who are new to SEO, it’s important to note that most search engine crawlers discover new webpages through links.
Let’s throw search engines a bone and help them discover our content!
Google Publishers Search API
Google has opened up tools to allow publishers to submit their content directly to Google’s index. This requires some advanced reading, but is well worth the time!
Optimize Internal Links
Optimizing your internal links is a vital part of every essential SEO checklist. Having a structured linking scheme on your site helps search engines discover new pages on your site.
Ramp Up Your Link Building Efforts
Yes, Link building should already be on your SEO checklist.
However, ramping up your link building efforts can help Google find pages on your site. Try earning new links or even reclaim broken backlinks.
The Wrap Up
Some SEO experts may consider implementing all of these tactics to be a bit overkill. However, every little bit helps.
Just like anything in SEO, better performance comes from a culmination of factors, not just one.
More Resources:
