Multimodal search describes the range of inputs search engines now accept from users. Gone are the days when we would have to type in a description of a product we were looking at in a shop when we wanted to run a price comparison. Now, we can simply take a photo of it and use technology like Google Lens to search based purely on that image alone.
Multimodal Search is wider than just using images as inputs. Modern search engines accept searches via keywords, images, voice, or a combination of these. Beyond that, we can use multimodal search to bring back content in a different format type. Use voice to request an image of a dog, and receive back Image Search results. Take a photo of a green dress and use text to ask Google to search for a version in blue.
Whereas in years past, images were just an augmentation of a page’s content, now they are an important entry point in a search journey.
So beyond standard image optimization, what do we need to do differently to make sure our images are ranking well and can be found in this new frontier of multimodal search?
Ensure Images Are Findable By The Search Engines
The very first step to ensuring images can be retrieved as part of a multimodal search is to ensure they are crawlable and indexable.
This is a simple step that can be accomplished in two ways:
Check Google Search Console
Using Google Search Console, you can identify whether your site is getting visibility in the Image SERPs by checking the “Search results” report under “Performance” and filtering by “Search type: Image.” From here, you will be able to see which pages are getting impressions and clicks from image search, rather than performance for individual image URLs. No impressions for a page might mean that its images haven’t been indexed, or that they aren’t appearing for relevant searches during the selected period. It does not, by itself, confirm an indexing or optimization problem.
Similarly, the newly introduced “Multimodal” option under “Web” in the Search type filter will allow you to see how your content is performing in searches that use an image as part of the search. These include Google Lens searches, Circle to Search, images uploaded to Google Search, and Chrome’s “Search this image” functionality.
If you notice that your pages aren’t getting impressions from image search and want to investigate potential crawling or indexing issues, use the “Inspect any URL” option in Google Search Console to inspect the URL of the page hosting the image. You can also run a live test of the image URL itself to help check whether Google can access it. However, URL Inspection does not reliably confirm whether an individual image is indexed in Google Images, and a page being indexed does not establish that its images are also indexed.
Manually Check The Search Engine Results
A great way to get an idea of how your images are appearing in multimodal search is to perform some test searches yourself.
If, for example, you have an ecommerce store that has many bespoke photos of your products you could take a photo of one of your products yourself and use it as a search input. This could include via Circle to Search, Google Lens, or upload to a Google search. By doing this, you will be able to see how easily Google can determine that your photo of your product, and your company’s photo of the product are referencing the same item. From there, you will also see if your company’s images are the ones dominating the SERPs.
You could also search for the image in Google Images using the src: operator. To do this, find an image on your website that you want to search for. Copy the URL of the image itself, not the page it is hosted on. Then, in Google Images, type src: followed by the image URL, e.g., src:https://www.examplewebsite.com/media/happy-cat.jpg.
The search results should bring back pages on the web that reference that image URL. If there are no pages from your own website in the SERPs, that gives you a clue that there might be something wrong with the indexing of your images on your own website. Do note that as with the site: command, the list of pages in the SERPs is not definitive, just because a page isn’t there doesn’t mean Google hasn’t indexed it.
See also: Multimodal Search Is Reshaping The Funnel For SEOs And Marketers
Examine Googlebot Image Crawler Activity
Alongside the above it is wise to check individual search engine crawlers to see if they are experiencing any blocks in crawling the images on your site. This is not necessarily the same as whether they can access the page the image is on. You specifically need to see if the bots can find and parse the images themselves.
There are two key ways of doing this. Firstly, you can check your website’s log files. Look specifically for image crawlers like Googlebot-Image. This will help you to see if specific images on your site have been crawled recently. If you notice that bots are always skipping certain image URLs, or some bots are not accessing your image at all, that could give you an indication that there is something blocking their crawling.
You should also use a crawling tool like Screaming Frog, and set the user-agent configuration to “Googlebot Image.” Then have the bot crawl a list of your image URLs and the URLs of the pages they are hosted on. This will give you an idea of whether any of them are bringing back a 3XX, 4XX, or 5XX status code. If so, Googlebot may not be able to access the image.
Check For Technical Issues That Could Be Preventing Crawling, Indexing Or Serving
Configuration Issues
The above checks may well flag that some of your images are not being crawled or indexed. This can be for a number of reasons. For instance, it could be that your robots.txt file is actually preventing the bots from crawling the image URLs themselves.
There may be x-robots header control preventing the indexing of the page the image is on, or even the <noimageindex> directive preventing the indexing of the image itself.
These configuration issues are fairly easy to check. The robots.txt is as simple as using a robots.txt validator like Dentsu’s robots.txt validator and checking the image and landing page URLs against the live robots.txt.
To check the page-level tags, check the page source, or in Chrome Developer Tools, check the Elements panel for <meta name=”robots” content=”noimageindex”> or noindex. Then, in Developer Tools, the Network tab, select both the page request and the image request and check the response headers for X-Robots-Tag.”
Server Level Issues
You may also find that the image URLs are being blocked at a server level. This is something that is unlikely to be obvious without some digging into your website’s CDN, botwall, or server stack.
In some cases, images are hosted on separate CDN domains. This means that issues with them may not be apparent from looking in Google Search Console or other tools unless you specify the subdomain or separate domain, the images are on. It is worth clarifying with your development team if that is the case for your images. You can often tell by the URL of your images, however.
For example, if your website is www.examplewebsite.com but the URL of your images starts with media.examplewebsite.com, that would suggest your images are being hosted on a separate subdomain. Add that subdomain to your tracking tools to keep an eye on any errors.
You should also enquire with whoever controls the server at your company whether they are blocking any bots that might be used for image crawling. The most obvious one is Google’s GoogleBot-Image, but it is a good idea to request a full list of blocked bots and any specific areas they are being blocked from, e.g., the media sub-domain in the example above.
JavaScript Rendering
Although Google has made great progress in rendering JavaScript, images that rely on it to load may not be fully discoverable. This could be due to the script being blocked or failing to load. You may also find that important images are being set as background images via CSS, which can cause problems for the bots. It is imperative that you check if the images are accessible by using the Inspect Tool in Google Search Console. Using the tool, examine the rendered HTML of the page with the images on. Check if the images are present as <img> elements with valid image URLs in the src attribute. This way, the images should be picked up by bots without requiring the rendering of JavaScript.
Lazy Loading
Lazy loading is a popular way to improve the load speed of a page. However, it can cause issues with discoverability of content if not implemented correctly. For example, Googlebot does not interact with a page in the same way as a human does. It will not scroll down the page or swipe through a gallery. If an image is only loaded once a user scrolls down the page, it may not be accessible to the bots.
To check if your images can be accessed by the bots without the need for interaction with the page, look at the rendered HTML of the page. If you can see the image URLs in the HTML, then they are accessible without needing further interactions.
Image URLs
One reason bots may not be ranking your images in multimodal searches is because the image URLs linked to on your site are broken or redirecting to another image or resource. Make sure image references return a 200 directly. Broken image URLs won’t be indexed. Redirects are followed, but chains and redirects to non-image resources waste crawl and can stop the image from being picked up. Check for redirects and broken URL references by running a crawl on your site. Make sure to include any subdomain or domain that your images are hosted on.
Another consideration for good image optimization is keeping image URLs consistent. It could be that your site references a new parameter on an image URL each time a page is loaded, for example:
- www.examplewebsite.com/media/12345.jpg
- www.examplewebsite.com/media/12345.jpg?session=a1b2c3
- www.examplewebsite.com/media/12345.jpg?session=d1e2f3
By doing that, you are essentially creating duplicate images for the bots to discover and crawl. Instead, it would be prudent to always reference the one URL per image. To identify if your site is doing this, you should look across a sample of image URLs and see if there are any elements of the URL that appear dynamically generated.
Assess If You Are Making It Easy For Bots To Discover Your Images
Image Sitemaps
XML image sitemaps are a clear signal to search engines that images are on the site and where to find them.
Image sitemaps are similar to standard URL XML sitemaps. However, they need to contain slightly more metadata in order to be useful to Google.
If you have sitemaps dedicated to your images already on your site, check if they use <image: image> (any information specific to the image) and <image: loc> (the URL where the image can be found.
For large websites with catalogs of images (think social media sites and stock image providers), it is necessary to have image sitemaps. Without them, it is difficult for the bots to know which images are available, and it makes it harder for you to debug issues.
Image Accessibility
We’ve already touched on making sure image URLs stay consistent, but don’t neglect the format of the images either. It is important to use supported file formats like JPEG, PNG, WebP, and AVIF. These are file types the search engines recognize as images.
For responsive images, ones that change depending on the device they are being viewed on, make sure you are using <picture> or srcset. This tells the bots that there are multiple sizes of this image available. Using an src fallback means that bots that don’t understand <picture> or srcset still have a location at which to find the image. This isn’t so much an issue for more standard search bots like Google Imagebot, but could be necessary for answer engine bots.
Paywalling Images
When looking at accessibility of images, it is important to note the nuance of paywalling.
Content gating is the process by which content is restricted, for example behind a registration form, login, or paid subscription. This is typically used for content like whitepapers or lead generation magnets. It is also commonly used by news publications that require a reader to register or pay before reading an article.
Google has clearly defined methods of gating content in a way that allows its bots to still access the content, while keeping it hidden from logged-out users.
On the surface, there appear to be different treatments of gated images and gated text. Gated text can be crawled and indexed by Google as long as its bots can access the content without needing to log in. Showing bots the content available to authorized users while showing logged-out users a restricted version is not considered cloaking when implemented in accordance with Google’s paywall guidance, including declaring the restricted content using Paywall Structured Data.
There is, however, a qualification for images. Google states in its image metadata guidelines that, to help Google discover and index images, publishers should: “Make sure people can access and view your pages that contain images without needing an account or logging in.” It doesn’t conclusively establish that all images on paywalled pages are ineligible for indexing or ranking, but it does mean publishers should not assume that paywall structured data alone satisfies Google’s guidance for image discovery and metadata features.
Determine How The Bots Are Perceiving Your Images
For multimodal search, it’s not just enough to know that your images can be retrieved by bots. You also need to confirm they understand what the images are of, and their relevance to the search input.
Audit Surrounding Text And Page Context
The bots need to understand what your images are of. They do this primarily by considering the context of the page the image is on and any additional information you supply about the image.
Check that your images are appropriately titled, have a relevant alt attribute, and are on a page that describes the image well.
Structured Data
A key way to give information about the image to search engines is through the use of structured data.
Using schema.org‘s ImageObject type gives you the opportunity to communicate key aspects of the image that could be useful in giving additional context. It can also help make your image eligible for specific SERP features. For example, using schema to communicate licensing details means your image may be eligible for the “licensable” badge when it is displayed in search results.
If there is more than one image on the page, then you can select the one you want Google to consider the primary for the page by using primaryImageOfPage schema markup.
A Final Checklist To Ensure Images Are Optimized For Multimodal Search
- Check existing image visibility in search engine tools like Google Search Console and Bing Webmaster Tools.
- Verify crawler access, including image-specific bots, indexing directives, and HTTP responses.
- Check rendered HTML for important image elements.
- Review JavaScript and lazy loading implementations.
- Test CDN and server bot restrictions.
- Check image discoverability aids like XML sitemaps.
- Review landing page context for items like schema, alt-text, and relevant copy.
Summary
In the end, multimodal search makes the visual content on your website even more important. The bots need to be able to access and interpret those images in order to serve them as results in a multimodal search.
The foundations of multimodal search are not different from the standards of image SEO. Even looking at AI search bots, we can see that the critical aspects of multimodal optimization for images lie in the technical implementation of your site.
More Resources:
- Google Search Console Adds Multimodal Filter For Image-Based Searches
- The Technical Signals AI Search Uses That Most SEOs Still Aren’t Optimizing
- Google Clarifies How It Picks Thumbnails For Search, Discover
Featured Image: Paulo Bobita/Search Engine Journal
