Webinar: Will AI shopping agents recommend your top products?

Save Your Spot
  1. SEJ
  2.  ⋅ 
  3. SEO

Google Takes Applications For A UGC Pipeline Separate From Crawling

Approved forums and social platforms can send content and interaction signals through a separate pipeline. Here's who can apply and how it compares to crawling.

Google Takes Applications For A UGC Pipeline Separate From Crawling

For most websites, crawling is still how you get into Google Search. Googlebot finds a URL and retrieves the page; then, independently, Google’s systems determine if they want to add this page to their index.

On Oct. 8, Google added Search Central documentation describing how approved forums and social platforms can submit user-generated content directly to Google.

The UGC Fresh Data Program allows approved platforms to submit content and interaction signals through a “dedicated data ingestion pipeline” that accepts data submission from those platforms. Google states that the pipeline is separate from both traditional crawlers and the Indexing API. According to Google, participation in the UGC Fresh Data Program does not ensure your content will be included in Google Search.

Google has partnered with individual social platforms before, including Twitter in 2009 and Reddit in 2024. This time the criteria are posted and there’s an application form, so you can check most of Google’s requirements against a platform’s own pages before applying. The form asks how many monthly active users the platform has and when it launched.

What Google Posted

The Search Central documentation changelog links to the page under the entry for Oct. 8: “Added documentation on the UGC Fresh Data Program.” Google wrote that the page is meant to explain the program’s purpose and help applicants “understand the eligibility criteria and how to participate.”

The program page opens with this description:

The UGC Fresh Data Program is an initiative designed to help platforms display fresh, high quality, and authentic user-generated content (UGC) across various features on Search.

Google says sending content through the pipeline “helps ensure that fresh content is processed and quickly updated in Search.” The page gives no time frame for “quickly.” It also says the program “doesn’t guarantee that content will appear in Search.”

The page limits the program to “trending UGC content.” It describes the pipeline as “independent of the Google Indexing API and traditional organic crawling for general web content.”

Oct. 8 is when the page went up. Neither the page nor the changelog entry says when the pipeline started operating.

Who Google Says Can Apply

Google lists six criteria, using the labels below.

  • Content focus. The platform must primarily host “UGC, social postings, or forum discussions.” Content should sit on dedicated pages with stable URLs, “not on profile or feed pages.”
  • Volume and audience. “A high volume of UGC content and a significant user base.”
  • Content accessibility. Content must be on public pages that Googlebot and users can reach. Anything behind a login or paywall doesn’t qualify, and all UGC must be tied to “a creator with a public profile.”
  • Technical readiness. The platform must be able to implement “secure OAuth 2.0 API authentication” and build “strictly validated JSON-LD payloads.” It must already use, or be prepared to implement, valid on-page schema.org markup. Google’s example is SocialMediaPosting or DiscussionForumPosting with interactionStatistic sub-fields.
  • Content health and moderation. No illegal content, plus active moderation that includes a way for users to report posts.
  • Content freshness. Content must be “as fresh as possible, ideally sent within minutes,” and the platform must be able to submit “regular updates of engagement counters (within 72 hours of creation).”

Submitting an application “doesn’t guarantee participation,” according to the page, and Google will share developer documents “once accepted into the program.”

The form has seven required fields: company name, service domain, 10 sample URLs with UGC, monthly active users, launch date, a Search Console owner’s email and a contact email. It calls a submission “merely an expression of interest” and says Google “has no obligation to incorporate your content in our features.”

The form goes further than the page on where content could show up, saying submitted content “is eligible to serve on Google Search and other surfaces, especially for time-sensitive or fresh content.” It adds, “we never guarantee serving as that is determined entirely by ranking algorithms.”

How It Compares With Crawling And The Indexing API

Any site can ask Google to crawl its pages, according to Google’s recrawl documentation. Owners and full users of a Search Console property can request crawls of individual URLs in the URL Inspection tool, within a quota, and sitemaps cover larger sets of URLs. Google says crawling “can take anywhere from a few days to a few weeks,” and that a request “does not guarantee that inclusion in search results will happen instantly or even at all.”

The Indexing API is narrower. It covers pages with JobPosting, or BroadcastEvent embedded in a VideoObject, and lets their owners tell Google when a page is added, updated or removed. This “allows Google to schedule pages for a fresh crawl.” There is a “default 200 quota” available for API onboarding and submission testing; however, the API “requires additional approval for usage and resource provisioning.” In a September post I noted several job boards had gone months without a reply to their approval requests.

Recrawls, sitemaps, the Indexing API provide the URL for crawling. Platforms submit what Google calls “content and interaction signals” through the UGC program, but this does not replace public URLs. The criteria still require that the content be accessible via Googlebot.

The Indexing API has a default quota for onboarding and testing, with additional approval required for further usage. The UGC program requires Google to approve applicants. Neither guarantees that submitted content will appear in Search. Of the three options being compared here, recrawl requests and sitemaps do not include a request for approval. The program’s “within minutes” wording covers sending, not indexing.

Google Has Partnered With Social Platforms Before

On Dec. 7, 2009, Google launched real-time search results that showed “live updates from people on popular sites like Twitter and FriendFeed.” The post named Facebook, MySpace, FriendFeed, Jaiku and Identi.ca as new partners, along with Twitter, which Google said it had announced “a few weeks ago.”

In July 2011, Google said “that agreement expired on July 2,” as Datamation reported at the time. In May 2015, Google said it was bringing Tweets to Search on mobile, and it added desktop that August.

In February 2024, Google stated that it “now has access” to Reddit’s Data API. According to Google, this Data API delivers “real-time, structured, unique content.” Google noted that accessing the Data API would provide “enhanced signals that will help us better understand Reddit content and display, train on, and otherwise use it.” Furthermore, Google states that this partnership does not affect its use of publicly accessible (crawl-able) content from the Internet for indexing, training or displaying such content in Google products.

Those partnerships mentioned above are the ones Google announced with named entities. However, those earlier posts did not provide sufficient technical information so as to allow comparison with the new pipeline. While the 2024 description notes “access to Reddit’s Data API,” the Oct. 8 posting describes approved platforms providing data through a Google ingestion pipeline. There was no reference to the Reddit partnership in the program page or the form.

As noted previously, one of the uses listed in the 2024 post is training. However, neither the program posting nor the posted application form referenced “training.”

There is a clear distinction between what has been published about the new program and the earlier partnerships. Google has made available to the general public both the eligibility criteria for participation in the new program and an online application. As previously discussed, Google provided identification of individual social platform partners through prior announcements.

What Forum Markup Already Carries

Google’s support for the markup the program names predates it by almost three years. In November 2023, Google added discussion forum and profile page structured data, which I covered at the time. Google said the markup works with features “designed to show first-person perspectives from social media platforms, forums, and other communities,” and that using it “does not guarantee appearance.”

In June 2024, the changelog confirmed that SocialMediaPosting markup is also supported, “to more accurately reflect how the data ingestion for these features work.” In March 2026, Google added forum and Q&A properties “for providing more clarity on comment thread structure to Google ingestion systems.” The update included labels for AI-generated and bot content, which I covered in March.

Google’s current discussion forum markup page lists five interaction types it supports in interactionStatistic: likes or upvotes, dislikes or downvotes, views, comments, and reshares. The program page names the same markup, with interactionStatistic, as its example of what applicants should have on their pages.

Neither the forum documentation nor the program page says how Google uses those counts in ranking.

Why This Matters

Google documents point to a separate intake for trending posts from forums and social platforms, apart from the crawling it uses for general web content. Approved platforms send the posts and their interaction signals themselves, while recrawl requests, sitemaps and the Indexing API all hand Google a URL to crawl.

Any platform that includes only feed items; requires login credentials for post views; or has user generated posts that cannot be linked back to a public profile does not qualify under the current specifications regardless of the size of their platform. The ten example URLs provided are intended to represent the format the platform plans to send to Google.

Whether or not the platform chooses to apply, the technical SEO can focus on the markup. Valid on-page structured data is one of the requirements, and in 2023 Google provided Search Console reporting and Rich Results Test support for both forum and profile markup. Those reports show errors, warnings, and valid items related to marked-up forum and profile pages.

If a site’s primary source of content is not user-generated content, then it fails the first qualification. However, these types of sites may still be competing against User Generated Content (Forum/Social) on the same searches, but Google did not name any participants. If you find your searches yielding more forum results, currently, there is no way within either the program page or form to determine whether the program is responsible for those increased results.

What Google Hasn’t Said

As of Oct. 15, none of the program page, form, or changelog entry (Oct. 8) names a participating platform. None of these items establish a numerical value for “a significant user base” despite this being one of the values requested via the Form to be reported as Monthly Active Users.

The program page references “various features on Search,” while the form says “Google Search and other surfaces.” They don’t identify a feature. The 72-hour counter window isn’t explained either.

The three listed documents also do not indicate how Google utilizes the interaction signals/engagement counts provided by participants; nor do they specify if participants will receive any reporting.

Looking Ahead

Applications are open through the form linked from the program page. Google says it will get back to applicants “within 6-8 weeks with a response on the status of your application.” Selected platforms then get “detailed technical integration documentation and onboarding instructions,” with no stated schedule for that step.

Google’s Search Central changelog records the Oct. 8 addition. It’s a place to watch for future changes to the documentation.

More resources:


Featured Image: Tetiana Yurchenko/Shutterstock. Source: Google.

Category SEO
SEJ STAFF Matt G. Southern Senior News Writer at Search Engine Journal

Follow on YouTube. Matt G. Southern is the Senior News Writer at Search Engine Journal, where he’s covered Google, SEO, ...