Connect: advanced
- What you need
- 1. Config
- 2. Blog index page
- 3. Post page
- 4. Caching
- 5. Sitemap and robots.txt
- 6. llms.txt
- 7. IndexNow key file
- 8. Publish webhook
- 9. Check it
- 10. Finish in the dashboard
The advanced way: your site gets each post as data and builds every part of the page itself. You get full control of the markup, and also the full job: SEO tags, structured data, sitemap and the rest.
Most sites should use the simple way instead. There, we send finished HTML and you only style it.
Using a coding agent? On your dashboard's "Connect your website" page, choose Advanced and copy the prompt.
What you need
From your dashboard's "Connect your website" page:
| Value | What it is | Secret? |
|---|---|---|
| Site key | 32 characters; identifies your site in the API | No, safe to commit |
| Post URL pattern | Where posts live, e.g. https://example.com/blog/{slug} |
No |
| IndexNow key | 32 characters; lets search engines hear about new posts | No |
| Signing secret | Proves webhooks come from us | Yes: keep it in an env var |
Set the post URL pattern in the dashboard first: canonical URLs, structured data and internal links are all built from it.
In this guide, {API} means https://blogsforseo.com/api/v1/sites/{SITE_KEY}.
1. Config
Use these environment variables:
BLOGSFORSEO_SITE_KEY: required.BLOGSFORSEO_API_URL: optional, defaults tohttps://blogsforseo.com.BLOGSFORSEO_WEBHOOK_SECRET: only for the webhook in step 8.
Never hardcode the secret.
Everything below is plain HTTP and JSON, so any language works. Field details are in the Content API reference.
2. Blog index page
GET {API}/posts?page=1&pageSize=12 returns { posts, page, totalPages, total }, newest first. pageSize can be up to 50.
Show a card for each post, linking to its URL. Add pagination, and an empty state for when nothing is published yet.
3. Post page
Serve posts at your post URL pattern, and fetch each one with GET {API}/posts/{slug}.
- Missing posts: when the API returns 404, your page must return a real 404 status.
- Server rendering: the title and body must be in the raw HTML response. Client-side fetching alone fails, because AI crawlers don't run JavaScript. On a single-page app, use server rendering, prerendering at build time, or a server route.
Render the post in this order:
titleas the<h1>. Under it:author.name,publishedAt,updatedAt(when different) andreadingMinutes.heroImage, when present, with itsalt,widthandheight.shortAnswerin a highlighted box, thenkeyTakeawaysas a list.- A table of contents from
toc({ id, text, level }, linking to#id), when it has 3 or more entries. bodyHtml, inserted as raw HTML. It is already sanitised; don't escape it, and keep its heading ids and links.faqs,sources(as links) andrelated(links to related posts).
Style the body's h2, h3, p, ul, ol, table, blockquote, a, img and code to match your site.
In <head>:
<title>frommetaTitle<meta name="description">frommetaDescription<link rel="canonical">fromcanonicalUrl, when it isn't null- Open Graph tags:
og:title,og:description,og:image(the hero image) andog:typeset toarticle <script type="application/ld+json">holdingJSON.stringify(post.jsonLd), with every<replaced by\u003c
Some frameworks stream metadata after the body to bots they don't recognise. If yours does, make sure GPTBot, ClaudeBot, PerplexityBot and Googlebot get it in <head>.
4. Caching
Cache API responses for up to 15 minutes, using ISR, a fetch cache or a CDN. The webhook in step 8 clears your cache straight away on every publish.
Static sites with no server: skip step 8. Instead, paste your host's deploy hook URL into the dashboard, and every publish will rebuild your site.
5. Sitemap and robots.txt
- Sitemap: add every entry from
GET {API}/sitemap([{ url, lastmod }]) and your blog index to your sitemap. Merge them with what's already there. - robots.txt: it must not block Googlebot, Bingbot, GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot or Perplexity-User.
6. llms.txt
- No
/llms.txtyet: serveGET {API}/llms.txt(plain text) at/llms.txt. - Already have one: add a "Blog" section that links each post.
7. IndexNow key file
Serve /{INDEXNOW_KEY}.txt at your site root. Its body must be exactly the key, sent as text/plain. Once it's there, we notify Bing (which ChatGPT search also uses) within minutes of every publish.
8. Publish webhook
Add a POST endpoint, e.g. /api/blogsforseo. Before trusting any request:
- Check the
x-blogsforseo-signatureheader. It looks liket=<unix seconds>,v1=<hex>. - Compute the HMAC-SHA256 of
t + "." + rawBody, usingBLOGSFORSEO_WEBHOOK_SECRETas the key. Use the raw request body, before any JSON parsing. - Compare the result with
v1in constant time. - Reject with 401 if they don't match, or if
tis more than 300 seconds from now.
Once the request checks out:
- Clear the cache for your blog index,
post.slugandpreviousSlug, if there is one. - Return 200 quickly.
Events and code samples for Node, PHP and Python are on the Webhooks page.
9. Check it
Run these checks against your site:
curl -A GPTBot https://your-site/blogreturns 200 and lists your posts, or shows the empty state.curl -A GPTBot <a post URL>returns raw HTML with the<h1>, the body text, the canonical link, the meta description and theapplication/ld+jsonscript.- A post URL with a made-up slug returns 404.
/sitemap.xml,/robots.txt,/llms.txtand/{INDEXNOW_KEY}.txtare all correct.- A request signed with the right secret gets 200 from your webhook. A wrong signature gets 401.
10. Finish in the dashboard
- Set the post URL pattern to the exact pattern your site uses.
- Set the webhook URL to your endpoint, or set a deploy hook for a static site.
- Set
BLOGSFORSEO_WEBHOOK_SECRETin production to the dashboard's signing secret, then deploy. - Press Send test, then Run health check. The health check loads a live post the way AI crawlers do, and lists anything left to fix.