Engineering10 min read

GEO in Practice: Optimising a Next.js Site for AI Answer Engines

Generative engine optimisation sounds like a new discipline, but most of what gets you cited by ChatGPT, Perplexity or AI Overviews is classic indexing done properly. Here is what I actually changed on this site, what I skipped, and how I check whether any of it works.

Gopal Yendluri
Contents
  1. Why I Bothered
  2. How Answer Engines Find and Cite Sources
  3. Step One: Get Indexed (The Boring Part That Matters Most)
  4. Step Two: Make the Entity Unambiguous
  5. Step Three: Write Content That Gets Cited
  6. Step Four: Decide What AI Crawlers May Do
  7. llms.txt: Useful, Cheap, Not Magic
  8. Measuring It Without Fooling Yourself
  9. What I Would Not Waste Time On
  10. Checklist
  11. The Takeaway

Why I Bothered

More of the people who might read this blog now ask an assistant before they search. They type the question into ChatGPT, Perplexity, Claude, Copilot or Google's AI Mode and get a synthesised answer with a handful of cited links. If your page is not one of those links, the visit never happens.

That is the pitch for GEO (generative engine optimisation): making your content easy for AI answer engines to find, trust and cite. There is a lot of hype around it, so rather than theorise, I treated this site (yendluri.com, a statically generated Next.js App Router site on Vercel) as the test bed and did the work myself. This post is what I changed, what I deliberately did not, and how I measure it.

The headline finding is unglamorous: most of GEO is SEO done properly. The extra layer on top is real but thin.

How Answer Engines Find and Cite Sources

At a high level, an answer engine does three things when it decides to ground a response in the web:

  1. Retrieval. It issues one or more queries against a search index. That index is either a classic search engine (Google for AI Overviews and AI Mode, Bing for Copilot, and Bing has historically been an important source for ChatGPT search) or the provider's own crawl, or a mix.
  2. Fetching. It may fetch some of the returned pages live, using a user-initiated agent, to read the current content.
  3. Synthesis and citation. The model writes an answer and attaches links to the passages it leaned on.

The implication is the part people skip: if a page is not in the index the engine queries, no amount of "AI optimisation" will get it cited. Training data shapes what a model "knows" about you in general, but citations in live answers come from retrieval.

So step one was not anything AI-specific. It was getting indexed.

Step One: Get Indexed (The Boring Part That Matters Most)

When I looked at Google Search Console, a batch of newer posts were sitting under "Discovered, currently not indexed". Worse, URL Inspection on a couple of the newest pages returned "URL is unknown to Google". Those pages were invisible to Google Search, and therefore to AI Overviews and AI Mode.

What I did:

  • Submitted the sitemap in Search Console and confirmed it was being read without errors.
  • Requested indexing for the most important new URLs through URL Inspection. This is rate limited and not a guarantee, but it nudges discovery.
  • Checked canonicals so that every page declares exactly one canonical URL on the www host.
  • Fixed the Lighthouse findings that made pages look low quality: meta descriptions that were too long, colour contrast failures, and accessibility tree issues such as missing names on links and an inconsistent heading order.
  • Improved performance: fonts self-hosted at build time through next/font, and inline CSS for the critical path, which removed render-blocking requests.

The sitemap is generated from the same Markdown that builds the posts, so it cannot drift. Each post carries its real last-modified date:

// src/app/sitemap.ts (trimmed)
const blogPosts: MetadataRoute.Sitemap = getAllPosts().map((post) => ({
  url: `${siteUrl}/blog/${post.slug}`,
  lastModified: post.updated,
  changeFrequency: "monthly" as const,
  priority: 0.7,
}));

Topic and series pages are in the sitemap too, giving crawlers more internal paths to every article.

The meta description fix became a build rule rather than a one-off clean-up. The post loader now refuses to build if a description is too long:

// src/lib/posts.ts (trimmed)
const MAX_DESCRIPTION = 160;
 
const description = typeof data.description === "string"
  ? data.description.trim()
  : excerpt;
 
if (description.length > MAX_DESCRIPTION) {
  throw new Error(
    `${file}: search description is ${description.length} characters; ` +
    `keep it to ${MAX_DESCRIPTION} or fewer`,
  );
}

Lighthouse caught the problem once; the build catches it every time.

Performance is not a direct "cite me" signal for answer engines, but it is part of page quality in Search, and Search is the front door. If you want the measurement side of that, I wrote about field versus lab data in CrUX and Core Web Vitals.

Step Two: Make the Entity Unambiguous

Answer engines are much better at citing a source when they are confident who is behind it. For a personal site, the entity is a person, and the job is to describe that person once, consistently, with a stable identifier, then point everything else at it.

The root layout emits a site-wide JSON-LD graph with two nodes: the WebSite and the Person. Both have fixed @id values derived from the site URL:

// src/lib/seo.tsx
export const siteUrl = process.env.NEXT_PUBLIC_SITE_URL || "https://www.yendluri.com";
export const personId = `${siteUrl}/#person`;
export const websiteId = `${siteUrl}/#website`;
export const profiles = ["https://www.linkedin.com/in/gopalyendluri"];
// src/app/layout.tsx (trimmed)
{
  "@type": "Person",
  "@id": personId,
  name: siteName,
  url: siteUrl,
  sameAs: profiles,
  knowsAbout: ["Software architecture", "Platform engineering", "Payments", "Kubernetes" /* ... */],
  makesOffer: services.map((s) => ({
    "@type": "Offer",
    itemOffered: { "@id": `${siteUrl}/consulting#${s.id}` },
  })),
}

A few things I think matter here:

  • sameAs links the on-site entity to external profiles describing the same person, so a crawler can join "the author of this blog" to "the person on LinkedIn".
  • knowsAbout does not make you an authority, but it states what the entity is associated with. Keep it matched to what you actually write about.
  • makesOffer connects the person to the consulting services, each with its own anchor on the consulting page, so the relationship is explicit rather than inferred.
  • One source of truth. Roles change. Mine has: after over six years as CTO at Freddie's Flowers I now work as a Principal Consultant through Krigo Ltd. Because the entity lives in one place, that is a single edit rather than a hunt through templates. Stale role descriptions are exactly what assistants repeat back months later.

Every article then references the same person by @id instead of redefining it:

// src/app/blog/[slug]/page.tsx (trimmed)
{
  "@type": "TechArticle",
  headline: post.title,
  description: post.description,
  author: { "@type": "Person", "@id": personId, name: "Gopal Yendluri", url: siteUrl },
  publisher: { "@id": personId },
  isPartOf: { "@id": websiteId },
  datePublished: post.date.toISOString(),
  dateModified: post.updated.toISOString(),
  articleSection: post.category,
}

Each post also emits a BreadcrumbList, and topic and series pages emit a CollectionPage whose mainEntity is an ItemList of the posts. None of this is exotic Schema.org; the point is consistency. A pageMetadata helper gives every other page a canonical URL and matching Open Graph tags in one call.

If you are on the App Router and want the broader production picture, my notes on running the Next.js App Router in production cover the rendering and caching side.

Step Three: Write Content That Gets Cited

This is where I changed the most, and none of it is technical. When I look at which pages assistants actually quote, they share a shape:

  • A clear claim near the top. "Core Web Vitals are metrics; CrUX is a dataset" is the kind of sentence that survives being lifted out of context. Long scene-setting introductions do not.
  • Definitions. If a post introduces a term, define it in one sentence, in bold, early. Assistants are constantly answering "what is X" questions.
  • Tables. Comparisons in tables are easy to extract and easy to cite accurately. Prose comparisons get paraphrased, and sometimes mangled.
  • First-hand experience. "When we did this, here is what broke" is something a model cannot synthesise from the rest of the web, which makes it the content most worth citing.
  • Dates and authorship. Visible publish dates, a dateModified in the structured data, and a named author. An undated page about a fast-moving topic is a liability to an engine trying to give a current answer.
  • Descriptive headings that give retrieval systems clean passage boundaries.

None of this is writing for robots. It is the advice you would give anyone writing documentation people need to find under pressure.

Step Four: Decide What AI Crawlers May Do

This is the one genuinely AI-specific decision. The crawlers do not all control the same outcome.

User agent / token Operator What it controls Blocking it means
GPTBot OpenAI Crawling for model training Content less likely to be used for training; does not by itself remove you from ChatGPT search
OAI-SearchBot OpenAI Indexing for ChatGPT search results You may not appear as a cited source in ChatGPT search
ClaudeBot Anthropic Crawling for model training Content less likely to be used for training
Claude-SearchBot / Claude-User Anthropic Search indexing and user-initiated fetches Reduced visibility when Claude searches or a user asks it to read your page
PerplexityBot Perplexity Indexing for Perplexity's answers You are unlikely to appear in Perplexity citations
Google-Extended Google A robots.txt token, not a separate crawler: whether content Googlebot fetches may be used for Gemini model training and grounding No effect on Google Search indexing or ranking, and it does not opt you out of AI Overviews
Googlebot / Bingbot Google / Microsoft Classic search indexing, which feeds AI Overviews, AI Mode and Copilot You disappear from search and the answer features built on it

First, Google-Extended is not the switch for AI Overviews. AI Overviews and AI Mode are Search features, governed by Googlebot and the usual Search controls such as nosnippet. If you block Google-Extended hoping to stay out of AI Overviews, you will be disappointed; if you block Googlebot to achieve it, you have also left Search. Second, the providers have increasingly split training crawlers from search and user-triggered agents, so you can now make different choices for each. Check each provider's current crawler documentation, because names do change.

My decision for this site was simple: everything here is public writing I want read and cited, so everything is allowed. The robots file says so in one rule:

// src/app/robots.ts
// Everything on the site is public, for search engines and AI assistants alike.
export default function robots(): MetadataRoute.Robots {
  return {
    rules: [{ userAgent: "*", allow: "/" }],
    sitemap: `${siteUrl}/sitemap.xml`,
    host: siteUrl,
  };
}

For a business with paid content, proprietary data or licensing concerns, the sensible middle ground is often to block training crawlers while allowing search and user-initiated agents:

User-agent: GPTBot
Disallow: /
 
User-agent: ClaudeBot
Disallow: /
 
User-agent: Google-Extended
Disallow: /
 
User-agent: *
Allow: /

Remember that robots.txt is a request, not access control. If something must not be read, put it behind authentication.

llms.txt: Useful, Cheap, Not Magic

llms.txt is a community proposal (documented at llmstxt.org) for a Markdown-flavoured plain-text file at the root of a site that gives language models a concise map of its most important content. It is not a web standard, and support among the major AI providers is uneven. I have seen no reliable public evidence that any of the large answer engines uses it as a ranking or citation signal. Treat it accordingly.

I added one anyway: it was cheap, and it is useful when someone points an agent at the site and says "read this". The important design choice is that it is generated from the same data as the pages, so it never goes stale:

// src/app/llms.txt/route.ts (trimmed and simplified)
export const dynamic = "force-static";
 
export async function GET() {
  const posts = getPostSummaries();
  const lines = [
    "# Gopal Yendluri",
    "",
    "> Technology leader in London writing about architecture, AWS, data and AI-assisted engineering.",
    "",
    "## Recent articles",
    ...posts.slice(0, 15).map((p) => `- [${p.title}](${siteUrl}/blog/${p.slug}) (${p.date}): ${p.excerpt}`),
  ];
  return new Response(lines.join("\n"), {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  });
}

A hand-maintained llms.txt is worse than none: it will drift from the site and start describing pages or roles that no longer exist. If you add one, generate it.

Measuring It Without Fooling Yourself

GEO measurement is immature. What I actually do:

Referrals in analytics

AI assistants that link out often send a referrer. In GA4 and Vercel Analytics I group sessions whose source matches the obvious assistant domains (for example chatgpt.com, perplexity.ai, copilot.microsoft.com, claude.ai, gemini.google.com) into one channel. A simple regex on session source does the job:

(chatgpt\.com|chat\.openai\.com|perplexity\.ai|copilot\.microsoft\.com|claude\.ai|gemini\.google\.com)

Some clicks arrive with no referrer, and Google does not reliably separate AI Overviews or AI Mode clicks from organic search, so treat the channel as a floor, not a count. If you want a cleaner way to wire up channel definitions across tools, I have written about analytics tracking architecture with a data layer and adapters.

Manual prompt checks

Once a month I run a short, fixed list of questions in each assistant, ones where a post of mine is a reasonable answer, and note whether the site is cited and what is said about me. It is anecdotal and results vary by session and model version, but it catches the important failures: not being cited at all, or being described with out-of-date facts.

Search Console

Still the most useful tool. Watch the page indexing report for "Discovered, currently not indexed" and "Crawled, currently not indexed", and use URL Inspection on anything new. If a page is not indexed, fix that before thinking about anything else on this list.

What I Would Not Waste Time On

  • AI-specific keyword stuffing or forcing every page into a question-and-answer format. There is no good evidence it helps.
  • Hidden text or "instructions to AI" in the page. At best ignored; at worst it looks like manipulation.
  • Separate "AI versions" of pages. Serving different content to crawlers than to people is cloaking by another name.
  • Dashboards promising precise AI share-of-voice numbers. Answers are non-deterministic and personalised.
  • Obsessing over llms.txt. Ship a generated one and move on.

Checklist

  • Sitemap generated from source data, submitted in Search Console, read without errors
  • Key pages show as indexed in URL Inspection
  • One canonical URL per page, on one host
  • Meta descriptions within limits, enforced at build time
  • Lighthouse SEO and accessibility issues fixed; fonts self-hosted, no render-blocking CSS
  • One Person (or Organization) entity with a stable @id, sameAs and accurate role
  • Articles reference that entity by @id, with datePublished and dateModified
  • Posts open with a clear claim, define terms and use tables for comparisons
  • A deliberate robots.txt decision for training, search and user-initiated AI agents
  • llms.txt, if you want one, generated rather than hand-written
  • An "AI assistants" channel in analytics and a monthly manual prompt check

The Takeaway

GEO is real, but it is mostly a new name for old discipline. Answer engines cite what their retrieval returns, and retrieval still runs largely on search indexes, so the biggest gains on this site came from fixing indexing, canonicals, descriptions and page quality. The AI-specific layer is thin: a clear, consistent entity in structured data, content that makes crisp first-hand claims with dates and authorship, a deliberate robots.txt policy that distinguishes training from search crawlers, and an honest, generated llms.txt that I do not expect miracles from. Measure with referrals, manual checks and Search Console, accept that the numbers are a floor, and spend the time you save writing things that are actually worth citing.

GEOSEOAI searchNext.jsstructured datallms.txtrobots.txt