Case study · Custom AI Systems
Engineering a site so search and AI can understand it
Most sites are built for a human skimming a homepage, not for the systems that decide whether anyone finds them. The work is treating search visibility as an engineering problem, not a content chore, so a site is legible to Google and to the AI systems now reading it.
- 01Search-intent map
- 02Content architecture
- 03Internal linking
- 04Structured data
- 05Entity signals
- 06Measurement
- Client
- a B2B software company
- Problem
- A site ranks for nothing because search engines cannot tell what it is about or who it is for.
- Approach
- Technical SEO, content architecture, search-intent mapping, internal links, structured data, and entity signals.
The problem
Most sites are built for a human skimming a homepage, not for the systems that decide whether anyone finds it. Pages target no particular query, the information architecture is a flat list of marketing pages, internal links are decorative, and there is no structured data telling a search engine what the entity is or what it does. The result is a site that is invisible for the searches that matter.
Adding more blog posts does not fix this. The problem is structural: search engines and AI answer systems cannot map the site to intent, entities, and topics, so there is nothing for them to rank or cite.
The reality behind the problem
Publishing more does not move the needle when the underlying structure is missing. A search engine ranks a page it can confidently map to an intent, an entity, and a place in a topic. A site that is a flat list of marketing pages gives it none of that, so volume adds noise, not visibility, and the pages compete with each other instead of ranking.
Being found now spans two different systems. A classic search crawler ranks pages, while an AI answer engine retrieves and synthesizes passages. A site tuned only for the first can be invisible to the second, because the signals that make a page rankable are not the same as the ones that make it machine-readable and quotable. Both have to be engineered deliberately.
What was assumed
We need more content to rank.
What Stallwart asked
Can a search engine even tell what this site is about, and who each page is for?
The desired outcome
The target is not a traffic number. It is a site that both Google and AI systems can map cleanly to intent, entities, and topics, so each page is legible to the systems that decide whether it is found.
- Every page maps to one clear search intent, so an engine knows who it is for.
- A topic architecture a crawler can follow, with authority routed to the pages meant to rank.
- Structured data that states the entity, its offerings, and how pages relate, in machine-readable form.
- One consistent entity across the site and the third-party profiles engines trust.
- Content legible to AI answer engines, not only to a classic search crawler.
- Coverage measured by intent and topic, not by vanity metrics.
The system we designed
Stallwart treats search visibility as engineering. It maps the real queries buyers use, from informational to commercial intent, and gives each intent a page designed to satisfy it, rather than one homepage trying to answer everything. Content architecture is organized into topic clusters with descriptive internal links, so authority flows to the pages that should rank and a crawler can see how topics relate.
Structured data (JSON-LD) and entity signals make the site machine-readable: what the company is, what it offers, how pages relate, and which real-world entity it maps to. Stallwart runs exactly this on its own site, where structured data, an llms.txt summary, and an explicit AI-crawler allowlist make the site legible to answer engines as well as to Google. All of it is publicly verifiable.
How it works
- 01
Map intent before touching copy
The first step is the search ecosystem around the business: what buyers actually type, and the different intents behind similar phrases. A page is then designed to satisfy one intent well, instead of a homepage trying to rank for everything and ranking for nothing.
- 02
Build architecture, not a pile of pages
Topic clusters and descriptive internal links give a crawler a map of how the site's topics relate and route authority to the pages meant to rank. This is the structural work that adding posts alone never does.
- 03
Make the site machine-readable
Structured data and consistent entity signals let a search engine, and an AI answer system, state plainly what the site is and how its pages connect. On Stallwart's own site this extends to an llms.txt summary and an AI-crawler allowlist.
The approach
From invisible pages to a legible search surface.
- 01
Search-intent map
Real queries, informational to commercial
- 02
Content architecture
Topic clusters, one intent per page
- 03
Internal linking
Descriptive links, authority routed
- 04
Structured data
JSON-LD: entity, offerings, relationships
- 05
Entity signals
Consistent, disambiguated, machine-readable
- 06
Measurement
Coverage and intent, not vanity metrics
Each layer makes the site more understandable to search and AI systems.
Architecture
What sits underneath.
Underneath the approach are four layers, each engineered so that both a search crawler and an answer engine can read the same site the same way.
Search-intent and topic model
The real queries buyers use are mapped to the intent behind them and grouped into topics, so each page is built to satisfy one intent and the site covers a subject rather than scattering across unrelated keywords.
Content architecture and internal linking
Topic clusters and descriptive internal links give a crawler a map of how pages relate and route authority to the pages meant to rank, instead of a flat list where every page competes with every other.
Structured data and entity signals
JSON-LD states what the company is, what it offers, and how its pages relate, and consistent entity signals map the site to one real-world entity, so an engine parses the facts directly rather than inferring them from prose.
AI-legibility surfaces
An llms.txt summary and an explicit AI-crawler allowlist make the site readable to answer engines as well as to Google, the setup Stallwart maintains on its own site so the content is legible to both classes of system.
The hard parts
Where the engineering judgment was.
Engineer for intent and entities, not keyword counts
Modern search and AI retrieval rank on understanding, not repetition. Intent coverage and clear entity signals are what make a site legible and rankable.
The trade-off
Slower than publishing volume, but it builds a surface that compounds instead of a pile of thin pages.
One intent per page
A page that tries to serve several intents serves none well and confuses ranking. Splitting intent across purpose-built pages is clearer for both users and crawlers.
The trade-off
More pages to design, each earning its place rather than padding a sitemap.
Engineer for both Google and AI answer engines
The signals that rank a page in a results list and the signals that make it retrievable by an answer engine overlap but are not identical. Building for only one leaves the site invisible to the other.
The trade-off
More surfaces to maintain, structured data and an llms.txt alongside the classic signals, in exchange for legibility to both systems.
Reliability & guardrails
How it avoids the wrong call.
The guardrails keep the site legible as it grows and keep the framing honest, so the structure never quietly works against the pages it is meant to lift.
One consistent entity everywhere
What the company is, what it offers, and how it is named are stated identically across pages, structured data, and the profiles engines trust, because a contradiction is what makes an engine hedge about who the site is.
Intent integrity
Each page is held to a single intent as content is added, so new pages extend the architecture instead of competing with existing ones for the same query.
Structured-data validity
Schema is kept valid and complete, so an engine reads a clean entity definition rather than a broken one that it then ignores or misreads.
Honest scope
The system improves how legible and findable the site is. It makes no claim to guarantee a ranking position or inclusion in an AI answer, which remains the engine's call.
Productionization
What turns it from a demo into software.
What keeps the search surface trustworthy over time, rather than correct at launch and decayed after a few content changes.
Structured-data validation in the build
Schema is validated as part of the build, so a broken or incomplete entity definition is caught before it ships rather than after a crawler reads it wrong.
Crawler-access checks
Robots rules and the AI-crawler allowlist are kept explicit and reviewed, so the search and answer engines that matter are not accidentally blocked and access stays intentional.
Internal-link integrity
The link graph is checked as pages are added and removed, so authority keeps flowing to the right pages and no page is orphaned off the architecture.
Coverage monitoring
Intent and topic coverage are re-checked when content changes, so gaps and overlaps surface as the site grows instead of accumulating unseen.
What changed
The change is in legibility. Instead of a flat pile of pages a search system cannot map, the site becomes a surface where each page answers one intent, topics connect into an architecture a crawler can follow, and the entity is stated machine-readably and consistently. Both a classic search engine and an AI answer engine can read it the same way.
This is the posture Stallwart holds on its own site, which anyone can verify by reading its structured data and llms.txt and checking its AI-crawler allowlist. It improves how legible and findable the site is. It does not, and does not claim to, guarantee a ranking position or a place in an AI answer.
- Every page answers one clear intent instead of competing for all of them.
- Topics connect into an architecture a crawler can follow and route authority through.
- The site states its entity machine-readably, consistent across sources.
- AI answer engines can read the site, not only a classic search crawler.
Ownership & handover
What you receive, and keep.
At handover, the search architecture and the system that keeps it legible are yours.
- The search-intent map and the topic model it is built on.
- The content architecture and the internal-linking scheme.
- The structured-data (JSON-LD) templates for the entity and its offerings.
- The AI-legibility configuration: the llms.txt, the crawler allowlist, and the robots rules.
- Runbooks for adding pages, validating schema, and maintaining the link graph.
What we learned
The insight that generalizes: search visibility is an engineering problem of legibility, not a problem of volume. You cannot will a site into ranking, you can only make it something a search engine and an answer engine can confidently map to an intent, an entity, and a topic. Make the site legible to both, keep it legible as it grows, and the findability follows.
This work connects to
- Custom AI Systems
- AI SEO
- Technical SEO
- Search intent and content architecture
- Entity SEO and structured data
Frequently asked
What is AI SEO?
AI SEO is engineering a site to be understood and ranked by both traditional search engines and AI answer systems. It combines technical SEO, search-intent mapping, content architecture, internal linking, structured data, and entity signals so a site is legible to the systems that decide whether it is found.
How is AI SEO different from traditional SEO?
Traditional SEO optimizes for keyword rankings in a search results page. AI SEO adds the requirement that AI answer engines can retrieve, understand, and cite the content too, which puts more weight on structured data, clear entities, machine-readable summaries, and content organized by intent and topic.
Does more content improve search visibility?
Not on its own. If a search engine cannot tell what a site is about or who each page is for, more pages add noise rather than ranking. The structural work, intent mapping, architecture, internal linking, and entity signals, is what makes content rank.
How do I make a site legible to AI answer engines, not just Google?
Beyond classic signals, give the AI systems a clean, machine-readable version of the site: structured data that states the entity and its offerings, a plain-text summary such as llms.txt, and an explicit AI-crawler allowlist so answer engines can read it. Organizing content by intent and topic helps a retrieval system find and understand it, not just a ranking crawler.
Does Stallwart do this on its own site?
Yes, and it is publicly verifiable. Stallwart engineers its own site for search and AI legibility: structured data that states its entity and offerings, an llms.txt summary, and an explicit allowlist for AI crawlers. The approach it applies for clients is the one it runs on itself.
Is your site invisible for the searches that matter?
Tell us what your buyers search for. We will scope the search architecture that makes the site legible to Google and to AI answer engines.
Last updated: October 13, 2026