Web Development and SEO: Why It Starts Before the First Line of Code

A site goes live. Everyone’s proud of how it looks. And then nothing. Weeks go by, traffic trickles in at a pace that would embarrass a brochure, and by the third month it clicks: search engines barely know the site exists.

I’ve watched this play out more times than I’d like to admit, and it’s almost never a design problem or a content problem on its own. It’s a timing problem. Most teams treat SEO as something you bolt on after launch, when the decisions that matter most, the ones that are painful and expensive to undo, all happen before anyone writes code. Build with SEO in the plan and you skip the cleanup that otherwise eats three to six months after launch, and you dodge the structural rewrites that kill ranking potential. This is how my team runs every build, web development and SEO as one process, and we write the first batch of content during development, scored by our own analyzer before a word goes live. Some of the screenshots here come from that tool, Algorithm. Most of what ranks for this query is a pre-launch checklist; this is the set of decisions the checklist assumes someone already made.

How Web Development and SEO Fit Together

People picture SEO and web development as two separate jobs handed to two separate teams: the developers build the thing, then the SEO people show up afterwards to optimize it. That hand-off is where ranking potential leaks away. Think about what gets decided in code. How the site is structured. How URLs are formed. Whether a page renders in a way a crawler can read, or only after a JavaScript bundle executes in the browser. How fast it loads on a mid-range phone. What markup describes the content to a machine. None of those are “add it later” tasks. They’re architecture, and architecture is the developer’s domain. By the time an SEO specialist is brought in post-launch to fix them, the cheap window has closed and you’re paying for rewrites in place of decisions.

Developers don’t have to become SEO experts, and the reverse is also true. The two have to share a plan before anyone writes code: the structure, the URL patterns, the rendering approach, the content the site needs to own. Get that agreed and SEO becomes a property of a well-built site, the way accessibility is on a team that put it in the brief.

The Three Windows Where SEO Gets Decided

SEO is a set of decisions spread across the project, and they land in three windows.

Before development. Your keyword strategy (the semantic core), your site architecture and your rendering approach get locked in. Change your mind later and you’re re-pouring the foundation.

During development. URL patterns get encoded. Meta tags, schema, the whole technical skeleton gets wired up here.

Before launch. Everything gets tested and audited. A problem caught in this window is a quick fix. The same problem caught a month after launch is an emergency, complete with re-crawling and lost ground.

Building the Semantic Core (Before You Commit Code)

The semantic core is the structured answer to one question: what is this site about, and what are people searching for when they need it?

Market intelligence. Look at what competitors rank for, find the gaps they’ve left open, spot seasonal patterns. Semrush, Ahrefs, AnswerThePublic and Google Trends each show a different slice.

Collect keywords. Pull your primary topics (around 5–15 core themes), the long-tail variations that reveal intent, and the question-shaped queries, the “how to,” “what is,” “where to find” ones. That last bucket matters more than it used to, because those are the prompts people type into an assistant verbatim.

Cluster and tag by intent. Group keywords that mean roughly the same thing, then label each cluster by what the searcher wants: to learn, to find a site, to compare, or to buy. Intent is the part most people skip, and it’s the part that makes everything downstream easier. Loose clustering (lumping “best X” with “what is X”) wrecks your architecture, because you build one page for two intents and it ranks for neither. We built our own clustering engine to keep intent boundaries clean at scale, though the principle holds regardless of tool.

Map topics to the site. Every cluster should correspond to an area of the site. This step dictates your URL structure and navigation, because it tells you which pages need to exist and how they relate.

Write it down. One reference sheet with topic pillar, primary keyword, search volume, difficulty, long-tail variations, target URL and intent, shared with everyone touching the project. A semantic core that lives in one person’s head is a liability.

Site Architecture: Easy to Crawl, Easy to Trust

Your structure decides whether search engines can reach your content efficiently, which pages accumulate authority, and how quickly new pages get indexed. The principles I hold to:

• Keep the hierarchy shallow and logical. Three to five primary categories, subcategories under those, individual pages under those, with everything within three clicks of the homepage.
• Keep URLs flat. Three levels deep, max. /resources/blog/seo-techniques is fine. /resources/guides/content/seo/techniques/best-practices is a maze.
• Organize by theme. Pillar pages that cover a major topic, each supported by cluster pages that go deep on subtopics, interlinked deliberately.
• Leave no page orphaned. Every page you want indexed should be reachable from navigation, footer, or an internal link. If nothing points to a page, search engines treat it as if it doesn’t exist, and there’s a rendering trap hiding in that rule, which the next section gets to.

A visual sitemap drawn before development is the blueprint for all of it.

JavaScript, Rendering, and What the Crawler Sees

This is the decision that lives entirely in code, gets made by default when nobody names it, and is the single biggest gap in the guides that rank for this topic. It turns on one question: is your content in the HTML the server returns, or does it only appear after a JavaScript bundle runs in the browser?

Google handles JavaScript in two waves, and most indexing problems live in the gap between them. In the first wave, Googlebot fetches the raw HTML (what the server returns before any script executes) and reads the links, the metadata, and whatever text is already there. If your content isn’t in that response, wave one sees an empty shell. In the second wave the URL goes into a render queue, where Google’s Web Rendering Service (an evergreen Chromium engine) executes the JavaScript and produces the final DOM. Only then does Google see client-side content. The catch is the queue: rendering is expensive, so Google defers it. On a high-priority site the gap is minutes; on a large or low-authority site it stretches to days, and on the slowest it’s been observed at a week or more. Publish time-sensitive content client-side and it can be stale before it’s indexed.

Three rendering strategies sit behind this. Server-side rendering (SSR) builds the full HTML on the server and ships it complete, so the crawler gets everything in wave one, no queue. Static site generation (SSG) pre-builds the HTML at deploy time, faster still and ideal for content that doesn’t change per request. Client-side rendering (CSR) ships a near-empty shell plus a bundle and builds the page in the browser: fine for an app behind a login, dangerous for anything that needs to rank. The rule I hold: if a page earns organic traffic, its primary content belongs in the server HTML. Reserve pure CSR for dashboards and authenticated states Google was never meant to index.

A client-side-routing trap catches teams who think they’ve solved rendering. In a single-page app, navigation often happens in JavaScript without an <a href>, so the internal links that carry authority between pages are invisible to wave one. The pages might render eventually, but the link graph that tells Google how they relate never forms. Test for it by crawling with JavaScript rendering switched off in Screaming Frog: every page that comes back thin or empty, and every link that doesn’t appear, is a page or a path the crawler can’t rely on.

Then the part that changed the calculus, because “Google renders it eventually” stopped being enough. The AI answer engines do not execute JavaScript. Vercel’s December 2024 analysis of crawler traffic across its network found GPTBot downloading JavaScript files in about 11.5% of its requests and ClaudeBot in about 23.84%, and neither executing any of it; a server-log study in January 2026 replicated the finding, with GPTBot’s script execution “extremely limited” rather than literally zero. PerplexityBot behaves the same way. The one exception is Google’s Gemini, which rides on Googlebot’s rendering service. One nuance from Vercel’s data that matters for modern stacks: content that sits in the initial HTML response can still be ingested even when it’s inside a JSON payload or a serialized server component, because the models read non-HTML text; what they can’t do is run the script that would fetch it. So a client-rendered page can rank perfectly well in classic Google search while being a blank shell to ChatGPT, Claude and Perplexity, cited by none of them, because the crawler that feeds them only ever saw a loading state. Bingbot, which ChatGPT’s search draws on, renders JavaScript only partially, so a CSR site can be cut off from two directions at once.

The test costs two minutes. Open the page and view source (the served HTML, not the DevTools Elements panel, which shows the rendered DOM after scripts run), then search for a sentence of your main content. If it’s in view source, both Google and the AI crawlers can read it. If it only appears in DevTools, you’re depending on a render some crawlers do late and most don’t do at all.

URL Structure and the Technical Foundation

A few URL habits that pay off for the life of the site: separate words with hyphens (on-page-seo, never on_page_seo); skip special characters and stray parameters; keep them descriptive but short, three to five words; mirror the site hierarchy in the path; and decide the strategy before you build, because changing URLs later means a pile of 301 redirects and the mistakes that come with them.

The technical checklist that should be in place at launch:

• robots.txt — Block what crawlers don’t need (admin, staging, internal search) while keeping important resources open, and point to your sitemap. One 2026 addition: decide, by agent name, how you treat the AI crawlers, because they aren’t interchangeable. OpenAI’s GPTBot is the training crawler and OAI-SearchBot is the one that feeds ChatGPT search; a rule written in 2023 to keep GPTBot out doesn’t touch search, and a blanket block written last year keeps you out of the answers. Google-Extended controls Gemini training and grounding, and has no effect on AI Overviews or AI Mode, which run on Googlebot. Then ClaudeBot and PerplexityBot on the same terms. Allowing the search agents is how your content becomes eligible to be cited, but per the section above, only if that content is in the HTML they can read.
• sitemap.xml — Every page that matters, auto-generated through your CMS so it stays current. On large sites, split it before you hit the 50,000-URL / 50 MB per-file limit and gather the parts under a sitemap index.
• Canonical tags — Self-referencing on unique pages, so you don’t split authority across duplicate URLs. On JavaScript stacks, verify the canonical in the served HTML matches the one after render; they can diverge, and the CMS section explains why.
• HTTPS — SSL, clean HTTP→HTTPS redirects, and an HSTS header.
• Mobile — Responsive is the floor. Test on devices, not the emulator: content reachable without zooming, tap targets at least 48px. Over half of all search traffic is mobile.

Core Web Vitals. These are confirmed ranking factors, and a couple of numbers people memorized are still circulating wrong. The current thresholds, measured on field data at the 75th percentile:

INP trips people up most. If your checklist still references “First Input Delay under 100ms,” it’s two years out of date, and INP is stricter because it watches every interaction across a session. And these numbers come from the Chrome User Experience Report, users in the field at the 75th percentile, which is why a page that flies on your laptop can still fail. Read them per template: the listing page, the category page and the homepage fail differently, and a site-wide score hides all three.

The revenue reason to care outweighs the raw ranking factor. In Rakuten 24’s Core Web Vitals A/B test, the faster variant earned 53.37% more revenue per visitor, with the two versions differing only in performance (web.dev, 2023). Portent’s analysis of over 100 million page views found pages loading in one second convert at roughly three times the rate of pages loading in five. The bar is lower than it looks: only about 48% of sites deliver a good Core Web Vitals experience on mobile (Web Almanac), so merely passing already beats half the web. The older Deloitte/Google study still holds the shape of it: a 0.1-second mobile speed improvement was associated with retail conversions 8.4% higher and travel conversions 10.1% higher, across 37 brands and 30 million sessions (2020).

Schema Markup: Talking to Machines in Their Own Language

Schema turns plain text into structured data that search engines, and increasingly AI models, can read without guessing. It powers rich results, featured snippets, and the answers that surface in voice and AI search. The types most sites need:

• Organization (homepage) — name, logo, contact, social profiles. The backbone of a knowledge panel.
• Article / BlogPosting (posts) — headline, publish date, author, image.
• FAQPage (high-intent pages) — question-and-answer pairs that make you eligible for snippet placement.
• BreadcrumbList (every page) — communicates hierarchy and tends to improve both click-through and crawl efficiency.

Use JSON-LD. It lives separately from your HTML, and it’s the format Google explicitly prefers. The shape of it:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "Article",
  "headline": "Web Development and SEO",
  "datePublished": "2026-06-08",
  "author": {
    "@type": "Person",
    "name": "Roman Makuev",
    "url": "https://seo-algorithm.com/"
  },
  "image": "https://example.com/cover.png"
}
</script>

Schema matters more each year for a reason that has little to do with rich results: grounding. When a page names an entity (a person, a product, an organization), schema is how a machine ties those words to a thing it can trust, in place of inferring one and sometimes getting it wrong. That matters doubly for the AI engines, which lean on structured data harder than classic crawlers because it lets them extract a fact rather than guess at it, and because, per the rendering section, they can’t run the JavaScript that would reveal the same fact in the visible page. The rule that trips people up: your schema must match what’s visible on the page. Don’t mark up an FAQ that isn’t there. Before launch, run everything through Google’s Rich Results Test.

CMS Architecture: Can Your CMS Even Do SEO?

A failure I see land after every other decision on this page was made correctly: the strategy is sound, the architecture is clean, the content adds something, and the CMS won’t let anyone ship it. The platform decides what you can control, and if it can’t edit the SEO primitives, no amount of strategy gets past it. Before you commit to a platform, the question is whether an editor can set, per page, without a developer: the title tag and meta description, the canonical, the H1 independent of the visual headline, the URL slug, the structured data, and a redirect when the URL changes. If any of those routes through an engineering ticket, your SEO velocity is capped at the speed of the deploy queue.

The platforms differ in where they pinch. WordPress with Rank Math or Yoast gives you the deepest control (20-plus schema types through a visual builder, per-page everything) at the cost of plugin sprawl and the classic conflict where two plugins each set a canonical and Google picks the wrong one. Webflow exposes title, meta, canonical and Open Graph natively with no plugin, canonical lives in page settings, and CMS collections auto-canonicalize to their template; it’s clean, but native schema depth is thin, so anything past the basics needs a code embed or a marketplace app. Headless (Contentful, Sanity, Storyblok on a Next.js or Nuxt front end) gives engineers total control and gives marketers nothing by default: meta tags, canonicals, structured data and sitemaps don’t exist until someone builds them, which is why so many headless launches turn into invisible websites. Headless also carries a failure mode monolithic platforms don’t: the canonical in the CMS field and the canonical in the rendered output can diverge, because a component injects a page-level tag the template already set, so the page claims two different canonicals and Google resolves the conflict for you, usually the wrong way. On headless, fixing a canonical is a deploy.

None of these is wrong. The mistake is choosing one without mapping it against who has to run SEO day to day. If the team is non-technical and needs to move fast, a platform that surfaces the SEO fields natively beats one that’s theoretically more powerful and practically blocked on engineering. Decide that before the build, because migrating CMS after launch is its own painful project.

The Part the Old Playbooks Skipped: AI Search

A growing share of queries get answered directly, inside Google’s AI Overviews or inside ChatGPT, Perplexity, Gemini and Claude, without the user ever clicking through. If your content can’t be read, understood and cited by those systems, you’re invisible in a channel that’s eating a chunk of your traffic.

The ranking-to-citation link has loosened. Ahrefs’ March 2026 analysis of 863,000 keywords and around four million AI Overview URLs found about 38% of cited pages also ranking in Google’s top 10 for the same query; in its July 2025 study the figure was 76%. The rest split almost evenly between positions 11–100 and pages outside the top 100 entirely. Two cautions before you build a strategy on the drop. Ahrefs says its citation detection improved between the two runs, so part of the fall is better measurement, and it attributes the rest to query fan-out, where Google splits a query into sub-queries and cites pages that perform across the cluster, which it hasn’t confirmed. Hold the exact number loosely and the direction firmly: ranking #1 no longer guarantees a seat in the AI answer, and being cited is becoming its own discipline, rewarding depth and source-worthiness over position. The reassuring part is that there’s no separate dark art. Google’s own guidance on AI features (the “AI features and your website” page, May 2025) says the foundation for AI Overviews and AI Mode is the same as for Search: original content, a crawlable and well-structured site, a clear answer near the top.

What the layer adds on top of that:

• Answer the question early. AI systems extract facts. Put a clear, self-contained answer in the first hundred words, and phrase headings as the questions people ask.
• Make the answer readable without JavaScript. This is the rendering section cashing out. The AI crawlers don’t execute JS, so the answer has to be in the server HTML, the most common reason a page that ranks on Google gets cited nowhere else.
• Make authority verifiable. AI systems cite authors with a digital footprint: a profile, mentions in the Knowledge Graph, external references. E-E-A-T becomes a technical requirement.
• Lean harder on structured data. LLMs rely on schema more than traditional crawlers do, because it’s how they extract facts rather than infer them.
• Offer something that isn’t a commodity. When a model can summarize the generic version of any topic in two sentences, the only content it has a reason to cite is content that says something it couldn’t generate on its own.

Information Gain: The Standard the Index Now Uses

If you take one idea from this piece, take this one. Topic authority in 2026 is awarded for added value. Google has gotten ruthless about excluding rehashes of whatever’s already in the top 10. If an article doesn’t bring new data, a fresh case, a different angle, or a connection between ideas that wasn’t already out there, it doesn’t earn authority, and it can sit at zero no matter how long or well-formatted it is. The term is Information Gain: how much new you added to the index. Google holds a patent on scoring a document by the information it adds beyond what the user has already seen on the results before it, and that framing has carried straight into the AI-answer era, where “does this page add anything” is the question a citation engine asks.

The consequence most people underestimate: Google evaluates the quality of your domain as a whole. Since the March 2024 core update, the helpful-content assessment is a site-wide signal folded into core ranking. Google’s own wording is that a site with relatively high amounts of unhelpful content overall is less likely to do well in Search, and “overall” includes the service and money pages that sell. A blog full of thin, AI-spun articles published for volume doesn’t only fail to rank itself; it drags down the trust of the entire domain. I’ve audited sites where the services pages were good but couldn’t rank because a neglected blog of sixty low-effort posts had poisoned the host-level signals. The recovery clock is unforgiving, too: the re-rating lands at a later core update, months out, so a domain dragging today keeps dragging until Google runs the next pass and notices you cleaned up.

This is why my team writes scored content during development rather than spinning up filler later. Before anything publishes, we run it through our own analyzer: it flags AI-pattern density, checks whether the piece adds information versus paraphrasing the pages already ranking, and grades it against the genre. A page that scores as a rehash never goes live, because one weak cluster can cost the domain. The tooling is ours; the habit travels: measure information gain before you publish.

On-Page and Internal Links, Briefly

Titles, descriptions and heading order are covered in every guide that ranks for this topic and I won’t repeat them. Two things from that layer do belong in a build article. First, internal linking as architecture: the pillar-and-cluster model, where a pillar page covers the topic and cluster pages go deep on its parts, each linking back, is how you tell a search engine you’ve covered the ground. What makes it work is anchor text that describes the destination, the important links high on the page, restraint, and the habit of pointing your strongest existing pages at anything new. Second, the rendering caveat again: if those links are injected by client-side routing without an href, wave one never sees them and the equity never flows. Build them as links.

Pillar: "Content Marketing Strategy: Complete Guide"
├── Cluster: "Content Marketing for SaaS"
├── Cluster: "Content Calendar Creation"
└── Cluster: "Measuring Content ROI"

Measuring What You Built

Once a month is plenty. Watch organic traffic, the average position of your target keywords, click-through rate, how many pages are in the index, and your Core Web Vitals field data. Add one more: whether the AI tools mention you at all. The low-tech check is to open ChatGPT, Gemini, Perplexity and Copilot, ask them the questions your customers ask, and see whether your name comes up, and whether what they say is even right. The analytics side is simpler than it looks: ChatGPT appends utm_source=chatgpt.com to the links it hands out, Perplexity and Copilot pass a referrer, so the assistant channel is a segment you can build before launch. Search Console and Analytics 4 do most of the heavy lifting and they’re free; add a rank tracker and a speed tool so you can see the gap between your lab numbers and the field. Make your changes, then leave them alone for four to six weeks before deciding whether they worked.

Rough Timelines, by Project Size

None of this is a rule, but after enough projects the shapes are consistent.

SEO as Acceptance Criteria

The way to keep all of this from becoming a post-launch cleanup is to write it into the definition of done, the same way accessibility and security go into the brief. A template doesn’t pass review until it passes these, and each one has an owner who isn’t “the SEO person.” The list I hand to the engineering lead before the first sprint:

The failures I see most often are the rows above, missed. The site ships client-side rendered, so the content that should rank sits behind JavaScript, visible in the browser, late to Google, invisible to every AI engine. Client-side routing strips the link graph, so authority never flows and nobody can see why rankings sag. The checklist still optimizes for FID or chases a 2.0s LCP number Google never set. The blog gets stuffed with filler to hit a word count and drags the domain’s trust down. And the one that hurts most because it’s so avoidable: the site launches with no analytics or Search Console in place, so for the first weeks nobody can see anything at all.

FAQ

Is SEO cheaper if you do it during development?

Much cheaper. Fixing it on a live site usually means three to six months of cleanup plus the structural rewrites nobody enjoys: changing URLs, building redirect chains, waiting on re-crawls. The decisions you make before any code exists, like architecture and the semantic core, cost almost nothing to set and a small fortune to unpick.

Does client-side rendering hurt SEO?

It can, and the damage is uneven. Google will usually render a client-side page eventually, on a delay that runs from minutes to days, and time-sensitive content can be stale before it’s indexed. The bigger problem in 2026 is that the AI crawlers don’t render JavaScript, so a client-rendered page can rank on Google and be a blank shell to ChatGPT, Claude and Perplexity. If a page needs to rank or be cited, put its content in the server HTML.

Is FID still a Core Web Vital?

No. It was retired in March 2024 when Interaction to Next Paint took over, and Chrome dropped support entirely. Hit INP under 200ms, LCP under 2.5s, and CLS under 0.1, measured on field data. Ignore any checklist still naming FID, or one claiming Google lowered LCP to 2.0s.

Does ranking #1 get me into Google’s AI Overview?

Less than it used to. In Ahrefs’ March 2026 study about 38% of AI Overview citations came from top-10 pages, against 76% the previous July, with the caveat that detection improved between the two runs. What earns the citation is depth, an author Google can verify, content that adds something, and being readable without JavaScript.

Can a bad blog hurt my service pages?

It can. Since March 2024 Google judges quality at the level of the whole domain, so a blog packed with thin or AI-spun posts pulls down the trust signals across the site and holds back your commercial pages even when those pages are good.

What is Information Gain?

How much new your content brings to the index: original data, first-hand experience, an angle nobody’s published. Google patented a way to score it, and in 2026 it’s what builds topic authority; straight rehashes of the top 10 tend to get left out.

Where This Goes

The development stage is the most leverage over search visibility you will ever have.

The sites that end up owning their search results almost never got there by luck. Someone decided, before a line of code was written, how the thing would be structured, which topics it would own, how it would render, and how information would move through it. A small edge at the foundation turns into a wide gap a year down the line.

My bet for the next two years: by 2028 the view-source test will sit in every serious front-end team’s definition of done next to the accessibility audit, and “it ranks on Google but no assistant has ever cited it” will be the diagnosis that ends the client-side-rendering argument for marketing sites for good.

If carrying it yourself isn’t realistic, that’s what our web development team does: design, build and search as one process, with the rendering and the CMS decided before the first sprint.