What an SEO Agency Does in Month One (And How to Do It Yourself With AI)

Every SEO agency starts a new small-business client the same way. Before any keyword research, before a single blog post, before "content strategy" even comes up — there's a technical setup pass. It's the unglamorous part of the invoice, usually buried in the proposal as "Technical SEO Audit & Setup" or "Month 1 Onboarding."

It almost always covers the same five things:

• robots.txt — telling search engines what they're allowed to crawl
• llms.txt — a newer, contested file aimed at AI crawlers instead of search engines
• sitemap.xml — the map of every page you want indexed
• JSON-LD structured data — machine-readable labels that tell Google what your business actually is
• Google Search Console — the free dashboard that proves you own the site and shows what's happening in search

None of it is secret. None of it requires a developer. If you can log into your hosting account and copy-paste text into a file, and you have an AI assistant to help you get the syntax right, you can do all five yourself — usually faster than an agency gets to it, since you're not waiting in someone else's onboarding queue.

1. robots.txt — Telling Search Engines What They're Allowed to Touch

robots.txt is a plain text file that lives at yoursite.com/robots.txt. It's the first thing most search engine crawlers check before they touch anything else on your site. It doesn't control rankings directly — it controls access.

Here's why it matters more than it sounds like it should: a shocking number of small business sites accidentally block themselves from Google entirely. It happens most often during a site migration or a WordPress relaunch, where a "Discourage search engines from indexing this site" checkbox gets left on, or a developer's staging robots.txt (which correctly blocks everything) gets pushed to the live site by mistake. The business owner has no idea — the site looks fine in a browser, it's just invisible to Google.

What an agency actually does in month one is pull up your current robots.txt, check it isn't quietly blocking the whole site, and replace it with a clean version that allows crawlers into everything that should be public, blocks the handful of paths that shouldn't be indexed (admin logins, cart/checkout pages, internal search results), and points to your sitemap.

The exact syntax changes depending on whether you're on WordPress, Shopify, Squarespace, or a custom site — and getting the wildcard rules wrong is an easy way to block more than you meant to. That's Day 2 of the email course below.

Curious what a real one looks like? Here's ours.

2. llms.txt — Yes, It's Contested. Agencies Set It Up Anyway.

llms.txt is a proposed standard, first floated in late 2024, for a plain-Markdown file at yoursite.com/llms.txt that gives AI systems — ChatGPT, Claude, Perplexity, and Google's AI Overviews — a clean, curated summary of your site instead of forcing them to parse messy HTML.

The honest disclosure: this one is genuinely contested. Unlike robots.txt, which every major crawler has respected for nearly 30 years, there's no confirmed agreement that any major AI system actually reads llms.txt today. Some prominent voices at Google have been openly skeptical it does anything at all.

So why do agencies keep adding it to the setup checklist? Two reasons. First, it costs almost nothing to create — a few hundred words of Markdown, maybe fifteen minutes of work. Second, traffic from AI answer engines is a real and growing referral source, and a well-written llms.txt is a cheap hedge on a channel nobody wants to have skipped if it turns out to matter. Plenty of agencies now bill this under a new line item: "AI Search Optimization" or "Generative Engine Optimization (GEO)."

A good llms.txt isn't a dump of every URL on your site — it's a short, curated summary: your business name and what you do in one line, then a handful of your most important pages, each with a one-sentence description of what's on it.

Which pages to include, how it's structured differently from a sitemap, and how to keep it from going stale — that's Days 8–9 of the email course.

Ours is live at variablesquirrel.com/llms.txt if you want to see the format.

3. sitemap.xml — The Map You Hand to Google

sitemap.xml is a structured XML file listing every URL on your site you want search engines to know about, along with when each page was last updated. Think of it as handing Google a table of contents instead of making it discover your site page by page through links.

This matters most for newer or smaller sites. A site with lots of inbound links gets crawled and discovered quickly on its own. A small business site with a handful of pages and not many external links pointing at it can sit un-indexed for weeks without a sitemap doing the introducing.

What an agency does: generates the sitemap (through a CMS plugin or a small script), makes sure it excludes pages that shouldn't be indexed — thank-you pages, duplicate filtered/sorted URLs, anything marked noindex — and submits it directly inside Google Search Console so Google knows exactly where to look.

Whether to hand-maintain your sitemap or generate it automatically so it never goes stale is covered on Day 6.

Here's this site's sitemap.xml as a working example.

4. JSON-LD Structured Data — Speaking Google's Language Directly

JSON-LD is a block of code — usually a <script type="application/ld+json"> tag — embedded in a page's HTML that describes what the page actually is, using a shared vocabulary called schema.org. It's not visible to human visitors. It's a label written specifically for machines.

It doesn't move rankings by itself, but it unlocks things that do move clicks: star ratings under a search result, an expandable FAQ dropdown right in the search listing, your business hours and address showing up in a knowledge panel. It also matters increasingly for AI answer engines, which lean on structured data to quote a business accurately instead of guessing from body text.

What an agency typically adds first: Organization or LocalBusiness schema sitewide (name, address, phone, hours), Article schema on blog posts, FAQ schema on any page with a real FAQ section, and BreadcrumbList schema so Google can show your page's location in the site hierarchy instead of just a bare URL.

Which schema types actually pay off for a small business site — and how to check your markup validates instead of silently failing — is Days 10–12 of the email course.

5. Confirming Google Search Console — Claiming Your Seat at the Table

Google Search Console is a free tool that shows you exactly how Google sees your site: what's indexed, what's not, what search terms are actually bringing people in, and whether Google has flagged any technical problems or manual penalties. Without it, you're guessing. With it, you're looking at the same data an agency would show you in a monthly report.

What an agency does in month one: verifies ownership of the property (usually via a DNS TXT record, an HTML file upload, or a meta tag — whichever fits your hosting setup), submits the sitemap from step three, checks the Coverage report for pages that failed to index and why, and sets up email alerts so problems get flagged automatically instead of discovered three months later.

The fastest verification method for most small-business hosting setups, and which two reports to check first, are covered on Days 4–5 and Day 13.

Frequently Asked Questions

Prefer to skip straight to having it done for you? Get in touch and we'll take care of it.