Long-Tail Keyword Research: Low-Competition Discovery
skillFinds long-tail keywords through intent analysis, query expansion, and competitor-page review. Use for prioritized keyword briefs and content-gap research.
instructions
Overview
Most keyword research dies at the head term. A DR-25 site chasing "project management software" (KD 72) loses to Asana and Monday.com forever, but the same audience also searches "free task tracker for two-person agencies" (KD 8, weak SERPs, transactional intent), the keyword class that converts on month-three traffic instead of year-three. This skill is the funnel that finds those reliably: seed, modifier expansion, SERP-mined patterns, and competitor gap, then a SERP-authority rubric that beats raw KD, then a prioritized brief mapped to content types and intent.
Mental model
Long-tail wins on four compounding axes that head-term chasing loses:
- SERP authority profile beats KD. KD is a dumb average; what matters is whether the actual top 10 are beatable. A "KD 35" keyword topped by three generic 1,500-word AI posts is more winnable than a "KD 18" one held by Wikipedia, a 5-year-old definitive guide, and the official product page.
- Intent specificity is the conversion lever. "[head term] for [audience] under [constraint]" attracts only buyers. A 100-volume long-tail at 8% conversion beats a 10,000-volume head term at 0.2%.
- Long-tail clusters earn topical authority. Five 200-volume long-tails on one sub-topic, internally linked to a pillar, signal more expertise than one head-term page. Google's core ranking (which absorbed the Helpful Content System in March 2024) rewards original information, depth, and entity-coverage breadth, not surface listicles.
- Query fan-out rewards coverage, not page count. Google's AI features issue concurrent related sub-queries (Google AI optimization guide, May 2026). A cluster that genuinely answers them gets surfaced; a page per variation built to manipulate AI answers is scaled content abuse, and Google says AI search needs no extra long-tail keywords. Map fan-out questions to sections first, new pages only for distinct intents.
2026 algorithmic context (kill-rules upstream of the funnel)
| Reality | Implication for keyword selection |
|---|---|
| Core updates: March 2026 (Mar 27–Apr 8), May 2026 (May 21–Jun 2); Helpful Content signals in core ranking since March 2024 | Google gave no specifics; third-party analyses saw aggregation-model sites (comparison, directory, job board) lose. Pick keywords where your unique angle, data, or experience fills a gap. |
| Site reputation policy (enforced since May 2024; manual actions no longer apply to EEA searchers since Aug 30, 2026) | "Best [X] [year]" third-party sections on big publishers still rank in some markets. Score these SERPs conservatively unless DR ≥ 50 or a niche-brand-page play. |
| AI Overviews / AI Mode can change informational search journeys; published CTR studies describe their own samples, not this site's expected change | Search Console's Generative AI performance report shows their impressions, not clicks, and Web totals also include them; verify live dimensions before claiming query-level presence. Keep actual engine captures and scoped vendor samples separate from search totals. |
| Branded web mentions correlate with AI Overview visibility more than backlinks (Ahrefs 2025; correlation, not cause) | Long-term, brand + modifier keywords become a defensive cluster. Short-term, long-tails still convert. |
| Reddit / forum threads hold many SERP slots and AI-answer citations (shares vary widely by study) | When the top 5 organic results are Reddit threads, the keyword is community-owned; engage there in parallel with publishing. |
Rules
Core rules
- Long-tail = 4+ words OR a clear intent-specific modifier on a head term. "Best CRM" is short-tail; "best CRM for solo financial advisors" is long-tail. Specificity matters, not word count.
- SERP review is non-negotiable. Inspect the actual top 10 for every keyword that survives expansion. KD from any tool (Ahrefs, SEMrush, SE Ranking, Serpstat) is a starting estimate, not a verdict.
- Realism rule (heuristic): target keywords whose top-10 average DR is within roughly +15 of your DR. A DR-25 site can credibly rank against DR 10–40; a top 10 averaging DR 60+ is a multi-year project. Widen the band when the top 10 are weak or old, narrow it when they are definitive.
- Information gain is the on-page kill rule (Google's self-assessment asks for "original information, reporting, research, or analysis"; "information gain" itself is patent language, not a documented ranking system). If you can't name what the new piece adds over the top 10 (data, perspective, structure, freshness), the keyword isn't yours yet; picking it without an angle is the same mistake as picking a head term without DR.
- Intent must match the site's commercial position. Informational keywords on a transactional product page, or commercial-investigation keywords on a thin glossary page, neither rank nor convert. Classify intent before assigning content type.
- Five+ candidates per sub-topic, not one. Single-keyword targeting anchors to whatever came first; even if four are dead ends, the comparison clarifies why the survivor wins.
- Zero-volume keywords are a real 2026 category. AI search shifts demand to long-tails that tools mark "0"; one ranking for 12 related semantic queries can earn 200+ visits/month. Check related-keyword overlap instead of auto-striking.
- GEO requests start from buyer decisions, not a keyword score. Use the
geo-visibility-auditskill for question discovery, observed citation gaps, or retests; DR and keyword volume aren't prerequisites. Conventional keywords may carry a provisional AI-answer relevance note, not a predicted citation score. Google AI-feature eligibility follows indexing and snippet eligibility, with no additional technical requirements. - Output a brief, not a verdict (see Stage 6); writing, schema, internal-link wiring, and outreach are downstream.
seo-datadefaults to keyless DuckDuckGo search plus public-page inspection; it supplies no measured volume, difficulty, backlink metrics, or DR. Optional providers include DataForSEO, Ahrefs, Semrush, SE Ranking, Serpstat, and authorized Search Console. Inspect the connected provider's tools/docs, confirm scope/budget before billable calls, and record provider, endpoint, locale, timestamp, and returned cost. Only Ahrefs data establishes Ahrefs DR. Search Console is first-party performance, not competitor intelligence; unofficial autocomplete/Trends sources provide suggestions and relative interest, not measured volume. Missing credentials leave the public-evidence path available.- Compound over spike. Trend keywords decay within 12 months; "how to [permanent task]" compounds for years.
The funnel
Stage 1 — Profile the site (kill or proceed)
This funnel qualifies conventional Google keywords. For a GEO/AEO request, use the GEO skill's bounded mode instead of forcing this funnel or its DR gate; an overlapping activation doesn't change the requested deliverable.
Get concrete answers before generating any list; if unclear, stop and ask.
- Niche: one sentence, plus sub-niche.
- Audience: the specific segment that buys, not "small businesses."
- Ahrefs DR if available: obtain Ahrefs data or a supplied export; label missing DR Unknown. Moz and other authority scores are separate metrics, not Ahrefs DR.
- Existing content: count and coverage. For an existing site, check what already ranks (Search Console queries) before generating overlap.
- Commercial position: informational, affiliate, SaaS, e-commerce, local services. Decides which intent classes to prioritize.
- Geo + language: global English, US-only, multi-region, non-English. Affects SERP review and modifier set.
Ask one focused question if the buyer or conversion goal is unclear. Missing DR or Search Console does not block a provisional public-evidence brief; qualify intent and content gaps, leave numerical realism Unknown, and name the evidence needed before claiming ranking feasibility.
Stage 2 — Generate seed keywords (parallel)
Fire all seed sources in ONE parallel block:
- Core product/service nouns, in buyer language, not internal jargon.
- Problem phrases: "[problem] when [context]", "[symptom] in [audience]", "stop [bad thing]".
- Solution phrases: "how to [task]", "[task] without [obstacle]", "alternative to [incumbent]".
- Audience qualifiers: "[product] for [segment / team size / use case]".
- Industry / vertical terminology from the buyer's world.
- Existing-content mining: Search Console queries pulling impressions but ranking page 2–5; each seeds related long-tails.
- Competitor SERP gaps: 3 competitors at the same DR tier, their top-200 ranking keywords (Ahrefs Site Explorer / SE Ranking competitor research / SEMrush Domain Overview), filtered to those the user's site doesn't rank for.
Output: 30–80 seed terms, unqualified.
Stage 3 — Expand to long-tail (modifier patterns + SERP mining)
Generate 5–15 variants per seed, all seeds in parallel, from the modifier buckets in references/modifier-library.md: question (highest GEO value), intent (commercial/transactional), constraint (highest yield for low-DR sites; every constraint kills a competing page), outcome, comparison (mid-funnel, AI-citation-friendly), and problem-trigger (error messages, support-forum phrasings; highest conversion when the product solves the pain).
Fire the SERP-mined sources listed there (autocomplete a–z, PAA, Related searches, Trends rising, Reddit/forums, Lowfruits, Glimpse / Exploding Topics) in one block. The web search tool isn't Google: autocomplete, PAA, and Related searches come from the user, a connected SERP tool, or public PAA tools; list any source you couldn't reach as unused in Method Notes.
Output: 200–500 candidates, deduped at Stage 4.
Stage 4 — Qualify each long-tail (SERP-first rubric)
Walk the rubric top to bottom. Unavailable metrics stay Unknown, not 0 or a failed check; keep those candidates provisional and name the missing verification before calling them qualified.
| # | Check | How to verify | Strike condition |
|---|---|---|---|
| 1 | Intent match | What does the searcher actually want? | Mismatch with the site's commercial position |
| 2 | Volume realism | Tool estimate + Google Trends sanity check | Volume 0 AND no related-keyword cluster |
| 3 | SERP authority profile | Top 10: domain DRs, word counts, content age, brand vs independent | Top-10 average DR > user DR + 15 AND no obvious content gap |
| 4 | Top-10 content quality | Read 3–5 results: comprehensive, recent, structured? | Already definitive (a perfect Wikipedia page, official docs, a 5k-word evergreen pillar) |
| 5 | Content-fit gap | Sub-angle the top 10 miss (audience, constraint, use case)? | Every angle covered well |
| 6 | Paid ads dominance | Count ads above organic | 4 ads above organic; organic CTR drops below 20% |
| 7 | AI Overview presence | Only from an actual dated capture or an explicitly documented SERP-data field; Search Console Web aggregates and ordinary web search don't establish query-level presence | Record observed presence or Unverified; don't forecast this site's CTR loss from another study |
| 8 | Site reputation risk | Top 3 are third-party sections on Forbes/CNN-style publishers? | Yes AND user DR < 50: demote (outside the EEA these sections face manual actions, so re-check periodically) |
| 9 | Branded SERP | Top 10 mostly one brand's pages? | Searcher wants a specific brand, not a category answer |
| 10 | Featured snippet | Who holds it? | Holder DR 70+ with strong content; a weak holder is a snippet-steal opportunity |
| 11 | Trend direction | Google Trends 5-year view | Sharp decline |
| 12 | Cluster fit | Fits a planned cluster? | Orphan with no internal-link path |
A fully qualified candidate must pass the applicable checks. Missing evidence leaves it provisional, not struck. For reported zero-volume candidates, look for corroborating buyer questions, related queries, PAA, autocomplete, or relevant community examples; record the evidence without treating mention counts as demand volume.
Stage 5 — Cluster, map entity coverage, prioritize
Pick pillar candidates: broad seeds with the most survivors around them.
Group survivors under pillars by sub-topic (one pillar + 4–8 cluster pages).
Map entity coverage per cluster: named entities, sub-topics, and concepts the pillar must cover, from the top-10 pages' H2/H3 structure (n-gram-mined), Wikipedia "See also" for the seed, and knowledge-panel related entities. A pillar missing 3+ entities the whole top 10 covers signals shallow coverage.
Score each survivor:
- Realism: top-10 authority vs user DR (1 = unwinnable, 5 = clearly beatable).
- Intent value: informational 1, navigational 1, commercial 2, transactional 3, multiplied by relevance to the site's commercial position.
- GEO potential: definition / how-to / comparison / list, 1 each (max 4).
- Cluster fit: pillar 5, primary cluster page 4, secondary 3, orphan 1.
- AIO discount: subtract 2 only when an AI Overview is observed on an informational keyword AND the page has no information-gain angle. Unknown presence stays Unverified, not a negative observation.
Composite priority = (realism × 3) + (intent × 2) + GEO + cluster_fit − AIO_discount. Range 9–35.
Bucket: P0 quick wins (≥ 28, publish first month), P1 high-value (22–27, next sprint), P2 cluster fillers (16–21, batch with pillar), P3 monitor (< 16, not this quarter). Realism 1 forces P3 regardless of total.
Stage 6 — Output the keyword brief
Fill every section of references/brief-template.md (P0–P3 tables with intent, content type, and calendar slot; clusters with entity coverage, GEO notes, risk notes, existing-content audit, downstream open items). The brief is the deliverable; anything outside it is downstream.
Decision guide — common edge cases
| Situation | Action |
|---|---|
| No DR estimate and no Search Console | Proceed with public intent/content-gap evidence; mark DR, performance, and numerical realism Unknown, with verification as an open item. |
| Top 10 includes Wikipedia | Strike; cluster around adjacent long-tails. |
| Volume 0 but 12 Reddit threads on the question | Keep; score realism normally. |
| User pushes a head term ("just give me 'project management software'") | Refuse and explain: a 24-month project at their DR. Offer P0 long-tails in the same cluster as the first target. |
| Top 10 are 4+ years old and thin (under 1,000 words) | Quick win; fresh, comprehensive content leapfrogs. |
| Featured snippet held by DR 70+ with strong content | Strike, unless it's wrong/incomplete and the user can demonstrably do better. |
| 4+ paid ads | Demote (organic CTR 15–20%); pick a less-monetized adjacent long-tail or pay. |
| Non-English market | Re-run Stages 2–4 with native-language seeds on the local Google domain. |
| Brand names in seeds | Treat separately: brand terms convert at 5–10× organic, but the play is brand defense + comparison content. |
Tools
Free-vs-paid tool table: references/modifier-library.md. Use a connected metrics provider for its documented keyword, difficulty, or SERP data. Search Console supplies authorized first-party clicks, impressions, CTR, and position, not market-wide volume, KD, or competitor SERPs. No connection is assumed; paid tools never replace manual SERP review.
Examples
Example 1: SaaS, mid-authority, finds quick wins
Open-source self-hosted CRM, DR 28: the rubric keeps "self-hosted CRM for sales teams under 10" (KD 14, top-10 avg DR 22) as P0 and zero-volume "CRM Docker compose" as P1, strikes "best CRM software" (KD 78) and "CRM" (branded), and clusters survivors under a "Self-hosted CRM: complete guide" pillar. Full walkthrough: references/brief-template.md (worked example).
Example 2: New site, no DR — refuse the head term
User: "We just launched. We sell project management software. Find me 50 keywords."
Stop. At a brand-new domain (DR ~0–5), 80% of "project management software" keywords are unwinnable for 12+ months; Asana, Monday, ClickUp, and Notion own them with decade-old backlink graphs. Fifty generic PM keywords would waste your first quarter. Two questions first:
- Which segment converts that the incumbents underserve (e.g. "construction subcontractors with 3–15 person crews", "law firms doing matter management")? Narrower segment, more long-tail surface.
- What's the 1–2-sentence reason to switch (feature gap, pricing model, integration, vertical workflow)? That becomes the Stage 3 modifier set.
If "we don't know yet", positioning is upstream of keyword research: pick the niche, then run the funnel.
Example 3: GSC mining for free quick wins
For any site live ≥ 3 months with Search Console, run this first; it's the highest-ROI single source. Pull queries with impressions > 100 and average position 8–20 (page 2), confirm intent against the existing page, and SERP-review for room to climb. Matching intent: a 200-word section plus an intent-aligned H2 typically lifts it into the top 10 in 4–8 weeks. Mismatched intent or incidental overlap: a new dedicated page that outranks the parent.
Checklist
- Stage 1 profile captured: niche, audience, DR if available, existing coverage, commercial position, geo
- Buyer/conversion goal confirmed; unavailable DR, performance, or volume left Unknown with a provisional qualification
- Seeds from ≥ 4 of the 7 source categories, run in parallel
- 5–15 variants per seed from the modifier library
- Manual SERP review for every candidate surviving volume + intent (rubric check 3)
- DR realism heuristic applied only with Ahrefs evidence; otherwise numerical realism remains Unknown
- AIO presence recorded from real evidence or marked Unverified; informational AIO keywords without an information-gain angle demoted
- Site Reputation Abuse risk flagged on "best [thing] [year]" keywords dominated by Forbes/CNN/Times brand pages
- Information gain articulated per P0 keyword
- Zero-volume candidates kept with Reddit/PAA validation or struck, never auto-skipped
- Survivors clustered into pillar-and-cluster structures, not flat lists
- Entity coverage mapped per cluster (top-10 H2/H3 mining, Wikipedia "See also", knowledge-panel entities)
- P0–P3 bucketing via the composite formula incl. AIO discount; realism 1 demotes to P3
- GEO potential noted per keyword (question / definition / comparison / list)
- Brief follows
references/brief-template.md; downstream concerns listed as Open Items - Cannibalization checked: any P0 keyword already targeted by an existing page?
- AI search visibility tracking listed in Open Items for GEO-flagged P0/P1 keywords
Composition / References
In-skill references (load on demand):
references/modifier-library.md— full modifier library, SERP-mined sources, and free-vs-paid tool table for Stage 3references/brief-template.md— canonical Stage 6 brief template, field definitions, and the full Example 1 walkthrough
Within-domain pairings:
- The sibling SEO skill that prospects backlinks for cluster pillar pages (priority for any P0 cluster lacking link equity).
geo-visibility-audit— buyer-question discovery, captured-answer diagnosis, and retests; its scope and no-DR gate take precedence for GEO requests.- The sibling SEO skill that validates free-tool ideas: a cluster surfacing a recurring user problem (calculator, checker, generator) is a tool-build trigger.
Downstream concerns (other domains):
- GEO content writing (clear answers, information gain, crawlable text); the brief only flags GEO-eligible keywords.
- Content briefs, internal-link wiring, on-page schema, outreach copy consume this brief.
- AI search visibility tracking setup (Profound / Otterly / AthenaHQ / Peec); the brief lists which keywords to track.
External authoritative sources:
- Ahrefs — long-tail keyword research
- Backlinko — keyword research definitive guide
- Google Search Central — Helpful Content guidance
- Google Search Central — Site Reputation Abuse policy
- SparkToro — zero-volume keyword research framework
- Search Engine Journal — SERP analysis for keyword qualification
- Search Engine Land — AI Overviews CTR impact studies 2024–2026
- Google Search Central — AI features and your website — eligibility and AI-feature traffic within Search Console Web performance.
- Google Search Central — Optimizing for generative AI features (May 2026) — query fan-out, scaled content abuse, mythbusting
- Princeton/Georgia Tech — GEO research paper (Aggarwal et al) — original framework for AI-citation optimization