Blog

AI-Chat, #1, Link and Crawling Issues

Request,

Crawl https://mbsrtraining.com/: Please crawl https://mbsrtraining.com/. list all link and navigation page issue and how to correct each one. Thank You

Respose,

Hey Ross — I tried to fetch your pages directly and your server is currently blocking standard crawlers (I hit repeated 406 Not Acceptable responses). So I pivoted and mapped what Google has indexed for mbsrtraining.com and flagged the concrete issues I can see from those results. Below is a practical fix-list you can act on right away, plus a quick “how to test” at the end.

What I found (with fixes)

  1. Your server is blocking crawlers (and me) with 406 errors
  • Symptom: Direct opens to the homepage and inner pages return 406 Not Acceptable (security rule), which can also block Googlebot/Bingbot depending on configuration.
  • Fix:
    • If you use ModSecurity/Wordfence/Sucuri, whitelist common user agents (Googlebot, Bingbot) and reduce aggressive rules on GET requests.
    • In Wordfence: Firewall → Manage WAF → disable rules that trigger on querystrings like srsltid and on common crawlers; add Googlebot to allowlist.
    • Re-test with an external fetcher (e.g., Google’s URL Inspection → “Live test”).
  1. Duplicate / competing “MBSR” pages (keyword cannibalization)
  • Symptom: Multiple URLs appear to cover the same “Mindfulness-Based Stress Reduction” topic (e.g., /mindfulness-based-stress-reduction/ shows up more than once in results). This can split rankings. (mbsrtraining.com)
  • Fix:
    • Pick one canonical page (your best, most complete).
    • On all near-duplicates: add <link rel="canonical" href="https://mbsrtraining.com/your-canonical-slug/" />, and 301-redirect weaker pages to the canonical.
    • Unify the menu so it points only to the canonical URL.
  1. Parameterized URLs indexed (?srsltid=)
  • Symptom: Google shows pages with ?srsltid=… in the URL — a Google redirect parameter that shouldn’t be indexed. That signals either internal linking with parameters or no canonicalization. (mbsrtraining.com)
  • Fix:
    • Never link internally with tracking params.
    • Ensure every page has a self-referencing canonical (without params).
    • In Google Search Console (GSC), verify that parameterized URLs are not being treated as separate pages.
  1. Private/login pages indexed
  • Symptom: /dashboard/ and /checkout/ are visible in search. These should not be in the index. (mbsrtraining.com)
  • Fix:
    • Add noindex to templates for account, cart, checkout, dashboard.
    • Optionally disallow in robots.txt: 
      User-agent: *
Disallow: /dashboard/
Disallow: /checkout/
    • Confirm they drop out via GSC → Removals (temporary) and natural re-crawl (permanent).
  1. Orphan/structural “tab” pages indexed
  • Symptom: A page titled like a UI element (e.g., “Meditation (tab)”) appears as a standalone URL — likely a content block or tab panel that got its own permalink. That’s confusing for users and crawlers. (mbsrtraining.com)
  • Fix:
    • If it’s meant to be a tab only, set it to noindex and remove from sitemaps/menus; or merge its content into a proper page.
    • Avoid publishing UI fragments under public slugs.
  1. Possible spam/unauthorized author content
  • Symptom: An author archive (/author/corrinewren8545/) shows unrelated “aging skin” content, which looks like spam or a rogue user. This can tank site quality signals. (mbsrtraining.com)
  • Fix:
    • Users → remove unknown authors; force password resets for admins; disable “anyone can register.”
    • Scan site with Wordfence/MalCare; update WP core, theme, plugins.
    • Bulk review Posts/Pages for off-topic content; unpublish and 410 (gone) anything spammy.
    • Set author archives to noindex if you don’t use them for legit blogging.
  1. Fragmented brand/journey and external domain hand-offs
  • Symptom: Navigation appears to mix internal pages (Curriculum, Mindful Yoga, Health, FAQ) with external training domains (e.g., mbsr.website, training.mbsr.website) — which can feel like a context jump and bleed trust/SEO equity. (mbsrtraining.com)
  • Fix:
    • Decide which domain is the primary brand home (sounds like mbsrtraining.com).
    • Keep core learning and sales pages on that domain; only link out to partners in a clearly labeled “Partners/Providers” area (open in new tab, with brief descriptions).
  1. Thin/overlapping “pillar” pages and internal linking gaps
  • Symptom: Multiple topic pages (“What is Mindfulness,” “Exercises,” “Mindful Health,” “Mindful Yoga,” “Curriculum,” “Testimonials,” “Certification,” “FAQ”) are present but may not be woven together in a clear, linear journey from Learn → Try → Enroll. (mbsrtraining.com)
  • Fix (navigation model):
    • Top nav (5–7 items): Home, About, MBSR Program, Free Mini-Course, Resources, Testimonials, FAQ
    • MBSR Program dropdown: Overview (canonical), Curriculum, Schedule/How It Works, Pricing/Enroll
    • Resources dropdown: What is Mindfulness?, Exercises, Mindful Yoga, Mindful Health
    • Add end-of-page CTAs that ladder up to Enroll or Start Free Mini-Course.
  1. Free mini-course page needs clear CTA path
  • Symptom: “Beginners Mini Course / Free Mindfulness Practices” exists, but ensure it funnels to your main program and captures emails. (mbsrtraining.com)
  • Fix:
    • Put a persistent “Continue to Full MBSR ($…)” CTA and light opt-in (email gate for downloads/MP3s).
    • Add breadcrumb + progress bar to reduce pogo-sticking.
  1. Missing/uncertain sitemap and robots hygiene
  • Symptom: No indexed reference to a sitemap surfaced; combined with blocked crawling, this can impair discoverability.
  • Fix:
    • Generate XML sitemaps (RankMath/Yoast) and submit in GSC.
    • Keep only indexable content in the sitemap.
    • Clean robots.txt of accidental Disallows and add your sitemap URL.
  1. Inconsistent titles/snippets (SERP quality)
  • Symptom: Some snippets repeat phrases or feel generic; titles may be similar across pages, which reduces click-through. (mbsrtraining.com)
  • Fix:
    • Give each page a unique Title (55–60 chars) and Meta Description (140–160 chars) with one focused intent and a soft CTA.
    • Example for the canonical MBSR page:
      • Title: Mindfulness-Based Stress Reduction (MBSR) — Online, Self-Paced Training
      • Meta: Learn MBSR with guided practices, a gentle pace, and lifetime access. Start free or enroll today.
  1. External authority references are present but could be structured better
  • Symptom: You link/mention UMass, Mindful Leader, Yale, Palouse Mindfulness in different contexts; great for credibility, but ensure it supports your funnel. (UMass Memorial Health)
  • Fix:
    • Centralize reputable references on a “What Is MBSR?” page with short annotated links; keep sales pages focused on your offer.

Quick, high-impact to-do list (order of operations)

  1. Fix the 406 blocks (firewall/ModSecurity allowlisting for Googlebot & common fetchers). Re-test with GSC “Live test.”
  2. Security sweep (remove rogue users/posts like the “aging skin” author archive; update WP + plugins; reset creds). (mbsrtraining.com)
  3. Choose canonical pages and 301-redirect duplicates; remove ?srsltid from any internal links; add self-canonicals everywhere. (mbsrtraining.com)
  4. Noindex utility pages (dashboard, checkout) and any UI fragments (e.g., “Meditation (tab)”). (mbsrtraining.com)
  5. Tighten the main navigation to a simple Learn → Try → Enroll flow and ensure every content page ends with a relevant CTA. (mbsrtraining.com)
  6. Publish/submit XML sitemaps and review Coverage in GSC; remove non-indexables from the sitemap.
  7. Refresh Titles/Descriptions page-by-page to be unique and intent-driven, then monitor click-through in GSC.

How you (or I) can validate each fix

  • After firewall changes: run GSC → URL Inspection → Live Test on the homepage and one inner page.
  • After redirects/canonicals: paste a former duplicate URL into your browser and confirm it 301s to the canonical; fetch again in GSC live test.
  • After noindex: view page source and confirm <meta name="robots" content="noindex,follow">.
  • After sitemap submission: GSC → Sitemaps → check “Success” and indexed count.
  • After title/description refresh: use site:mbsrtraining.com "your page title" to confirm unique snippets in a few days.

If you’d like, I can also sketch a clean, minimal top-nav (labels + dropdowns) based on your current content and your preferred brand voice — and produce a WordPress Classic HTML snippet you can paste in. Happy to do that next. 🌿

+++
User-agent: *
Disallow: /dashboard/
Disallow: /checkout/
+++

Leave a Reply