Join my community / free newsletter — sign up here

crawler-governance

By Greg Nowak , 7 July, 2026

AI search is eating the click: measure the queries before rewriting pages

AI summaries are cutting into search clicks. Start with a practical dashboard that shows query risk, bot purpose, page value, and crawler policy.

AI crawler permissions now belong in a licensing register

AI crawler controls now affect search visibility, AI answers, model training, and licensing. A register keeps policy, evidence, and enforcement aligned.

Tags

crawler-governance

Review Greg on Google

Written recommendations from Trafik og Veje, Aarhus Municipality (2011) and AgroTech (2010) — read them on LinkedIn.

MCP 2026-07-28 Is an Auth Migration, Not a Version Bump

2026-08-29

MCP’s July 2026 release removes protocol sessions and tightens OAuth. Here’s a practical plan for migrating clients, servers and enterprise access safely.

Turn a Technician’s Voice Note into a Work Order—Not Raw Audio

2026-08-28

Voice input can reduce the technician’s documentation burden when hours, materials and status are validated before the information is saved in the work order system.

ChatGPT Disabled Personal Knowledge Sync. What Broke on Your Team?

2026-08-27

ChatGPT retired personal sync connections for Enterprise and Edu. Here is how to find affected workflows, migrate access, and test permissions.

Cloudflare’s September Bot Defaults Could Quietly Cut AI Visibility

2026-08-26

Cloudflare’s September bot defaults give publishers more control, but one training block could also cut search crawling and AI-driven discovery.

Does Your AI Chatbot Clearly Identify Itself?

2026-08-25

The EU’s transparency requirements for AI chatbots now apply. Here is how to make your bot’s identity clear, limit its system access and provide a genuine route to a member of staff.

Should publishers add Google’s new Preferred Sources button?

2026-08-24

Google’s Preferred Sources button is worth a controlled test for eligible publishers, with careful choices around placement, performance and measurement.

Search Console Can See TikTok Now. Your Reporting Has to Catch Up

2026-08-23

Google can now report how social profiles appear in Search. Here is how to measure cross-channel discovery without mistaking visibility for business results.

Your AI workflow has logs. Can they explain one bad decision?

2026-08-22

Logs can show that every service worked while leaving a bad AI decision unexplained. See how connected traces and careful redaction close the gap.

Security Questionnaires Eat Into Selling Time—Let AI Find the Evidence

2026-08-21

NIS 2 is generating more supplier questionnaires. A controlled AI assistant can find approved answers and sources—and route uncertain cases for review.

Locked out of your Apple developer account? Fix it before October 1

2026-08-20

Apple's updated developer agreement must be accepted by October 1, 2026, and many small app owners cannot even log in. Here is where Apple's two-factor codes really go, and how to fix your access before the deadline.