Some developers have been experimenting with bot-specific Markdown delivery as a way to reduce token usage for AI crawlers. Google Search Advocate John Mueller pushed back on the idea of serving raw Markdown files to LLM crawlers, raising technical concerns on Reddit and calling the concept “a stupid idea” on Bluesky. What’s Happening A developer […]
Information Retrieval Part 2: How To Get Into Model Training Data
If a language model has never encountered your brand, it cannot recommend you. It guesses, and a guess usually means it names a competitor. That is the mechanic behind every conversation about AI visibility, and it starts long before anyone types a prompt. It starts with model training data. Understanding how that data gets gathered, […]