DEVOURED
Does a URL in a prompt steer an LLM's output toward its content?
URLs in prompts can steer model output, but only if the content at the URL was already memorized during training; bare links to client-rendered apps are largely invisible.
What: Paul Kinlan discovered that models often cannot 'see' JavaScript-rendered content because training crawlers (like ClaudeBot) rarely execute JS. Consequently, URLs for modern single-page applications (like ChromeStatus) fail to provide context, while famous identifiers (like arXiv IDs or CVEs) decode perfectly because their content is baked into the model's weights.
Why it matters: The 'web' that AI models 'know' is biased toward older, server-rendered content, effectively ignoring large swaths of the modern JavaScript-heavy internet.
Takeaway: If you want your documentation or site to be 'known' by AI models, ensure your core content is server-rendered in the static HTML rather than injected via JavaScript.
Deep dive
- Bare URLs act as keys into a model's latent weights rather than live pointers to external data.
- Training crawlers generally do not execute JavaScript, leading to an 'empty shell' phenomenon where modern sites are invisible to AI training.
- 0.45% of the web is effectively an 'empty mount' (blank shell) to crawlers; this rate is increasing for top-tier sites.
- Famous identifiers (e.g., arXiv/SHA/CVE) act as strong 'anchors' for specific knowledge because of their frequency in training data.
- Including a link to a known topic in a prompt can bias model output by 40+ percentage points even if the model isn't explicitly told to use it.
Decoder
- Common Crawl: An open repository of web crawl data used by most major AI labs to train large language models.
- Latent space: The high-dimensional conceptual space where an AI model stores the associations and relationships between data learned during training.
- Single-Page Application (SPA): A web app that loads a single HTML page and dynamically updates content via JavaScript rather than loading new pages from the server.
Original article
Full article content is not available for inline reading.
Explore all 67 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi