Back to Devoured - November 19, 2026
DEVOURED
Context API extracts full context from any web page

Context API extracts full context from any web page

AI Perplexity
Perplexity released a Context API that extracts clean Markdown and metadata from up to 10 URLs simultaneously for agent-based workflows.
What: The API fetches content from web pages using Perplexity's index or live requests, filters boilerplate like ads and cookie banners, and returns relevant passages with source metadata for $1 per 1,000 URLs.
Why it matters: Developers are moving toward RAG (Retrieval-Augmented Generation) pipelines that demand higher precision and cleaner data than standard web scraping, shifting the burden of content processing from the model to the data retrieval layer.
Takeaway: Use the Context API to feed clean, formatted text into your LLM agents by providing a list of URLs to the `/v1/context` endpoint.
Deep dive
  • Provides clean Markdown, HTML, and metadata from target URLs
  • Supports batch processing for up to 10 URLs
  • Offers 'freshness' control via max_age_seconds
  • Includes semantic snippet extraction based on query relevance
  • Uses block-level scoring to filter out navigation and boilerplate
  • Priced at $1 per 1,000 successful URL fetches
Decoder
  • RAG (Retrieval-Augmented Generation): A technique where an AI model retrieves external, up-to-date data (like search results or documentation) to improve the quality of its response, rather than relying solely on training data.
  • Boilerplate: Standardized, repetitive HTML code found on websites—such as navigation menus, sidebars, and footers—that is often useless for an AI agent trying to parse page content.
Original article

When someone asks a chatbot a question, an AI agent may need to read several web pages to answer it. Standard search results often return only snippets. Fetching a page directly can return raw HTML, which is full of navigation, cookie banners, and ads. Agents need the page’s context in a format a model can use, while developers need a way to control how fresh that context must be.

Today we’re launching the Perplexity Context API, which reads up to 10 URLs and returns clean markdown, HTML, links, and query-based snippets for each page. The information comes from Perplexity's vast web index or a live fetch. Agents can specify the freshness of each request, and validate the source of each result.

Context API supports a variety of formats

Agents can initiate a request by providing a list of URLs and specifying the output formats. The API returns a result for each URL in the same order.

Context API supports a variety of response formats:

  • Markdown: The page content as clean Markdown, ready to place in a model’s context.
  • HTML: The page's HTML, for applications that run their own parsing.
  • Links: The links on the page as absolute URLs, without duplicates and in page order so an agent can decide where to go next.
  • Snippets: The passages of the page most relevant to a query provided, within the token limit set by the developer for each page.

Context API selects passages from each page that are relevant to the query, rather than generating a summary. As a result, the model receives the page’s original wording, but only the selected passages, so there is less text to process.

Every successful result also includes metadata such as the title, description, language, author, and published date, plus the page's Open Graph tags. To build the Markdown, the Context API splits a page into blocks and a model scores each block as main content or boilerplate. Only the content blocks are kept.

Essential for tasks that need the full picture

Finding a relevant page isn’t always enough to complete a job. Using Context API, agents can retrieve the content they need from selected URLs to complete complex tasks.

  • Implementing an API integration: A search result may identify the right documentation without including the required parameters, authentication instructions, or error cases. Context API can help a coding agent retrieve the documentation and API references to write an integration based on those details.
  • Comparing current prices: A listed price may depend on usage limits, billing terms, or eligibility conditions that are missing from a search excerpt. An agent can request a live fetch of selected pricing pages and read those conditions before making a comparison.
  • Answering policy questions: A return-policy excerpt may state the standard return window but omit exceptions for particular products or regions. A support agent can retrieve the policy page so its answer accounts for those exceptions.
  • Researching across multiple sources: A search excerpt may surface a finding without the methodology, qualifications, or surrounding argument. A research agent can retrieve the selected articles and reports, then compare the evidence rather than relying on excerpts alone.

Control freshness and handle URLs reliably

By default, the Context API first checks Perplexity’s web index for a copy of the page fetched within the last two days. If there isn't one, it fetches the page live. Developers can change max_age_seconds to 0 to always read the live pages, for prices, scores, and breaking news. For reference material that rarely changes, they can raise it. Set cache_only to true for workloads that should never trigger a live request.

Each result records whether the content came from a stored copy or a live fetch, and when it was fetched.

URLs in a request are fetched in parallel, each with its own time budget and its own status. One slow or broken page doesn’t fail the others. The status indicates what happened with each URL, so an agent can retry only the pages that timed out and keep the rest.

Context API returns usable, complete content

We tested the Context API against Firecrawl, Exa, Parallel, Keenable and Context.dev on thousands of URLs sampled from real agent fetch requests. The set covers 715 sites and includes government and academic sources and non-English pages. A separate judge model then scored the outputs without knowing which provider produced each one.

The chart measures performance on three criteria:

  • Usable: The share of URLs that came back with the page's real content, not an error page or an empty result.
  • Fact recall: The judge collects the key facts found on a page across all providers, then checks which of them each output contains. A failed fetch scores zero.
  • Content precision: The share of returned text that is page content, not cookie notices, navigation, or ads, averaged over all URLs. A failed fetch scores zero.

In this evaluation the Context API returned usable content for 92.3% of URLs, recovered 87.9% of the key facts, and scored 80.3% precision on content precision.

Pricing

Context API costs $1 per 1,000 URLs fetched. Developers pay for each URL that returns ok. URLs that fail aren’t billed.

Getting started

curl --request POST https://api.perplexity.ai/v1/context \  --header "Authorization: Bearer $PERPLEXITY_API_KEY" \  --header "Content-Type: application/json" \  --data '{  "urls": ["https://info.cern.ch/hypertext/WWW/TheProject.html"],  "formats": ["markdown"]  }'

Each result comes back with the page's Markdown, the passages that match the query, the page metadata, and a status. Check the status of each result before using its content.

Read the Context API docs or see the API reference.

Explore all 60 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi