Back to Devoured - June 03, 2026
DEVOURED
TinyFish Bigset turns text prompts into live datasets

TinyFish Bigset turns text prompts into live datasets

AI opensourceagentsdata TestingCatalog
TinyFish launched Bigset, an open-source, multi-agent system that converts text prompts into self-refreshing, structured datasets pulled from the live web using agents powered by Claude Sonnet 4.6 and Qwen3.7-max.
What: TinyFish, a Palo Alto-based company with $47 million in Series A funding, released Bigset, an AGPL-3.0 licensed open-source system that generates self-refreshing structured datasets from plain-language prompts. It deploys an orchestrator and sub-agents, powered by models like Claude Sonnet 4.6 and Qwen3.7-max, to research and verify data from the live web, then exports it as CSV or XLSX, with scheduled refreshes from 30 minutes to weekly.
Why it matters: This move by TinyFish, a well-funded company with enterprise clients, signals a trend towards making advanced multi-agent systems and web data extraction capabilities accessible via open-source tools, potentially disrupting traditional data scraping and market research by automating dataset creation and maintenance.
Takeaway: Developers interested in automated web data collection can self-host TinyFish Bigset via Docker, test it with their own prompts, or explore its 9 curated public datasets for AI companies and GPU prices.
Deep dive
  • TinyFish Bigset is an open-source (AGPL-3.0) multi-agent system that converts natural language prompts into structured, self-refreshing datasets from the live web.
  • It uses an orchestrator agent for discovery and sub-agents for data collection, with sub-agents operating under a tight budget of 6 tool calls to ensure efficiency.
  • Agents are instructed not to fabricate values, leave unconfirmed fields blank, and reject duplicate primary keys.
  • Schema inference is handled by Claude Sonnet 4.6, and agent roles by Qwen3.7-max by default, routed through OpenRouter and configurable.
  • The system is self-hostable via Docker and allows setting refresh cadences from 30 minutes to weekly.
  • TinyFish, backed by $47 million in Series A funding from ICONIQ, offers Bigset as an open-source counterpart to its enterprise agent products, built on the same web infrastructure (TinyFish Search and Fetch).
  • The team acknowledges it's experimental, with datasets taking 2-5 minutes to build and working best on topics with public web data.
  • A free tier allows 2,500 row operations per month, and 9 curated public datasets are available.
Decoder
  • AGPL-3.0: The GNU Affero General Public License version 3, a free software license that requires anyone who runs a modified program over a network to also make the source code available.
  • Orchestrator agent: A primary AI agent responsible for coordinating and directing the tasks of other, smaller sub-agents within a multi-agent system.
  • Sub-agents: Smaller, specialized AI agents that perform specific, focused tasks under the direction of an orchestrator agent.
  • Claude Sonnet 4.6: A specific model in Anthropic's Claude family of large language models, likely a mid-tier offering optimized for speed and cost.
  • Qwen3.7-max: A specific model from the Qwen series of large language models developed by Alibaba Cloud, indicating a high-performance version.
  • OpenRouter: A platform that provides a unified API for accessing various large language models from different providers, often allowing users to select and route requests to specific models.
Original article

TinyFish has launched Bigset, an open-source multi-agent system that turns a plain-language sentence into a structured dataset pulled from the live web. You describe what you want, and Bigset infers the schema, sends autonomous agents to research it on real web pages, verifies their findings against sources, deduplicates, and hands back a clean table you can export as CSV or XLSX. Set a refresh cadence from 30 minutes to weekly, and have the agents rerun on schedule so the dataset stays current without anyone needing to touch a script.

The work is split across two agent roles. An orchestrator agent does breadth-first discovery, identifying which rows belong in the dataset and where on the web to find them, then dispatches sub-agents to fill each one. The orchestrator holds no write access of its own. Each sub-agent researches a single entity under a tight budget of 6 tool calls, pulls real data via TinyFish Search and Fetch, and inserts one verified row with its source URLs and a record of how the data was found.

Sub-agents are instructed never to fabricate values, to leave fields blank when they cannot be confirmed, and to reject duplicate primary keys automatically. The orchestrator runs until the dataset reaches its row target, building faster as it learns where the data lives.

Bigset is licensed under AGPL-3.0 and runs self-hosted through Docker, with schema inference on Claude Sonnet 4.6 and the agent roles on Qwen3.7-max by default, all routed through OpenRouter and configurable per role. The team is candid that the project is experimental: a dataset takes 2 to 5 minutes to build, it works best on topics with public web data, and the free tier covers 2,500 row operations per month. It ships with 9 curated public datasets covering AI companies hiring, GPU prices, model pricing, and top open-source repositories, browsable without an account.

TinyFish is the Palo Alto-based company behind the platform, backed by $47 million in Series A funding led by ICONIQ, and counts Google, DoorDash, and Amazon among its enterprise clients, having processed more than 40 million agent operations. Bigset is built directly on TinyFish Search and Fetch, the same web infrastructure underneath the company's enterprise agent products, and arrives as the open-source answer to proprietary natural-language dataset tools, with no per-seat pricing, no domain restrictions, and full pipeline ownership for anyone who runs it themselves.

Star it on Github and grab an API key! 🔥

Explore all 53 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi