BotBrowser

BotBrowser is a token-efficient web browser designed specifically for large language model (LLM) agents that need to process web content. It addresses the challenge of web pages containing excessive non-essential data by extracting and returning only the core content in clean markdown format, leading to significant reductions in token usage—typically saving 90–95% compared to raw HTML. This efficiency is particularly relevant for developers and AI workflows that rely on LLMs, where minimizing token consumption is crucial.

The tool operates by fetching a given URL, identifying the main content using Mozilla Readability for JavaScript or Trafilatura for Python, cleaning out scripts, styles, advertisements, navigation elements, footers, cookie banners, and tracking components, and then converting the result to markdown. The markdown output preserves important structural elements such as headings, lists, links, tables, and code blocks. BotBrowser also provides metadata, including the percentage of token savings achieved during extraction.

BotBrowser offers native SDKs for both JavaScript and Python, with real implementations rather than thin wrappers, allowing developers to use the tool within their preferred technology stack. Installation is straightforward via npm or pip, requiring no API keys, accounts, or server infrastructure, and the tool can function entirely offline. For advanced use cases and larger-scale deployments, BotBrowser provides a hosted MCP server with features such as JavaScript page rendering, batch processing, targeted extraction of specific page sections, web search and extraction, summarization to a token budget, and the ability to browse multiple pages simultaneously. Pro and Enterprise plans offer additional capabilities, including increased request limits, custom caching policies, webhook callbacks, audit logs, and dedicated support with a service-level agreement.

The platform is open source and MIT licensed, enabling users to self-host, fork, or embed the extraction engine without vendor lock-in. BotBrowser is built by Amplify and is available for free as a self-hosted solution, with paid plans for hosted services that scale to enterprise needs.