markcrawl is an open-source Python CLI tool for crawling websites, extracting content, and generating embeddings for AI and Retrieval-Augmented Generation (RAG) workflows. It streamlines data ingestion and supports markdown export for downstream processing.
markcrawl is an Other AI product. It focuses on automating website crawling and content extraction for AI and RAG workflows. markcrawl is an open-source project aimed at AI developers and data engineers. The project is open source (MIT). It runs on the web and the command line.
AIMLPM builds and maintains markcrawl, and the product first shipped in 2026. The project is developed in the open on GitHub with 234 commits in the last 90 days. Across PulseGate's embedding index, markcrawl has few near neighbours, marking it as relatively distinct. Among its 5 catalogued features are web crawling, content extraction, and embedding generation. It exposes integrations via an MCP server.
Latest indexed changes and source events
Other apps tracked under the same category.
