Your AI agent needs live web data. It doesn't need you writing glue code every time a URL changes or a new source appears. That's the gap the NeuroAPI MCP server fills — it exposes scraping, crawling, extraction, and search as tools your LLM can call directly through the Model Context Protocol. I set this up recently for a RAG pipeline that needed to pull structured data from product pages on demand. Took about 15 minutes end-to-end. Here's how it works, what to watch out for, and a few patterns I picked up along the way. What MCP Actually Does Here The Model Context Protocol is an open standard that lets LLMs interact with external tools through a structured interface. Instead of writing custom function-calling schemas for every API, you point your model at an MCP server and it discovers what's available. NeuroAPI's MCP server lives at https://neuroapi.me/api/mcp and exposes every endpoint — scrape, crawl, map, search, extract, screenshot, interact, and more — as MCP-compatible tools. Your agent sees them as callable functions. No API wrapper, no SDK boilerplate. Prerequisites A NeuroAPI account with an API key (sign up here) A client that supports MCP — Claude Desktop, Cursor, or any custom agent using an MCP client library Node.js 18+ if you're using the mcp-remote bridge (most setups need this) Step 1: Get Your API Key Head to the dashboard and generate a key. Free-tier keys work fine for getting started. Copy it somewhere safe — you'll need it in the next step. Step 2: Configure Your MCP Client Claude Desktop Open your Claude Desktop config. On macOS, that's at: ~/Library/Application Support/Claude/claude_desktop_config.json Add the NeuroAPI MCP server to the mcpServers block: { "mcpServers": { "neuroapi": { "command": "npx", "args": [ "mcp-remote", "https://neuroapi.me/api/mcp", "--header", "Authorization: Bearer ${NEUROAPI_KEY}" ], "env": { "NEUROAPI_KEY": "your-api-key-here" } } } } Restart Claude Desktop. You should see the NeuroAPI tools listed in the tool picker. Cursor In Cursor, open Settings → MCP Servers and add the same configuration. The format is identical — Cursor uses the same mcp-remote bridge under the hood. Custom Agent (TypeScript) If you're building your own agent, use the official MCP client SDK: import { Client } from "@modelcontextprotocol/sdk/client/index.js"; import { SSEClientTransport } from "@modelcontextprotocol/sdk/client/sse.js"; const transport = new SSEClientTransport( new URL("https://neuroapi.me/api/mcp"), { requestInit: { headers: { Authorization: "Bearer YOUR_API_KEY", }, }, } ); const client = new Client({ name: "my-agent", version: "1.0.0" }); await client.connect(transport); // List available tools const { tools } = await client.listTools(); console.log(tools.map((t) => t.name)); This prints something like: [ "scrape", "batch_scrape", "crawl", "map", "search", "extract", "screenshot", "interact", "summary", "question", "highlights", "parse", "branding" ] Step 3: Call a Tool Once connected, calling a tool is straightforward. Here's how your agent would scrape a page and get clean markdown back: const result = await client.callTool({ name: "scrape", arguments: { url: "https://news.ycombinator.com", formats: ["markdown"], }, }); console.log(result.content[0].text); The response contains the page content in the format you specified. If you need structured data instead of raw text, use the extract tool with a JSON schema — that's where things get interesting for agent workflows. A Real Example: Structured Extraction for RAG Suppose your agent needs to pull pricing tables from competitor sites and store them in a vector database. Instead of writing a custom scraper per site, you use extract with a schema: const result = await client.callTool({ name: "extract", arguments: { url: "https://example-saas.com/pricing", schema: { type: "object", properties: { plans: { type: "array", items: { type: "object", properties: { name: { type: "string" }, price: { type: "string" }, features: { type: "array", items: { type: "string" }, }, }, }, }, }, }, }, }); const data = JSON.parse(result.content[0].text); console.log(data.plans); The extract endpoint uses AI to parse the page and pull data matching your schema. It handles messy HTML, nested structures, and even some JavaScript-rendered content. The extract docs cover the full parameter set. Authentication: How the Key Flows Worth understanding the plumbing here. The MCP server authenticates via the Authorization header passed through mcp-remote. Your API key never leaves the transport layer — the LLM itself never sees it. It just knows it has a tool called scrape that works. If you're running the agent in production, store the key in environment variables or a secrets manager. Don't hardcode it in config files that end up in version control. Tips from Production Use A few things I learned after running this for a few weeks: Use search before scrape. If your agent is discovering URLs on its own, have it call search first to find relevant pages, then scrape or extract the ones that matter. Two-step workflows produce better results than asking the agent to guess URLs. Batch when you can. If you need 20 pages, batch_scrape is faster and more credit-efficient than 20 individual scrape calls. It returns a job ID you poll until it's done. Check your usage. The usage tool (mapped to /api/v1/usage) shows credit consumption in real time. Call it from your agent