BoundBench

Vane (Perplexica)

Self-hosted AI answering engine (formerly Perplexica) that searches the web through a bundled SearxNG, reads pages and uploaded files, and writes cited answers with a choice of LLM providers.

github.com/ItzCrazyKns/Vane · 2026-10-05 · 348feca

Defense-in-depth score

3.2 / 10

Minimal

A search-and-answer agent with a small tool set: it can search, fetch web pages and read uploaded files, but it cannot write files, run commands or send messages. Around that core there are almost no safeguards: the web app has no authentication (the README lists it as upcoming), so anyone who can reach the port can use the agent, read every chat and change provider settings and keys. Web content enters the model unmarked while a URL-fetch tool can reach any address, and the whole app runs as root in one container with the provider keys.

Key gaps (2)

  1. A hijacked session can send data out through the always-on URL fetch with no gate, and with no user separation it can reach other users' chats. C5 · Untrusted input blast radius
  2. Model-influenced execution (headless browser, math evaluator) shares a root process with the provider keys and all chat data, with no dedicated isolation. C4 · Code-execution isolation

Criteria

C1 Identity & least privilege

Minimal 0.17 / 1.00

The agent acts with the deployment's own authority: the LLM provider keys entered at setup, and the network position of the server, which its URL-fetch tool can use to reach any host, including internal ones. There is no user identity at all, because the web app and its API have no authentication, so every request is treated the same and anyone who can reach the port can change providers and settings. The shipped image runs the app as root. A misused instance can spend the provider accounts and read from the network the server sits on, but it holds no cloud or source-control credentials.

C2 Approval gates

Minimal 0.10 / 1.00

There is no approval step anywhere, and nothing marks any tool as needing one. The agent's tools only search, fetch pages and read uploaded files, so it cannot write files, send messages or change data directly. Fetching a model-chosen URL does load that page in a full browser from the server's network position, which can have side effects on services that act on page loads, and nothing previews or confirms those fetches.

C3 Tool & action scoping

Minimal 0.30 / 1.00

The tools are narrow by design: web, academic and social search through SearxNG, a search over uploaded files, and a URL fetch. Each caps how many queries or URLs one call can use (three), but nothing else is checked: the declared argument schemas are sent to the model and not enforced, and the URL fetch accepts any address, scheme or host, with no allowlist and no block on internal or local addresses. Which tools are offered depends on the chosen sources and mode, but the URL fetch is always on.

C4 Code-execution isolation

Minimal 0.25 / 1.00

The agent never runs model-written programs, but two paths interpret text it does not control: the calculation widget evaluates a model-extracted expression with the mathjs expression parser inside the server process, and the URL fetch loads model-chosen pages, running their JavaScript, in a headless Chromium. Neither has dedicated isolation. The only separation is the recommended Docker container, a stock image that runs as root, gives the bundled search service's account passwordless sudo, and holds the provider keys and all chat data; non-Docker installs run everything directly on the host.

C5 Untrusted input blast radius

Minimal 0.25 / 1.00

Web search results, fetched pages and uploaded documents go into the model's context wrapped in plain result tags, and tool results go back in as ordinary tool messages; nothing marks them as untrusted or changes what the agent may do after reading them. The only limit on the URL-fetch tool is a sentence in its description. A hijacked session can therefore send data out unattended by fetching an attacker URL with data in it, and because the instance has no users or authentication, it can also reach other people's chats through the app's own local API. It cannot take destructive actions.

C6 Memory, context & configuration integrity

Minimal 0.25 / 1.00

The agent has no long-term memory tool and loads no instruction files from a workspace. What persists is chat history in SQLite, which the browser sends back as conversation history on follow-ups, and uploaded files with their embeddings. Neither is validated or tagged as untrusted, and both are shared by everyone using the instance: chats are listed globally and an uploaded file is found by its ID alone. Poisoned text in a chat persists for that conversation and can steer later tool use there, but chats are visible in the library and can be deleted.

C7 Third-party extensions

N/A · full credit 1.00 / 1.00

Vane loads no plugins, MCP servers or third-party tools at runtime: its model providers and tools are a fixed list in the source. The optional local embedding provider downloads ONNX model weights from Hugging Face, which are data rather than code. The Docker build clones SearxNG unpinned, but that is a build-time dependency rather than an extension the agent loads.

C8 Secrets & sensitive-data protection

Minimal 0.13 / 1.00

Provider API keys are typed into the setup screen and stored in plaintext in a JSON config file in the data volume. There is no redaction helper anywhere, and with no authentication on the instance, anyone who can reach it can open the provider settings. The settings form masks key fields on screen and API errors return a generic message, but nothing else protects keys on server paths. There is no telemetry. A leaked key is long-lived and stays valid until the operator rotates it at the provider.

C9 Audit & traceability

Minimal 0.25 / 1.00

Each answer is saved in SQLite with its response blocks, which include the research steps (search queries issued and pages read) and sources. That gives a partial, after-the-fact record, but the full blocks are only written when an answer completes, tool arguments and timings are not recorded as such, there is no actor or user attribution, and any client can delete a chat and its record through the open API. Everything else is console logging of errors.

C10 Limits & kill switch

Minimal 0.45 / 1.00

The research loop has a hard iteration cap of 2, 6 or 25 rounds depending on the mode the request chooses, SearxNG queries time out after 10 seconds and page loads after 20. LLM calls have no timeout, there is no token or cost cap, and nothing rate-limits requests to the unauthenticated API. Stopping does not stop much: when the client disconnects only the response stream closes, and the agent keeps searching, fetching and calling the model until it finishes.