Local Embedder + OpenCode RAG Plugin

This repository provides a local embedding service (FastAPI + SentenceTransformer) and a Node.js plugin that wires OpenCode RAG tools to Qdrant using a YAML-first configuration.

Goals

• Run embeddings locally with a stable HTTP API.
• Keep RAG configuration explicit, validated, and YAML-first.
• Provide OpenCode tools for search, ingest, and reindexing.
• Preserve consistent data contracts for client integrations.

Architecture

• FastAPI embedder in app.py, loading BAAI/bge-small-en-v1.5 once at startup.
• RAG plugin in opencode-rag-local/ that calls the embedder and Qdrant.
• YAML config at opencode-rag-local/config/rag.yaml with strict validation.

Repository Layout

• app.py: FastAPI embedder service.
• requirements.txt: Python runtime dependencies.
• logging.yaml: Uvicorn logging configuration.
• opencode-rag-local/: OpenCode RAG plugin (Node.js).
• instructions/: Versioned operational notes and runbooks.

Requirements

• Python 3.9+ (type-hinting uses list[str]).
• Node.js 18+ for the RAG plugin.
• Qdrant instance reachable at the qdrant_url configured in rag.yaml.
• OpenCode configured to load the local RAG plugin.
• Apple Silicon or an environment that supports torch MPS if you keep device="mps" in app.py.

Quick Start

1. Start the embedder service.

python -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
uvicorn app:app --reload --log-config logging.yaml --port 8010

2. Start Qdrant (example Docker command).

docker run --rm -p 6333:6333 -p 6334:6334 qdrant/qdrant:latest

3. Set up the RAG plugin.

cd opencode-rag-local
npm run setup

4. Update the OpenCode config at ~/.config/opencode/opencode.json.

Embedder API

POST /embed expects:

{
  "texts": ["hello world", "another input"],
  "normalize": true
}

Response shape:

{
  "vectors": [[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]],
  "dim": 384,
  "model": "BAAI/bge-small-en-v1.5"
}

Notes:

• normalize defaults to true.
• Vectors are returned as float32 lists in the input order.
• dim is derived from the embedding width.

RAG Plugin Configuration

Edit opencode-rag-local/config/rag.yaml and keep it in sync with your runtime services. There are no environment variable fallbacks.

Key settings:

• embed_url: URL to the FastAPI embedder (default: http://localhost:8010/embed).
• qdrant_url: Qdrant endpoint (default: http://localhost:6333).
• collection: Qdrant collection name (default: opencode_memory).
• vector_size: Must match the embedder output (default: 384).
• distance: Qdrant distance metric (default: Cosine).

Operational sections:

• embed: embedding defaults (including normalize).
• search: top_k and truncate_chars.
• chunking: chunk_size and chunk_overlap for ingest.
• ingest: source/extension filters and size limits.
• retry: retry and backoff behavior.
• ids: namespace UUID and user ID policy.
• logging: plugin log level.

Run npm run setup after edits to validate the YAML.

OpenCode Configuration

OpenCode loads plugins from ~/.config/opencode/opencode.json. Ensure the local RAG plugin is listed. If you already have a plugins array, append the path.

Example:

{
  "plugins": [
    "opencode-rag-local"
  ]
}

If your OpenCode version uses a different key or structure, follow your existing schema and ensure the plugin path is registered.

RAG Tools (OpenCode)

• rag_search: search Qdrant with query, optional top_k, collection.
• rag_upsert: upsert documents (id, text, optional metadata).
• rag_ingest: ingest files from disk with chunking and filtering options.
• rag_reindex: drop and re-ingest a collection.

Testing

• Node tests: npm test in opencode-rag-local.
• Python tests: not currently present; use python -m compileall app.py for a quick syntax check.

Logging

• Uvicorn uses logging.yaml for the embedder.
• The RAG plugin logs through pino.

Development Notes

• The embedder model is loaded once at import time; avoid re-loading in request handlers.
• Batch size is set to 32 to keep throughput predictable.