comsift-new

Desktop Obsidian plugin for Canvas mind mapping, folder based investigations, Sherlock and Maigret, native Graph View relationships, local entity resolution, Ollama analysis, and embedded spreadsheets.

Install

Requires Obsidian 1.8.7 or newer. Install the release assets main.js, manifest.json, and styles.css in Vault/.obsidian/plugins/comsift/, then enable the plugin. The Python analysis worker is embedded in main.js; no extra plugin file is needed. Spreadsheet support is bundled; .sheet.md files inside a user folder are mirrored automatically to hidden CSVs (<user>/.comsift/sheets/) that analysis and chat use, so spreadsheets and CSV files can be mixed. Disable the separate Excel community plugin to avoid duplicate view registration.

From source:

npm ci
npm run build
npm run typecheck
npm test
npm run test:python

Users and migration

Leave Users folder blank to create people at the vault root, or set a vault-relative parent folder. Choose New user from the File Explorer's New menu (or run Comsift: New user from the command palette), enter a name, and the plugin creates:

On first load after updating from the old Comsift/Users default, that unchanged legacy default is converted to blank. Choose Comsift/Users again in settings if you explicitly want to keep it. Existing person folders are not moved automatically.

examplePerson/
  examplePerson.md
  examplePerson.canvas
  Sherlock/
  Maigret/
  Analysis/

The tool and analysis folders appear when used. The primary note has a managed <!-- comsift:title --> marker followed immediately by its H1. Editing that H1, or renaming the folder, primary note, or Canvas, synchronizes the other primary names. The comsift_id frontmatter property and plugin registry keep the investigation identity separate from its path. Arbitrary headings do not trigger renames. Canvas JSON owns node positions, dimensions, and arrows. Markdown prose is independent.

For an existing same named Markdown and Canvas pair, use the Comsift: Migrate command or file menu item. Migration shows the destination before moving files and copies both originals to Comsift/Backups/<name>-<timestamp>/ with the root setting, or <Users folder>/Backups/ when configured. It removes old whereami JSON sections only when their nodes and edges are confirmed in the Canvas snapshot. Unrecognized or divergent content stays in Markdown. Migration never runs on startup. If migration stops midway, use the snapshot to recover manually.

Canvas and Graph View

Existing Canvas mind map shortcuts remain available. Sherlock and Maigret run on the selected text node. Their result groups can independently spawn above, below, left, or right of the source node; settings prevent selecting the same direction. Results are written inside the active investigation folder, with unique filenames for repeated scans. Sherlock results become file nodes in Canvas.

The primary note links to its Canvas and Markdown result notes through a managed, folded callout. In Reading View the callout is hidden by plugin CSS. This uses ordinary [[wiki links]], so Obsidian can index the relationships for its built in Graph View. The plugin never serializes Canvas into Markdown. Arrows between resolvable Markdown file nodes also create managed links from the source note to the target note, including across user folders. Other node types do not create files. Handwritten links outside the managed block remain untouched. Deleting an arrow removes only its generated relationship. Graph View does not encode Canvas arrow direction, labels, or styling.

The link generation and cleanup logic has automated tests. Actual Graph View indexing, especially links to .canvas files and folded callouts, still needs validation in a running Obsidian vault. Obsidian's Graph View may omit Canvas files depending on its own indexing and filters.

Comsift does not include a vault or Canvas exporter. Use an Obsidian community extension for HTML export.

Local analysis

Open Comsift: Open analysis and chat or use the ribbon icon. Choose a user folder, put at least two CSV datasets anywhere inside it, and choose Splink, Jina, or Combined. Results are saved as JSON under Analysis/run-<timestamp>.json. The analysis panel displays original records, candidate pairs, Splink scores and evidence, and Jina rankings independently. Nothing is merged automatically.

CSV columns can differ. The worker recognizes common aliases for username, display name, bio, platform, profile URL, website, and location. Settings accept a JSON column map, for example:

{
  "*": { "username": "handle" },
  "Comsift/Users/examplePerson/tiktok.csv": {
    "display_name": "nickname",
    "profile_url": "profile_link"
  }
}

The analysis worker requires a user managed Python environment. The plugin does not install packages or download model weights. A suggested environment is:

python3 -m venv .venv
.venv/bin/pip install 'splink>=5,<6' pyarrow transformers torch accelerate

Set Python command in plugin settings to that environment's Python executable. Splink 5 uses its DuckDB backend for probabilistic linking and trains on the imported datasets; tiny datasets below 20 records return a clear error. Jina only mode remains available for small datasets with an exact field overlap. Configure Jina local model path to a downloaded jinaai/jina-reranker-v3.5 directory containing config.json. The worker runs Transformers in offline mode and offers CPU or automatic device placement. The model's trust_remote_code requirement means you should trust the local model files you chose. Jina relevance scores are ranking signals, not calibrated identity probabilities.

To download Jina into a folder outside the vault, install the Hugging Face CLI in that Python environment, then run:

.venv/bin/pip install huggingface_hub
.venv/bin/hf download jinaai/jina-reranker-v3.5 --local-dir ~/Documents/comsift-models/jina-reranker-v3.5

Enter that absolute directory as Jina local model path. Splink has no separate model to download. For Ollama, install Ollama separately and run ollama pull qwen3.5:9b or ollama pull gemma4:e4b; set the same tag as Ollama model in plugin settings.

Ollama reasoning and chat

Configure the Ollama API URL (default http://127.0.0.1:11434), installed model, context length, timeout, and CPU or automatic device use. Load models queries Ollama's /api/tags; no model name is hardcoded. Ask Ollama to reason sends one source record, its candidates, Splink evidence, Jina ranking, and an explicit NO_MATCH option to /api/chat with a JSON schema. The plugin validates the returned candidate ID and stores the structured decision. Optional Jina interpretation ranks the model's answer against every candidate and NO_MATCH; disagreements remain visible and never override a valid structured result.

Chat is a separate mode that uses only Ollama. It reads a limited set of text files from the selected user folder, answers questions, and may propose creating or replacing Markdown, CSV, JSON, Canvas, or text files. Generated Analysis/ data is excluded. The proposal shows the complete new content and requires Approve and apply. If the source file changed since the proposal, approval fails and asks for a fresh proposal. The model cannot edit files directly.

Review buttons save Confirmed, Rejected, Uncertain, or Review later to Analysis/reviews.json. Confirming creates profile notes for the two CSV records and a managed Markdown relationship for native Graph View. Reversing the review removes that generated relationship.

Imported CSV, biographies, and notes are treated as data, never instructions. The default Ollama URL is local. Changing it to a remote endpoint explicitly sends investigation context to that endpoint.

Checks and limitations

npm run build
npm run typecheck
npm test
npm run test:python

The automated suite covers link generation and cleanup, cross folder Canvas edges, placement, legacy block preservation, CSV normalization, conservative Jina candidate generation, Ollama schema validation, and a mocked local Ollama API. Full Splink and Jina inference require user installed dependencies and weights, so those paths need an integration run in the target environment. Canvas interaction and Graph View indexing need a running Obsidian vault; compilation does not establish their runtime behavior.

License

GPL 3.0 or later; see LICENSE, COPYING, and NOTICE. Embedded spreadsheet code derives from obsidian-excel (GPLv3). The Jina model has its own CC BY NC 4.0 license; it is not bundled.