# IWE — full documentation > A Markdown knowledge base you and your AI agent work on together, in the iWe Mac app or your editor. Your agent reads and edits the same files over CLI and MCP. This file contains every documentation page from https://iwe.md/ as plain markdown, section by section. The short index is at https://iwe.md/llms.txt. --- # Documentation URL: https://iwe.md/docs/ IWE is a Markdown knowledge base you and your AI agent work on together. It organizes your notes as a hierarchical knowledge graph and makes them accessible to AI agents. Write in Markdown, structure with links, give AI agents the tools to navigate your knowledge. ## Why IWE? - **Local-first** — Your data stays on your machine as a directory with Markdown files - **Hierarchical knowledge graph** — Same note in multiple contexts without duplication using [inclusion links](/docs/concepts/inclusion-links/) - **External memory for AI** — [CLI tools](/docs/agentic/) let AI agents retrieve and update your knowledge with full context - **Editor integration** — Search, navigate, refactor notes via LSP ([VS Code](/docs/editors/vscode/), [Neovim](/docs/editors/neovim/), [Zed](/docs/editors/zed/), [Helix](/docs/editors/helix/)) IWE has no built-in AI — it's designed to work with external AI tools like Claude, Codex, Gemini, and others. The CLI provides structured access to your knowledge graph, whether you're scripting your own workflows or giving AI agents the context they need. --- # Getting Started URL: https://iwe.md/docs/getting-started/ IWE works two ways: through your text editor for writing and organizing knowledge, and through CLI tools that give AI agents navigation capabilities. --- # How to Install URL: https://iwe.md/docs/getting-started/installation/ Installation instructions are below. Editor integration is covered in the [editor setup](/docs/editors/) section. ## Using Homebrew (macOS/Linux) The easiest way to install IWE on macOS or Linux is using Homebrew: ``` sh brew install iwe-org/iwe/iwe ``` This installs the CLI (`iwe`), the LSP server (`iwes`), and the MCP server (`iwec`). ## Using npm (macOS/Linux/Windows) Prebuilt binaries are published to npm, including Windows — no toolchain needed: ``` sh npm install -g @iwe-org/iwe ``` This installs the same three binaries (`iwe`, `iwes`, `iwec`). You can also run the CLI without installing: ``` sh npx -y @iwe-org/iwe --help ``` For AI agents, [@iwe-org/mcp](https://www.npmjs.com/package/@iwe-org/mcp) starts the MCP server directly via `npx -y @iwe-org/mcp` — see the [agentic docs](https://iwe.md/docs/agentic/). ## From Crates.IO - Rust and Cargo must be installed on your system. You can get them from [rustup.rs](https://rustup.rs). IWE is available at [crates.io](https://crates.io/crates/iwe). You can install IWE using cargo (and [iwes](https://crates.io/crates/iwes) for LSP server) ``` sh cargo install iwe cargo install iwes cargo install iwec ``` The binaries will be installed to `$HOME/.cargo/bin`. You may need to add it to your `$PATH`. ## From Source Clone the repository, navigate into the project directory, and build the project: ``` sh git clone git@github.com:iwe-org/iwe.git cd iwe cargo build --release ``` This will create executables located in the `target/release` directory. --- # How to Use with Your Text Editor URL: https://iwe.md/docs/getting-started/usage/ ## Purpose This page intends to teach you how to trigger IWE's features from inside your text editor. ## Background IWE's features are implemented as Language Server Protocol (**LSP**) capabilities. This makes IWE *editor-agnostic*; it's intended to work the same across all text editors that support the LSP standard. What this means for you is that to interact with IWE from inside your editor, you need to use *LSP requests*. These may be accessed differently across editors, and it's up to each editor to implement them properly. **If you've ever used something like "Find References" or "Go To Definition" before, then you're already familiar with LSP requests.** ## Primary Features IWE provides comprehensive features for markdown-based knowledge management: ### Core LSP Features - 🤖 **Text Transformation**: Generate, rewrite, or modify text using configurable CLI commands - 🔍 **Global Search**: Search through all notes using fuzzy matching on document paths and content - 🧭 **Link Navigation**: Follow links between documents with Go To Definition - 👁️ **Hover Preview**: Preview linked notes without navigating away - 📥 **Extract/Inline Notes**: Split sections into separate files or merge them back - 📝 **Auto-Format**: Normalize document structure, headers, lists, and link titles - 🔄 **Rename Refactoring**: Rename files while automatically updating all references - 🔗 **Backlinks Discovery**: Find all documents that reference the current document - 💡 **Inlay Hints**: Display parent document references and link usage counts - ✨ **Auto-Complete**: Smart completion for links as you type - 🗂️ **Document Symbols**: Navigate document outline via table of contents - 🔧 **Text Manipulation**: Transform lists to headers and vice-versa, change list types ## LSP Feature Reference Here's a reference connecting each LSP request with IWE features: | IWE Feature | LSP Request | Description | | -------------------- | ------------------- | ---------------------------------------------------- | | Extract/Inline Notes | Code Action | Split sections into files or merge them back | | Text Transformation | Code Action | Generate, rewrite, or modify text using CLI commands | | Text Transformation | Code Action | Convert lists to headers, change list types | | Link Navigation | Go To Definition | Follow markdown links to target documents | | Hover Preview | Hover | Preview linked notes without opening the file | | Backlinks | Go To References | Find all documents referencing current document | | Document Outline | Document Symbols | View table of contents for navigation | | Global Search | Workspace Symbols | Search through all notes with fuzzy matching | | Auto-Format | Document Formatting | Normalize structure, headers, and links | | File Renaming | Rename Symbol | Rename files and update all references | | Link Completion | Completion | Auto-complete links as you type | | Visual Hints | Inlay Hints | Show parent references and link counts | ## Usage Example > **Editor Compatibility**: Most editors have keybindings for LSP requests. Common patterns include: > > - VS Code: `Ctrl+Shift+P` (Command Palette) → search for LSP commands > - Neovim: `ca` (code actions), `gd` (go to definition), `gr` (references) > - Helix: `space+a` (code actions), `gd` (go to definition), `gr` (references) > - Zed: `Cmd+.` (code actions), `F12` (go to definition), `Shift+F12` (references) Suppose that you have the following in a Markdown file: ``` markdown # My First Note There's some content here. ## Another section With a list inside it: - list item - another item ``` ### Extracting a Section 1. Move your cursor to the `## Another section` line 2. Invoke the **Code Action** command (varies by editor) 3. Select "Extract section" from the options 4. Your file will now look like this: ``` markdown # My First Note There's some content here. [Another section](2sbdlvhe) ``` The `2sbdlvhe` refers to the name of a new file IWE generated for you. ### Following the Link 1. Move your cursor anywhere on the `[Another section](2sbdlvhe)` link 2. Use **Go To Definition** command 3. Your editor will open the new file containing: ``` markdown # Another section With a list inside it: - list item - another item ``` ### Finding Backlinks 1. In the extracted file, move your cursor to the `# Another section` line 2. Use the **Go To References** command 3. You'll see a list of all files that link to this document 4. Select the original file to navigate back ## Advanced Features ### Text Transform Commands IWE supports configurable text transformation commands that can: - Rewrite and improve text - Generate new content based on prompts - Expand on ideas and concepts - Add formatting and structure Commands work by piping content through external CLI tools. Configure them in your `.iwe/config.toml`: ``` toml [commands.claude] run = "claude -p" timeout_seconds = 120 [actions.rewrite] type = "transform" title = "Improve Text" command = "claude" input_template = "Improve this text: {{context}}" ``` You can use any CLI tool that reads from stdin and writes to stdout, including external tools like Claude CLI, custom scripts, or Unix commands. [Configuration](/docs/configuration/) ### Text Transformations Use **Code Actions** to transform document structure: - Convert bullet lists to numbered lists - Transform lists into header hierarchies - Convert headers back to lists - Change outline organization ### Auto-Formatting The **Document Formatting** command will: - Normalize header formatting and spacing - Standardize list formatting - Update link titles automatically - Fix markdown syntax issues - Ensure consistent document structure ### Global Search Use **Workspace Symbols** to: - Search across all documents - Find content by fuzzy matching - Navigate to specific sections - Explore document relationships Results show full paths like: ``` Journal, 2025 ⇒ Week 3 - Coffee week ⇒ Jan 26, 2025 - Cappuccino My Coffee Journey ⇒ Week 3 - Coffee week ⇒ Jan 26, 2025 - Cappuccino ``` ## Inlining Extracted Sections To reverse section extraction: 1. Move your cursor to a link like `[Another section](2sbdlvhe)` 2. Invoke **Code Action** 3. Select "Inline section" 4. The content returns to the original document: ``` markdown # My First Note There's some content here. ## Another section With a list inside it: - list item - another item ``` **Note**: Inlining automatically deletes the separate file after merging content back. ## Working with New Files When IWE creates new files (via extraction): - Files are initially created in memory/buffer - Save them using your editor's save command - In some editors, use "Save All" to ensure all new files are written to disk - Files use unique identifiers as filenames for reliable linking ## Best Practices 1. **Use Meaningful Headers**: Clear section titles improve navigation and search 2. **Link Liberally**: Create connections between related concepts 3. **Regular Formatting**: Use document formatting to maintain consistency 4. **Organize with Extraction**: Break large documents into focused, linked sections 5. **Leverage Search**: Use global search to discover connections and content 6. **Configure Commands**: Set up transform actions that match your writing workflow 7. **Use Inlay Hints**: Enable hints to understand document relationships at a glance --- # Features URL: https://iwe.md/docs/features/ IWE provides a rich set of features for managing your Markdown notes through LSP integration. Start with the [Feature Overview](/docs/features/overview/) — what IWE adds beyond a markdown language server. ## Navigation & Discovery - [Search](/docs/features/search/) - Find notes by content path - [Navigation](/docs/features/navigation/) - Go to definition, backlinks, document symbols - [Hover Preview](/docs/features/hover-preview/) - Preview linked notes without navigating ## Editing Assistance - [Autocomplete](/docs/features/autocomplete/) - Link suggestions as you type - [Inlay Hints](/docs/features/inlay-hints/) - Visual annotations showing note relationships - [Auto-format](/docs/features/auto-format/) - Automatic document formatting ## Content Management - [Renaming](/docs/features/renaming/) - Rename files and update all references - [Subdirectories](/docs/features/subdirectories/) - Organize notes in folders - [Text Manipulation](/docs/features/text-manipulation/) - Convert lists to sections and more - [Visualization](/docs/features/visualization/) - Graph visualization of your knowledge base ## AI Integration - [Text Commands](/docs/features/text-commands/) - Transform text with CLI tools and AI assistants --- # Feature Overview URL: https://iwe.md/docs/features/overview/ IWE turns a directory of markdown files into a knowledge graph — a connected structure you browse from your editor and your AI queries from the command line. This page is the catalogue: the capabilities that do not exist in a markdown language server at all, and where each one is documented. For how IWE stacks up against marksman, markdown-oxide, Obsidian, zk, and mdbase — including where IWE is weaker — see [How IWE Compares](/docs/concepts/comparison/). ## Graph Transformations - **[Extract](/docs/code-actions/extract/) and [inline](/docs/code-actions/inline/) notes**: LSP code actions that split a section into its own document or merge a referenced document back in, rewriting every affected link - **Section-to-list and list-to-section conversions**: restructure a document without retyping it - **Sub-section extraction**: break a long note into linked components, one action at a time - **Reference inlining**: turn a link into a quote, or embed the target section directly ## Query Language - **MongoDB-style YAML filters**: match documents by frontmatter with `$eq`, `$in`, `$gt`, `$exists`, `$all`, and the rest of the operator set - **Graph operators**: `$includes`, `$includedBy`, `$references`, `$referencedBy` walk inclusion and reference edges at bounded or unbounded depth - **Bulk mutations**: `iwe update` and `iwe delete` apply the same filters across the whole workspace with `$set` / `$unset` - **Projection, sort, and limit**: shape results as JSON, YAML, or Markdown for piping into other tools See [Query Language](/docs/concepts/query-language/) and the [formal specification](/docs/concepts/spec/). ## MCP Server for AI Agents - **14 native tools**: agents call `iwe_find`, `iwe_retrieve`, `iwe_create`, `iwe_extract`, `iwe_normalize`, and more over the Model Context Protocol - **Built-in prompts**: `explore`, `review`, and `refactor` guide agents through common workflows - **Resources**: documents, tree, stats, and config exposed as MCP resources - **File watching**: the in-memory graph stays in sync with editor edits — no restart needed - **Dry-run support**: every write and refactoring tool can preview its changes before applying See [MCP Server](/docs/agentic/mcp/). ## Search and Ranking - **Titles and path context, not filenames**: a note called `20260903-a7f.md` is found by its heading and its position in the graph - **Two rankers, fused**: skim fuzzy matching over title and key, BM25 lexical relevance over title and body, combined with Reciprocal Rank Fusion - **One ranking everywhere**: the CLI, the LSP workspace symbols, and the MCP tools read the same index - **Incremental**: the index is maintained per edit, so long editor sessions stay current See [Notes Search](/docs/features/search/) and [Search Ranking](/docs/architecture/search-ranking/). ## Schema Inference - **Automatic frontmatter analysis**: `iwe schema` reports field names, type distributions, coverage, and value breakdowns across the workspace - **Filter-aware**: inspect the schema of any subset of documents using the query language - **JSON/YAML output**: pipe schema data into automation, dashboards, or validation pipelines See [IWE Schema](/docs/cli/schema/) and [Document Schema](/docs/concepts/document-schema/). ## External Command Integration - **Configurable command pipeline**: connect any CLI tool through a custom template - **Block-level transformations**: apply a command to one section with the surrounding structure as context - **Template-based input**: tune command behavior per content type See [Custom Text Commands](/docs/features/text-commands/). ## Markdown Normalization - **Batch operations**: normalize thousands of files in under a second - **Auto-formatting on save**: link titles, header levels, list numbering, and wrapping fixed in place - **Header hierarchy management**: keep document structure consistent as it grows - **Link title synchronization**: keep link text in step with the target document's title See [Text Structure Normalization](/docs/features/auto-format/) and [Header Levels Normalization](/docs/features/normalization/). ## Hierarchical Notes - **[Inclusion links](/docs/concepts/inclusion-links/)**: the same note can belong to several parents without copying the file - **Context-aware search**: results carry the note's position in the graph, not just its name - **[Inlay hints](/docs/features/inlay-hints/)**: see parent context without leaving the document you are reading - **[Hover preview](/docs/features/hover-preview/)**: inspect a linked note inline, frontmatter stripped - **Flexible organization**: flat Zettelkasten and deep directory trees both work - **Path-based navigation**: reach the same content through different conceptual paths ## Cross-Editor LSP Integration - **Native LSP**: documented setup for [VS Code](vscode.md), [Neovim](neovim.md), [Zed](zed.md), and [Helix](helix.md); any LSP-compatible editor can be wired up - **Consistent behavior**: the same features and the same performance in every client - **No lock-in**: switch editors without losing anything ## Developer-Focused Architecture - **Rust core**: [20,000 files in under a second](/docs/architecture/benchmark/) - **Shared core library**: the CLI, the LSP server, and the MCP server are thin shells over one domain model, so a new graph operation appears in all three at once - **Rich graph processing**: the transformations above are graph traversals, not text munging - **Cross-platform**: identical behavior on every supported operating system The shared library is the reason the rest of this page is possible. See [Data Model](/docs/architecture/data-model/) for how documents become a graph. --- # Notes Search URL: https://iwe.md/docs/features/search/ Notes search is a key feature in IWE. IWE allows you to organize documents hierarchy just by adding **[Inclusion Links](/docs/concepts/inclusion-links/)**. Then you can search for the documents taking into account the hierarchy. Search can be used via the LSP `Workspace Symbols` command. For every note, IWE will generate full paths. And allow you to do a fuzzy matching to filter the search results. So you can find both entries just by typing `cappu`. The workspace-symbol ranking also blends in BM25 full-text relevance, so a term that appears in a note's body helps surface it even when the header does not match — see [Search Ranking](/docs/architecture/search-ranking/) for the details. ``` Journal, 2025 ⇒ Week 3 - Coffee week ⇒ Jan 26, 2025 - Cappuccino My Coffee Journey ⇒ Week 3 - Coffee week ⇒ Jan 26, 2025 - Cappuccino ``` Since `Week 3` is included in two notes it shown in both contexts. Note that you don't have to deal with the file names at all, as everything is based on the headers from your notes! ## Custom Document Titles By default, IWE uses the first header of each document as its title in search results. You can configure IWE to use a YAML frontmatter field instead by setting `frontmatter_document_title` in your configuration. See the [Configuration](/docs/configuration/#frontmatter-document-title) documentation for details. ## Structured search via the CLI The CLI [`iwe find`](/docs/agentic/find/) command pairs BM25 full-text ranking over document title and body with a YAML-based filter language. Use `--filter` for frontmatter predicates (`status: draft`, `priority: { $gt: 3 }`), and the structural anchor flags (`--includes`, `--included-by`, `--references`, `--referenced-by`) to scope the search to a subtree or set of references. A text query then ranks the scoped documents by relevance. See the [Query Language](/docs/concepts/query-language/) reference for the filter syntax and [Search Ranking](/docs/architecture/search-ranking/) for how relevance is scored. [`iwe count`](/docs/agentic/count/) returns the integer count for the same filter shape. --- # Navigation URL: https://iwe.md/docs/features/navigation/ IWE provides several ways to navigate your documents using standard LSP features. This allows you to move through your knowledge graph efficiently using familiar editor commands. ## Links Navigation (Go To Definition) Jump directly to linked documents using the LSP "Go To Definition" command. ### Usage 1. Place your cursor on a markdown link like `[Topic](topic-file)` 2. Trigger Go To Definition: - **VS Code**: `F12` or `Ctrl+Click` / `Cmd+Click` - **Neovim**: `gd` or `:lua vim.lsp.buf.definition()` - **Helix**: `gd` 3. The linked document opens automatically ### Example ``` markdown # My Notes Check out [Project Ideas](project-ideas) for brainstorming. ``` With your cursor on "Project Ideas", triggering Go To Definition opens `project-ideas.md`. ## Table of Contents (Document Symbols) View the structure of your current document using the LSP "Document Symbols" command. ### Usage 1. Open any markdown file 2. Trigger Document Symbols: - **VS Code**: `Ctrl+Shift+O` / `Cmd+Shift+O` - **Neovim**: `:lua vim.lsp.buf.document_symbol()` or use Telescope - **Helix**: `space` + `s` 3. Navigate through headers and sections This provides an outline view of your document, showing all headers in a hierarchical tree. You can quickly jump to any section. ## Backlinks (Find References) Find all documents that link to the current document using the LSP "Find References" command. ### Usage 1. Open a document you want to find references to 2. Trigger Find References: - **VS Code**: `Shift+F12` or right-click and select "Find All References" - **Neovim**: `:lua vim.lsp.buf.references()` or `gr` - **Helix**: `gr` 3. View all documents containing links to this file ### Example If you're viewing `project-ideas.md` and trigger Find References, you'll see a list of all documents that contain links like `[Project Ideas](project-ideas)`. This is essential for understanding how your notes are connected and for discovering relationships in your knowledge graph. ## Navigation Tips - **Preview before jumping**: Use [Hover Preview](/docs/features/hover-preview/) to see linked content without leaving your current document - **Use search for discovery**: Combine navigation with [Notes Search](/docs/features/search/) to find documents by content path - **Track your trail**: Some editors maintain a navigation history, allowing you to go back to previous locations --- # Hover Preview URL: https://iwe.md/docs/features/hover-preview/ IWE supports the standard LSP `textDocument/hover` request to preview the contents of a linked note without navigating away from the current document. ## Supported links - Wiki links: `[[note]]` - Markdown links: `[title](note)` External links (e.g. `https://...`, `mailto:...`) are ignored. ## Preview content - Returns the target note as Markdown (`MarkupContent` with kind `markdown`). - Strips frontmatter at the top of the note (delimited by `---` and terminated by `---` or `...`). - Returns the rest of the document without truncation so your editor can decide how to render/clip it. ## Editor usage - Helix: place the cursor on a link, then `space` + `k` - Neovim: place the cursor on a link, then `K` (or run `:lua vim.lsp.buf.hover()`) - VS Code: hover the link (or run “Show Hover”) --- # Auto-Complete URL: https://iwe.md/docs/features/autocomplete/ IWE can suggest links as you type using the standard LSP code completion feature. ## Trigger Characters IWE asks the editor to open the completion popup as soon as the user types one of the configured trigger characters. The default is `[`, so typing `[` (or `[[`) immediately offers the list of documents. ``` toml [completion] trigger_characters = ["["] ``` The value is a list, so you can add any number of characters: ``` toml [completion] trigger_characters = ["[", "+", "@"] ``` Most editors also open the popup automatically as you type word characters (letters and digits), independently of this list — the list is only needed for non-word symbols. To restore the pre-0.1.7 behavior, set the list to `["+"]`. ### Suppressing word-character completions To make completions appear *only* after `[` / `[[` and never on word characters, configure the editor: - **Neovim** — with the built-in client, use `vim.lsp.completion.enable(..., { autotrigger = true })` (Neovim 0.11+) — its autotrigger only fires on the server's trigger characters. With `nvim-cmp`, set the LSP source's `keyword_length = 999`. With `blink.cmp`, set the LSP provider's `min_keyword_length = 999`. Trigger characters still pass through in all three cases. - **VS Code** — turn off word-character auto-suggestions for markdown while leaving trigger characters on: ``` json "[markdown]": { "editor.quickSuggestions": { "other": "off", "comments": "off", "strings": "off" }, "editor.suggestOnTriggerCharacters": true } ``` Typing `[` still opens the popup; typing letters no longer does. `Ctrl+Space` always invokes completions manually. ### Bracket prefixes When the cursor is preceded by `[` or `[[`, the inserted completion replaces those brackets and the link shape is chosen by the typed prefix: - `[` → markdown link `[title](key)` - `[[` → wiki link `[[key]]` This happens regardless of `link_format` — the typed brackets always win. When no bracket precedes the cursor, `link_format` decides the shape (see below). If your editor auto-pairs `[` to `[]` (or `[[` to `[[]]`), the completion still works cleanly: the inserted range extends through any `]` / `]]` immediately after the cursor, so the result is a single well-formed link with no stray closing bracket. ## Link Format By default, completions insert Markdown-style links `[title](key)`. You can configure IWE to use WikiLinks `[[key]]` instead: ``` toml [completion] link_format = "wiki" ``` Available options: - `"markdown"` (default): Creates `[title](key)` style links - `"wiki"`: Creates `[[key]]` style WikiLinks This setting only applies when the cursor is not preceded by `[` or `[[`; an explicit bracket prefix always overrides it. ## Minimum prefix length By default, IWE shows completions as soon as the editor requests them (no minimum length). Raise the threshold to suppress the popup until the user has typed a few characters: ``` toml [completion] min_prefix_length = 3 ``` The length is measured against the search query after any leading `[` or `[[` is stripped — `[ab` is treated as 2 characters, not 3. ## Document Titles By default, IWE uses the first header of a document as its title in completion suggestions. You can configure IWE to use a YAML frontmatter field instead: ``` toml [workspace] frontmatter_document_title = "title" ``` With this configuration, documents with frontmatter like: ``` markdown --- title: Custom Document Title --- # Header ``` Will appear in completions as "Custom Document Title" and insert `[Custom Document Title](key)` when selected. If the frontmatter field is missing, IWE falls back to using the first header. --- # Inlay Hints URL: https://iwe.md/docs/features/inlay-hints/ Inlay hints are visual annotations that appear inline within your document, providing contextual information without requiring navigation. IWE provides two types of inlay hints. ## Note Header Hints At the top of each document, IWE displays information about the note's position in your knowledge graph. ### What's Shown - **Parent document title**: Shows which document links to this note - **Links counter**: Number of incoming references (backlinks) ### Example When viewing `project-ideas.md`: ``` [linked from: Index] [3 links] # Project Ideas Your content here... ``` This tells you that "Index" links to this document and there are 3 total backlinks. ## Inclusion Hints When viewing an [Inclusion Links](/docs/concepts/inclusion-links/), inlay hints show which notes directly reference the linked content. ### What's Shown - **Parent notes list**: The direct parent notes (one level up) that link to the referenced content ### Example ``` markdown # Daily Notes [Meeting Notes](meeting-2024) [Index, Journal] ``` The hint `[Index, Journal]` shows that the linked note is referenced from both "Index" and "Journal" documents. ## Enabling Inlay Hints ### VS Code Inlay hints should work automatically. If not visible, check: 1. Open Settings (`Ctrl+,` / `Cmd+,`) 2. Search for "inlay hints" 3. Enable "Editor: Inlay Hints" ### Neovim Enable inlay hints in your LSP configuration: ``` lua vim.lsp.inlay_hint.enable(true) ``` Or toggle with a keybinding: ``` lua vim.keymap.set('n', 'ih', function() vim.lsp.inlay_hint.enable(not vim.lsp.inlay_hint.is_enabled()) end) ``` ### Helix Inlay hints are enabled by default. Configure in `config.toml`: ``` toml [editor.lsp] display-inlay-hints = true ``` ## Benefits - **Context at a glance**: Understand note relationships without navigating away - **Track connections**: See how many documents link to the current note - **Navigate hierarchy**: Understand where content fits in your knowledge structure --- # Text Structure Normalization / Formatting URL: https://iwe.md/docs/features/auto-format/ IWE provides auto-formatting through the LSP formatting feature. This typically triggers when you save your document (if your editor is configured for format-on-save) or manually via a formatting command. ## What Gets Normalized ### 1. Link Title Updates Link titles are automatically updated to match the header of the linked document. **Before:** ``` markdown See [old title](my-note) for details. ``` **After** (if `my-note.md` has header "# My Updated Note"): ``` markdown See [My Updated Note](my-note) for details. ``` ### 2. Header Level Adjustment Header levels are adjusted to maintain a proper tree structure. This ensures that nested sections have correct relative header levels. **Before:** ``` markdown # Main Document #### Incorrectly Deep Section Some content. ``` **After:** ``` markdown # Main Document ## Incorrectly Deep Section Some content. ``` ### 3. Ordered List Numbering Ordered lists are renumbered sequentially, regardless of the original numbers. **Before:** ``` markdown 1. First item 5. Second item 3. Third item ``` **After:** ``` markdown 1. First item 2. Second item 3. Third item ``` ### 4. List Indentation and Spacing List structure is normalized with consistent indentation and proper newlines. **Before:** ``` markdown - Item one - Badly indented - Inconsistent -No space after marker ``` **After:** ``` markdown - Item one - Badly indented - Inconsistent - No space after marker ``` ### 5. Whitespace Cleanup Excess blank lines and trailing whitespace are cleaned up. ## Usage ### Format on Save Most editors can be configured to format on save: - **VS Code**: Enable "Format On Save" in settings - **Neovim**: Configure autocommand or use plugin like `conform.nvim` - **Helix**: Set `auto-format = true` in `languages.toml` ### Manual Formatting Trigger formatting manually: - **VS Code**: `Shift+Alt+F` / `Shift+Option+F` - **Neovim**: `:lua vim.lsp.buf.format()` - **Helix**: `:format` ## Configuration Link title updates, header level adjustment, list renumbering, and whitespace cleanup are always part of formatting and cannot be toggled individually. The markdown syntax used in the formatted output is configurable in `.iwe/config.toml`. ### Custom Formatting Tokens You can customize the markdown syntax used during formatting. For example, to use underscores for emphasis and plus signs for lists: ``` toml [markdown.formatting] emphasis_token = "_" strong_token = "__" list_token = "+" ``` See [Configuration](/docs/configuration/) for all available formatting options. ## Tips - **Enable format-on-save** for consistent formatting across your workspace - **Use with version control** to easily review formatting changes - If you notice unexpected formatting, check [Header Levels Normalization](/docs/features/normalization/) for detailed header adjustment rules --- # Header Levels Normalization URL: https://iwe.md/docs/features/normalization/ Header level adjustment is one of the normalizations applied by [document formatting](/docs/features/auto-format/); this page describes its rules in detail. IWE reads and understands nested structures based on headers. It identifies sub-header relationships. Markdown allows header structures where the nesting isn't clear, like: ``` markdown ## First Header # Second Header ``` IWE automatically fixes the header levels to ensure they're nested correctly. So the example above corrects to: ``` markdown # First Header # Second Header ``` ## Removing unnecessary levels IWE can normalize the headers structure by dropping unnecessary header levels, for example: ``` markdown # First header ### Second header ``` Will be normalized by dropping unnecessary levels and will look like: ``` markdown # First header ## Second header ``` --- # Files Renaming URL: https://iwe.md/docs/features/renaming/ IWE provides file renaming through the LSP `rename` refactoring feature. When you rename a note file, IWE automatically updates all references throughout your entire workspace. ## How It Works When you trigger a rename operation on a markdown file: 1. **Rename the file** - The file is renamed to your specified name 2. **Update all references** - Every link pointing to the old filename is updated to use the new name 3. **Preserve link titles** - Link display text remains unchanged This ensures your knowledge graph stays consistent without manual search-and-replace operations. ## Usage ### In Your Editor 1. Open a markdown file you want to rename 2. Trigger the LSP rename command: - **VS Code**: `F2` or right-click and select "Rename Symbol" - **Neovim**: `:lua vim.lsp.buf.rename()` or your configured keybinding - **Helix**: `space` + `r` 3. Enter the new filename 4. Confirm the rename ### Example **Before renaming:** `old-topic.md`: ``` markdown # Old Topic Some content here. ``` `index.md`: ``` markdown # Index See [Old Topic](old-topic) for details. Check also [Old Topic](old-topic) in another context. ``` **After renaming to `new-topic.md`:** `new-topic.md`: ``` markdown # Old Topic Some content here. ``` `index.md`: ``` markdown # Index See [Old Topic](new-topic) for details. Check also [Old Topic](new-topic) in another context. ``` Note that the link text ("Old Topic") is preserved while the link target is updated. ## Command Line Usage You can also rename documents using the CLI: ``` bash # Basic rename iwe rename old-topic new-topic # Preview changes first iwe rename old-topic new-topic --dry-run # Get affected document keys iwe rename old-topic new-topic -f keys ``` See [IWE Rename](/docs/agentic/rename/) for full documentation. ## Benefits - **Safe refactoring** - No broken links after renaming - **Bulk updates** - All references updated in a single operation - **Undo support** - Most editors support undoing the rename operation --- # Sub-Directories URL: https://iwe.md/docs/features/subdirectories/ IWE supports organizing your markdown files in subdirectories while maintaining full functionality across all features. This allows you to structure your knowledge base hierarchically without losing the ability to link, search, and process files across directory boundaries. ## How It Works ### Recursive Directory Scanning IWE recursively scans the configured workspace path and all its subdirectories: - **Includes**: All `.md` files in any subdirectory level - **Excludes**: Hidden files and directories (starting with `.`) - **File Keys**: Include the relative path from workspace root ### File Path Resolution Files in subdirectories get keys that include their relative path: ``` Project Structure: your-project/ ├── .iwe/config.toml ├── docs/ │ ├── guide.md → Key: "docs/guide" │ ├── api/ │ │ └── reference.md → Key: "docs/api/reference" │ └── examples/ │ └── basic.md → Key: "docs/examples/basic" └── README.md → Key: "README" (if workspace.path = "") ``` ### Cross-Directory Linking Links use relative paths based on each file's location in the directory structure: ``` markdown See the [guide](docs/guide.md) for details. Back to [index](../index.md) or see [API reference](api/reference.md). Check out the [basic example](../examples/basic.md) or [guide](../guide.md). ``` **Path Resolution Rules:** - From root to subdirectory: `subdirectory/file.md` - From subdirectory to root: `../file.md` - Between subdirectories at same level: `../other-directory/file.md` - Within same directory: `file.md` --- # Text Manipulation URL: https://iwe.md/docs/features/text-manipulation/ IWE offers actions for context-aware transformations on your notes. These actions are available through your editor's code actions menu (usually `Ctrl+.` or `Cmd+.`). ## List to Sections Convert a list into a series of headers/sections. Each list item becomes a section header with its nested content preserved. ### Example **Before:** ``` markdown # Topics - Project A - Details about project A - More info - Project B - Details about project B ``` **After:** ``` markdown # Topics ## Project A Details about project A More info ## Project B Details about project B ``` ### When to Use - Converting brainstorm lists into structured documents - Expanding outline notes into full sections - Promoting list items to top-level content ## Sections to List Convert a series of headers back into a list. This is the reverse of "List to Sections". ### Example **Before:** ``` markdown # Topics ## Project A Details about project A ## Project B Details about project B ``` **After:** ``` markdown # Topics - Project A - Details about project A - Project B - Details about project B ``` ### When to Use - Condensing detailed sections into an overview - Creating summary lists from expanded content - Reorganizing document structure ## Change List Type Toggle between bullet lists and ordered (numbered) lists. ### Example **Before (bullet list):** ``` markdown - First item - Second item - Third item ``` **After (ordered list):** ``` markdown 1. First item 2. Second item 3. Third item ``` ### When to Use - Converting unordered lists to numbered steps - Changing numbered lists back to bullet points - Adjusting list style based on content type ## Sort List Items Sort list items alphabetically or by other criteria. ### Example **Before:** ``` markdown - Zebra - Apple - Mango - Banana ``` **After:** ``` markdown - Apple - Banana - Mango - Zebra ``` ### When to Use - Organizing alphabetical lists (glossaries, indexes) - Sorting task lists - Maintaining consistent ordering ## Usage 1. Place your cursor on the list or section you want to transform 2. Open the code actions menu: - **VS Code**: `Ctrl+.` / `Cmd+.` - **Neovim**: `:lua vim.lsp.buf.code_action()` - **Helix**: `space` + `a` 3. Select the desired transformation from the menu --- # Graph Visualization URL: https://iwe.md/docs/features/visualization/ IWE provides powerful graph visualization capabilities through DOT format export, allowing you to create visual representations of your knowledge graph structure. This helps you understand the relationships between documents, sections, and references in your markdown collection. ## Export Command The `iwe export -f dot` command generates graph data in DOT format, which can be processed by Graphviz and other visualization tools. ### Basic Usage ``` bash # Export all root documents iwe export -f dot # Export specific document by key iwe export -f dot --key project-notes # Export with depth limit iwe export -f dot --depth 3 ``` ### Advanced Visualization with Headers Use the `--include-headers` flag to create detailed visualizations that show document structure with sections grouped in colored subgraphs: ``` bash # Include sections and subgraphs iwe export -f dot --include-headers # Detailed view of specific document iwe export -f dot --key documentation --include-headers # Combined with depth limit iwe export -f dot --key meetings --depth 2 --include-headers ``` ## Visualization Modes ### Basic Mode (Default) Shows document-to-document relationships with clean node styling: ``` dot digraph G { rankdir=LR fontname=Verdana 1[label="Project Notes",fillcolor="#ffeaea",fontsize=16,shape=note,style=filled] 2[label="Meeting Notes",fillcolor="#f6e5ee",fontsize=16,shape=note,style=filled] 1 -> 2 [color="#38546c66",arrowhead=normal,penwidth=1.2] } ``` Rendered for this documentation directory (regenerate with `make` in `docs/`): ![](docs-basic.svg) ### Detailed Mode (--include-headers) Shows document structure with sections grouped in colored subgraphs: ``` dot digraph G { rankdir=LR 1[label="Project Notes",shape=note,style=filled] 2[label="Introduction",shape=plain] 3[label="Requirements",shape=plain] subgraph cluster_0 { labeljust="l" style=filled color="#fff9de" fillcolor="#fff9de" 2 3 } 2 -> 1 [arrowhead="empty",style="dashed"] 3 -> 1 [arrowhead="empty",style="dashed"] } ``` Rendered for this documentation directory (regenerate with `make` in `docs/`): ![](docs-detailed.svg) ## Key Features - **Color Coding**: Each document key gets a unique, consistent color scheme - **Shape Differentiation**: Documents use `note` shape, sections use `plain` shape - **Subgraph Clustering**: Sections are grouped in colored clusters with document keys - **Edge Styles**: Different styles for document vs section relationships - **Automatic Layout**: Left-to-right layout optimized for readability ## Integration with Graphviz ### Generate PNG Images ``` bash # Basic visualization iwe export -f dot | dot -Tpng -o knowledge-graph.png # Detailed with sections iwe export -f dot --include-headers | dot -Tpng -o detailed-graph.png # Focus on specific topic iwe export -f dot --key project --include-headers | dot -Tpng -o project-structure.png ``` ### Generate SVG for Web ``` bash # Scalable vector graphics iwe export -f dot | dot -Tsvg -o interactive-graph.svg # With better layout for complex graphs iwe export -f dot --include-headers | neato -Tsvg -o network-view.svg ``` ### Different Layout Engines ``` bash # Hierarchical layout (default) iwe export -f dot | dot -Tpng -o hierarchical.png # Force-directed layout iwe export -f dot | neato -Tpng -o network.png # Circular layout iwe export -f dot | circo -Tpng -o circular.png # Spring-based layout iwe export -f dot | fdp -Tpng -o spring.png ``` ## Filtering and Focusing ### By Document Key ``` bash # Show only documents related to 'meetings' iwe export -f dot --key meetings --include-headers # Multiple levels of related documents iwe export -f dot --key architecture --depth 2 ``` ### By Content Depth ``` bash # Show only immediate relationships iwe export -f dot --depth 1 # Show deeper connections iwe export -f dot --depth 3 --include-headers ``` ## Workflow Examples ### Daily Documentation Review ``` bash #!/bin/bash # Generate today's knowledge graph iwe export -f dot --include-headers > today.dot dot -Tpng today.dot -o daily-review.png open daily-review.png # macOS ``` ### Project Structure Analysis ``` bash #!/bin/bash # Analyze specific project structure iwe export -f dot --key $PROJECT_NAME --include-headers | \ dot -Tsvg -o "project-${PROJECT_NAME}.svg" ``` ### Knowledge Base Overview ``` bash #!/bin/bash # Create multiple views of your knowledge base iwe export -f dot > overview.dot iwe export -f dot --include-headers > detailed.dot # Generate both views dot -Tpng overview.dot -o overview.png dot -Tpng detailed.dot -o detailed.png ``` ## Customization Tips ### Layout Optimization For large graphs, experiment with different Graphviz engines: - **`dot`**: Best for hierarchical structures - **`neato`**: Good for network-like relationships - **`fdp`**: Spring model, useful for clustered data - **`circo`**: Circular layout for cyclic structures ### Output Formats Graphviz supports many output formats: - **PNG/JPG**: For presentations and documents - **SVG**: For interactive web displays - **PDF**: For high-quality prints - **DOT**: For further processing or debugging ### Performance Considerations - Use `--depth` limits for large knowledge bases - Filter by `--key` to focus on specific areas - Use `--include-headers` for detailed structure visualization when needed ## Troubleshooting ### Large Graphs ``` bash # Reduce complexity with depth limits iwe export -f dot --depth 2 | dot -Tpng -o simplified.png # Use different layout engine iwe export -f dot | fdp -Tpng -o alternative-layout.png ``` ### Missing Graphviz Install Graphviz on your system: ``` bash # macOS brew install graphviz # Ubuntu/Debian sudo apt install graphviz # Windows winget install graphviz ``` ### Complex Layouts For complex graphs, try different approaches: ``` bash # Increase node separation iwe export -f dot | dot -Tpng -Gnodesep=1.0 -o spaced.png # Adjust DPI for clarity iwe export -f dot | dot -Tpng -Gdpi=200 -o high-res.png ``` The visualization feature makes IWE's knowledge management capabilities tangible, helping you understand and navigate your documentation structure at a glance. --- # Custom Text Commands URL: https://iwe.md/docs/features/text-commands/ IWE includes **text transformation** capabilities that can be accessed right from your text editor. You can effortlessly **rewrite** text, **expand** on ideas, **highlight** important words, or apply any custom transformation using CLI tools. Transform actions work by piping content through external commands. You can use custom scripts, standard Unix tools, or AI assistants like `claude -p` (Claude CLI in pipe mode) - any command-line tool that reads from stdin and writes to stdout. ## How It Works 1. Select or place cursor on a text block in your editor 2. Open the code actions menu (`Ctrl+.` or `Cmd+.`) 3. Select a transform action (e.g., "Rewrite", "Expand") 4. The text is processed through your configured command 5. The result replaces the original content ## Configuration Transform actions require two parts: 1. **Command definition** - the CLI tool to run 2. **Action definition** - how to use the command ``` toml [commands.claude] run = "claude -p" timeout_seconds = 120 [actions.rewrite] type = "transform" title = "Rewrite" command = "claude" input_template = """ Rewrite this text to improve clarity and readability. Keep links intact. {{context}} """ ``` Check [Configuration](/docs/configuration/) section for detailed instructions on how to set up custom commands. ## Example Commands **Using Claude CLI:** ``` toml [commands.claude] run = "claude -p" timeout_seconds = 120 ``` **Using a Python script:** ``` toml [commands.summarize] run = "python ~/scripts/summarize.py" timeout_seconds = 30 ``` **Using standard Unix tools:** ``` toml [commands.uppercase] run = "tr '[:lower:]' '[:upper:]'" timeout_seconds = 5 ``` --- # Code Actions URL: https://iwe.md/docs/code-actions/ IWE provides powerful code actions for restructuring your notes. These are available through your editor's code actions menu (`Cmd+.` in VS Code, `space+a` in Helix/Neovim). ## Available Actions - [Extract](/docs/code-actions/extract/) - Break sections into separate notes - [Inline](/docs/code-actions/inline/) - Merge extracted notes back into the parent - [Attach](/docs/code-actions/attach/) - Link content to other documents - [Link](/docs/code-actions/link/) - Create links from selected text --- # Extract Actions URL: https://iwe.md/docs/code-actions/extract/ Extract actions enable the creation of new documents from markdown sections (headers). IWE provides two types of extract actions: 1. **Extract** - Extracts a single section into a new file 2. **Extract All** - Extracts all direct subsections of a section into separate files Both operations: - Create new files containing the selected content - Add [Inclusion Links](/docs/concepts/inclusion-links/) (like `[Section Title](new-file)`) to the newly created files - Automatically adjust header levels to maintain proper document structure - Support relative path preservation The reverse operation, known as **inline**, allows you to: 1. Embed the content back into the document via the inclusion link 2. Remove the link and the extracted file ## Extract Single Section ### Basic Configuration The extract action is configured in `.iwe/config.toml` under the `[actions]` section: ``` toml [actions] extract = { type = "extract", title = "Extract section", key_template = "{{id}}", link_type = "markdown" } ``` ### Configuration Options - **`type`**: Must be `"extract"` for single section extraction - **`title`**: Display name shown in editor's code actions menu - **`key_template`**: Template for generating the new file's key/name (see Template Variables below) - **`link_type`**: Optional link format - `"markdown"` for `[text](key)` or `"wiki"` for `[[key]]` ### Example: Basic Section Extraction **Source document** (`document.md`): ``` markdown # Main Document ## Important Section This content will be extracted. ### Subsection More content here. ``` **After extraction** (`document.md`): ``` markdown # Main Document [Important Section](extracted-123) ``` **Extracted file** (`extracted-123.md`): ``` markdown # Important Section This content will be extracted. ## Subsection More content here. ``` ## Extract All Subsections ### Configuration ``` toml [actions] extract_all = { type = "extract_all", title = "Extract all subsections", key_template = "{{title}}", link_type = "markdown" } ``` ### Configuration Options - **`type`**: Must be `"extract_all"` for extracting all subsections - **`title`**: Display name shown in editor's code actions menu - **`key_template`**: Template for generating new file keys/names - **`link_type`**: Optional link format - `"markdown"` for `[text](key)` or `"wiki"` for `[[key]]` ### Example: Extract All Subsections **Source document** (`guide.md`): ``` markdown # User Guide Introduction content here. ## Installation How to install the software. ## Configuration How to configure settings. ## Usage How to use the application. ``` **After extract all** (`guide.md`): ``` markdown # User Guide Introduction content here. [Installation](Installation) [Configuration](Configuration) [Usage](Usage) ``` **Extracted files**: `Installation.md`: ``` markdown # Installation How to install the software. ``` `Configuration.md`: ``` markdown # Configuration How to configure settings. ``` `Usage.md`: ``` markdown # Usage How to use the application. ``` ## Template Variables The `key_template` supports several template variables for flexible file naming: ### Basic Variables - **`{{id}}`**: Random unique identifier (e.g., `123`, `456`) - **`{{today}}`**: Current date formatted using `date_format` from `[workspace]` section (default: `%Y-%m-%d`) - **`{{title}}`**: The section title being extracted (automatically sanitized for filenames) - **`{{slug}}`**: URL-friendly version of the title (lowercase, alphanumeric characters only, non-alphanumeric replaced with dashes, no consecutive dashes) ### Parent Section Variables - **`{{parent.title}}`**: Title of the parent section containing the extracted section - **`{{parent.slug}}`**: URL-friendly version of the parent section title (lowercase, alphanumeric characters only, non-alphanumeric replaced with dashes, no consecutive dashes) - **`{{parent.key}}`**: Key of the parent document ### Source Document Variables - **`{{source.key}}`**: Full key of the source document - **`{{source.file}}`**: Filename portion of the source document key - **`{{source.title}}`**: Title (first header) of the source document - **`{{source.slug}}`**: URL-friendly version of the source document title (lowercase, alphanumeric characters only, non-alphanumeric replaced with dashes, no consecutive dashes) - **`{{source.path}}`**: Directory path of the source document ## Relative Path Behavior IWE preserves directory structure when extracting sections. The key generation follows these rules: 1. **Generated keys are relative to the source document's directory** 2. **`Key::combine(&key.parent(), &relative_key)` creates the final path** 3. **The parent directory is automatically preserved** ### Example: Relative Path Extraction **Source document** (`docs/tutorial/basics.md`): ``` markdown # Basic Tutorial ## Getting Started Content to extract. ``` **Configuration**: ``` toml extract = { type = "extract", title = "Extract", key_template = "extracted-{{title}}" } ``` **Result**: - Source: `docs/tutorial/basics.md` - Extracted file: `docs/tutorial/extracted-Getting Started.md` - Link: `[Getting Started](extracted-Getting Started)` The extracted file is created in the same directory (`docs/tutorial/`) as the source document. ## Advanced Configuration Examples ### 1. Simple Numeric Keys ``` toml extract = { type = "extract", title = "Extract", key_template = "{{id}}" } ``` Creates files like: `123.md`, `456.md` ### 2. Date-based Extraction ``` toml extract = { type = "extract", title = "Extract to today", key_template = "{{today}}" } ``` Creates files like: `2024-01-15.md` ### 3. Title-based Extraction ``` toml extract = { type = "extract", title = "Extract section", key_template = "{{title}}" } ``` Creates files like: `Getting Started.md`, `Configuration.md` ### 4. Slug-based Extraction (URL-friendly) ``` toml extract = { type = "extract", title = "Extract as slug", key_template = "{{slug}}" } ``` Creates files like: `getting-started.md`, `configuration.md` From titles like "Getting Started", "User's Guide/Setup", "API*Reference" → `getting-started.md`, `users-guide-setup.md`, `api-reference.md` ### 5. Hierarchical Extraction ``` toml extract = { type = "extract", title = "Extract with context", key_template = "{{parent.title}}-{{title}}" } ``` From a document with structure: ``` markdown # User Guide ## Installation ``` Creates: `User Guide-Installation.md` ### 6. Hierarchical Extraction (URL-friendly) ``` toml extract = { type = "extract", title = "Extract with slug context", key_template = "{{parent.slug}}-{{slug}}" } ``` From a document with structure: ``` markdown # User Guide & Setup ## Installation/Configuration ``` Creates: `user-guide-setup-installation-configuration.md` ### 7. Source-aware Extraction ``` toml extract = { type = "extract", title = "Extract from source", key_template = "{{source.file}}-{{title}}" } ``` From `user-guide.md`: ``` markdown ## Installation ``` Creates: `user-guide-Installation.md` ### 8. Source-aware Extraction (URL-friendly) ``` toml extract = { type = "extract", title = "Extract with source slug", key_template = "{{source.slug}}-{{slug}}" } ``` From `User Guide & Manual.md`: ``` markdown # User Guide & Manual ## Installation/Setup ``` Creates: `user-guide-manual-installation-setup.md` ### 9. Path-based Organization ``` toml extract = { type = "extract", title = "Extract to subfolder", key_template = "extracted/{{title}}" } ``` From `docs/guide.md`, creates: `docs/extracted/Installation.md` ### 10. Wiki-style Links ``` toml extract = { type = "extract", title = "Extract (wiki)", key_template = "{{id}}", link_type = "wiki" } ``` Creates links like: `[[123]]` instead of `[Section Title](123)` ### 11. Complex Template Example ``` toml extract = { type = "extract", title = "Extract with full context", key_template = "{{source.path}}/{{today}}-{{parent.title}}-{{title}}" } ``` From `docs/tutorials/advanced.md` on 2024-01-15: ``` markdown # Advanced Tutorial ## Complex Features ``` Creates: `docs/tutorials/2024-01-15-Advanced Tutorial-Complex Features.md` ## Key Collision Handling When the generated key already exists, IWE automatically appends numeric suffixes: 1. First attempt: `extracted-section.md` 2. If exists: `extracted-section-1.md` 3. If exists: `extracted-section-2.md` 4. And so on... ### Example: Handling Collisions **Multiple sections with same title**: ``` markdown # Document ## Section Content 1 ## Section Content 2 ## Section Content 3 ``` **With `key_template = "{{title}}"`, extract all creates**: - `Section.md` (first section) - `Section-1.md` (second section) - `Section-2.md` (third section) ## Date Formatting The `{{today}}` variable uses the `date_format` setting from the `[workspace]` section: ``` toml [workspace] date_format = "%Y-%m-%d" # Results in: 2024-01-15 [actions] extract = { type = "extract", title = "Daily extract", key_template = "{{today}}-{{title}}" } ``` Common date formats: - `%Y-%m-%d`: `2024-01-15` - `%Y%m%d`: `20240115` - `%b %d, %Y`: `Jan 15, 2024` - `%A, %B %d, %Y`: `Monday, January 15, 2024` ## Filename Sanitization Special characters in titles are automatically sanitized using the `sanitize_filename` crate: - `Section/With*Special:Chars` → `SectionWithSpecialChars` - `"Quoted Section"` → `Quoted Section` - `Section | With | Pipes` → `Section With Pipes` ## Command Line Usage You can also extract sections using the CLI: ``` bash # List all sections with block numbers iwe extract my-document --list # Extract section by title iwe extract my-document --section "Architecture" # Extract section by block number iwe extract my-document --block 2 # Preview changes iwe extract my-document --section "Design" --dry-run ``` See [IWE Extract](/docs/agentic/extract/) for full documentation. --- # Inline Notes URL: https://iwe.md/docs/code-actions/inline/ The `inline` action is a powerful feature for embedding content from one document directly into another. It replaces a link to a document with the actual content of that document. This is useful for consolidating notes, embedding sources, or restructuring your knowledge base. There are two primary ways to inline content, configured via the `inline_type` property: 1. **`section`**: Inlines the entire content of the linked document into the current document's section that contains the link. 2. **`quote`**: Replaces the link with the content of the linked document, formatted as a markdown blockquote. ## Configuration You can define custom `inline` actions in your `config.toml` file within the `.iwe` directory. ### Example Configuration: Here is an example of how to configure two different inline actions: one for inlining as a section and another for inlining as a quote. ``` toml [actions] # Inlines the content of the linked document and deletes the original file. inline_section = { type = "inline", title = "Inline Section", inline_type = "section", keep_target = false } # Inlines the content as a blockquote and keeps the original file. inline_quote = { type = "inline", title = "Inline as Quote", inline_type = "quote", keep_target = true } ``` ### Configuration Keys: - `type`: Must be set to `"inline"`. - `title`: The text that will appear in the code action menu in your editor (e.g., "Inline Section"). - `inline_type`: Determines how the content is embedded. - `"section"`: Embeds the full content. - `"quote"`: Wraps the content in a blockquote. - `keep_target` (optional, defaults to `false`): - If `false`, the original source file of the link will be deleted after its content is inlined. The action will also clean up any other references to the deleted file across your workspace. - If `true`, the original source file is left untouched. ## Command Line Usage You can also inline references using the CLI: ``` bash # List all [inclusion links](/docs/concepts/inclusion-links/) with numbers iwe inline my-document --list # Inline by reference key iwe inline my-document --reference "architecture" # Inline by block number iwe inline my-document --block 1 # Inline as blockquote iwe inline my-document --reference "notes" --as-quote # Keep the target document iwe inline my-document --block 2 --keep-target # Preview changes iwe inline my-document --reference "design" --dry-run ``` See [IWE Inline](/docs/agentic/inline/) for full documentation. --- # Linking and Note Templates URL: https://iwe.md/docs/code-actions/attach/ The "attach" code action allows you to link content under your cursor to another document. This feature is perfect for creating daily notes, collecting ideas in an inbox, or organizing thoughts into topic-specific documents. Note: Enable [Inlay Hints](/docs/features/inlay-hints/) to see the action results immediately. (the list of backlinks will be updated automatically) ## How It Works When you place your cursor on an [Inclusion Links](/docs/concepts/inclusion-links/) and trigger a code action, IWE can: 1. **Create or open** a target file based on your configuration 2. **Append** inclusion link under cursor to the target file The target file is determined by templates you configure, allowing for dynamic file creation based on dates or static collection files. ## Common Use Cases ### Daily Note Collection Automatically link interesting thoughts, tasks, or notes to today's daily note: ``` toml [actions.daily] type = "attach" title = "Add to Daily Note" key_template = "daily/{{today}}" document_template = "# Daily Note - {{today}}\n\n{{content}}\n\n" ``` **Workflow:** 1. While reading or writing, place cursor on any content 2. Trigger code action and select "Add to Daily Note" 3. Content gets appended to `daily/2024-01-15.md` (or created if doesn't exist) 4. Continue your work knowing the important bits are captured ### Inbox System Create an inbox for collecting random thoughts and ideas: ``` toml [actions.inbox] type = "attach" title = "Send to Inbox" key_template = "inbox" document_template = "# Inbox\n\n{{content}}\n\n" ``` **Workflow:** 1. Come across something interesting while working 2. Place cursor on the content and select "Send to Inbox" 3. All collected items accumulate in a single `inbox.md` file 4. Review and organize your inbox regularly ### Topic Collections Organize content by themes or projects: ``` toml [actions.research] type = "attach" title = "Add to Research Notes" key_template = "research/general" document_template = "# Research Notes\n\n{{content}}\n\n" [actions.meeting_notes] type = "attach" title = "Add to Meeting Notes" key_template = "meetings/{{today}}" document_template = "# Meeting Notes - {{today}}\n\n{{content}}\n\n" ``` ## File Creation Behavior - **New files**: If the target file doesn't exist, it's created with the `document_template` content - **Existing files**: Content is appended to the end of existing files - **Duplicate prevention**: If the exact same inclusion link already exists in the target file, the code action will not be suggested ## Template Variables Attach actions support two template variables: - `{{today}}`: Current date formatted using your configured date format - In `key_template`: Uses `workspace.date_format` (default: `"%Y-%m-%d"`) - In `document_template`: Uses `markdown.date_format` (default: `"%b %d, %Y"`) - `{{content}}`: The actual content being attached ## Editor Integration The attach code action appears in your editor's code action menu (usually triggered by Ctrl+. or Cmd+.) when: 1. Your cursor is positioned on an inclusion link 2. You have attach actions configured in your `.iwe/config.toml` ## Best Practices ### 1. Start Simple Begin with a basic daily note or inbox system: ``` toml [actions.capture] type = "attach" title = "Quick Capture" key_template = "capture" document_template = "# Capture\n\n{{content}}\n" ``` ### 2. Use Descriptive Titles Make it clear what each action does: ``` toml title = "Add to Today's Notes" # Good title = "Attach" # Less clear ``` ### 3. Consider File Organization Use path separators in key templates to organize files: ``` toml key_template = "daily/{{today}}" # Creates files in daily/ folder key_template = "projects/{{today}}" # Creates files in projects/ folder ``` ## Integration with PKM Workflows The attach action works well with common Personal Knowledge Management approaches: - **GTD (Getting Things Done)**: Use inbox actions for quick capture - **Zettelkasten**: Create topic-specific collections and daily notes - **PARA Method**: Organize attachments into Projects, Areas, Resources, Archives folders - **Daily Notes**: Build a consistent journaling practice with date-based collection --- # Creating Links from Text URL: https://iwe.md/docs/code-actions/link/ The "link" code action allows you to quickly convert text under your cursor or selected text into a link to a new note. This feature streamlines the process of creating new notes while maintaining connections in your knowledge graph. ## How It Works When you place your cursor on a word or select text and trigger a code action, IWE can: 1. **Create a new note** based on the text at your cursor or selection 2. **Replace the text** with a link to the newly created note 3. **Use templates** to control where the new note is created and how it's named The link action works on a single line - you can either place your cursor on a word (and IWE will detect the word boundaries) or select a specific range of text. ## Configuration Configure link actions in your `.iwe/config.toml`: ``` toml [actions.link] type = "link" title = "Link word" key_template = "{{id}}" # Template for the new note's key link_type = "Markdown" # Optional: "Markdown" or "WikiLink" ``` ### Configuration Options - **title**: The name displayed in your editor's code action menu - **key_template**: Template for generating the new note's key (supports template variables) - **link_type**: Optional link format - `"Markdown"`: Creates `[text](key)` style links (default) - `"WikiLink"`: Creates `[[key]]` style links ## Common Use Cases ### Simple ID-Based Links Create notes with auto-generated numeric IDs: ``` toml [actions.link] type = "link" title = "Link word" key_template = "{{id}}" ``` **Workflow:** 1. Place cursor on "important" or select "important concept" 2. Trigger code action and select "Link word" 3. Text becomes `[important](2)` and a new note `2.md` is created with `# important` 4. Or if you selected "important concept", it becomes `[important concept](2)` with `# important concept` ### Slug-Based Links Use slugified versions of the text as the note key: ``` toml [actions.link] type = "link" title = "Link word" key_template = "{{slug}}" ``` **Example:** - Cursor on "Important Concept" → `[Important Concept](important-concept)` → `important-concept.md` - Selected "My Idea" → `[My Idea](my-idea)` → `my-idea.md` ### Title-Based Links Use the exact text (sanitized for filenames) as the key: ``` toml [actions.link] type = "link" title = "Link word" key_template = "{{title}}" ``` **Example:** - Cursor on "MyWord" → `[MyWord](MyWord)` → `MyWord.md` - Selected "Project Ideas" → `[Project Ideas](Project Ideas)` → `Project Ideas.md` ### WikiLink Format Use WikiLink style instead of Markdown: ``` toml [actions.wiki_link] type = "link" title = "Link word (wiki)" key_template = "{{id}}" link_type = "WikiLink" ``` **Example:** - Cursor on "concept" → `[[2]]` → `2.md` - Selected "important idea" → `[[2]]` → `2.md` with `# important idea` ## Text Selection vs Cursor The link action supports two modes: ### Cursor Mode (Word Detection) - Place your cursor anywhere within a word - IWE automatically detects word boundaries - Words can include alphanumeric characters, underscores, hyphens, and Unicode characters - Examples: - Cursor on "wo|rd" → detects "word" - Cursor on "multi-w|ord" → detects "multi-word" - Cursor on "some_|function" → detects "some_function" ### Selection Mode - Select any text range on a single line - IWE uses exactly what you selected - Perfect for phrases with spaces or partial words - Examples: - Select "important concept" → creates link for "important concept" - Select "very important" → creates link for "very important" - Select part of a sentence → creates link for that specific text ## Template Variables Link actions support several template variables: - **{{id}}**: Auto-generated unique numeric ID - **{{slug}}**: Slugified version of the text (lowercase, hyphens instead of spaces) - **{{title}}**: Sanitized version of the text (safe for filenames) - **{{today}}**: Current date formatted using your configured date format from `workspace.date_format` (default: `"%Y-%m-%d"`) ## Key Collision Handling If a note with the generated key already exists, IWE automatically appends a numeric suffix: **Example:** - First link: `concept` → `concept.md` - Second link: `concept` → `concept-1.md` - Third link: `concept` → `concept-2.md` ### Use Multiple Link Actions Configure different link actions for different purposes: ``` toml [actions.link] type = "link" title = "Quick Link" key_template = "{{id}}" [actions.concept_link] type = "link" title = "Link Concept" key_template = "concepts/{{slug}}" [actions.daily_link] type = "link" title = "Link to Today" key_template = "daily/{{today}}" ``` --- # Delete Action URL: https://iwe.md/docs/code-actions/delete/ The delete action allows you to cleanly remove a referenced section and automatically update all files that reference it. ## How It Works When you place your cursor on an [Inclusion Links](/docs/concepts/inclusion-links/) (like `[Important Topic](file)`) and trigger the delete action, IWE will: 1. **Delete the target file** - The referenced section/file is completely removed 2. **Clean up inclusion links** - All inclusion links to the deleted section are removed from other files 3. **Convert inline links** - Inline links to the deleted section are converted to plain text, preserving readability ## Usage 1. Position your cursor on any inclusion link in your markdown file 2. Open the code actions menu (typically `Ctrl+.` or `Cmd+.`) 3. Select "Delete" from the refactor actions ## Example **Before deletion:** ``` markdown # My Notes Some text with an inline link to [Important Topic](file). [Important Topic](file) ``` **After deleting the reference on line with `[Important Topic](5)`:** ``` markdown # My Notes Some text with an inline link to Important Topic. ``` The referenced file `Important Topic` is completely deleted, the inclusion link is removed, and the inline link becomes plain text. ## When Delete Action Is Available - The delete action only appears when your cursor is on an **inclusion link** - It will not appear on regular text, headers, or other content types - The action ensures safe deletion by updating all referencing files automatically ## Command Line Usage You can also delete documents using the CLI: ``` bash # Delete a single document iwe delete my-document # Preview changes first iwe delete my-document --dry-run # Get affected document keys iwe delete my-document -f keys # Bulk delete by filter iwe delete --filter 'status: archived' ``` `iwe delete` does not prompt before writing — pass `--dry-run` first if you want to inspect the matched set. See [IWE Delete](/docs/agentic/delete/) and the [Query Language](/docs/concepts/query-language/) reference for filter syntax. --- # Agentic Tools URL: https://iwe.md/docs/agentic/ IWE provides CLI commands that turn your personal knowledge graph into AI memory. These tools give AI agents the same navigation capabilities you have — discovering entry points, following relationships, and retrieving knowledge with appropriate context. ## The Context Bridge When AI agents work with traditional files, they face three problems: 1. **Fragmented context** — Knowledge scattered across files with no structure 2. **Retrieval ineffectiveness** — Search finds text but misses relationships 3. **No memory iteration** — Each conversation starts from scratch IWE solves this by giving AI agents real navigation capability. Your knowledge graph isn't just text to search through — it's a structure AI can traverse, following the same relationships you created. This is the **context bridge**: your personal knowledge graph that both you and AI can navigate and build upon. ## How AI Navigates Your Knowledge AI agents use these commands to collaborate with your knowledge: **Navigation commands:** - **`iwe find`** — Discover entry points with fuzzy search - **`iwe retrieve`** — Get content with graph expansion and context - **`iwe tree`** — Display document hierarchy as navigable structure **Analysis commands:** - **`iwe stats`** — Generate knowledge base statistics **Refactoring commands:** - **`iwe extract`** — Extract sections to new documents - **`iwe inline`** — Inline referenced content - **`iwe delete`** — Delete documents and clean up references - **`iwe rename`** — Rename documents and update references Together, these commands let AI: - Start from any topic and discover related knowledge - Follow graph relationships to build understanding - Retrieve exactly the context needed for a task - Navigate your knowledge the way you do - Refactor and reorganize knowledge programmatically ## Available commands ## Typical agent workflow ```bash iwe find "authentication" --roots -f keys ``` ```bash iwe retrieve -k authentication -d 2 -c 1 ``` ```bash iwe retrieve -k login-flow -e authentication ``` 1. **Find entry points** using `iwe find` with fuzzy search 2. **Retrieve content** with appropriate depth and context 3. **Follow relationships** by retrieving linked documents 4. **Exclude known content** to avoid duplication ## Output formats Both commands support multiple output formats: | Format | Flag | Use case | |--------|------|----------| | Markdown | `-f markdown` | Human-readable output with full content | | Keys | `-f keys` | Piping to other commands | | JSON | `-f json` | Programmatic processing and parsing | ## Integration examples ### Claude Code / MCP integration ```bash iwe find --roots -f json | jq '.results[].key' iwe retrieve -k "$KEY" -d 1 -c 1 ``` ### Building context for LLMs ```bash iwe find "$QUERY" -l 5 -f keys | while read key; do iwe retrieve -k "$key" -d 0 -c 0 --dry-run done iwe retrieve -k "$SELECTED_KEY" -d 2 -c 1 ``` ## Why Graph Structure Matters Traditional file search treats your knowledge as a bag of text. IWE's graph structure enables real navigation: - **Relationships are explicit** — Links between documents create navigable paths - **Context is preserved** — Parent and child documents provide understanding - **Structure is shared** — AI follows the same paths you created - **Memory accumulates** — Knowledge builds on itself over time Learn more about the underlying concepts in [The Context Bridge](/docs/concepts/context-bridge/). --- # Open Knowledge Format (OKF 0.2) URL: https://iwe.md/docs/agentic/okf/ [OKF](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) is an open format for agent-maintained knowledge, published by Google Cloud: a directory of markdown files with YAML frontmatter, where provenance (`sources`, `generated`), trust (`verified`), and lifecycle (`status`, `stale_after`) are first-class frontmatter families. No SDK, no registry — if you can `cat` a file, you can read OKF; if you can `git clone` a repo, you can ship it. That is also a precise description of an IWE workspace. IWE has no OKF mode because it doesn't need one: markdown, frontmatter, and links are its native data model, so OKF bundles are readable, queryable, and refactorable with the standard tools. ## Scaffold a bundle ``` bash iwe init --okf ``` This initializes a workspace with links written in OKF's resolvable form (`refs_extension = ".md"`), a bundle-root `index.md` carrying `okf_version: "0.2"`, and three conformance schemas wired into validation: | Schema | Checks | |---|---| | `okf.yaml` | every concept document carries a non-empty `type`; the optional OKF families (`status`, `stale_after`, `generated`, `verified`, `sources`) are well-formed when present | | `okf-index.yaml` | reserved `index.md` files carry no frontmatter (bar the bundle root's `okf_version`) and keep the SPEC §8 shape — sections of link bullets | | `okf-log.yaml` | reserved `log.md` files keep the SPEC §9 shape — date-grouped history under ISO 8601 headings | ## Make an existing workspace conformant OKF conformance (SPEC §11) is three rules: every non-reserved `.md` file has parseable frontmatter, every frontmatter block has a non-empty `type`, and `index.md` / `log.md` follow their prescribed shapes. To adopt it in a workspace you already have: 1. Set `refs_extension = ".md"` in `.iwe/config.toml` and run `iwe normalize` so every internal link resolves for consumers outside IWE. 2. Add `type` to any document missing it. 3. Copy the three schemas from a scaffolded project (or [`schematter/examples/okf/`](https://github.com/iwe-org/schematter/tree/main/examples/okf)) into `.iwe/schemas/` and bind them in `.iwe/config.toml`. A binding for the concept schema needs to skip the reserved files, which is what `!` negation is for — the last matching pattern decides, gitignore-style: ``` toml [schemas.okf] match = ["**", "!index", "!**/index", "!log", "!**/log"] [schemas.okf-index] match = ["index", "**/index"] [schemas.okf-log] match = ["log", "**/log"] ``` All matching schemas apply to a document, so `okf.yaml` stacks on top of whatever per-type schemas you already have. ## Validate ``` bash iwe schema validate ``` Validation is mechanical and CI-ready — it exits non-zero on any violation and names the exact document, field, and rule. Because document schemas describe body structure as well as frontmatter, this covers all three conformance criteria, including the index and log shapes a frontmatter-only check can't see. ## Query OKF frontmatter OKF fields are ordinary frontmatter, so `iwe find` filters on them directly: ``` bash iwe find --filter '{type: Playbook}' iwe find --filter '{status: draft}' -f keys iwe find --filter '{type: Metric, status: deprecated}' ``` ## Reference bundles Both IWE workspace templates ship as conformant OKF v0.2 bundles, validated in CI — clone one to see every family in use, or browse them online: - [marketing-workspace](https://github.com/iwe-org/marketing-workspace) — campaign memory for a marketing agent ([browse](https://iwe.md/templates/marketing-workspace/data/)) - [dev-workspace](https://github.com/iwe-org/dev-workspace) — project memory for a coding agent ([browse](https://iwe.md/templates/dev-workspace/data/)) --- # IWE Find URL: https://iwe.md/docs/agentic/find/ Search and discover documents in your knowledge base. Combines a text query — `--fuzzy` (title and key) or `--lexical` (BM25 full-text) — with a YAML-based filter language. `--fuzzy` / `--lexical` are the CLI surface of the query language's [`search`](/docs/concepts/query-language/#search-find-only) stage: they select (a document must match to appear) and order by relevance, jointly with the filter. The same stage is available in an operation document via `search: { lexical, fuzzy }` and through the MCP `iwe_query` tool. ## Usage ``` bash iwe find [OPTIONS] iwe find --fuzzy [OPTIONS] iwe find --lexical [OPTIONS] ``` ## Options | Flag | Description | Default | | ------------------------------- | -------------------------------------------------------------------------------------------- | ---------- | | `--fuzzy ` | Fuzzy match on document title and key. | none | | `--lexical ` | Lexical (BM25) full-text match on title and body. | none | | `[QUERY]` | Deprecated: bare positional query. Behaves as `--fuzzy` and prints a warning. | none | | `--filter ` | Inline YAML filter expression. See [Query Language](/docs/concepts/query-language/). | none | | `-k, --key ` | Match by document key. Repeatable: 1 key uses `$eq`, 2+ uses `$in`. Unknown keys match nothing. | none | | `--includes ` | `$includes` anchor. Repeatable; anchors are ANDed. | none | | `--included-by ` | `$includedBy` anchor. Repeatable; anchors are ANDed. | none | | `--references ` | `$references` anchor. Repeatable; anchors are ANDed. | none | | `--referenced-by ` | `$referencedBy` anchor. Repeatable; anchors are ANDed. | none | | `--max-depth ` | Session default for inclusion anchor flags without a colon-suffix. `0` = unbounded. | 1 | | `--max-distance ` | Session default for reference anchor flags without a colon-suffix. `0` = unbounded. | 1 | | `--project ` | Projection: comma-list (`name`, `name=path`, `name=$selector`, `$selector`) or inline YAML mapping. Replaces the default fields. | none | | `--add-fields ` | Additive projection: same grammar as `--project`, extends the defaults instead of replacing. | none | | `--blocks ` | Locate blocks: adds a `blocks` field listing each block matching the inline block predicate. | none | | `--matches ` | Grep over blocks: restricts results to documents whose content matches the Rust regex `PATTERN` and adds a `matches` field with the matching lines. | none | | `--sort ` | Sort by frontmatter field. `DIR` is `1` (asc) or `-1` (desc). `COLLATION` is `binary` for codepoint order or a BCP-47 locale tag such as `sv-SE`; default is the workspace locale, and a language without its own collation data uses the root rules. | none | | `-l, --limit ` | Maximum number of results (`0` = unlimited). | unlimited | | `--max-tokens ` | Cap total projected `$content` tokens across all results (`0` = unlimited). | unlimited | | `--max-document-tokens ` | Cap projected `$content` tokens per result, head-truncating with a marker (`0` = unlimited). | unlimited | | `-f, --format ` | Output format: `markdown`, `keys`, `json`, `yaml`. | `markdown` | All filter clauses (the text query plus every flag above) are AND-composed at the top level. For OR or NOT, write it inside `--filter`. See [Query Language](/docs/concepts/query-language/). `--lexical` stems its terms and drops stop words; a query with no searchable terms left matches nothing (a warning suggests `--fuzzy` for common or partial words). `--max-tokens` and `--max-document-tokens` only act when the projection includes a `$content`-shaped field — bare `$content` or a narrowed `{ $content: PREDICATE }`; a narrowed body is counted and capped like a full one, while `$blocks` / `$matches` entries cost nothing. A metadata index carries no content tokens, so bound it with `--limit`. Token budgets are off by default, count body text only, and print a `warning:` to stderr when they trim the output. ## Filter language `--filter` accepts a YAML mapping. Bare scalars become equality predicates; `$`-prefixed keys are operators. ``` bash # Bare equality on a frontmatter field iwe find --filter 'status: draft' # Multiple top-level keys are ANDed iwe find --filter '{status: draft, priority: { $gt: 3 }}' # OR composition iwe find --filter '$or: [{ status: draft }, { status: review }]' # Comparison and ordering iwe find --filter 'priority: { $gte: 3, $lte: 7 }' --sort modified_at:-1 # Membership iwe find --filter 'tags: rust' # array membership iwe find --filter 'status: { $in: [draft, review] }' # any of these # Presence iwe find --filter 'reviewed_by: { $exists: false }' ``` See [Query Language](/docs/concepts/query-language/) for the full operator vocabulary. ## Structural anchors The four `$`-prefixed graph operators have CLI shortcuts. They walk inclusion edges (block-reference inclusion links) or reference edges (inline links), starting at the named anchor key. ``` bash # Direct sub-documents only — scalar shorthand fixes maxDepth: 1 iwe find --included-by projects/alpha # Walk inclusion edges down 5 levels iwe find --included-by projects/alpha:5 # Every descendant (unbounded — depth `0` is the unbounded sentinel) iwe find --included-by projects/alpha:0 # Anchors are ANDed: documents under both alpha and dmytro iwe find --included-by projects/alpha --included-by people/dmytro # Per-anchor depth: alpha within 3 levels, dmytro direct only iwe find --included-by projects/alpha:3 --included-by people/dmytro:1 # Session default for un-suffixed anchors iwe find --max-depth 5 --included-by projects/alpha --included-by research/q2 # Documents that reference a specific document (inline links) iwe find --references people/alice # Walk reference edges within 3 hops iwe find --referenced-by archive/index:3 # OR composition of structural anchors via --filter iwe find --filter '$or: [{ $includedBy: projects/alpha }, { $includedBy: projects/beta }]' ``` The same selector flags are accepted by [`iwe count`](/docs/agentic/count/), [`iwe retrieve`](/docs/agentic/retrieve/), [`iwe tree`](/docs/agentic/tree/), [`iwe export`](/docs/cli/export/), [`iwe schema`](/docs/cli/schema/), [`iwe update`](/docs/agentic/update/), and [`iwe delete`](/docs/agentic/delete/). ## Block projection `--project` and `--add-fields` can address blocks — the structural nodes inside each matched document — through three sources, each taking a block predicate. See [Query Language](/docs/concepts/query-language/#block-projection) for the predicate grammar. A block predicate is structured, so these forms are written as an inline YAML mapping (the comma list reaches only bare `$blocks` and `$content`): ``` bash # Body narrowed to one section, header included iwe find -k projects/roadmap --project 'notes: { $content: { $section: Unreleased } }' # Table of contents: the whole argument is a block predicate (--project only) iwe find -k guides/handbook --project '{ $header: {} }' # Located blocks as data: type, section path, own text iwe find --add-fields 'hits: { $blocks: { $within: Goals, $text: "Q3" } }' # Grep: matching lines with their locations iwe find --add-fields 'found: { $matches: "(?i)todo|fixme" }' ``` Two dedicated flags shortcut the common reads: ``` bash # --blocks PRED lowers to: addFields: { blocks: { $blocks: PRED } } iwe find --blocks '{ $within: Goals, $text: "Q3" }' # --matches PATTERN lowers to BOTH a membership clause and a projection entry: # filter: { $content: { $matches: PATTERN } } # addFields: { matches: { $matches: PATTERN } } # — one-flag grep: only matching documents return, each carrying its lines iwe find --matches '(?i)todo|fixme' ``` `--blocks` adds the located blocks without restricting membership; combine it with `--filter '$content: …'` (usually the same predicate) to drop non-matching documents. `--matches` restricts membership on its own. In markdown output, `$blocks` and `$matches` entries print one grep line each — `key › section path › text` — under the result's index line: ``` - [Roadmap](projects/roadmap) projects/roadmap › Goals › Q3 Milestones › Ship the editor integration TODO confirm date ``` In JSON / YAML, a narrowed `$content` field is a string (empty when no block matches) and `$blocks` / `$matches` fields are arrays of entries carrying `path` (enclosing section titles), `text` (own text), and — for `$blocks` — `type`. ## How it works 1. **Text matching** — `--fuzzy` matches the key and the title using SkimMatcherV2; `--lexical` runs a BM25 full-text query over title and body. 2. **Filter** — `--filter` and the structural-anchor flags evaluate per document; results are intersected. 3. **Sort** — `--sort field:DIR` orders the matched set; ties are broken by document key. 4. **Limit** — applied last. 5. **Project** — `--project` / `--add-fields` shape each result: frontmatter fields, system fields (`$key`, `$content`, edge selectors), or block-addressed sources. Without a text query, results are sorted by incoming-reference popularity. With `--fuzzy`, they are sorted by fuzzy match score; with `--lexical`, by BM25 relevance. ## Output formats ### Markdown (default) A compact index — one line per document, meant to be scanned: ``` markdown - [User Authentication](authentication) - [Login Flow](login-flow) <- [User Authentication](authentication) - [Session Management](session-management) <- [User Authentication](authentication) ``` Each line is `- [title](key)`, followed by any projected edge and scalar fields: - **Link text** is the `title`. When `title` is not projected (e.g. `--project key`), the key string is used instead. - **Edge fields** render as arrows after the link, by direction: incoming (`includedBy`, `referencedBy`) as `<- [Title](key)`, outgoing (`includes`, `references`) as `-> [Title](key)`. Multiple targets in one field are comma-joined after a single arrow. Empty edges are omitted. - **Scalar fields** render as ` · name: value`. Arrays of scalars join with `, `; richer values fall back to compact inline YAML (use `-f json`/`yaml` for those). The index never prints document bodies. Projecting a `$content`-shaped field (via `--project` or `--add-fields`, bare or narrowed) switches the whole invocation to the fenced document block that [`iwe retrieve`](/docs/agentic/retrieve/) emits, with content as the body and other projected fields as frontmatter. A narrowed `{ $content: PREDICATE }` puts the narrowed rendering in the body; several `$content`-shaped fields concatenate in projection order. `$blocks` / `$matches` fields render as grep lines instead (see [Block projection](#block-projection)). ``` markdown $ iwe find --fuzzy auth --add-fields '$content' ````markdown #authentication --- title: User Authentication --- # User Authentication …body… ```` ``` To read full documents, use [`iwe retrieve`](/docs/agentic/retrieve/). ### Keys (`-f keys`) ``` authentication login-flow session-management ``` One key per line; suitable for piping. ### JSON (`-f json`) ``` json [ { "key": "authentication", "title": "User Authentication", "includedBy": [] } ] ``` The top-level value is a bare array of result objects. Each object carries the system fields `key`, `title`, `includedBy`, and every user-frontmatter field merged at the top level. `includedBy` entries are `EdgeRef { key, title, sectionPath }`. `--project title,status` projects only those fields into each result, in the listed order. System fields and user frontmatter fields are projectable interchangeably. ### YAML (`-f yaml`) Same shape as JSON, rendered as YAML. ## Examples ``` bash # All documents, default markdown iwe find # Fuzzy search on title and key iwe find --fuzzy authentication # Full-text search on title and body iwe find --lexical "session token" # Text query AND a frontmatter filter iwe find --fuzzy auth --filter 'status: draft' # Roots — documents with no incoming inclusion edges iwe find --filter '$nor: [{ $includedBy: { match: {} } }]' # Limit iwe find --limit 10 # JSON for programmatic use, project two fields iwe find --project title,modified_at -f json # Corpus-wide grep with locations iwe find --matches '(?i)todo|fixme' # Locate blocks before mutating them with iwe update iwe find --blocks '{ $within: Goals, $text: "Q3" }' # One section of one document iwe find -k projects/roadmap --project 'notes: { $content: { $section: Goals } }' # Pipe keys to retrieve iwe find --filter 'status: draft' -f keys | xargs -I {} iwe retrieve -k {} ``` ## Deprecated aliases The following flags pre-date the query language and remain accepted for backward compatibility. Each invocation prints a one-line `warning: ... is deprecated` to stderr. | Deprecated | Use instead | | ------------------ | --------------------------------------------------------------------------- | | bare positional `QUERY` | `--fuzzy QUERY` (or `--lexical QUERY` for full-text) | | `--in KEY[:N]` | `--included-by KEY[:N]` | | `--in-any K1 K2` | `--filter '$or: [{ $includedBy: K1 }, { $includedBy: K2 }]'` | | `--not-in KEY` | `--filter '$nor: [{ $includedBy: KEY }]'` | | `--refs-to KEY` | `--references KEY` (legacy semantics: ORs `$includes` and `$references`) | | `--refs-from KEY` | `--referenced-by KEY` (legacy semantics: ORs `$includedBy` and `$referencedBy`) | | `--roots` | `--filter '$nor: [{ $includedBy: { match: {} } }]'` | ## Technical notes - All filter flags AND together at the top level. To compose with OR or NOR, wrap inside `--filter`. - The colon-suffix on an anchor flag (`KEY:N`) overrides `--max-depth` / `--max-distance` for that anchor only. `0` is the unbounded sentinel. - Combining `-k KEY` with a `--filter` whose top level also contains `$key` is a parse-time error. Use `-k a -k b` for multi-key match (lowers to `$key: { $in: [a, b] }`), or write the OR inside `--filter`. - Both [Inclusion Links](/docs/concepts/inclusion-links/) and inline references count toward `incoming_refs`. --- # IWE Count URL: https://iwe.md/docs/agentic/count/ Count documents in your knowledge base that match a filter. Output is a single integer on stdout. ## Usage ``` bash iwe count [OPTIONS] ``` ## Options | Flag | Description | Default | | -------------------------- | ------------------------------------------------------------------------------------------------------------ | ------------- | | `--filter ` | Inline YAML filter expression. See [Query Language](/docs/concepts/query-language/). | none | | `-k, --key ` | Match by document key. Repeatable: 1 key uses `$eq`, 2+ uses `$in`. | none | | `--includes ` | `$includes` anchor. Repeatable; anchors are ANDed. DEPTH defaults to `--max-depth`. | none | | `--included-by ` | `$includedBy` anchor. Repeatable; anchors are ANDed. | none | | `--references ` | `$references` anchor. Repeatable; anchors are ANDed. DIST defaults to `--max-distance`. | none | | `--referenced-by ` | `$referencedBy` anchor. Repeatable; anchors are ANDed. | none | | `--max-depth ` | Session default for inclusion anchor flags without a colon-suffix. `0` = unbounded. | 1 | | `--max-distance ` | Session default for reference anchor flags without a colon-suffix. `0` = unbounded. | 1 | | `-l, --limit ` | Cap the number of matches counted (`0` = unlimited). | none | `count` does not accept `-f / --format` or `--project`. Output is always a single integer terminated by a newline. ## How it works `count` runs the same filter pipeline as [`iwe find`](/docs/agentic/find/) but returns just the integer count of matched documents. All filter flags AND together at the top level; combine with `--filter '$or: [...]'` for OR composition. The `KEY[:DEPTH]` / `KEY[:DIST]` colon-suffix on an anchor flag overrides the session default for that anchor only; `0` is the unbounded sentinel. ## Examples ``` bash # Total documents iwe count # Drafts iwe count --filter 'status: draft' # Drafts with a high priority iwe count --filter '{status: draft, priority: { $gt: 3 }}' # Descendants of an anchor, within 10 levels iwe count --included-by projects/alpha:10 # Every descendant of an anchor (unbounded) iwe count --included-by projects/alpha:0 # Multi-key match iwe count -k projects/alpha -k projects/beta # OR-composition via --filter iwe count --filter '$or: [{ status: draft }, { status: review }]' # Content membership — documents holding a block matching a predicate iwe count --filter '$content: { $matches: "(?i)todo|fixme" }' iwe count --filter '$nor: [{ $content: { $header: Status } }]' # Cap the count for very large corpuses iwe count --filter 'tags: rust' -l 1000 ``` ## Use cases ### Quick health checks ``` bash # Are there still drafts in the queue? iwe count --filter 'status: draft' # Has the archive grown beyond a threshold? iwe count --filter 'status: archived' ``` ### Exit-code-friendly assertions in CI ``` bash # Fail CI if any draft remains test "$(iwe count --filter 'status: draft')" -eq 0 ``` ### Coverage of a structural set ``` bash # How many documents are below this hub? iwe count --included-by projects/alpha:0 ``` ## Related - [Query Language](/docs/concepts/query-language/) — the YAML filter syntax. - [`iwe find`](/docs/agentic/find/) — same filter shape, returns the matched documents. - [`iwe update`](/docs/agentic/update/) — apply mutations to the same matched set. - [`iwe delete`](/docs/agentic/delete/) — remove the same matched set. --- # IWE Retrieve URL: https://iwe.md/docs/agentic/retrieve/ Retrieves document content with graph expansion and relationship context. Designed for AI agents to navigate and understand the knowledge graph. `retrieve` assembles reading context around a set of documents in a single call. It has two moving parts: **seeds** — the documents to read, given directly by `-k` / `--filter` / anchors, or found by a `--lexical` / `--fuzzy` search — and **expansion** — the graph directions to follow out from each seed, given by the `--expand-*` flags. ## Usage ``` bash iwe retrieve [OPTIONS] iwe retrieve -k KEY --expand-includes 1 --expand-included-by 1 iwe retrieve --lexical "QUESTION" --limit 5 --expand-references 1 --expand-included-by 1 ``` Without `-k` (and without a search flag), reads the document key from stdin (for piping). ## Options | Flag | Description | Default | | --------------------- | ---------------------------------------------------------------------- | -------- | | `-k, --key ` | Document key(s) to retrieve, or the candidate set searched within when a search flag is present (repeatable). 1 key = `$eq`, 2+ = `$in`. Unknown keys are an error unless searching — see [Named Keys vs Filters](/docs/architecture/keys/#named-keys-vs-filters). | stdin | | `--expand-includes [N]` | Follow inclusion edges downward, pulling child (sub-)documents to depth `N`. Bare = `1`; `0` = unbounded; omitted = not followed. | not followed | | `--expand-included-by [N]` | Follow inclusion edges upward, pulling parent documents to depth `N`. Bare = `1`; `0` = unbounded; omitted = not followed. | not followed | | `--expand-references [N]` | Follow outbound reference links, pulling documents this seed links to within `N` hops. Bare = `1`; `0` = unbounded; omitted = not followed. | not followed | | `--expand-referenced-by [N]` | Follow inbound reference links, pulling documents that link to this seed within `N` hops. Bare = `1`; `0` = unbounded; omitted = not followed. | not followed | | `--lexical ` | Seed search: BM25 full-text query on title and body. Runs a seed query and switches to the search-indexed loader. | none | | `--fuzzy ` | Seed search: fuzzy query on title and key. Combine with `--lexical` to fuse (RRF). | none | | `-e, --exclude ` | Exclude document key(s) from results (repeatable). | none | | `-b, --backlinks` | Populate the `referencedBy` edge list in output (does not add documents). | true | | `--children` | Populate the `includes` edge list in output (does not add documents). | false | | `-f, --format ` | Output format: `markdown`, `keys`, `json`, `yaml`. | `markdown` | | `--filter ` | Inline YAML filter expression restricting the candidate set. See [Query Language](/docs/concepts/query-language/). | none | | `--includes ` | `$includes` anchor restricting the candidate set. Repeatable; anchors are ANDed. | none | | `--included-by ` | `$includedBy` anchor restricting the candidate set. Repeatable; anchors are ANDed. | none | | `--references ` | `$references` anchor restricting the candidate set. Repeatable; anchors are ANDed. | none | | `--referenced-by ` | `$referencedBy` anchor restricting the candidate set. Repeatable; anchors are ANDed. | none | | `--max-depth ` | Session default for inclusion anchor flags without a colon-suffix. `0` = unbounded. | 1 | | `--max-distance ` | Session default for reference anchor flags without a colon-suffix. `0` = unbounded. | 1 | | `--limit ` | Cap the number of seed documents kept **before** expansion — top-N by relevance when searching, the first N of the selection otherwise. `0` = unlimited. | unlimited | | `--max-documents ` | Cap the number of documents returned **after** expansion, trimming periphery documents first. `0` = unlimited. | unlimited | | `--max-tokens ` | Cap total content tokens across all documents (whole documents are dropped). `0` = unlimited. | unlimited | | `--max-document-tokens ` | Cap content tokens per document (body is head-truncated with a marker). `0` = unlimited. | unlimited | > **Note.** The colon-suffix (`KEY:DEPTH`) on the anchor *selection* flags above and the depth values on the `--expand-*` flags are unrelated: the anchor flags **restrict which documents are seeds**, the `--expand-*` flags **follow edges out from the seeds**. ## Expansion (`--expand-*`) Expansion pulls related documents into the result set. Each direction is its own flag; the value is a depth (bare flag = `1`, `0` = **unbounded**, omitted = **not followed**): | Flag | Follows | Adds | | --- | --- | --- | | `--expand-includes [N]` | inclusion edges downward | child (sub-)documents | | `--expand-included-by [N]` | inclusion edges upward | parent documents | | `--expand-references [N]` | outbound reference (inline) links | documents this seed links to | | `--expand-referenced-by [N]` | inbound reference links | documents that link to this seed | ``` bash --expand-references 1 --expand-included-by 1 # one hop of outbound refs + one level of parents --expand-includes 2 # children two levels deep --expand-included-by 0 # every ancestor (0 = unbounded) --expand-includes # bare flag = one level ``` A non-integer or negative value is an error. **Expansion is doc-only by default.** With no `--expand-*` flag (and no deprecated `-d` / `-c` / `-l`), `retrieve` returns exactly the requested document(s) with no expansion. The old implicit default is now written explicitly as `--expand-includes 1 --expand-included-by 1`. `includedBy` is always listed in the output frontmatter regardless of expansion; `-b` and `--children` toggle the `referencedBy` and `includes` edge lists; `references` is listed when a `--expand-references` expansion is requested. Edge-list toggles never add documents. ## Search seeds When `--lexical` or `--fuzzy` is present, `retrieve` first runs a **seed query** — a `find` with that search over the candidate set (`-k` / `--filter` / anchors) — then reads and expands the ordered seeds. Seeds come out first, in relevance order, followed by their expansion; seeds survive token-budget trimming first. `--limit` caps the seeds kept (top-N by relevance) before expansion. Because search **restricts**, `-k a -k b -k c --lexical Q` searches *within* those three keys: any that do not match the query drop from the output. `-k` is candidate restriction here, not a guaranteed read. The standing/hub pages an agent needs are reached structurally rather than pinned: search finds the episode pages, and one expansion direction brings the hubs in — `includedBy` when hubs include their episodes, `referencedBy` when a hub timeline links to them, `references` when episodes link back to the hub. ### Output limits `--max-documents`, `--max-tokens`, and `--max-document-tokens` bound output for context-limited callers and are **off by default**. (`--limit` is not an output budget — it caps the *seeds* before expansion.) - **What is counted.** Each document's body plus its serialized edge lists. JSON/YAML structural overhead (field names, escaping) is not counted, and the counts are an approximate (OpenAI-BPE) proxy — so treat the budget as a soft bound with headroom, not an exact byte cap. The `⋯ truncated` marker appended to a clipped body is itself uncounted, so a clipped document lands slightly over `--max-document-tokens`. - **What survives.** The requested document(s) are collected first, so they always survive; `--max-documents` and `--max-tokens` trim from the periphery (deepest expansion and parent context) inward. A clipped result set is therefore **not relationally closed** — a kept child may lose the parent-context document it appeared under. - **Signal.** When any limit trims the output, a `warning:` line is printed to **stderr** while stdout stays a clean array/markdown. Page through a large result set by combining `--exclude` with the truncation signal: fetch, note the returned keys, re-run excluding them. ### Filter and structural anchors When filter or anchor flags are provided, `retrieve` reads the documents that match. With both `-k` and a filter expression, the result is the **intersection** — `-k` clauses AND with the filter at the top level. ``` bash # Retrieve every doc under projects/alpha (unbounded), no expansion iwe retrieve --included-by projects/alpha:0 # Sub-documents of two anchors (intersection) iwe retrieve --included-by projects/alpha --included-by people/dmytro # Restrict explicit keys to those inside an anchor's subtree iwe retrieve -k x -k y -k z --included-by projects/alpha # Filter by frontmatter iwe retrieve --filter 'status: draft' ``` See [Query Language](/docs/concepts/query-language/) for the full filter syntax. ## How It Works The `retrieve` command collects documents in a specific order: 1. **Seeds** - the requested documents (`-k` / `--filter` / anchors, or the ordered results of a `--lexical` / `--fuzzy` seed query) 2. **Children** (`--expand-includes N`) - Documents included via [Inclusion Links](/docs/concepts/inclusion-links/), expanded N levels deep 3. **Context** (`--expand-included-by N`) - Parent documents of each seed, up N levels 4. **Sub-document parents** - Parents of direct sub-documents (only when both `--expand-includes` and `--expand-included-by` are set) 5. **References** (`--expand-references N`) - Documents referenced inline, within N hops 6. **Referenced by** (`--expand-referenced-by N`) - Documents that reference the seed, within N hops Documents are deduplicated - each document appears only once in the output. ### Depth Expansion (`--expand-includes`) Controls how deep to follow children (sub-documents): - **omitted**: Only the seed, no downward expansion (the default) - **`--expand-includes 1`**: Seed + direct sub-documents - **`--expand-includes 2`**: Seed + sub-documents + their sub-documents - **`--expand-includes 0`**: Every descendant (unbounded) ### Context Expansion (`--expand-included-by`) Controls how many levels of parent documents to include: - **omitted**: No parent context documents included (the default) - **`--expand-included-by 1`**: Direct parents of the seed and of its immediate sub-documents - **`--expand-included-by 0`**: Every ancestor (unbounded) ### Sub-document Parent Context When you retrieve a document with both `--expand-includes` and `--expand-included-by` set, the command also fetches parent documents of any sub-documents. This provides important context: the parent document often defines what *type* of thing the sub-document is. **Example: A component used in multiple projects** Consider a `button` component embedded in two different projects. When you retrieve `mobile-app`: ``` iwe retrieve -k mobile-app --expand-includes 1 --expand-included-by 1 ``` The output for the `button` sub-document shows its other parents: ``` markdown # Button [Button](button) <- [Web Dashboard](web-dashboard) A reusable button component. ``` The `<-` notation lists other documents where `button` is embedded. Since we're already viewing `button` from within `mobile-app`, only `web-dashboard` is shown — revealing that this component is shared across projects. **Result includes:** 1. `mobile-app` — the seed document 2. `button` — sub-document (via `includes: 1`) 3. `web-dashboard` — another parent of `button` (via `includedBy: 1`) The `web-dashboard` document is fetched because it's a parent of `button`. This context helps understand what the component is and where else it's used. **Note:** Only parents of *direct* sub-documents are included. ### References (`references`) When expanded, includes documents that are referenced inline (not as inclusion links): ``` markdown # Main Document See [Related Topic](related-topic) for more details. ``` With `--expand-references 1`, `related-topic` would be included in the output. Distances greater than 1 follow the reference chain transitively. ## Document Relationships ### Parent Documents vs Back Links **Parent Documents** - Inclusion links (document embedded in another): ``` markdown # Parent Document ## Overview [child](child) <- Inclusion link creates parent-child relationship ``` The child document shows: `**Parent documents:** [Parent Document](parent) > Overview` **Back Links** - Inline references (links within text): ``` markdown # Some Document See [target](target) for more details. <- Inline reference creates backlink ``` The target document shows: `**Back links:** [Some Document](some-document)` ## Output Format ### Markdown Format (default) Documents are rendered with YAML frontmatter containing metadata, followed by the document content: ``` markdown --- document: key: my-document title: My Document includedBy: - key: index title: Index referencedBy: - key: related-doc title: Related Doc --- # My Document Original document content preserved exactly as written. ``` Each document in the result set includes: - YAML frontmatter with `key`, `title`, `includedBy`, and `referencedBy` - Document content with original headers preserved - Two empty lines after each document for easy parsing Multiple documents are separated by their frontmatter delimiters (`---`), no horizontal rules. ### Keys Format (`-f keys`) ``` my-document child-document parent-document ``` One key per line, suitable for piping to other commands or building exclude lists. ### JSON Format (`-f json`) ``` json [ { "key": "my-document", "title": "My Document", "content": "# My Document\n\nContent here...", "includedBy": [ { "key": "index", "title": "Index", "sectionPath": ["Topics", "My Topic"] } ], "includes": [], "referencedBy": [ { "key": "related-doc", "title": "Related Doc", "sectionPath": [] } ] } ] ``` The top-level value is a bare array of document objects. Each carries `key`, `title`, `content`, and three edge arrays — `includedBy`, `includes`, `referencedBy` — using the same `EdgeRef { key, title, sectionPath }` shape. Empty arrays are emitted explicitly. `includes` is populated only when `--children` is passed. ## Examples ``` bash # Retrieve a single document (doc only, no expansion) iwe retrieve -k my-document # Retrieve multiple documents at once iwe retrieve -k doc1 -k doc2 -k doc3 # Immediate context: direct children + direct parents iwe retrieve -k my-document --expand-includes 1 --expand-included-by 1 # Deep expansion (3 levels of sub-documents) iwe retrieve -k my-document --expand-includes 3 # More parent context iwe retrieve -k my-document --expand-included-by 2 # Include inline referenced documents iwe retrieve -k my-document --expand-references 1 # One-shot search + expand: find seeds, then pull structure iwe retrieve --lexical "auth token rotation" --limit 5 \ --expand-references 1 --expand-included-by 1 -f json # Exclude documents which you don't need iwe retrieve -k my-document -e already-loaded -e another-loaded # Populate the `includes` edge list with child document edges iwe retrieve -k my-document --children # Get JSON output for programmatic processing iwe retrieve -k my-document -f json # Or YAML iwe retrieve -k my-document -f yaml # Pipe keys from stdin (one per line) echo -e "doc1\ndoc2\ndoc3" | iwe retrieve # Without backlinks (cleaner output) iwe retrieve -k my-document -b false ``` ## Use Cases ### AI Agent Context Building Build rich context for AI agents navigating the knowledge base: ``` bash # Get a document with immediate context iwe retrieve -k authentication --expand-includes 1 --expand-included-by 1 # Assemble a dossier from a question in one call iwe retrieve --lexical "how does session expiry work" --limit 5 \ --expand-references 1 --expand-included-by 1 ``` ### Understanding Document Relationships ``` bash # See where a document is used (parent documents in output) iwe retrieve -k my-topic # See the full context including inline references iwe retrieve -k my-topic --expand-references 1 ``` ### Exploring Knowledge Base Structure ``` bash # Start from an entry point and expand iwe retrieve -k index --expand-includes 2 # Get JSON for analysis iwe retrieve -k project-overview --expand-includes 1 -f json ``` ### Chaining Retrievals Use the `keys` format to chain retrieval operations and exclude already-fetched documents: ``` bash # Get keys from first retrieval KEYS_A=$(iwe retrieve -k topic-a -f keys) # Retrieve for topic-b, excluding keys from topic-a iwe retrieve -k topic-b -e $KEYS_A # Or using command substitution with xargs iwe retrieve -k topic-b $(iwe retrieve -k topic-a -f keys | xargs -I {} echo "-e {}") ``` ## Deprecated aliases The following flags pre-date the query language and remain accepted as hidden aliases for backward compatibility. | Deprecated | Use instead | | ------------------ | --------------------------------------------------------------------------- | | `-d, --depth N` | `--expand-includes N` (legacy `0` = off, not unbounded) | | `-c, --context N` | `--expand-included-by N` (legacy `0` = off) | | `-l, --links` | `--expand-references 1` | | `--in KEY[:N]` | `--included-by KEY[:N]` | | `--in-any K1 K2` | `--filter '$or: [{ $includedBy: K1 }, { $includedBy: K2 }]'` | | `--not-in KEY` | `--filter '$nor: [{ $includedBy: KEY }]'` | | `--refs-to KEY` | `--references KEY` (legacy semantics: ORs `$includes` and `$references`) | | `--refs-from KEY` | `--referenced-by KEY` (legacy semantics: ORs `$includedBy` and `$referencedBy`) | `-d` / `-c` / `-l` keep their legacy **`0` = off** meaning — they never expressed unbounded, whereas the `--expand-*` flags treat `0` as the unbounded sentinel. Passing a legacy alias together with the `--expand-*` flag for the same direction (e.g. `-d` and `--expand-includes`) is an error. ## Technical Notes - Documents use YAML frontmatter for metadata, content follows with original formatting - Empty `includedBy` or `referencedBy` fields are omitted from markdown frontmatter (JSON/YAML output always emits them as `[]`) - Original document headers are preserved (no level shifting) - Two empty lines separate documents for easier parsing - Duplicate documents are automatically filtered out - Sub-document parent context only includes parents of direct (first-level) sub-documents, not nested ones --- # IWE Tree URL: https://iwe.md/docs/agentic/tree/ Display document hierarchy as a tree structure. ## Usage ``` bash iwe tree [OPTIONS] ``` ## Options | Option | Default | Description | | ------------------------------- | ---------- | ------------------------------------------------------------------------------------ | | `-f, --format ` | `markdown` | Output format: `markdown`, `keys`, `json`, `yaml`. | | `-d, --depth ` | `4` | Maximum depth to traverse. | | `--project ` | - | Projection for `json` / `yaml` nodes: same grammar as [`iwe find`](/docs/agentic/find/). | | `--add-fields ` | - | Additive projection: extends each node's default fields. Same grammar as `--project`. | | `--filter ` | - | Inline YAML filter expression. See [Query Language](/docs/concepts/query-language/). | | `-k, --key ` | - | Start tree from specific document(s); repeatable. Unknown keys are an error. | | `--includes ` | - | `$includes` anchor. Repeatable; anchors are ANDed. | | `--included-by ` | - | `$includedBy` anchor. Repeatable; anchors are ANDed. | | `--references ` | - | `$references` anchor. Repeatable; anchors are ANDed. | | `--referenced-by ` | - | `$referencedBy` anchor. Repeatable; anchors are ANDed. | | `--max-depth ` | `1` | Session default for inclusion anchor flags without a colon-suffix. `0` = unbounded. | | `--max-distance ` | `1` | Session default for reference anchor flags without a colon-suffix. `0` = unbounded. | | `-v, --verbose ` | `0` | Verbosity level (1=info, 2=debug). | When filter or anchor flags are provided, the selector resolves to a set of keys and those keys are used as the tree roots. Combining `-k` with the selector intersects the two — empty intersection yields an empty tree. ## Output Formats ### Markdown (default) Nested list with links: ``` - [Main Document](main) - [Child Document](child) - [Nested Item](nested) - [Another Root](another) ``` ### Keys Document keys only with tab indentation: ``` main child nested another ``` ### JSON Nested JSON array structure: ``` json [ { "key": "main", "title": "Main Document", "children": [ { "key": "child", "title": "Child Document", "children": [] } ] } ] ``` `children` is always present (empty array on leaves). `--project` / `--add-fields` accept the same grammar as [`iwe find`](/docs/agentic/find/) — frontmatter fields, `$`-selectors, and block-addressed sources — and the projected fields are emitted alongside `key`, `title`, `children` per node. Markdown and keys output ignore projection. ``` bash iwe tree --project pillar,status -f json iwe tree --add-fields 'found: { $matches: "(?i)todo" }' -f yaml ``` ## Starting from Specific Documents Use `-k` to start the tree from specific document(s): ``` bash iwe tree -k my-doc iwe tree -k doc-a -k doc-b ``` This is essential for documents involved in circular references that have no natural root. ## Handling Circular References When documents form circular references (A→B→C→A), they have no natural root and won't appear in the default tree output. Use `-k` to start from any document in the cycle: ``` bash iwe tree -k doc-a ``` Output shows the cycle: ``` - [Doc A](doc-a) - [Doc B](doc-b) - [Doc C](doc-c) - [Doc A](doc-a) ``` ## Examples ``` bash iwe tree iwe tree -f keys iwe tree -f json iwe tree -f yaml iwe tree -k my-doc iwe tree -k doc-a -k doc-b iwe tree --depth 2 iwe tree | grep -i api iwe tree -f keys | grep cli # Roots inside an anchor's subtree iwe tree --included-by projects/alpha:0 # Tree restricted to drafts iwe tree --filter 'status: draft' ``` ## Depth Impact | Depth | Shows | | ----- | ------------------------------- | | 1 | Root documents only | | 2 | Roots and their direct children | | 3 | Up to grandchildren | | 4+ | Deeper nested relationships | ## AI Agent Tips - Use `tree` to understand the overall structure of a knowledge base - Use `-f keys` for programmatic processing - Use `-f json` or `-f yaml` for structured data consumption - Pipe through `grep` to filter results - Use `-k` to explore documents involved in circular references - Use `--depth 2` to quickly identify major topic areas ## Deprecated aliases The following flags pre-date the query language and remain accepted for backward compatibility. Each invocation prints a one-line `warning: ... is deprecated` to stderr. | Deprecated | Use instead | | ------------------ | --------------------------------------------------------------------------- | | `--in KEY[:N]` | `--included-by KEY[:N]` | | `--in-any K1 K2` | `--filter '$or: [{ $includedBy: K1 }, { $includedBy: K2 }]'` | | `--not-in KEY` | `--filter '$nor: [{ $includedBy: KEY }]'` | | `--refs-to KEY` | `--references KEY` (legacy semantics: ORs `$includes` and `$references`) | | `--refs-from KEY` | `--referenced-by KEY` (legacy semantics: ORs `$includedBy` and `$referencedBy`) | --- # IWE Stats URL: https://iwe.md/docs/agentic/stats/ Generates comprehensive statistics about your knowledge graph. ## Usage ``` bash iwe stats [OPTIONS] iwe stats similarity [-t ] ``` ## Options - `-f, --format `: Output format (default: `markdown`) - `markdown`: Human-readable formatted statistics - `csv`: Machine-readable CSV format with per-document statistics - `json`: JSON output (aggregate stats only; ignored when `-k` is given — per-key stats always serialize as JSON) - `-k, --key `: Document key for per-document statistics (always JSON output). Omit for aggregate graph statistics. ## Subcommands - `similarity`: list pages that have a near-identical, mutually-similar counterpart elsewhere in the store (see [Detecting similar pages](#detecting-similar-pages)). - `-t, --threshold `: how close a pair must be to be reported (default: `0.85`). Lower values list looser matches, higher values only closer ones. ## What it shows The stats command provides detailed analytics across multiple dimensions: ### Overview - Total documents in your knowledge base - Total nodes (all content elements) - Total paths through the graph ### Document Statistics - Total sections/headers across all documents - Average sections per document - Top 10 documents by section count - Paragraph counts per document ### Reference Statistics - Inclusion links (embedded documents) - Inline references (wiki-links) - Total references count - Orphaned documents (no incoming references) - Leaf documents (no outgoing references) - Top 10 most referenced documents ### Lines Statistics - Total lines across all documents - Average lines per document - Top 10 largest documents by line count ### Words Statistics - Total words across all documents - Average words per document - Top 10 largest documents by word count ### Structure Statistics - Root-level sections - Maximum and average path depth - Counts of bullet lists, ordered lists, code blocks, tables, and quotes ### Orphans - A list of every document with no incoming references (inclusion or inline), by key. `index` pages (the root `index` or any `/index`) are treated as intentional entry points and are never reported as orphans. The markdown, JSON, and YAML outputs carry this list; the aggregate count also appears under Reference Statistics. ### Network Analysis - Average references per document - Top 10 most connected documents (by total incoming + outgoing references) ### Per-document Similar pages When you pass `-k `, the output also lists **similar pages** — other documents whose content is near-identical to this one (see [Detecting similar pages](#detecting-similar-pages) for how the match is decided). The markdown output appends a `- **Similar page:** ()` line per match; the JSON/YAML output carries a `similarPages` array of `{ key, score }`. ## Detecting similar pages `iwe stats similarity` scans the whole store and reports pages that duplicate one another. A page is only reported when the match is a genuine duplicate, not merely a related page: - **Mutual.** The similarity must hold in both directions — the two pages must each be mostly made of the other's content. This rules out a short page that is wholly contained in a longer one. - **Comparable size.** Both pages must be at least ~50 tokens and within ~2× the length of each other. Real duplicates are the same content, so they are the same size. - **Near-identical.** The bar is set high (a self-normalized BM25 score of at least `0.85`), so the same page saved under two keys is caught while related-but-distinct pages are not. Forward matches are computed once per page and run concurrently, so the scan stays fast on large stores. Output lists each mutually-similar pair once, tab-separated, in alphabetical order: ``` text people/ada-and-kai people/kai-and-ada notes/2019-budget notes/2019-budget-copy ``` ### Tuning the match level `-t, --threshold` moves the near-identical bar. Lower it to surface rewrites and paraphrases that the default misses, raise it to keep only the closest copies: ``` bash # Default: only near-identical copies iwe stats similarity # Looser: also catches rewritten and paraphrased pages iwe stats similarity -t 0.5 ``` The threshold applies to both directions of the match; the mutuality, size, and length gates stay in place at every level. The per-document similar pages shown by `iwe stats -k ` always use the default level. ## Examples ``` bash # Generate human-readable statistics (default) iwe stats # Export per-document statistics as CSV iwe stats --format csv > stats.csv # Analyze CSV data with standard tools iwe stats -f csv | cut -d, -f1,2,3 | column -t -s, ``` ## Sample Markdown Output ``` markdown # Graph Statistics ## Overview - **Total documents:** 39 - **Total nodes:** 1552 - **Total paths:** 302 ## Document Statistics - **Total sections:** 844 - **Average sections/doc:** 21.64 ### Top Documents by Sections 1. **VS Code** (102 sections) 2. **Neovim** (89 sections) 3. **Extract Actions** (79 sections) ... ## Lines Statistics - **Total lines:** 3404 - **Average lines/doc:** 87.28 ### Top Documents by Lines 1. **Neovim** (429 lines) 2. **Extract Actions** (342 lines) ... ## Words Statistics - **Total words:** 14204 - **Average words/doc:** 364.21 ### Top Documents by Words 1. **Neovim** (1337 words) 2. **Extract Actions** (1200 words) ... ``` ## CSV Format Details The CSV format provides per-document statistics with the following columns: - `key`: Document identifier/filename - `title`: Document title (first heading) - `sections`: Number of heading sections - `paragraphs`: Number of paragraph blocks - `lines`: Total line count - `words`: Total word count - `includedByCount`: Documents that include this one - `referencedByCount`: Inline references pointing to this document - `incomingEdgesCount`: Total incoming references - `includesCount`: Documents included by this one - `referencesCount`: Inline references from this document - `totalEdgesCount`: Total references (incoming + outgoing) - `bulletLists`: Number of unordered lists - `orderedLists`: Number of numbered lists - `codeBlocks`: Number of code/raw blocks - `tables`: Number of tables - `quotes`: Number of quote blocks ## Using CSV Output The CSV format enables programmatic analysis and integration with data tools: ``` bash # Import into spreadsheet applications iwe stats -f csv > knowledge-base-stats.csv # Find most connected documents iwe stats -f csv | tail -n +2 | sort -t, -k12 -nr | head -5 # Calculate total word count iwe stats -f csv | tail -n +2 | cut -d, -f6 | paste -sd+ | bc # Filter documents with many references iwe stats -f csv | awk -F, '$9 > 5 {print $1, $2, $9}' OFS=, # Generate reports with Python/pandas import pandas as pd df = pd.read_csv('stats.csv') print(df.describe()) print(df.nlargest(10, 'totalEdgesCount')) ``` --- # IWE Extract URL: https://iwe.md/docs/agentic/extract/ Extract a section from a document into a new document with an [Inclusion Links](/docs/concepts/inclusion-links/). ## Usage ``` bash iwe extract [OPTIONS] ``` ## Arguments | Argument | Description | | -------- | ---------------------------------------------- | | `` | Document key containing the section to extract | ## Options | Flag | Description | | ------------------- | --------------------------------------------- | | `--section ` | Section title to extract (case-insensitive) | | `--block <N>` | Block number to extract (1-indexed) | | `--list` | List all sections with block numbers | | `--action <NAME>` | Action name from config to use for extraction | | `--dry-run` | Preview changes without writing to disk | | `--quiet` | Suppress progress output | | `-f, --format <FMT>` | Output format: `markdown` (default) or `keys` (print affected document keys, one per line) | ## How It Works The `extract` command creates a new document from an existing section: 1. **Select a section** - Use `--list` to see available sections, then select with `--section` or `--block` 2. **Create new document** - The section content is moved to a new document 3. **Add inclusion link** - The original section is replaced with a link to the new document 4. **Adjust headers** - Header levels are adjusted to maintain proper document structure ## Workflow ### Step 1: List Available Sections ``` bash $ iwe extract my-document --list 1: Introduction 2: Getting Started 3: Configuration 4: Advanced Topics ``` ### Step 2: Extract by Title or Block Number ``` bash # By title (case-insensitive, partial match) $ iwe extract my-document --section "configuration" Extracting section 'Configuration' to 'configuration' Done # By block number (unambiguous) $ iwe extract my-document --block 3 Extracting section 'Configuration' to 'configuration' Done ``` ## Configuration The `--action` flag uses extraction settings from `.iwe/config.toml`: ``` toml [actions] extract = { type = "extract", title = "Extract", key_template = "{{slug}}", link_type = "markdown" } ``` Without `--action`, the command uses the first `extract` action found in config, or defaults to: - `key_template = "{{slug}}"` - URL-friendly version of the section title - `link_type = "markdown"` - Standard markdown links ### Key Template Variables See [feature-extract.md](/docs/code-actions/extract/#template-variables) for all available template variables. ## Output Modes ### Default Output Shows progress: ``` bash $ iwe extract my-document --section "Architecture" Extracting section 'Architecture' to 'architecture' Done ``` ### Dry Run (`--dry-run`) Preview what would happen: ``` bash $ iwe extract my-document --section "Design" --dry-run Would extract section 'Design' to 'design' Would update 'my-document' ``` ### Keys Output (`-f keys`) Print affected document keys: ``` bash $ iwe extract my-document --block 2 -f keys my-document getting-started ``` ## Examples ``` bash # List all sections with block numbers iwe extract notes/project --list # Extract section by title iwe extract notes/project --section "Architecture" # Extract section by block number (unambiguous) iwe extract notes/project --block 2 # Preview changes without writing iwe extract notes/project --section "Notes" --dry-run # Use a specific action from config iwe extract notes/project --section "Design" --action "my-extract" # Get keys for scripting iwe extract notes/project --section "API" -f keys ``` ## Use Cases ### Breaking Down Large Documents Extract sections to create a modular document structure: ``` bash # See what sections exist iwe extract large-document --list # Extract each major section iwe extract large-document --section "Introduction" iwe extract large-document --section "Implementation" iwe extract large-document --section "Testing" ``` ### Organizing by Topic Move related content to dedicated documents: ``` bash # Preview the extraction iwe extract project-notes --section "Authentication" --dry-run # Extract to a dedicated document iwe extract project-notes --section "Authentication" ``` ### Scripting Use with other commands: ``` bash # Extract and get the new document key NEW_KEY=$(iwe extract doc --section "API" -f keys | tail -1) echo "Created: $NEW_KEY" ``` ## Error Handling The command fails with an error if: - The document does not exist - No section matches the provided title (with `--section`) - Multiple sections match the title (use `--block` instead) - The block number is out of range (with `--block`) - Must specify `--section`, `--block`, or `--list` ## Key Collision Handling When the generated key already exists, IWE automatically appends numeric suffixes: 1. First attempt: `architecture.md` 2. If exists: `architecture-1.md` 3. If exists: `architecture-2.md` ## Technical Notes - Header levels are adjusted automatically (H2 in source becomes H1 in extracted document) - The original section is replaced with an inclusion link - The command uses the same extraction logic as the LSP code action - Directory structure is preserved based on the source document location --- # IWE Inline URL: https://iwe.md/docs/agentic/inline/ Replace an [Inclusion Links](/docs/concepts/inclusion-links/) with the referenced document content. ## Usage ``` bash iwe inline <KEY> [OPTIONS] ``` ## Arguments | Argument | Description | | -------- | ----------------------------------------------- | | `<KEY>` | Document key containing the reference to inline | ## Options | Flag | Description | | ------------------- | ------------------------------------------- | | `--reference <KEY>` | Reference key or title to inline | | `--block <N>` | Block number to inline (1-indexed) | | `--list` | List all inclusion links with numbers | | `--action <NAME>` | Action name from config to use for inlining | | `--as-quote` | Inline as blockquote instead of section | | `--keep-target` | Keep the target document after inlining | | `--dry-run` | Preview changes without writing to disk | | `--quiet` | Suppress progress output | | `-f, --format <FMT>` | Output format: `markdown` (default) or `keys` (print affected document keys, one per line) | ## How It Works The `inline` command embeds referenced content directly into the source document: 1. **Select a reference** - Use `--list` to see available references, then select with `--reference` or `--block` 2. **Embed content** - The referenced document's content replaces the inclusion link 3. **Delete target** (optional) - By default, the referenced document is deleted 4. **Clean up references** - Other references to the deleted document are updated ## Workflow ### Step 1: List Available References ``` bash $ iwe inline my-document --list 1: [Introduction](introduction) 2: [Getting Started](getting-started) 3: [Configuration](config) ``` ### Step 2: Inline by Reference or Block Number ``` bash # By reference key/title (case-insensitive, partial match) $ iwe inline my-document --reference "getting-started" Inlining [Getting Started](getting-started) into 'my-document' Done # By block number (unambiguous) $ iwe inline my-document --block 2 Inlining [Getting Started](getting-started) into 'my-document' Done ``` ## Inline Types ### Section (Default) Embeds the full content of the referenced document: Before: ``` markdown # My Document [Introduction](introduction) ``` After: ``` markdown # My Document ## Introduction Content from the introduction document... ``` ### Quote (`--as-quote`) Wraps the content in a blockquote: Before: ``` markdown # My Document [Quote Source](quote-source) ``` After: ``` markdown # My Document > Content from the quote source document... ``` ## Target Document Handling ### Default Behavior (Delete Target) By default, the target document is deleted and other references are cleaned up: ``` bash $ iwe inline index --reference "old-section" Inlining [Old Section](old-section) into 'index' Done ``` ### Keep Target (`--keep-target`) Preserve the target document after inlining: ``` bash $ iwe inline index --reference "shared-content" --keep-target Inlining [Shared Content](shared-content) into 'index' Done ``` ## Configuration The `--action` flag uses inline settings from `.iwe/config.toml`: ``` toml [actions] inline_section = { type = "inline", title = "Inline Section", inline_type = "section", keep_target = false } inline_quote = { type = "inline", title = "Inline as Quote", inline_type = "quote", keep_target = true } ``` ## Output Modes ### Default Output Shows progress: ``` bash $ iwe inline my-document --reference "config" Inlining [Configuration](config) into 'my-document' Done ``` ### Dry Run (`--dry-run`) Preview what would happen: ``` bash $ iwe inline my-document --reference "config" --dry-run Would inline [Configuration](config) into 'my-document' Would delete 'config' Would update 2 additional document(s) ``` ### Keys Output (`-f keys`) Print affected document keys: ``` bash $ iwe inline my-document --block 1 -f keys my-document introduction other-referencing-doc ``` ## Examples ``` bash # List all inclusion links with numbers iwe inline notes/index --list # Inline by reference key iwe inline notes/index --reference "architecture" # Inline by block number (unambiguous) iwe inline notes/index --block 1 # Preview changes without writing iwe inline notes/index --reference "design" --dry-run # Keep the target document after inlining iwe inline notes/index --block 2 --keep-target # Inline as blockquote instead of section iwe inline notes/index --reference "notes" --as-quote # Use a specific action from config iwe inline notes/index --reference "design" --action "inline_quote" ``` ## Use Cases ### Consolidating Documents Merge related documents into a single comprehensive document: ``` bash # See what references exist iwe inline comprehensive-guide --list # Inline each section iwe inline comprehensive-guide --reference "intro" iwe inline comprehensive-guide --reference "setup" iwe inline comprehensive-guide --reference "usage" ``` ### Embedding Quotes Add cited content as blockquotes: ``` bash iwe inline article --reference "source-material" --as-quote --keep-target ``` ### Preview Impact Check what would be affected before inlining: ``` bash # See all affected documents iwe inline index --reference "shared-doc" --dry-run # Get keys for further analysis iwe inline index --reference "shared-doc" -f keys --dry-run ``` ### Scripting Use with other commands: ``` bash # Inline all references in a document iwe inline my-doc --list | while read line; do NUM=$(echo "$line" | cut -d: -f1) iwe inline my-doc --block "$NUM" --quiet done ``` ## Error Handling The command fails with an error if: - The document does not exist - No reference matches the provided key/title (with `--reference`) - Multiple references match (use `--block` instead) - The block number is out of range (with `--block`) - Must specify `--reference`, `--block`, or `--list` ## Technical Notes - When inlining as a section, header levels are adjusted appropriately - When deleting the target, all other references to it are cleaned up - Inline references (not inclusion links) are converted to plain text - The command uses the same inline logic as the LSP code action --- # IWE Delete URL: https://iwe.md/docs/agentic/delete/ Delete one or many documents and clean up references to them across the knowledge base. ## Usage ``` bash iwe delete <KEY> [OPTIONS] iwe delete --filter "EXPR" [OPTIONS] iwe delete <KEY> --filter "EXPR" [OPTIONS] ``` Either a positional `KEY` or `--filter` is required. When both are given, the union is deleted. ## Arguments | Argument | Description | | -------- | ---------------------------------------------------- | | `<KEY>` | Document key to delete (sugar for `--filter '$key: K'`). | ## Options | Flag | Description | Default | | -------------------- | -------------------------------------------------------------------------------------------- | ---------- | | `--filter <EXPR>` | Inline YAML filter expression. See [Query Language](/docs/concepts/query-language/). | none | | `--expect <ARG>` | Document-level guard: assert the number of matched documents. `ARG` is `N` or `{ min: M, max: N }`. | none | | `--strict` | Require the `--expect` guard. Aborts before deleting if it is missing. Exempt under `--dry-run`. | false | | `--dry-run` | Preview changes without writing to disk. | false | | `--quiet` | Suppress progress output. | false | | `-f, --format <FMT>` | Output format: `markdown` or `keys`. `keys` prints affected keys (one per line). | `markdown` | `delete` does not prompt before writing. Use `--dry-run` to preview the matched set before applying. ## Guards and strict mode `--expect` asserts the number of documents the operation will delete. On violation nothing is deleted and the error lists each matched document. Under `--strict`, a missing `--expect` is an error before anything runs; `--dry-run` is exempt — it is how the count is learned. See [`expect` guards](/docs/concepts/query-language/#expect-guards). ``` bash # Learn the count, then pin it iwe delete --filter 'status: draft' --strict --dry-run iwe delete --filter 'status: draft' --strict --expect '{ max: 5 }' ``` ## How it works 1. **Resolve the target set** — positional `KEY` and `--filter` matches are unioned, deduplicated, and sorted. 2. **Delete the document files** — every target is removed from the filesystem. 3. **Remove inclusion links** — [Inclusion Links](/docs/concepts/inclusion-links/) pointing at any deleted document are dropped. 4. **Convert inline references** — inline links to deleted documents become plain text (link text is preserved). 5. **Maintain integrity** — the operation runs once across the whole matched set; no broken references remain after. The reference cleanup step runs once over the union of targets, not per document. Documents that reference each other are handled correctly even when both are in the matched set. ## Reference cleanup ### Inclusion Links Before: ``` markdown # Index [Overview](overview) [Deleted Topic](deleted-topic) [Other Topic](other-topic) ``` After deleting `deleted-topic`: ``` markdown # Index [Overview](overview) [Other Topic](other-topic) ``` ### Inline Links Before: ``` markdown For more details, see [Deleted Topic](deleted-topic) and [Other](other). ``` After deleting `deleted-topic`: ``` markdown For more details, see Deleted Topic and [Other](other). ``` ## Output modes ### Default (`-f markdown`) Shows progress as documents are deleted: ``` bash $ iwe delete my-document Deleting 'my-document' Updated 2 document(s) ``` ### Dry run (`--dry-run`) Preview without writing: ``` bash $ iwe delete my-document --dry-run Would delete 'my-document' Would update 2 document(s) index overview ``` ### Keys (`-f keys`) Print affected document keys (the deleted target plus every doc whose references were rewritten): ``` bash $ iwe delete my-document -f keys my-document index overview ``` Suitable for scripting. `--dry-run` combines with `-f keys` to preview affected keys. ### Quiet (`--quiet`) Suppress all non-error output: ``` bash $ iwe delete my-document --quiet ``` ## Examples ``` bash # Single doc iwe delete old-notes # Preview first iwe delete obsolete-doc --dry-run # Bulk delete by filter iwe delete --filter 'status: archived' # Bulk delete with preview iwe delete --filter 'status: archived' --dry-run # Delete every descendant of a hub (unbounded) iwe delete --filter '$includedBy: { match: { $key: archive/2024 } }' # Affected keys for downstream processing iwe delete my-doc -f keys --dry-run ``` ## Use cases ### Cleaning up obsolete documents ``` bash # Check what would be affected iwe delete old-feature --dry-run # Delete if satisfied iwe delete old-feature ``` ### Bulk archival cleanup Use a filter to match many documents at once: ``` bash # Archive everything under a hub, then verify iwe delete --filter 'status: archived' --dry-run -f keys > affected.txt iwe delete --filter 'status: archived' --quiet ``` ### Pipeline integration Pair with [`iwe find`](/docs/agentic/find/) for ad-hoc selections: ``` bash iwe find temp -f keys | while read key; do iwe delete "$key" --quiet done ``` For a single-pass equivalent, prefer `--filter`: ``` bash iwe delete --filter '$key: { $in: [a, b, c] }' ``` ## Deprecated aliases | Deprecated | Use instead | | ---------- | ---------------------------- | | `--keys` | `-f keys` (equivalent; `--keys` is still accepted silently) | ## Error handling The command fails (exit code 1) when: - Neither a positional `KEY` nor `--filter` is provided. - The `--filter` expression cannot be parsed. - The `--expect` guard is violated, or `--strict` is given without `--expect`. - A target document does not exist. - Filesystem permissions prevent writing. ## Technical notes - The operation is atomic over the full matched set — either every change succeeds or none are applied. - Inclusion links to deleted documents are removed entirely; inline references are converted to plain text (link text preserved). - `--dry-run` prints the would-be changes and exits without touching disk. ## Related - [Query Language](/docs/concepts/query-language/) — filter syntax. - [`iwe find`](/docs/agentic/find/) — preview which documents a filter selects. - [`iwe count`](/docs/agentic/count/) — count the matched set before deleting. --- # IWE Rename URL: https://iwe.md/docs/agentic/rename/ Rename a document and update all references to it across the knowledge base. ## Usage ``` bash iwe rename <OLD_KEY> <NEW_KEY> [OPTIONS] ``` ## Arguments | Argument | Description | | ----------- | -------------------- | | `<OLD_KEY>` | Current document key | | `<NEW_KEY>` | New document key | ## Options | Flag | Description | | ----------- | ------------------------------------------- | | `--dry-run` | Preview changes without writing to disk | | `--quiet` | Suppress progress output | | `-f, --format <FMT>` | Output format: `markdown` (default) or `keys` (print affected document keys, one per line) | ## How It Works The `rename` command performs a safe document rename with full reference tracking: 1. **Renames the document file** - Moves the document from old key to new key 2. **Updates [Inclusion Links](/docs/concepts/inclusion-links/)** - All inclusion links pointing to the old key are updated 3. **Updates inline links** - All inline references to the old key are updated 4. **Maintains integrity** - Ensures no broken references after renaming ## Output Modes ### Default Output Shows progress and summary: ``` bash $ iwe rename old-document new-document Renaming 'old-document' to 'new-document' Updated 3 document(s) ``` ### Dry Run (`--dry-run`) Preview what would happen without making changes: ``` bash $ iwe rename old-document new-document --dry-run Would rename 'old-document' to 'new-document' Would update 3 document(s) index overview related-topic ``` ### Keys Output (`-f keys`) Print affected document keys for scripting: ``` bash $ iwe rename old-document new-document -f keys old-document new-document index overview related-topic ``` ### Quiet Mode (`--quiet`) Suppress all output except errors: ``` bash $ iwe rename old-document new-document --quiet ``` ## Examples ``` bash # Basic rename iwe rename my-note renamed-note # Preview changes first iwe rename old-key new-key --dry-run # Get affected keys for processing iwe rename document new-document -f keys # Silent rename (for scripts) iwe rename doc new-doc --quiet ``` ## Use Cases ### Reorganizing Knowledge Base Rename documents to follow a new naming convention: ``` bash # Check what would be affected iwe rename user-auth authentication --dry-run # Perform the rename iwe rename user-auth authentication ``` ### Moving to Subdirectories Move a document into a subdirectory: ``` bash # Move from root to subdirectory iwe rename config settings/config ``` ### Scripting Use with other commands for batch operations: ``` bash # Get list of affected documents for further processing iwe rename old-api api-v2 -f keys | while read key; do echo "Affected: $key" done ``` ## Error Handling The command fails with an error if: - The source document does not exist - The target key already exists - There are filesystem permission issues ## Technical Notes - References include both inclusion links and inline links - The operation is atomic - either all changes succeed or none are applied - Directory structure is preserved when renaming with path components --- # IWE Update URL: https://iwe.md/docs/agentic/update/ Modify existing documents. `update` has two mutually-exclusive modes: - **Body-overwrite** — replace the full markdown content of one document. - **Mutation** — apply frontmatter operators (`$set` / `$unset`) and block operators (`$replace`, `$replaceText`, `$insertBefore`, `$insertAfter`, `$append`, `$delete`) to one or many documents matched by a filter. Combining body flags with mutation flags in one invocation is a parse-time error. ## Usage ``` bash # Body-overwrite mode — replace one document's content iwe update -k <KEY> -c <CONTENT> iwe update -k <KEY> -c - # read content from stdin # Mutation mode — frontmatter and block edits on matched documents iwe update -k <KEY> --set FIELD=VALUE [--set ...] [--unset FIELD ...] iwe update --filter "EXPR" --set FIELD=VALUE [--set ...] [--unset FIELD ...] iwe update --filter "EXPR" --replace-text "{ <selector>, to: ... }" ``` ## Options | Flag | Description | Mode | | -------------------------- | ------------------------------------------------------------------------------------------------------------ | ------------------ | | `-k, --key <KEY>` | Document key. Required for body-overwrite. Optional in mutation mode (combined with `--filter` via AND). Unknown keys are an error. | both | | `-c, --content <STR>` | The complete document. Use `-` to read from stdin. Replaces the existing frontmatter block when it carries one of its own. | body-overwrite | | `--filter <EXPR>` | Inline YAML filter. Required if `-k` omitted in mutation mode. | mutation | | `--set <FIELD=VALUE>` | `$set` assignment. `VALUE` is parsed as a YAML scalar. Repeatable. | mutation | | `--unset <FIELD>` | `$unset` field. Repeatable. | mutation | | `--replace <ARG>` | `$replace`: replace each selected block. `ARG` is `{ <selector>, content: <markdown> }`. | mutation | | `--replace-text <ARG>` | `$replaceText`: rewrite own text of each selected block. `ARG` is `{ <selector>, from: X, to: Y }`; omit `from` and `to` replaces the entire own text. | mutation | | `--insert-before <ARG>` | `$insertBefore`: insert sibling content before each selected block. `ARG` is `{ <selector>, content: <markdown> }`. | mutation | | `--insert-after <ARG>` | `$insertAfter`: insert sibling content after each selected block. `ARG` is `{ <selector>, content: <markdown> }`. | mutation | | `--append <ARG>` | `$append`: append child content to each selected container. `ARG` is `{ <selector>, content: <markdown> }`. | mutation | | `--delete <ARG>` | `$delete`: remove each selected block. `ARG` is the `{ <selector> }` mapping (`{}` selects every block). | mutation | | `--expect <ARG>` | Document-level guard: assert the number of matched documents. `ARG` is `N` or `{ min: M, max: N }`. | mutation | | `--strict` | Require an `expect` guard on every mutating application (document-level `--expect` and each block operator's `expect`). Aborts before writing if any is missing. Exempt under `--dry-run`. | mutation | | `--dry-run` | Preview changes without writing. | both | | `--quiet` | Suppress progress output. | both | ## Body-overwrite mode `update -k KEY -c CONTENT` overwrites the document at `<KEY>` with the provided content. The graph is not mutated by `update` itself; the file change is picked up on the next read. Use [`iwe normalize`](/docs/cli/normalize/) afterward if you want to canonicalize the output. ``` bash # Replace a doc with a fixed string iwe update -k notes/draft -c "# Draft\n\nNew content." # Pipe content from another command cat new-content.md | iwe update -k notes/draft -c - # Preview without writing iwe update -k notes/draft -c "..." --dry-run ``` `-c` is the **complete document**. If the content you pass begins with its own frontmatter block, that block replaces the existing one. If it does not, the existing document's frontmatter is preserved above it, so a body-only rewrite keeps the metadata untouched. ``` bash # Body only -- the existing frontmatter block stays iwe update -k notes/draft -c "# Draft New content." # Complete document -- the frontmatter block is replaced too iwe update -k notes/draft -c '--- type: page --- # Draft New content.' ``` This mode is also the escape hatch when block predicates cannot address a target — two byte-identical sibling blocks, for instance, are indistinguishable to any predicate. To edit individual frontmatter fields rather than the whole block, use mutation mode's `--set` / `--unset`. ## Mutation mode In mutation mode, `update` builds one `update` operation document of the [Query Language](/docs/concepts/query-language/) and applies it to every document matched by the filter. The selector can be a single key (`-k`), a filter expression (`--filter`), or both (ANDed). All operators present in one invocation — frontmatter and block — validate and apply atomically: per document, every edit commits as one rewrite, and any validation failure anywhere aborts the whole operation before anything is written. `update` does not prompt before writing. Use `--dry-run` to inspect the matched set before applying. ### `--set FIELD=VALUE` `VALUE` is parsed as a YAML 1.2 scalar. Type inference follows YAML 1.2 rules: `5` is an integer, `true` is a boolean, `2026-04-26` is a date, `draft` is a string, `[a, b]` is a list. To force a string, quote it as YAML: `--set 'count="5"'`. Note that `yes`, `no`, `on`, and `off` are plain strings in YAML 1.2 (not booleans as in YAML 1.1). Frontmatter is re-serialized as YAML 1.2 after mutation, so quote styles may change on untouched fields. ### `--unset FIELD` Removes `FIELD` from every matched document's frontmatter. Absent on a given document is a no-op. ### Frontmatter examples ``` bash # Single-doc frontmatter mutation iwe update -k notes/draft --set status=published # Bulk mutation across a filter iwe update --filter 'status: draft' --set 'reviewed=true' # Multiple set / unset in one call iwe update --filter 'status: archived' --unset draft_notes --unset temporary # Promote drafts to published, with a date iwe update --filter 'status: draft' --set status=published --set 'published_at=2026-04-26' # Preview only — no writeback iwe update --filter 'status: draft' --set status=published --dry-run # Combine -k and --filter (ANDed) iwe update -k projects/alpha --filter 'status: draft' --set reviewed=true ``` ### Field names you cannot target `$`-prefixed names are the query language's operator vocabulary. Targeting a `$`-prefixed segment in a `--set` or `--unset` path — at any depth — is a parse-time error. Every other field name, `_hidden` and `@user` included, is an ordinary target and survives writeback untouched. See `docs/spec.md` §2.3 / §9.2 for the full rules. ### Conflict detection `$set` and `$unset` paths are checked for prefix overlap; conflicts (e.g. `--set a.b=1 --unset a`) are parse-time errors. Mapping values in `$set` replace wholesale — use dotted shorthand (`--set 'author.name=alice'`) to write a leaf without dropping siblings. ## Block operators Each block operator has its own flag whose argument is that operator's `{ <selector>, <payload> }` mapping: the `$`-prefixed keys are a [block predicate](/docs/concepts/query-language/#block-predicates) selecting the blocks, the bare keys carry the payload (`content`, `from`, `to`) and the optional `expect` guard. The operator name is the flag, so the `$replaceText:` wrapper is dropped. Full semantics — targets and coalescing, header retitle/dissolve behavior, `$replaceText` anchor rules, disjointness — are in [Block update operators](/docs/concepts/query-language/#block-update-operators). ``` bash # Retitle a section: the heading line changes, contents stay iwe update -k projects/roadmap --replace '{ $header: Goals, content: "## Goals 2026", expect: 1 }' # Remove a section wholesale: header and everything below it iwe update -k projects/roadmap --delete '{ $section: Goals, expect: 1 }' # Dissolve a header: the heading line goes, its contents merge into the enclosing section iwe update -k projects/roadmap --delete '{ $header: Goals, expect: 1 }' # Precision text edit, guarded to exactly one block iwe update --filter '$content: { $text: "Q3 Milestones" }' \ --replace-text '{ $within: Goals, $text: "Q3 Milestones", from: "Q3 Milestones", to: "Q3 2026 Milestones", expect: 1 }' # Rename a header (from omitted: to replaces the entire own text) iwe update -k projects/roadmap --replace-text '{ $header: Goals, to: Aims, expect: 1 }' # Bulk maintenance: delete every paragraph referencing a retired doc iwe update --filter '{}' --delete '{ $paragraph: { $references: archive/old-plan } }' # Bounded cleanup: refuse a runaway match iwe update --filter 'type: meeting' --delete '{ $paragraph: { $matches: "^DONE " }, expect: { max: 20 } }' # Clear a document's body, keep its frontmatter iwe update --filter '$key: notes/scratch' --delete '{}' ``` The block-operator flags combine with each other and with `--set` / `--unset`; every present flag becomes one operator entry in a single `update` document. Each operator flag appears at most once per invocation. Frontmatter operators apply to every filter-matched document regardless of block selections — to restrict a combined edit to documents where the block edit lands, conjoin `$content` into the filter with the same predicate: ``` bash # Stamp frontmatter and append a status line under every active # project's Status section, in one atomic operation iwe update --filter '{ type: project, status: active, $content: { $header: Status } }' \ --set reviewed=true \ --append '{ $header: Status, content: "Reviewed 2026-07-06." }' ``` Use [`iwe find --blocks`](/docs/agentic/find/#block-projection) with the same predicate to locate the targets (and learn the counts) before mutating. ## Guards and strict mode `--expect` asserts the number of matched documents the operation will write; each block operator's `expect` key asserts the number of targets it will act on (selected blocks for `--replace-text`). The two are independent quantities. Any violated guard aborts the whole operation before anything is written, listing the actual targets. See [`expect` guards](/docs/concepts/query-language/#expect-guards). Under `--strict`, every mutating application must carry its guard — a missing one is an error before anything runs. `--dry-run` is exempt: it is how the counts are learned. ``` bash # Learn the counts, then pin them iwe update --filter 'status: draft' --set status=published --strict --dry-run iwe update --filter 'status: draft' --set status=published --strict --expect 3 ``` ## Composition order Within one invocation, the operation document is built in this order: 1. **Filter** — narrows by `--filter` and `-k`. 2. **Sort** / **limit** — not exposed on the CLI for `update`; the engine processes the matched set in deterministic key order. 3. **Update** — frontmatter and block operators validated together, then applied atomically per document. ## Relationship to MCP The `iwe_update` MCP tool exposes body-overwrite. The full mutation surface — frontmatter and block operators — is the `iwe_query` tool, which accepts an `update` operation document verbatim and is always strict: every mutating application must carry its `expect` guard. See [MCP Server](/docs/agentic/mcp/). ## Related - [Query Language](/docs/concepts/query-language/) — filter syntax, block predicates, and update operator semantics. - [`iwe find`](/docs/agentic/find/) — preview which documents a filter selects; locate blocks with `--blocks`. - [`iwe count`](/docs/agentic/count/) — count the matched set before mutating. - [`iwe delete`](/docs/agentic/delete/) — remove the matched set instead of mutating it. --- # MCP Server URL: https://iwe.md/docs/agentic/mcp/ IWE provides an MCP (Model Context Protocol) server that gives AI agents direct access to your knowledge graph. The MCP server exposes the same operations as the CLI — search, retrieve, create, refactor — through a standardized protocol that AI tools can use natively. ## Setup You configure the server by pointing your AI tool to the `iwec` binary and setting the working directory to your knowledge graph. By default it communicates over stdio, which is what most editor and agent integrations expect. ### Transport options `iwec` accepts two flags that control how it serves the protocol: | Flag | Default | Description | | --------------------------- | ----------- | ------------------------------------------------------------ | | `--transport <stdio\|http>` | `stdio` | Serve over stdio, or over HTTP | | `--host <HOST>` | `127.0.0.1` | Address to bind to (only used with `--transport http`) | | `--port <PORT>` | `8000` | Port to listen on (only used with `--transport http`) | With `--transport http` the server listens for Streamable HTTP connections at `http://<host>:<port>/mcp`: ```bash iwec --transport http --port 8000 ``` By default the HTTP server binds to `127.0.0.1`, so it only accepts connections from the local machine. To accept connections from other machines, bind to a reachable address: ```bash iwec --transport http --host 0.0.0.0 --port 8000 ``` The server speaks plain HTTP, so put a reverse proxy in front of it for TLS or authentication when exposing it beyond localhost. ## Tools The MCP server exposes 14 tools for reading, writing, querying, and refactoring documents. ### Reading | Tool | Description | | -------------- | -------------------------------------------------------------- | | `iwe_find` | Search documents with fuzzy matching and relationship filters | | `iwe_retrieve` | Fetch documents with search seeds and graph expansion | | `iwe_tree` | View hierarchical document structure | | `iwe_stats` | Get knowledge graph statistics and broken link reports | | `iwe_squash` | Expand block references into a single flat document | ### Writing | Tool | Description | | -------------- | ---------------------------------------------------------- | | `iwe_create` | Create a document at a key from a complete markdown document | | `iwe_update` | Replace the full content of an existing document | | `iwe_delete` | Delete a document and clean up all references | #### `iwe_create` `iwe_create` takes the **complete document** and writes it verbatim — the server adds nothing and moves nothing. ``` json { "key": "people/ada", "content": "---\ntype: person\ntags: [pioneer]\n---\n\n# Ada Lovelace\n\nFirst English computer programmer.\n", "if_exists": "fail" } ``` → `{ "key": "people/ada", "created": true }` | Parameter | Description | | ------------ | ------------------------------------------------------------------------------------------------- | | `key` | Required. The document's stable identity — draw it from metadata (an entity name, a session date), not the title wording. Subdirectory keys such as `people/ada` are allowed; omit the file extension. | | `content` | Required. Frontmatter block first (when there is one), then the markdown, normally starting with `# Title`. | | `if_exists` | `"fail"` (default) errors when the key is taken; `"skip"` leaves the existing document untouched and returns `created: false`, which makes retries idempotent. | Because `content` is the whole file, frontmatter belongs at its first byte — that is where other tools read it. There is no separate frontmatter parameter to place it for you, and nothing is inserted above or below what you send. `iwe_update` has the same contract, with the opposite existence precondition. The `template`, `variables` and `frontmatter` parameters are reserved for a later template mode and are rejected today. #### Stats warnings A successful `iwe_create`, `iwe_update`, or `iwe_query` (`update` / `delete`) may carry **stats warnings** alongside its result — one warning content block per finding, of the form `<key> › <rule>: <message>`: - **orphan** — a page nothing links to (no inclusion or inline reference points at it). `index` pages (root `index` or any `<dir>/index`) are intentional entry points and are never reported as orphans. - **dangling-link** — a link whose target document does not exist. - **similar-page** — a just-authored page that is near-identical to another (only on `create` / `update`; see [Detecting similar pages](/docs/agentic/stats/#detecting-similar-pages)). These warnings are **advisory** — nothing is ever blocked by them (schema validation remains the only hard reject). Each finding is reported **once per session**, so the first mutation surfaces the store's standing issues and later calls surface only what changed. Resolve reported warnings before ending the session; each carries the fix in its message. The per-document `iwe_stats` result (call `iwe_stats` with a `key`) also carries a `similarPages` array — other documents near-identical to that page. ### Query | Tool | Description | | ----------- | -------------------------------------------------------------------- | | `iwe_query` | Run a [Query Language](/docs/concepts/query-language/) operation document verbatim | `iwe_query` takes an `operation` kind (`find`, `count`, `update`, or `delete`) and the operation `document` as a YAML string, plus an optional `dry_run` for the mutating kinds. It exposes the full query surface: frontmatter and graph filters, the `$content` block-membership operator, the [`search`](/docs/concepts/query-language/#search-find-only) stage on `find` (`search: { lexical, fuzzy }`), the `$content` / `$blocks` / `$matches` projection sources, and the block update operators (`$replace`, `$replaceText`, `$insertBefore`, `$insertAfter`, `$append`, `$delete`). `find` and `count` read; `update` applies frontmatter and block edits atomically per document; `delete` removes documents with reference cleanup. The tool is **always strict**: every mutating application must carry an `expect` guard — the document-level `expect` on `update` / `delete`, plus one per block operator — or the operation is refused with the missing guards named. Use `find` with `$blocks` / `$matches` to locate targets and learn the counts before mutating. See [Strict mode](/docs/concepts/query-language/#strict-mode). ### `iwe_retrieve` search and expansion `iwe_retrieve` assembles reading context in one call. Beyond the selector parameters and token budgets, it accepts: | Parameter | Description | | --------- | ----------- | | `search` | BM25 full-text seed query (lexical). Present → the tool searches the candidate set (`keys` / selector) and reads the ordered seeds. | | `fuzzy` | Fuzzy seed query on title + key. Combine with `search` to fuse (RRF). | | `expand` | Object over `includes` / `includedBy` / `references` / `referencedBy` → integer depths (`0` = unbounded, omitted key = not followed). Follows those edges out from each seed. Expansion is doc-only when omitted. | | `limit` | Cap the number of seed documents kept **before** expansion — top-N by relevance when searching, the first N of the selection otherwise (`0` = unlimited). | | `max_documents` | Cap the number of documents returned **after** expansion, trimming periphery documents first (`0` = unlimited). | Output is seeds first (relevance order), then expansion. The edge-list toggles (`backlinks`, `children`) are unchanged. The pre-existing `depth`, `context`, and `links` parameters are **deprecated** aliases for `expand`'s `includes` / `includedBy` / `references`; passing `expand` together with any of them is an error. ### Refactoring | Tool | Description | | ---------------- | ---------------------------------------------------------- | | `iwe_rename` | Rename a document key with automatic link updates | | `iwe_extract` | Extract a section into a new document with block reference | | `iwe_inline` | Replace a block reference with the referenced content | | `iwe_normalize` | Re-format all documents for consistent formatting | | `iwe_attach` | Attach a document to a target using configured actions | All write and refactoring tools support a `dry_run` parameter to preview changes before applying them. ### Selector parameters `iwe_find`, `iwe_retrieve`, and `iwe_tree` accept a structural selector embedded in their tool input: `in`, `in_any`, `not_in`, and `max_depth`. Each entry is either a bare key or `{ key, depth }`. These are a convenience for the most common selection patterns; the full query surface — `--filter`-style documents, `$`-prefixed graph operators, block predicates, frontmatter and block mutation — is `iwe_query`, documented in the [Query Language](/docs/concepts/query-language/) reference. ### Unknown keys `keys` on `iwe_retrieve` and `iwe_tree`, and `key` on every single-document tool and on the `review` / `refactor` prompts, name documents: an unknown key is an `invalid_params` error that lists every missing key. Selector anchors, `exclude`, and `keys` used as the candidate set for `search` / `fuzzy` are filters and may match nothing. See [Named Keys vs Filters](/docs/architecture/keys/#named-keys-vs-filters). ## Prompts The server provides three built-in prompts that guide AI agents through common workflows: | Prompt | Description | | ---------- | --------------------------------------------------------- | | `explore` | Get an overview of the knowledge graph with key statistics | | `review` | Review a specific document with full context | | `refactor` | Analyze a document and suggest restructuring operations | ## Resources The server exposes knowledge graph data as MCP resources: | URI | Description | | ------------------------- | -------------------------------------- | | `iwe://documents/{key}` | Individual document content | | `iwe://tree` | Full hierarchical document tree | | `iwe://stats` | Aggregate knowledge graph statistics | | `iwe://config` | Configuration with templates and actions | ## File watching The MCP server watches the knowledge graph directory for changes. When you edit markdown files in your editor, the server automatically updates its in-memory graph. There is no need to restart the server after making changes. --- # Editor setup: VS Code, Neovim, Zed and Helix URL: https://iwe.md/docs/editors/ IWE is a language server for Markdown notes. Install it in the editor you already use and your folder of notes becomes a knowledge base: links work like code, every note knows what links to it, search runs on headings, and a code action turns a section into its own note or folds it back in. Your AI agent reads and edits the same files through the `iwe` CLI and MCP server, so nothing is copied anywhere. Pick your editor: ## What you get in every editor - **Links you can follow.** Go to definition on a link opens the note. Go back and forward through everything you opened. - **Backlinks.** Find references lists every note that links to the current one. - **Search by heading.** Workspace symbol search finds a note by any part of its title. - **Extract and inline.** A code action moves a section into its own note and leaves a link behind, or merges a linked note back in. - **Rename that follows.** Rename a note and every link to it is updated. - **Format on save.** Titles, headers and link text stay consistent across the whole graph. - **Completion.** Type `[` and pick an existing note. Any other editor with Language Server Protocol support works too: run `iwes` as the language server for Markdown files. ## Your AI agent works on the same folder The editor is one way into the notes. [Claude Code, Codex](/docs/agentic/) or any client of the Model Context Protocol (MCP) uses the [`iwe` CLI](/docs/cli/) and the [MCP server](/docs/agentic/mcp/) to search the graph, pull a note with its surrounding structure into context, and make guarded edits. Same files, same links. See [agent memory in plain Markdown](/blog/memory-that-lives-in-your-project/) for how that fits together. ## Next steps - [How to use IWE](/docs/getting-started/usage/) for the everyday workflow - [Features](/docs/features/) for what the language server can do - [Get started](/get-started/) if you have not installed IWE yet - [iWe for Mac](/app/) if you would rather not configure an editor at all --- # VS Code markdown knowledge base extension URL: https://iwe.md/docs/editors/vscode/ The IWE extension makes VS Code a Markdown knowledge base. Links work like code: `F12` follows a link, `Shift+F12` lists every note that links here, `Cmd+T` finds a note by heading, and a code action extracts a section into its own note or inlines it back. Rename a file and every reference follows. Format on save keeps titles and links consistent. Your AI agent, whether Claude Code, Codex or any MCP client, works on the same folder through the [`iwe` CLI and MCP server](/docs/agentic/), so what you write in VS Code is what it reads. ## Install ### Install the extension Open the Extensions view (`Ctrl+Shift+X` / `Cmd+Shift+X`), search for "IWE" and click Install. Or from a terminal: ```sh code --install-extension IWE.iwe ``` The extension is also on the [Visual Studio Code Marketplace](https://marketplace.visualstudio.com/items?itemName=IWE.iwe). ### Make sure the language server is available The extension talks to the `iwes` language server. If the status bar does not show "IWE" after you open a Markdown file, install the server and restart VS Code: ```sh brew install iwe-org/iwe/iwe ``` Or `npm install -g @iwe-org/iwe` on any platform, or `cargo install iwes`. More options in [how to install](/docs/getting-started/installation/). ### Initialize your notes Open your notes folder in VS Code. An existing folder of Markdown works as it is. In the integrated terminal: ```sh iwe init ``` This writes `.iwe/config.toml`, fitted to the conventions your notes already use. ### Check it works Open any `.md` file. The status bar shows "IWE" when the language server is running. Press `F12` on a link to follow it. ## Keyboard shortcuts | IWE feature | Shortcut | Also available as | | --- | --- | --- | | Code actions (extract, inline, transform) | `Ctrl+.` / `Cmd+.` | Right-click → Quick Fix | | Go to definition (follow link) | `F12` or `Ctrl+Click` / `Cmd+Click` | Right-click → Go to Definition | | Find all references (backlinks) | `Shift+F12` | Right-click → Go to References | | Document symbols (outline) | `Ctrl+Shift+O` / `Cmd+Shift+O` | Command Palette → Go to Symbol | | Workspace search (by heading) | `Ctrl+T` / `Cmd+T` | Command Palette → Go to Symbol in Workspace | | Format document | `Shift+Alt+F` / `Shift+Option+F` | Right-click → Format Document | | Rename (file and every reference) | `F2` | Right-click → Rename Symbol | | Peek definition (preview a linked note) | `Alt+F12` / `Option+F12` | | | Peek references (preview backlinks) | `Shift+Alt+F12` / `Shift+Option+F12` | | | Completion (type `[` for note links) | `Ctrl+Space` | | All IWE commands are also in the Command Palette (`Ctrl+Shift+P` / `Cmd+Shift+P`): type "IWE". ## Everyday use ### Extract a section into its own note 1. Put the cursor on a header line, for example `## Section title`. 2. Press `Ctrl+.` / `Cmd+.`. 3. Choose "Extract section". VS Code creates a new file with the section and replaces it with a link. Choose "Inline" on that link to merge it back. ### Follow links and come back Press `F12` or `Ctrl+Click` / `Cmd+Click` on a link. `Alt+Left` / `Ctrl+-` returns to where you were. ### See what links here Press `Shift+F12` anywhere in a note. The references panel lists every note that links to it. Click a result to open it. ### Find a note by heading Press `Ctrl+T` / `Cmd+T` and type part of the title. Results show matching notes and sections. ## Configuration Per-workspace settings go in `.vscode/settings.json` in your notes folder: ```json { "iwe.enable": true, "iwe.trace.server": "off", "files.associations": { "*.md": "markdown" } } ``` User-level options are under Settings (`Ctrl+,` / `Cmd+,`), search "IWE". Project-level behaviour, such as formatting rules and note templates, lives in `.iwe/config.toml`; see [configuration](/docs/configuration/). ## Troubleshooting **The language server does not start.** Check that `iwes` is on your `PATH` (`which iwes`), restart VS Code, and look at Output → "IWE Language Server" for errors. **Features do nothing.** The folder needs a `.iwe/config.toml`; run `iwe init` in it. The file must have the `.md` extension. **It is slow on a large workspace.** Point IWE at the notes folder rather than a repository that contains it, or use the library path setting in `.iwe/config.toml`. **Debug logging.** Set `IWE_DEBUG=true` in the environment, restart VS Code, and include the `iwe.log` from the workspace when reporting an issue. Questions and bugs: [GitHub issues](https://github.com/iwe-org/iwe/issues) and [discussions](https://github.com/iwe-org/iwe/discussions). ## Next steps - [How to use IWE](/docs/getting-started/usage/) for the everyday workflow - [Connect your AI agent](/docs/agentic/mcp/) over MCP to the same notes - [Features](/docs/features/) for everything the language server does - [iWe for Mac](/app/) if you want an app instead of an editor --- # Neovim markdown knowledge base plugin URL: https://iwe.md/docs/editors/neovim/ The iwe.nvim plugin makes Neovim a Markdown knowledge base. Links work like code: `gd` follows a link, `gr` lists every note that links here, Telescope searches notes by heading, and a code action extracts a section into its own note or inlines it back. Rename a note and every reference follows. Format on save keeps titles and links consistent. Your AI agent, whether Claude Code, Codex or any MCP client, works on the same folder through the [`iwe` CLI and MCP server](/docs/agentic/), so what you write in Neovim is what it reads. ## Install ### Install the language server The plugin runs the `iwes` language server, which must be on your `PATH`: ```sh brew install iwe-org/iwe/iwe ``` Or `npm install -g @iwe-org/iwe` on any platform, or `cargo install iwes`. More options in [how to install](/docs/getting-started/installation/). ### Install the plugin With [lazy.nvim](https://github.com/folke/lazy.nvim): ```lua { "iwe-org/iwe.nvim", dependencies = { "nvim-lua/plenary.nvim", "nvim-telescope/telescope.nvim", }, config = function() require("iwe").setup() end, } ``` With packer.nvim: ```lua use { "iwe-org/iwe.nvim", requires = { "nvim-lua/plenary.nvim", "nvim-telescope/telescope.nvim" }, config = function() require("iwe").setup() end, } ``` With vim-plug: ```vim Plug 'nvim-lua/plenary.nvim' Plug 'nvim-telescope/telescope.nvim' Plug 'iwe-org/iwe.nvim' " after plug#end() lua require("iwe").setup() ``` Or clone it into your pack directory: ```sh git clone https://github.com/iwe-org/iwe.nvim.git ~/.local/share/nvim/site/pack/plugins/start/iwe.nvim ``` The plugin lives at [iwe-org/iwe.nvim](https://github.com/iwe-org/iwe.nvim). ### Initialize your notes An existing folder of Markdown works as it is. In the folder: ```sh iwe init ``` Or open Neovim there and run `:IWE init`. This writes `.iwe/config.toml`, fitted to the conventions your notes already use. ### Check it works Open a `.md` file and run `:checkhealth iwe`. It checks the plugin, the `iwes` binary, the LSP attachment, Telescope and the project configuration. `:LspInfo` shows the server attached to the buffer. ## Keybindings Defaults set by the plugin. All of them can be changed in `setup()`. | IWE feature | Shortcut | Mode | What it does | | --- | --- | --- | --- | | Code actions | `<leader>ca` | Normal | Extract, inline and transform actions for the element under the cursor | | Go to definition | `gd` | Normal | Follow the link under the cursor | | Find references | `gr` | Normal | Backlinks to the current note | | Document symbols | `<leader>ds` | Normal | Outline of the current note | | Workspace search | `<leader>ws` | Normal | Search all notes by heading with Telescope | | Format document | `<leader>f` | Normal, Visual | Format the note | | Rename | `<leader>rn` | Normal | Rename the note and update every reference | | Hover | `K` | Normal | Preview the linked note | | Next, previous diagnostic | `]d`, `[d` | Normal | Jump between diagnostics | Telescope pickers: | Command | Shortcut | What it shows | | --- | --- | --- | | `:Telescope iwe search` | `<leader>ws` | All notes, searched by heading | | `:Telescope iwe backlinks` | `<leader>wb` | Notes linking to the current one | | `:Telescope iwe links` | `<leader>wl` | Links in the current note | Inside a picker, `<CR>` opens the note, `<C-x>` and `<C-v>` open it in a split, `<C-t>` in a new tab, `<C-u>` and `<C-d>` scroll the preview, and `<C-q>` sends the results to the quickfix list. ## Everyday use ### Extract a section into its own note 1. Put the cursor on a header line, for example `## Section title`. 2. Press `<leader>ca`. 3. Choose "Extract section". The section becomes a new note and a link takes its place. Choose "Inline" on that link to merge it back. ### Follow a link in nvim With IWE attached, `gd` on a Markdown link opens the linked note, whether the link is `[text](note)`, `[text](note.md)` or a link into a subfolder. `<C-o>` jumps back to where you were and `<C-i>` forward again, so you can walk a chain of notes and retrace it. Without a language server, Neovim's built-in `gf` opens the file path under the cursor. It only works when the cursor is on the path itself, and IWE-style links without an extension need `:set suffixesadd+=.md`. `gr` does the reverse: it lists every note that links to the one you are in. ### See what links here `gr` or `<leader>wb` lists every note that links to the current one. ### Find a note by heading `<leader>ws` opens the search picker. Type part of a title; the preview shows the note without opening it. ## Configuration The defaults are a good start. Everything below is optional. ```lua require("iwe").setup({ lsp = { cmd = { "iwes" }, settings = { iwe = { debug = false }, }, }, keybindings = { enable = true, code_action = "<leader>ca", goto_definition = "gd", find_references = "gr", document_symbols = "<leader>ds", workspace_search = "<leader>ws", format_document = "<leader>f", rename_symbol = "<leader>rn", }, telescope = { enable = true, search = { layout_strategy = "horizontal", layout_config = { preview_width = 0.6 }, }, }, }) ``` ### Your own keybindings Set `keybindings.enable = false` and map what you like: ```lua local map = vim.keymap.set map("n", "<leader>ia", vim.lsp.buf.code_action, { desc = "IWE code actions" }) map("n", "<leader>ig", vim.lsp.buf.definition, { desc = "IWE follow link" }) map("n", "<leader>ir", vim.lsp.buf.references, { desc = "IWE backlinks" }) map("n", "<leader>is", "<cmd>Telescope iwe search<cr>", { desc = "IWE search" }) map("n", "<leader>ib", "<cmd>Telescope iwe backlinks<cr>", { desc = "IWE backlinks" }) map("n", "<leader>if", vim.lsp.buf.format, { desc = "IWE format" }) map("n", "<leader>in", vim.lsp.buf.rename, { desc = "IWE rename" }) ``` With which-key.nvim, register the group so the shortcuts are discoverable: ```lua require("which-key").register({ ["<leader>i"] = { name = "IWE", a = "Code actions", g = "Follow link", r = "Backlinks", s = "Search", b = "Backlinks", f = "Format", n = "Rename", }, }) ``` ### Root directory and capabilities By default the server starts at the nearest `.iwe` directory. To change that, or to pass completion capabilities from nvim-cmp: ```lua require("iwe").setup({ lsp = { cmd = { "iwes" }, filetypes = { "markdown" }, root_dir = function(fname) return require("lspconfig.util").root_pattern(".iwe")(fname) or require("lspconfig.util").find_git_ancestor(fname) or vim.loop.os_homedir() end, capabilities = require("cmp_nvim_lsp").default_capabilities(), handlers = { ["textDocument/hover"] = vim.lsp.with(vim.lsp.handlers.hover, { border = "rounded" }), }, }, }) ``` ### Without the plugin The plugin is a convenience. Plain LSP works too: ```lua vim.api.nvim_create_autocmd("FileType", { pattern = "markdown", callback = function(args) vim.lsp.start({ name = "iwes", cmd = { "iwes" }, root_dir = vim.fs.root(args.buf, { ".iwe" }), flags = { debounce_text_changes = 500 }, }) end, }) ``` ## Works well with - [nvim-cmp](https://github.com/hrsh7th/nvim-cmp) with the `nvim_lsp` source gives link completion when you type `[`. - [nvim-treesitter](https://github.com/nvim-treesitter/nvim-treesitter) with `markdown` and `markdown_inline` for highlighting. - [render-markdown.nvim](https://github.com/MeanderingProgrammer/render-markdown.nvim) for in-buffer rendering. IWE does not render a Markdown preview. `K` on a link shows the linked note's text in a hover window. For a rendered view, use render-markdown.nvim inside the buffer, or [mpls](https://github.com/mhersson/mpls) for a live browser preview next to IWE. ## Troubleshooting **The server does not start.** `which iwes` must find the binary. `:LspInfo` shows whether it attached; `:LspLog` shows why not. **Telescope pickers are missing.** Load the extension: `require("telescope").load_extension("iwe")`. **A keybinding does nothing.** Check that default keybindings are enabled, then `:verbose map <key>` to see what else claims it. **Debug logging.** Start with `IWE_DEBUG=true nvim`, or set `vim.env.IWE_DEBUG = "true"` and `:LspRestart`. The log is `iwe.log` in the working directory. Questions and bugs: [iwe.nvim issues](https://github.com/iwe-org/iwe.nvim/issues), [IWE issues](https://github.com/iwe-org/iwe/issues) and [discussions](https://github.com/iwe-org/iwe/discussions). ## Next steps - [How to use IWE](/docs/getting-started/usage/) for the everyday workflow - [Connect your AI agent](/docs/agentic/mcp/) over MCP to the same notes - [Features](/docs/features/) for everything the language server does - [iWe for Mac](/app/) if you want an app alongside the terminal --- # Helix markdown knowledge base with LSP URL: https://iwe.md/docs/editors/helix/ Helix talks to language servers out of the box, so IWE needs no plugin. Links work like code: `gd` follows a link, `gr` lists every note that links here, `space` + `S` finds a note by heading, and a code action extracts a section into its own note or inlines it back. Rename a note and every reference follows. Format on save keeps titles and links consistent. Your AI agent, whether Claude Code, Codex or any MCP client, works on the same folder through the [`iwe` CLI and MCP server](/docs/agentic/), so what you write in Helix is what it reads. ## Install ### Install the language server The `iwes` binary must be on your `PATH`: ```sh brew install iwe-org/iwe/iwe ``` Or `npm install -g @iwe-org/iwe` on any platform, or `cargo install iwes`. Arch users can take it from the AUR. More options in [how to install](/docs/getting-started/installation/). ### Register it for Markdown Add this to `~/.config/helix/languages.toml`, creating the file if needed: ```toml [language-server.iwe] command = "iwes" [[language]] name = "markdown" language-servers = ["iwe"] auto-format = true ``` To keep another Markdown server alongside it, list both: `language-servers = ["iwe", "marksman"]`. Inlay hints show the parent note and link counts next to each heading. Turn them on in `~/.config/helix/config.toml`: ```toml [editor.lsp] display-inlay-hints = true ``` ### Initialize your notes An existing folder of Markdown works as it is. In the folder: ```sh iwe init ``` This writes `.iwe/config.toml`, fitted to the conventions your notes already use. ### Check it works ```sh hx --health markdown ``` lists the configured language servers for Markdown. Open a `.md` file and press `gd` on a link. ## Keybindings Helix's standard LSP keys drive everything: | IWE feature | Keybinding | | --- | --- | | Go to definition (follow link) | `gd` | | Find references (backlinks) | `gr` | | Hover preview of a linked note | `space` + `k` | | Code actions (extract, inline, transform) | `space` + `a` | | Document symbols (outline) | `space` + `s` | | Workspace symbols (search by heading) | `space` + `S` | | Rename (file and every reference) | `space` + `r` | | Format document | `:format` | ## Everyday use ### Extract a section into its own note 1. Put the cursor on a header line, for example `## Section title`. 2. Press `space` + `a`. 3. Choose "Extract section". The section becomes a new note and a link takes its place. Choose "Inline" on that link to merge it back. After an inline, the buffer of the deleted note stays open; close it with `:buffer-close`. ### Follow links and come back `gd` on a link opens the note. `Ctrl+o` returns to where you were. ### Peek at a linked note `space` + `k` on a link shows the linked note in a hover popup. More in [hover preview](/docs/features/hover-preview/). ## Helix markdown preview Helix has no built-in rendered Markdown preview, and IWE does not add one: it works on the structure of your notes, not on how they look. For a live rendered preview, run [mpls](https://github.com/mhersson/mpls), the Markdown Preview Language Server, next to IWE. It opens the preview in your browser and updates it as you type. Install `mpls` by following its README, then list both servers in `~/.config/helix/languages.toml`: ```toml [language-server.iwe] command = "iwes" [language-server.mpls] command = "mpls" [[language]] name = "markdown" language-servers = ["iwe", "mpls"] auto-format = true ``` Open the preview with `:lsp-workspace-command open-preview`, or bind it to a key in `~/.config/helix/config.toml`: ```toml [keys.normal] "C-m" = ":lsp-workspace-command open-preview" ``` IWE keeps links, backlinks and refactoring; mpls renders the page. In upstream Helix the preview does not follow you when you switch files; the mpls README explains why and what to do about it. ## Helix LSP for Markdown Helix starts a language server for each file type listed in `languages.toml`. For Markdown, the choice is which servers to run: | Server | What it adds | | --- | --- | | IWE (`iwes`) | Links and backlinks, heading search, extract and inline, rename with reference updates, formatting, and the same notes for your AI agent | | [mpls](https://github.com/mhersson/mpls) | Live rendered preview in the browser | | [Marksman](https://github.com/artempyanykh/marksman), [markdown-oxide](https://github.com/Feel-ix-343/markdown-oxide) | Other Markdown language servers; see [IWE vs markdown-oxide vs Marksman](/docs/compare/iwe-vs-markdown-oxide-vs-marksman/) | You can list several in `language-servers`; Helix asks each one for what it supports. ## Configuration ### Only for your notes You probably do not want IWE in every README you open. Helix reads a project-local `.helix/languages.toml`, so put the language server configuration from the install step in `.helix/languages.toml` at the root of your notes folder instead of the global file. ### Tables and auto-format If you keep Markdown tables and the formatter changes them in ways you dislike, turn formatting off for Markdown and run `:format` by hand: ```toml [[language]] name = "markdown" language-servers = ["iwe"] auto-format = false ``` Formatting rules and note templates live in `.iwe/config.toml`; see [configuration](/docs/configuration/). ## Troubleshooting **Nothing happens on `gd`.** `hx --health markdown` must list `iwe`, and `which iwes` must find the binary. Check `~/.cache/helix/helix.log` for server errors. **Features do nothing.** The folder needs a `.iwe/config.toml`; run `iwe init` in it. Questions and bugs: [GitHub issues](https://github.com/iwe-org/iwe/issues) and the [Helix integration discussion](https://github.com/iwe-org/iwe/discussions/25). ## Next steps - [How to use IWE](/docs/getting-started/usage/) for the everyday workflow - [Connect your AI agent](/docs/agentic/mcp/) over MCP to the same notes - [Features](/docs/features/) for everything the language server does - [iWe for Mac](/app/) if you want an app alongside the terminal --- # Zed markdown knowledge base extension URL: https://iwe.md/docs/editors/zed/ The IWE extension makes Zed a Markdown knowledge base. Links work like code: `F12` follows a link, `Shift+F12` lists every note that links here, `Cmd+T` finds a note by heading, and a code action extracts a section into its own note or inlines it back. Rename a note and every reference follows. Format on save keeps titles and links consistent. Your AI agent, whether Claude Code, Codex or any MCP client, works on the same folder through the [`iwe` CLI and MCP server](/docs/agentic/), so what you write in Zed is what it reads. ## Install ### Install the extension Open Extensions from the main menu (`Cmd+Shift+X`), search for "IWE" and click Install. The extension downloads the `iwes` language server the first time it starts. If you would rather manage the binary yourself, install it and make sure it is on your `PATH`; the extension uses the system binary when it finds one: ```sh brew install iwe-org/iwe/iwe ``` Or `npm install -g @iwe-org/iwe`, or `cargo install iwes`. More options in [how to install](/docs/getting-started/installation/). ### Initialize your notes An existing folder of Markdown works as it is. In the folder: ```sh iwe init ``` This writes `.iwe/config.toml`, fitted to the conventions your notes already use. ### Check it works Open a `.md` file and press `F12` on a link. If nothing happens, see troubleshooting below. ## Keyboard shortcuts | IWE feature | Shortcut | | --- | --- | | Go to definition (follow link) | `F12` or `Cmd+Click` | | Find references (backlinks) | `Shift+F12` | | Code actions (extract, inline, transform) | `Cmd+.` | | Format document | `Cmd+Shift+I` | | Document symbols (outline) | `Cmd+Shift+O` | | Workspace search (by heading) | `Cmd+T` | | Rename (file and every reference) | `F2` | ## Everyday use ### Extract a section into its own note 1. Put the cursor on a header line, for example `## Section title`. 2. Press `Cmd+.`. 3. Choose "Extract section". The section becomes a new note and a link takes its place. Choose "Inline" on that link to merge it back. ### Follow links and come back `F12` or `Cmd+Click` on a link opens the note. `Ctrl+-` returns to where you were. ### See what links here `Shift+F12` anywhere in a note lists every note that links to it. ## Zed markdown: what's built in and what the extension adds Zed already handles Markdown on its own: syntax highlighting, and a rendered preview from the command palette with `markdown: open preview` or `markdown: open preview to the side`. The IWE extension does not replace either. It adds what Zed doesn't do for `.md` files: | You want to | Zed alone | With the IWE extension | | --- | --- | --- | | See the rendered page | `markdown: open preview` | Same, IWE adds no preview of its own | | Follow a link to another note | Opens only if you navigate to the file yourself | `F12` or `Cmd+Click` | | See which notes link here | No | `Shift+F12` | | Find a note by heading | File finder by name | `Cmd+T` searches every heading | | Split a long note | Cut and paste | Extract a section into its own note, links kept | | Rename a note | References break | `F2` renames the file and every reference | | Let your AI agent read the same notes | No | The `iwe` CLI and MCP server work on the same folder | If you already use another Markdown extension in Zed, see [IWE vs markdown-oxide vs Marksman](/docs/compare/iwe-vs-markdown-oxide-vs-marksman/), or [Obsidian vs an editor setup](/docs/compare/obsidian-vs-editor/) if you're deciding between Zed and Obsidian. ## Configuration ### Only for your notes To run IWE for your notes folder and not for every Markdown file you open, put the language server configuration in `.zed/settings.json` at the root of the notes folder: ```json { "lsp": { "iwe": { "binary": { "path": "iwes" } } } } ``` Formatting rules and note templates live in `.iwe/config.toml`; see [configuration](/docs/configuration/). ## Troubleshooting **The extension does nothing.** Check it is installed and enabled, that the notes folder has `.iwe/config.toml` (run `iwe init`), and look at Zed's language server logs for errors. **The binary was not found.** If the download fails, install `iwes` yourself as above, make sure it is on your `PATH`, and restart Zed. **Platforms.** The extension supports macOS (Apple Silicon and Intel) and Linux (x86_64 and aarch64). Windows support is planned. Questions and bugs: [GitHub issues](https://github.com/iwe-org/iwe/issues) and the [Zed integration discussion](https://github.com/iwe-org/iwe/discussions/22). ## Next steps - [How to use IWE](/docs/getting-started/usage/) for the everyday workflow - [Connect your AI agent](/docs/agentic/mcp/) over MCP to the same notes - [Features](/docs/features/) for everything the language server does - [iWe for Mac](/app/) if you want an app instead of an editor --- # Concepts URL: https://iwe.md/docs/concepts/ Understanding the ideas and patterns behind effective note-taking with IWE. --- # Context bridge: AI agent memory you share in Markdown URL: https://iwe.md/docs/concepts/context-bridge/ IWE gives your AI agent a memory that you can read and edit too. Your notes stay plain Markdown files on your machine; you work on them in your editor or the [iWe Mac app](/app/), and agents such as Claude Code and Codex read and write the same files through the [CLI and MCP server](/docs/agentic/). That shared knowledge graph is what IWE calls the context bridge: one memory, two kinds of users. ## The Problem: Fragmented Context Knowledge workers face three interconnected problems: **Fragmented context** — Knowledge lives in scattered files, documents, and systems. No unified structure connects ideas across sources. **Retrieval ineffectiveness** — Traditional search finds text but misses relationships. You search for "authentication" and get results, but not the connection to your security architecture decisions or the login flow you designed. **Memory without iteration** — Each AI conversation starts fresh. Context built in one session doesn't carry to the next. Knowledge doesn't accumulate. These problems affect both humans and AI. And they share a common cause: treating knowledge as isolated text rather than connected structure. ## The Solution: Graph Structure IWE treats your knowledge as a graph. Documents connect through explicit relationships. Parents provide context. Children add detail. Links create paths between ideas. This structure enables navigation, not just search. When you or an AI agent retrieves a document, you can follow relationships to related knowledge. You can expand upward for context or downward for detail. You can trace connections across your entire knowledge base. The graph structure is the key insight: **relationships make knowledge navigable**. Traditional files are opaque — text to search through. A knowledge graph with explicit connections gives both humans and AI real navigation capability. You can follow paths, understand context, and retrieve related knowledge intelligently. ## The Context Bridge Your personal knowledge graph is a **context bridge** between your thinking and AI capabilities. When you write and organize knowledge, you're creating structure. When AI agents navigate that structure, they understand your knowledge the way you organized it — following the same relationships, seeing the same context. And when agents create documents, extract sections, or rename keys, they extend the same graph you built. This is genuine collaboration: - **You** write, organize, and shape knowledge through your editor - **AI agents** discover, retrieve, create, and restructure knowledge through CLI and MCP tools - **Both** work with the same graph, the same relationships, the same structure — reading *and* writing The knowledge graph connects your understanding to AI understanding. That's the bridge. ## Why Local Ownership Matters The context bridge stays under your control because IWE is local-first. Your knowledge graph lives on your machine. No cloud storage, no external processing, no vendor dependencies. You decide what context to share with AI tools. This matters because: **Privacy** — Your knowledge isn't uploaded anywhere. AI integration happens locally. **Control** — You own the bridge. You decide when and how AI accesses your knowledge. **Portability** — Plain markdown files. No proprietary formats. Your knowledge moves with you. **Persistence** — No subscription required. Your knowledge graph works offline forever. Local ownership means the context bridge is yours to maintain and share as you choose. ## The Dual-User Advantage Most tools optimize for either human use or AI use. IWE serves both: | You (via Editor) | AI Agents (via CLI / MCP) | |------------------|---------------------------| | Write and organize knowledge | Discover entry points (`iwe_find`) | | Navigate with go-to-definition | Retrieve with context (`iwe_retrieve`) | | Transform with code actions | Create and update documents (`iwe_create`, `iwe_update`) | | Search with fuzzy matching | Restructure the graph (`iwe_extract`, `iwe_rename`, `iwe_inline`, `iwe_normalize`) | Same knowledge base. Two access patterns. Both sides read *and* write. True collaboration. The advantage isn't just convenience — it's that both users build the same graph. When you create a relationship, AI can follow it. When an agent extracts a section into a new document, you can open it in your editor the next moment. Your work improves AI's capabilities, and AI's work extends knowledge you own. ## AI Memory Management IWE solves AI memory management for human-AI collaboration: **Unified context** — One knowledge graph instead of fragmented files **Structured retrieval** — Graph navigation instead of keyword search **Memory iteration** — Knowledge accumulates because both you and AI write to the graph, not just read from it **Relationship-aware** — AI follows connections you created and creates new ones **Local persistence** — Memory lives on your machine, available across sessions Unlike retrieval-only memory systems, AI agents don't just read your graph — they extend it. They can create documents, extract sections, rename keys, and normalize structure through the same tools you use. The graph grows through collaboration. This isn't AI storing its own data. It's AI navigating and contributing to your knowledge — knowledge you own, structure you and your agents shape together, context you control. --- # Inclusion Links URL: https://iwe.md/docs/concepts/inclusion-links/ When you place a markdown link on its own line, IWE treats it as a structural relationship—the current document becomes the "parent" of the linked document. This lets you organize knowledge into hierarchies without folders, and a single document can have multiple parents. ## Why Hierarchy Matters Hierarchy is one of the most natural ways humans organize knowledge. We instinctively break complex information into nested structures: - **Living things** are classified into kingdoms, classes, orders, and species - **Places** nest from continents to countries to cities to neighborhoods - **Organizations** have companies, departments, teams, and roles - **Projects** break down into epics, features, and tasks This isn't arbitrary—hierarchical thinking is how we manage complexity. It lets us zoom out to see the big picture or zoom in to focus on specifics—and switch between these views effortlessly. It provides context: knowing something is "under" a topic tells you what it relates to. What makes knowledge truly interesting is where hierarchies intersect. "Color Theory" belongs under both Art and Physics. "Game Theory" connects Mathematics and Economics. "Nutrition" bridges Chemistry and Health. These intersections are where insight lives. Good knowledge systems should support this natural way of thinking. ## The Problem Traditional ways of organizing information don't match how knowledge actually works. ### Directories: One Place Only File systems force everything into a single hierarchy. A note about "Meditation" must live in either `/health/` or `/productivity/`—it can't naturally exist in both. This leads to: - Arbitrary placement decisions - Broken references when folders change - Duplicates or "misc" folders when nothing fits But knowledge isn't a single tree—it's interconnected. ### Tags: Flexible but Shallow Tags allow multiple categories, but they lack structure: - No order or priority - No explanation of relationships - No hierarchy - No grouping within a category A note tagged `#health #productivity #mindfulness` doesn't explain how these topics relate or which one is primary. ## The Solution: Inclusion Links An **inclusion link** is a markdown link placed on its own line: ``` markdown # Photography [Composition](composition.md) [Lighting](lighting.md) [Post-Processing](post-processing.md) ``` When a link appears on its own line, it defines structure: "Photography" becomes the parent of the linked documents. This simple rule turns plain markdown into a structured, navigable system. ## What This Enables ### Multiple Contexts (Polyhierarchy) A document can belong to multiple parents. **In `health-practices.md`:** ``` markdown # Health Practices [Meditation](meditation.md) ``` **In `productivity-tools.md`:** ``` markdown # Productivity Tools [Meditation](meditation.md) ``` The same document appears in both contexts without duplication. ### Structure with Meaning Unlike tags, inclusion links allow ordering, grouping, and explanation: ``` markdown # Psychology ## Cognitive How we encode, store, and retrieve information [Memory](memory.md) Biases, heuristics, and rational choice [Decision Making](decision-making.md) ## Behavioral Cue, routine, reward loops [Habit Formation](habit-formation.md) ``` You're not just grouping items—you're adding context. ### Navigable Hierarchies By linking documents together, you naturally create paths through your knowledge: ``` Knowledge Base > Psychology > Cognitive > Memory Knowledge Base > Photography > Composition ``` This makes navigation and search more meaningful—you see not just what matches, but where it fits. ### Context Flows Downward When a document appears under a parent, it inherits meaning from that relationship. A "Memory" note under "Psychology" is understood differently than "Memory" under "Computer Architecture." The parent page can also add explicit context: ``` markdown # Cognitive Psychology Foundation of learning and recall [Memory](memory.md) Selective focus and its limits [Attention](attention.md) ``` These descriptions aren't part of the child documents—they live in the parent, explaining why each child belongs here. ### Flexible Level of Detail You can control how much information is visible. With `--depth 1`, you see only immediate children: ``` # Psychology - Memory - Decision Making - Habit Formation ``` With `--depth 3`, the full subtree expands: ``` # Psychology - Memory - Working Memory - Capacity Limits - Chunking Strategies - Long-term Memory - Decision Making - Habit Formation ``` Use [Extract Actions](/docs/code-actions/extract/) to move details into separate documents, or [Inline Notes](/docs/code-actions/inline/) to expand linked content in place. ## Inclusion Links vs Inline Links ### Inclusion Links (Structure) Inclusion links define parent-child relationships and are used for: - Navigation - Hierarchical traversal (`--depth`) - Structured views ### Inline Links (References) Links inside text create conceptual connections: ``` markdown # Habit Formation The [Habit Loop](habit-loop.md) consists of cue, routine, and reward. This process is driven by [Dopamine Pathways](dopamine.md) in the brain. ``` These links: - Create backlinks - Show relationships between ideas - Do not affect structure ### Why This Distinction Matters When retrieving content with depth: ``` bash iwe retrieve psychology --depth 2 ``` The `--depth` flag controls how many levels of inclusion links to expand. See [IWE Retrieve](/docs/agentic/retrieve/) for details. - Inclusion links expand into full content - Inline links remain references only This keeps structure clean while preserving connections between ideas. ## Summary Inclusion links give you structure without rigidity: - Documents can exist in multiple contexts - Ordering, grouping, and meaning are explicit - Navigation becomes contextual For a deeper understanding of how IWE represents these relationships internally, see [Data Model](/docs/architecture/data-model/). --- # How IWE Compares URL: https://iwe.md/docs/concepts/comparison/ IWE turns a directory of markdown files into a knowledge graph — a connected structure you browse from your editor and your AI queries from the command line. Same files, same links, two interfaces. That sentence is also the short answer to most questions on this page. The tools below are one of three things: a **markdown language server** (marksman, markdown-oxide), a **note-taking application** (Obsidian), or a **CLI notebook** (zk, telekasten.nvim). IWE overlaps all three because the graph is the product — the LSP server, the CLI, and the [MCP server](/docs/agentic/mcp/) are three views of one in-memory model. ## Which Tool Should You Use - **A markdown language server for almost any editor, with the least setup** — use **marksman**. It is the incumbent, it is small, and it does links, references, and diagnostics well. - **Obsidian-style daily notes, block references, and tags without leaving your editor** — use **markdown-oxide**. Natural-language dates are better there than in IWE today. - **A GUI, mobile apps, a plugin ecosystem, canvas, and a web clipper** — use **Obsidian**. - **A Zettelkasten CLI with templates and an fzf browser** — use **zk**. - **Notes as a graph you can query, refactor, and hand to an AI agent — from the editor *and* the command line** — use **IWE**. If you are weighing IWE against one specific tool, skip to that section. If you want the argument against IWE, read *Where IWE Is Weaker* below; it is not a short list. ## marksman vs markdown-oxide If you only want a markdown language server and are choosing between the two established ones, the sections below give the details. In short: | | marksman | markdown-oxide | | --------------------- | ----------------------------------------------------------------------- | ----------------------------------------------------------- | | **Aimed at** | Correct links, headings, and references in any editor | Obsidian's editing model inside a text editor | | **Link targets** | Documents and headings | Documents, headings, and blocks (`[[doc#^block]]`) | | **Daily notes** | None | `:Today`, `:Daily next monday`, relative `prev` / `next` | | **Diagnostics** | Live wiki-link diagnostics | Unresolved-reference diagnostics | | **Editors** | VS Code, Neovim, Vim, Emacs, Helix, Kakoune, Sublime Text, BBEdit, Zed | Neovim, VS Code, Zed, Helix, Kakoune | | **Implementation** | F#/.NET | Rust | **Pick marksman** for the widest editor support and the least setup, especially in Vim, Emacs, Sublime Text, or BBEdit. **Pick markdown-oxide** if you are coming from Obsidian and want daily notes, block references, and tags in your editor. **Pick IWE** if you also want to query the notes, restructure them, and let an AI agent work on them from the CLI or MCP. The two sections below say where each of them is ahead of IWE. ## IWE vs marksman *Verified 2026-09-03 against [marksman's feature documentation](https://github.com/artempyanykh/marksman/blob/main/docs/features.md) and its 2026-02-08 release.* [marksman](https://github.com/artempyanykh/marksman) is the default answer to "is there a markdown LSP?" It has the widest editor coverage of any tool on this page, it is the tool other markdown language servers get compared against, and for a large class of users it is enough. **The difference in one sentence: marksman is a markdown language server; IWE is a knowledge graph that speaks LSP.** marksman's job ends at making links, headings, and references work correctly inside an editor — and it does that job well. IWE's LSP is one interface onto a graph that also has a CLI, a query language, and an MCP server; the editor features fall out of the graph rather than being the point. | Capability | IWE | marksman | | ------------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | | **Link syntax** | ✅ Markdown links, `[[wiki]]`, `[[wiki\|piped]]`, and [inclusion links](/docs/concepts/inclusion-links/) | ✅ Markdown inline links, reference links, and wiki links | | **Link targets** | ⚠️ Document granularity — `#fragment` is dropped when resolving ([Keys](/docs/architecture/keys/)) | ✅ Documents *and* headings — `[[doc#heading]]`, `[[#heading]]` | | **Diagnostics** | ❌ None published by the LSP; `iwe stats` reports broken links in batch | ✅ Live wiki-link diagnostics in the editor | | **Workspace symbols** | ✅ Section-grain, carrying hierarchical path text (`Journal ⇒ 2026 ⇒ Jan 26`) | ✅ Heading-grain | | **Search ranking** | ✅ Fuzzy + BM25 fused with [RRF](/docs/architecture/search-ranking/), shared by CLI, LSP, and MCP | ⚠️ Subsequence matching, no relevance ranking | | **Reference counts** | ⚠️ Whole-document backlink count as an [inlay hint](/docs/features/inlay-hints/) | ✅ Per-heading counts via code lens | | **Graph transformations** | ✅ [Extract](/docs/code-actions/extract/) / [inline](/docs/code-actions/inline/) sections, section↔list, attach, squash | ❌ Not a goal | | **Query language** | ✅ [MongoDB-style YAML](/docs/concepts/query-language/) over frontmatter and graph edges | ❌ None | | **CLI** | ✅ [Full CLI](/docs/cli/) — find, retrieve, tree, stats, schema, normalize, bulk update/delete | ❌ None (a standalone check command is on the roadmap) | | **MCP for AI agents** | ✅ [14 tools](/docs/agentic/mcp/), prompts, resources, file watching | ❌ None | | **Table of contents** | ❌ Not supported | ✅ Code action generates and updates a ToC | | **Rename** | ✅ Rename a file, all references follow | ✅ Rename updates cross-references in the configured link style | | **Daily notes** | ⚠️ Template-based via [attach](/docs/code-actions/attach/); no date resolution | ❌ None | | **Folding** | ✅ Folding ranges over sections and lists | ❌ Not documented | | **Workspace scoping** | Directory the server starts in, plus `workspace.path`; honors `.gitignore` | VCS root or `.marksman.toml`; honors `.gitignore`, `.hgignore`, `.ignore` | | **Editors** | ✅ Documented setup for VS Code, Neovim, Zed, Helix; any LSP client works | ✅ VS Code, Neovim, Vim, Emacs, Helix, Kakoune, Sublime Text, BBEdit, Zed | | **Implementation** | Rust; pulldown-cmark AST with wikilinks enabled | F#/.NET; custom parser built on Markdig | **Choose marksman if** you want link and reference correctness in an editor marksman already supports, with a `.marksman.toml` and nothing else to learn — especially Emacs, Vim, Sublime Text, or BBEdit, where IWE has no documented setup. **Choose IWE if** the notes are something you also want to query, restructure, and expose to agents outside the editor. ## IWE vs markdown-oxide *Verified 2026-09-03 against [oxide.md](https://oxide.md/), the markdown-oxide repository, and release `v0.25.12`.* [markdown-oxide](https://github.com/Feel-ix-343/markdown-oxide) is a PKM language server aimed squarely at bringing Obsidian's editing model into a text editor: daily notes, block references, tags, callouts. It is an actively maintained solo project with a real community around it, and on the Obsidian-compatibility axis it is ahead of IWE. **The difference in one sentence: markdown-oxide brings Obsidian's model into your editor; IWE treats the notes as a graph with an editor attached.** They overlap on links, backlinks, and completion, and diverge on almost everything else. | Capability | IWE | markdown-oxide | | -------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Parsing** | AST — pulldown-cmark with wikilinks enabled | Regular expressions ([`src/vault/mod.rs`](https://github.com/Feel-ix-343/markdown-oxide/blob/main/src/vault/mod.rs)), a deliberate tradeoff for indexing speed ([#2](https://github.com/Feel-ix-343/markdown-oxide/issues/2)) | | **Link granularity** | Document — `#fragment` is dropped; the answer to "link to a section" is to [extract it](/docs/code-actions/extract/) into its own document | Document, heading, and block — `[[doc#heading]]` and `[[doc#^block]]` resolve directly | | **Search & ranking** | ✅ Section-grain workspace symbols with path context; fuzzy + BM25 fused with [RRF](/docs/architecture/search-ranking/) | ⚠️ Filename, heading, and block completion; no relevance ranking of its own — sorting is left to the editor's completion engine ([#409](https://github.com/Feel-ix-343/markdown-oxide/issues/409)) | | **Daily & periodic notes** | ⚠️ Template-based via [attach](/docs/code-actions/attach/) (`daily/{{today}}`); no date resolution | ✅ `:Today`, `:Daily next monday`, `:Daily two days ago`, relative `prev` / `next` / `+7` | | **Diagnostics** | ❌ None from the LSP; `iwe stats` reports broken links in batch | ✅ Unresolved-reference diagnostics in the editor | | **Reference counts** | ⚠️ Whole-document backlink count as an inlay hint | ✅ Per-referenceable counts via code lens (needs enabling in some clients) | | **Backlinks** | ✅ Go-to-references over reference and inclusion edges | ✅ Files, headings, and blocks | | **Hover preview** | ✅ Linked note rendered inline, frontmatter stripped | ✅ Hover preview | | **Tags** | ⚠️ Frontmatter `tags` are queryable; no inline `#tag` index or completion | ✅ Tag completion, and tag references through workspace symbols | | **Graph transformations** | ✅ Extract / inline sections, section↔list, attach, squash | ❌ Linking and rename only | | **Query language** | ✅ MongoDB-style filters over frontmatter *and* graph edges, with projection, sort, limit | ❌ None | | **CLI** | ✅ Full CLI, including bulk `update` / `delete` across the workspace by filter | ⚠️ Small CLI for opening daily notes and config files | | **MCP for AI agents** | ✅ 14 tools, prompts, resources, file watching, dry-run on every mutation | ❌ None | | **External commands** | ✅ [Configurable CLI pipeline](/docs/features/text-commands/) for block-level transformations | ❌ None | | **Auto-formatting** | ✅ [Full normalization](/docs/features/normalization/) — headers, lists, link titles, wrapping | ⚠️ Basic formatting | | **Editors** | ✅ VS Code, Neovim, Zed, Helix | ✅ Neovim, VS Code, Zed, Helix, Kakoune | | **Performance** | ✅ Rust; [20,000 files in under a second](/docs/architecture/benchmark/) | ✅ Rust; fast in the common case, no published benchmark | ### On parsing Both approaches work; they fail differently. Regex parsing is very fast to index and gets the common cases right. An AST costs more to build and settles a class of question by construction rather than by pattern: whether `[[` inside an inline-code span is a link, whether a fenced block nested in a list item is code, whether seven `#` characters open a heading. IWE pays the parse cost once at startup in the LSP and MCP servers, and per invocation in the CLI — [the benchmark](/docs/architecture/benchmark/) is where that cost is measured rather than asserted. ### On link granularity This is a design difference, not a scoreboard. markdown-oxide resolves heading and block anchors, so a link can point at a paragraph. IWE resolves at document granularity and drops `#fragment` when computing the target [key](/docs/architecture/keys/); if a section is worth linking to, the [extract](/docs/code-actions/extract/) code action turns it into its own document and rewrites the link for you. That is a real answer, but it is not the same answer, and if you want to point at a block *without* moving it, markdown-oxide does what you want and IWE does not. ## IWE vs Obsidian *Verified 2026-09-03 for pricing and platform support; feature rows carried forward from May 2026.* **Obsidian** is a GUI PKM application with strong visualization, an extensive plugin ecosystem, and Bases (typed database views), Canvas, and Web Clipper: | Feature | IWE | Obsidian | | ----------------------------------------- | --------------------------------------------------------------------- | -------------------------------------------------------------------------------- | | **Editor Integration** | ✅ Works with your preferred text editor (VS Code, Neovim, Zed, Helix) | ❌ Proprietary editor only | | **Cost** | ✅ Completely free and open source | ⚠️ Free for personal use; Sync $4–5/mo, Publish $8–10/mo, commercial $50/user/yr | | **Performance** | ✅ Rust-powered, instant operations on thousands of files | ⚠️ Electron-based, can be slower with large vaults | | **Graph Transformations** | ✅ Automated extract/embed operations, section-to-list conversions | ❌ Manual linking and organization | | **Query Language** | ✅ MongoDB-style YAML over frontmatter and graph edges | ✅ Bases (typed database views) and the Dataview plugin | | **MCP / AI Integration** | ✅ Native MCP server with 14 tools, prompts, and resources | ⚠️ Requires third-party plugins | | **External Commands** | ✅ Configurable CLI tools (AI agents, scripts, Unix tools) | ⚠️ Limited, requires third-party plugins | | **[Inclusion Links](/docs/concepts/inclusion-links/)** | ✅ Native support with automatic linking | ⚠️ Available via plugins | | **Auto-formatting** | ✅ Comprehensive markdown normalization on save | ⚠️ Basic formatting; plugins for advanced normalization | | **Batch Operations** | ✅ CLI for bulk transformations, update, delete with filters | ❌ No batch operation capabilities | | **Graph Visualization** | ⚠️ CLI-based dot export | ✅ Interactive graph view with customizable styling | | **Canvas / Whiteboard** | ❌ Not supported | ✅ Native infinite canvas | | **Web Clipper** | ❌ Not supported | ✅ Native browser extension | | **Plugin Ecosystem** | ⚠️ LSP capabilities, CLI extensions, and MCP tools | ✅ Thousands of community plugins | | **Learning Curve** | ⚠️ Requires terminal knowledge and editor setup | ✅ GUI-friendly with intuitive interface | | **Sync & Collaboration** | ✅ Git-based sync (free), works with any Git hosting | ⚠️ Obsidian Sync (paid) with shared vaults, or manual Git setup | | **Publishing** | ✅ Export to various formats via CLI | ⚠️ Obsidian Publish (paid) or manual export | | **Mobile Support** | ❌ Desktop and terminal only | ✅ Native mobile apps with sync | **The philosophical difference**: IWE is editor-agnostic and developer-focused, designed to sit inside existing technical workflows and to be readable by agents. Obsidian is a complete environment with its own ecosystem, better suited to people who want one application that does everything, with visual interfaces and plugin-level customization. If you are not already living in a text editor, Obsidian is very likely the right choice. ## IWE vs zk and telekasten.nvim *Feature rows carried forward from May 2026 — not re-verified on 2026-09-03.* **zk** (zk-org) is a CLI-driven Zettelkasten tool with an LSP server and integrations for Neovim, VS Code, and Emacs. **telekasten.nvim** is a Neovim-only plugin with calendar, image paste, and Telescope integration: | Feature | IWE | zk | telekasten.nvim | | -------------------------- | ------------------------------------------------------------- | ----------------------------------------- | ------------------------------- | | **Editor Support** | ✅ VS Code, Neovim, Zed, Helix, any LSP-compatible editor | ✅ Neovim, VS Code, Emacs via LSP | ❌ Neovim only | | **Graph Transformations** | ✅ Automated extract/embed, structural changes | ❌ Basic note creation and linking | ❌ Basic note creation | | **Query Language** | ✅ MongoDB-style YAML filters over frontmatter and graph edges | ⚠️ Filter flags for tags, links, mentions | ❌ None | | **MCP / AI Integration** | ✅ Native MCP server with 14 tools | ❌ None | ❌ None | | **External Commands** | ✅ Configurable CLI tools (AI agents, scripts, Unix tools) | ⚠️ Aliases and shell automation | ❌ Manual workflows only | | **Performance** | ✅ Rust-powered LSP | ✅ Go-based CLI | ⚠️ Lua-based, editor-dependent | | **Batch Operations** | ✅ CLI for bulk operations with filters | ⚠️ Limited (notebook housekeeping) | ❌ One-note-at-a-time workflow | | **Auto-formatting** | ✅ Built-in normalization | ❌ Requires external tools | ❌ Requires external tools | | **Note Templates** | ✅ Note templates via `attach` command | ✅ Template-based note creation | ✅ Static templates | | **Daily / Periodic Notes** | ⚠️ Template-based via `attach`; no date resolution | ⚠️ Via templates | ✅ Daily/weekly/monthly/yearly | | **Search Integration** | ✅ LSP-based with any picker | ✅ Built-in `fzf` browser | ✅ Telescope/fzf integration | | **Calendar View** | ❌ Not supported | ❌ Not supported | ✅ Calendar with note highlights | | **Image / Media Paste** | ❌ Not supported | ❌ Not supported | ✅ Clipboard image paste | | **Installation** | ✅ Single binary + editor extension | ✅ Single binary + LSP | ⚠️ Complex Neovim plugin setup | zk is a strong choice for Zettelkasten purists who want a CLI-first workflow with a simpler data model. telekasten.nvim is the best fit for Neovim users who want a polished journaling experience with calendar and image-paste workflows — neither of which IWE offers. ## IWE vs mdbase *Feature rows carried forward from May 2026 — not re-verified on 2026-09-03.* **mdbase** is a specification (v0.2.1) for treating folders of markdown files as typed, queryable data collections. It has a TypeScript reference implementation and a Go library, but no LSP server, no editor extensions, and no CLI. It overlaps with IWE on querying and frontmatter typing — but not on graph transformations, editor integration, or AI tooling. | Feature | IWE | mdbase | | ------------------------- | -------------------------------------------------------------------- | --------------------------------------------------------------- | | **Project Maturity** | ✅ Production tool: CLI, LSP, MCP shipping today | ⚠️ Pre-1.0 spec (v0.2.1) with reference library implementations | | **Editor Integration** | ✅ LSP for VS Code, Neovim, Zed, Helix | ❌ No LSP, no editor extensions | | **CLI** | ✅ `iwe` with find, retrieve, update, delete, normalize, schema, etc. | ❌ Library-only (TypeScript / Go) | | **MCP / AI Integration** | ✅ Native MCP server with 14 tools | ❌ None | | **Query Language** | ✅ MongoDB-style filters over frontmatter and graph edges | ✅ Expression language for filters, ordering, and link traversal | | **Frontmatter Schemas** | ✅ Schema inference from existing documents (`iwe schema`) | ✅ Explicit type definitions with validation and inheritance | | **Validation** | ⚠️ Implicit via schema inference | ✅ Configurable strictness (off / warn / error) | | **Graph Transformations** | ✅ Extract, inline, attach, normalize, rename with link updates | ⚠️ Rename updates references; no structural refactors | | **Inclusion Links** | ✅ Native block-reference inclusion | ❌ Wikilinks and markdown links only | | **Performance** | ✅ Rust core, in-memory graph | ⚠️ TypeScript reference impl with SQLite-backed cache | | **Batch Operations** | ✅ `iwe update`/`delete` with filters | ✅ `batchUpdate` / `batchDelete` library calls | | **File Watching** | ✅ Built into LSP and MCP servers | ✅ Watch-mode simulation in reference impl | **mdbase's advantage**: explicit, vendor-neutral schema definitions with strict validation suit teams that want to enforce frontmatter contracts across tools. If you primarily need typed records rather than a knowledge graph, mdbase's typed-collection model may fit better. ## Where IWE Is Weaker Everything above is easy to write in IWE's favor, so here is the other list — checked against IWE `0.23.0` on 2026-09-03. - **Daily and periodic notes.** IWE has templates: an [attach](/docs/code-actions/attach/) action with `key_template = "daily/{{today}}"` creates and links today's note. It has no natural-language date resolution, no `[[next monday]]` as a link target, no relative jump commands, and no weekly, monthly, or yearly periods. markdown-oxide is clearly ahead here; telekasten.nvim has the weekly, monthly, and yearly periods. - **Live diagnostics.** IWE's LSP publishes none. Broken links are detectable, but only in batch, through `iwe stats`. Both marksman and markdown-oxide flag unresolved references as you type. - **Section and block link targets.** IWE resolves links at document granularity and drops `#fragment`. marksman resolves heading anchors; markdown-oxide resolves headings and blocks. Extraction is IWE's answer, and it is a different answer, not a strictly better one. - **Per-heading reference counts.** IWE's inlay hint shows one backlink count for the whole document. marksman and markdown-oxide both show counts per heading. - **Tags.** IWE queries frontmatter `tags` arrays well, but has no index, completion, or go-to-definition for inline `#tag` syntax. markdown-oxide does. - **Aliases.** A document has one title. If you want a note findable under several names, IWE has no answer today. - **Editor breadth.** IWE documents VS Code, Neovim, Zed, and Helix. marksman additionally covers Emacs, Vim, Sublime Text, BBEdit, and Kakoune; markdown-oxide covers Kakoune. Any LSP client can be wired up by hand, but "by hand" is the operative phrase. - **Table of contents.** marksman generates and maintains one as a code action. IWE does not. - **Mobile.** Obsidian has native apps with sync. IWE is desktop and terminal only. - **Plugin ecosystem, canvas, web clipper, and interactive graph view.** Obsidian wins these outright, and it is not close. - **First-run clarity.** IWE is a binary, an editor config, and a directory of markdown. If you have never used an LSP-backed tool before, Obsidian will be working before IWE is configured. ## What IWE Adds Beyond a Markdown LSP The tables above compress a lot. The capabilities that have no counterpart in a markdown language server — graph transformations, the query language, the MCP server, search ranking, schema inference, external commands, normalization, and inclusion links — are catalogued with links to their documentation in the [Feature Overview](/docs/features/overview/). ## Method and Dates - **Each comparison block carries its own verification date.** A block dated May 2026 has not been re-checked since; treat it accordingly. marksman and markdown-oxide were re-verified on 2026-09-03 against their documentation, source, and releases. Obsidian pricing and platform support were re-checked the same day; its feature rows were not. - **Claims about IWE** were checked against IWE `0.23.0`. - **Claims about other tools** link to their documentation, source, or issue tracker wherever the claim is not obvious from the tool's front page. - **Corrections are welcome and will be applied.** If something here is wrong, out of date, or unfair — including by the maintainers of the tools compared — [open an issue](https://github.com/iwe-org/iwe/issues) and it will be fixed. This page is maintained, not marketing. --- # Query Language URL: https://iwe.md/docs/concepts/query-language/ > **Experimental.** The query language is under active development. Syntax, operators, defaults, and CLI flag names may change without warning. It is **not exposed as a library API** and has no stable on-disk format. The supported surfaces today are the CLI subcommands listed below and the MCP `iwe_query` tool, which accepts an operation document verbatim; build automation against them at your own risk. IWE has a YAML-based, MongoDB-style query language for selecting, shaping, and mutating documents in a workspace by their frontmatter, graph relationships, and content. It is reachable through the CLI subcommands `iwe find`, `iwe count`, `iwe update`, and `iwe delete` (plus the read-only selectors on `iwe retrieve`, `iwe tree`, and `iwe export`), and through the [MCP](/docs/agentic/mcp/) `iwe_query` tool. This page is the reference for the language. The [Query Language Specification](/docs/concepts/spec/) is the formal source of truth for the document-level core (operations, filter, graph operators, projection, output formats) and carries the full grammar, block productions included; the block surface's semantics are documented here. ## Operations | Operation | CLI subcommand | What it does | | --- | --- | --- | | `find` | [`iwe find`](/docs/agentic/find/) | Returns matched documents (subject to projection). | | `count` | [`iwe count`](/docs/agentic/count/) | Returns the integer count of matched documents. | | `update` | [`iwe update`](/docs/agentic/update/) | Mutates frontmatter and blocks on each matched document. | | `delete` | [`iwe delete`](/docs/agentic/delete/) | Removes each matched document and cleans up references. | `update` and `delete` require an explicit filter — passing `{}` on purpose is the only way to operate on the whole corpus. ## Filter syntax A filter document is YAML. A document matches when every top-level key matches; multiple top-level keys are AND-composed. ### Bare equality ```yaml status: draft ``` Matches documents whose `status` field equals `draft`. For arrays, a bare scalar tests membership: `tags: rust` matches when `rust` is in the `tags` array. Cross-type comparisons are always false (no implicit coercion: `priority: "3"` does not match an integer field). ### Operator expressions A mapping with `$`-prefixed keys is an operator expression: ```yaml priority: { $gt: 3 } priority: { $gte: 3, $lte: 7 } # closed range [3, 7] status: { $in: [draft, review] } status: { $nin: [archived, deleted] } reviewed: { $exists: true } tags: { $all: [rust, async] } tags: { $size: 0 } # exact array length tags: { $size: { $gte: 3 } } # $size also takes count comparisons ``` Operators in one expression are ANDed together. User frontmatter fields cannot start with `$`, so an operator and a field name never collide. ### Logical composition ```yaml $and: - status: draft - priority: { $gt: 3 } $or: - status: draft - status: review $nor: - status: archived - status: deleted ``` Top-level AND is implicit. Use explicit `$and` when you need the same field name on multiple sub-clauses (a YAML mapping cannot have duplicate keys). ### Nested fields Nested fields can be addressed via nested mapping or dotted shorthand; both forms are equivalent: ```yaml author.name: alice author: name: alice ``` Field names that themselves contain a literal `.` are not addressable in v1 — the engine always splits paths on `.`. ## Graph operators Graph operators live alongside frontmatter predicates inside the same filter. They walk inclusion edges (block-reference inclusion links) or reference edges (inline links). ### `$key` — identity ```yaml $key: notes/foo # implicit $eq $key: { $in: [a, b, c] } # any of these $key: { $nin: [drafts/scratch, drafts/temp] } # none of these $key: { $startsWith: notes/ } # everything under notes/ ``` Every form matches the key as stored, with the document extension already stripped. `$startsWith` is a plain string prefix, not a path-segment match, so `notes` also matches `notes-archive/gamma` and the hub note `notes` itself; write the trailing separator when you mean the directory alone. The prefix is case-sensitive, unlike wiki link resolution. ### Relational operators | Operator | Reads as | Edge type | Walk parameters | | --- | --- | --- | --- | | `$includes` | this doc includes an anchor | inclusion | `maxDepth`, `minDepth` | | `$includedBy` | this doc is included by an anchor | inclusion | `maxDepth`, `minDepth` | | `$references` | this doc references an anchor | reference | `maxDistance`, `minDistance` | | `$referencedBy` | this doc is referenced by an anchor | reference | `maxDistance`, `minDistance` | Each takes either a scalar key (shorthand for direct edges) or a mapping with an optional `match`, walk parameters, and an optional `$size`: ```yaml # Direct edges only — scalar shorthand for { match: { $key: projects/alpha } } $includedBy: projects/alpha # Walk inclusion edges from a single anchor, bounded $includedBy: { match: { $key: projects/alpha }, maxDepth: 5 } # Anchor by frontmatter predicate (every active project) $includedBy: match: type: project status: active maxDepth: 5 # Anchor set is two named documents $includedBy: match: $key: { $in: [projects/alpha, projects/beta] } maxDepth: 5 # Range bounds — documents 2 to 3 hops from archive/index $referencedBy: { match: { $key: archive/index }, minDistance: 2, maxDistance: 3 } ``` In the full mapping form, **omitting `maxDepth` / `maxDistance` means direct edges** (depth 1). Set `maxDepth: 0` / `maxDistance: 0` for an unbounded walk that reaches every transitively-related document. Walks are BFS and de-duplicate via a visited set, so cycles terminate. ```yaml $includedBy: { match: { $key: projects/alpha } } # direct children $includedBy: { match: { $key: projects/alpha }, maxDepth: 0 } # whole tree ``` `match` is optional and defaults to `{}` (any document); the empty mapping `$includedBy: {}` is still a parse error. A relational operator never matches a document in its own anchor set. To include the anchor, OR it in: ```yaml $or: - $key: projects/alpha - $includedBy: { match: { $key: projects/alpha }, maxDepth: 0 } ``` #### Cardinality — `$size` By default a relational operator holds when at least one document stands in the relation. Add `$size` to test the *count* of related documents instead. `$size` takes a non-negative integer (an `$eq` shorthand) or a mapping of count comparisons (`$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`); multiple comparisons AND together. The count is over the distinct documents in the walk's `[min, max]` band, matching `match`, always excluding the document itself. ```yaml $includedBy: { $size: 0 } # roots — nothing includes them $includes: { $size: 0 } # leaves — include nothing $referencedBy: { $size: 0 } # orphans — no inbound links $referencedBy: { $size: { $gte: 5 } } # hubs — 5+ direct referrers $includedBy: { $size: { $gte: 10 }, maxDepth: 0 } # 10+ transitive ancestors $includedBy: { match: { type: project, status: active }, $size: { $gte: 2 }, maxDepth: 3 } # counted, filtered, bounded ``` Because the default bounds count direct edges, `$size: 0` is the same whether bounded or unbounded: zero direct edges means zero at every depth. ### `$content` — content membership `$content` lifts a [block predicate](#block-predicates) into the filter: it matches documents containing **at least one** block satisfying the predicate. `$content: {}` matches any document with at least one block. ```yaml # Drafts, or documents whose content mentions TODO filter: $or: - status: draft - $content: { $text: "TODO" } ``` ```yaml # Documents under a project that LACK a Status section — absence is # a quantifier over the whole block set, expressible only in filter filter: $includedBy: { match: { $key: projects/alpha }, maxDepth: 3 } $nor: - $content: { $header: Status } ``` Like `$key`, it is a noun operator testing the document itself, so it composes with every other filter clause under `$and` / `$or` / `$nor` and inside the `match` of any relational operator. Like any filter clause, `$content` decides membership only — it selects nothing; which blocks an operation reads or mutates is designated at the projection or update site. ## Search (`find` only) `search` is a top-level clause of a `find` operation, beside `filter`. It selects documents by relevance to a full-text query and supplies the default ordering. Search leads, filter refines: ```yaml search: lexical: "broken links cleanup" filter: { type: note } limit: 5 ``` - `lexical: <string>` — BM25 full-text over title + body. `fuzzy: <string>` — skim subsequence match over title + key. Both present → RRF fusion, exactly as `iwe find --fuzzy --lexical`. At least one is required; `search: {}` is a parse-time error, as is any key other than `lexical` / `fuzzy`. - **Search selects and orders.** A document is in the result **iff** it matches the search **and** passes the `filter` (search matches ∩ filter matches), ordered by relevance, ties by key ascending. Candidates with no BM25 hit / no skim score are dropped — a text query is a filter here, not a sort variant. The two selections are joint; no evaluation order is implied, and `limit: 5` always means the 5 best documents that pass the filter (never the filtered survivors of a globally-truncated top list). - **`search` + `sort` is legal.** Search contributes membership and the default ordering; an explicit `sort` overrides the ordering only. "The 5 newest documents matching Q" is `search` + `sort: { modified_at: -1 }` + `limit: 5`. - **Corpus-global scores.** The BM25 index is fit to the whole workspace; IDF is not re-fit per filter (standard Lucene-style behavior). Searching within a filter is therefore not identical to searching a corpus of only the filtered documents. - **No searchable terms.** A `lexical` query that stems away to nothing (stop words only) matches nothing; the result carries a structured warning so callers see why it is empty. - **`find` only** — not `count` / `update` / `delete` — and it requires the search-indexed graph. The `iwe find --fuzzy` / `--lexical` flags and the [MCP](/docs/agentic/mcp/) `iwe_query` `search` clause are the two surfaces. ## Projection (`find` only) `project` shapes each result to exactly the listed fields; `addFields` keeps the default fields and adds to them. The two are mutually exclusive. Each entry maps an output name to a source: `1`, `true`, and YAML null project the frontmatter field of the same name, a dotted path projects a nested frontmatter field, and a `$`-selector projects a system field. ```yaml project: title: 1 priority: meta.priority key: $key body: $content parents: $includedBy ``` The `$`-selectors: `$key`, `$title`, `$titleSlug`, `$content`, `$frontmatter`, `$includes`, `$includedBy`, `$references`, `$referencedBy`. Projection can also address blocks inside each matched document — narrowed bodies, located blocks, grep lines. See [Block projection](#block-projection). ## Sort and limit ```yaml sort: { modified_at: -1 } # 1 = ascending, -1 = descending limit: 100 # 0 = no limit ``` ```yaml sort: { title: { order: 1, collation: binary } } # codepoint order for this field sort: { title: { order: -1, collation: sv-SE } } # Swedish rules for this field ``` v1 accepts exactly one sort key. Ties (and the no-sort case) are broken by document key in ascending lexicographic order. When run through the `iwe` CLI, the LSP server or the MCP server, text values sort with the collation rules of the workspace locale: case and accents only break ties, `Note 2` comes before `Note 10`, and emoji are ignored unless the remaining text is identical. The engine itself defaults to codepoint order. A per-field `collation` of `binary` or a locale tag overrides whichever default applies. ## Update operators ```yaml update: $set: reviewed: true audited_at: 2026-04-26 "review.reviewer": alice $unset: draft_notes: "" ``` `$set` adds the field if absent, replaces it otherwise. Mapping values replace wholesale; use dotted shorthand to write subset leaves without dropping siblings. `$unset` removes fields; values are ignored. Block update operators — `$replace`, `$replaceText`, `$insertBefore`, `$insertAfter`, `$append`, `$delete` — live in the same `update` document as siblings of `$set` / `$unset` and combine freely with them. See [Block update operators](#block-update-operators). ### Field names the language cannot address The engine preserves every frontmatter field a document carries, whatever its name starts with. One name shape cannot be addressed: `$`-prefixed names are the operator vocabulary, so a path segment beginning with `$` — in a filter key, a sort key, a projection source, or a `$set` / `$unset` target, at any depth — is a parse-time error. Such a field is still stored, returned and validated; it just cannot be reached by a path. ## Blocks The language also addresses **blocks** — the structural nodes inside a document: headers, paragraphs, lists, list items, code blocks, quotes, tables, references, horizontal rules. There is no separate block-selection clause; blocks enter through three sites the operation document already owns: | Site | Addition | Purpose | | --- | --- | --- | | `filter` | [`$content` operator](#content--content-membership) | Document membership by content — "documents containing a block matching P" | | `project` / `addFields` | `{ $content: P }` narrowing, `$blocks` and `$matches` sources | Read a slice of a document: narrowed body, located blocks, grep lines | | `update` | Block operators (`$replace`, `$replaceText`, `$insertBefore`, `$insertAfter`, `$append`, `$delete`) | Mutate blocks, each operator carrying its own selection | One grammar — the **block predicate** — is consumed at all three sites, and one convention governs every argument shape: **`$`-prefixed keys select, bare keys configure.** Inside an update operator's argument, the `$`-keys form the block predicate and the bare keys (`from`, `to`, `content`, `expect`) are the payload. Because there is no block-selection clause, membership stays `filter`'s alone: nothing outside `filter` adds or removes documents from the result. Deliberately out of scope: computed replacements (renumbering, case transforms), stateful walks, inline-element rewriting — the driving agent reads the selection, computes, and applies literal edits. The typical workflow is two operations sharing one predicate: locate with `project: { hits: { $blocks: P } }`, then mutate with an update operator carrying the same `P`. ### Block model Each block type has its own predicate operator. **Node** operators select the matching block alone; **tree** operators select the block together with everything below it. The **own text** column is what text predicates match against: | Operator | Sort | Markdown construct | Own text | | --- | --- | --- | --- | | `$header` | node | heading (`#`, `##`, …); the blocks of its section are its children | the heading text | | `$paragraph` | node | paragraph | the paragraph text | | `$list` | tree | bullet or ordered list — an empty head whose children are the items | none | | `$quote` | tree | block quote — an empty head wrapping the quoted blocks | none | | `$item` | node | list item | the item's own line — a loose item's leading paragraph folds into it — excluding nested children | | `$code` | node | fenced code block | the code text, excluding the fence lines | | `$table` | node | table | the rows as rendered, one line per row | | `$ref` | node | block reference (inclusion link) | the authored (piped wikilink) text, when present; a title derived from the target is not own text | | `$hr` | node | horizontal rule | none | The tree rooted at a header — the header together with every block below it — is a **section**: `$header` matches the heading line alone, `$section` selects the whole tree. Quotes and lists are containers with **empty heads**: the root carries no own text, only the wrapper, so their operators select the container with its contents, and `$within` peels the wrapper off. A **container** is a block that can carry children — a header, an item, a list, a quote; containers are the legal `$append` targets. Raw HTML blocks are outside the model — the parser drops them. **Own text.** Text predicates evaluate against a block's own text only, never descendant text. A header whose section contains `TODO` does not match `$text: TODO` — the paragraph carrying it does. To select a block *because of* its contents, use `$contains`. Own texts are pairwise disjoint, which the update semantics rely on. **Normalized form.** Matching operates on the normalized rendering of the document — the same form `iwe retrieve` and every other read emits — at the byte level: inline markup keeps its markdown syntax (`**bold**` stays `**bold**`, never `bold`), and inline link URLs are part of own text, rendered relative to the containing document. Text copied from a read matches byte-for-byte in a later predicate or `$replaceText` anchor. In normalized form every block's own text is a single line, except code blocks and tables — one line per code line or table row, with table cells padded to their column width. **Section path.** Every block has a section path: the titles of its enclosing sections, ordered from the document root down, with top-level headers omitted — the value block reads emit as `path`. Path elements are plain text (inline markup stripped); a heading's own text keeps its markdown syntax, so for a heading carrying inline markup the two differ — echo own text, not path elements, into text predicates. **No block identity.** No IDs are minted or written to files; addressing is by predicate against normalized text, guarded by [`expect`](#expect-guards). Two byte-identical sibling blocks under the same section path are indistinguishable — no predicate can select one and not the other; every mutation touches both or fails its `expect`. The escape hatch is the whole-body rewrite (`iwe update -k KEY -c CONTENT`). **Readable but not editable.** The task-checkbox marker (`[ ]` / `[x]`) sits outside an item's own text: text predicates never see it, no predicate selects items by state, and `$replaceText` cannot toggle it — toggling remains the editor's code-action surface. Tables select, render, and travel as units — membership, reads, `$delete`, `$replace`, and the insertion anchors all work — but `$replaceText` selecting a table is a validation error; editing inside a table means replacing the table. ### Block predicates A block predicate is a YAML mapping of `$`-prefixed operators. The empty predicate `{}` matches every block; top-level keys AND together; unknown `$`-names and bare keys are parse-time errors. | Operator | Matches | | --- | --- | | `$text: S` | own text contains `S`, case-insensitively; `$text: { $eq: S }` matches the whole own text, also case-insensitively | | `$matches: REGEX` | own text matches the pattern (Rust regex, case-sensitive — use an inline `(?i)` flag; no backreferences or lookaround) | | `$header`, `$paragraph`, `$item`, `$code`, `$table` | blocks of one type — the matching block alone, no implicit subtree; the argument is a nested predicate, and a scalar is exact-text shorthand (`$header: Status`) | | `$ref` | block references, selected by target (`$ref: { $references: KEY }`); the scalar shorthand is a parse-time error (a ref's rendered title is usually derived from its target), and text predicates match the authored (piped) link text alone | | `$hr` | horizontal rules; no own text — the scalar shorthand and direct text predicates in the argument are parse-time errors | | `$section: T` | a header matching `T` together with everything below it; scalar `T` is exact-text shorthand for the root header | | `$quote: P`, `$list: P` | a quote / a list together with its contents; the root has no own text, so scalars and direct text predicates in the argument are parse-time errors — scope with `$within` or `$contains` | | `$within: T` | blocks inside the selection — a section's body, a quote's content, at any depth; scalar `T` names a section; a mapping argument must select content: `{}`, or a predicate containing `$section` / `$quote` / `$list` | | `$contains: P` | blocks with a descendant matching `P`, at any depth | | `$references: KEY` | blocks whose own content links to `KEY` — a ref targeting `KEY`, or inline text linking to it | | `$and`, `$or`, `$nor` | logical composition, as in filters | ```yaml $header: {} # every header: the outline $section: Unreleased # one section, header included $within: Goals # inside Goals, header excluded $within: { $section: { $matches: "^Q[0-9]" } } # inside any quarterly section $paragraph: { $references: archive/old-plan } # paragraphs linking to a document $header: { $matches: "(?i)todo" } # headers whose heading line says TODO { $header: {}, $contains: { $matches: "(?i)todo" } } # headers whose section holds a TODO $within: { $quote: {} } # quoted content, wrapper peeled off ``` Deeper scoping is recursion, not extra syntax — each element of a section path becomes one `$section` predicate scoped by `$within` to the elements before it: ```yaml # Inside Q3, itself inside Goals (matches Goals > 2026 > Q3, not Q3 > Goals) $within: { $section: { $text: { $eq: Q3 }, $within: Goals } } ``` Conditions inside a type operator's argument and conditions beside it conjoin identically: `{ $header: { $matches: "^Q" } }` and `{ $header: {}, $matches: "^Q" }` match the same blocks. Type follows the block model, not the visual layout — a loose item's leading paragraph folds into the item, so its text matches `$item`, not `$paragraph`. When unsure of a block's type, locate with an untyped predicate (`$blocks: { $text: alpha }`) and read the entry's `type`; the type operators narrow a selection, they are not for asserting what kind of block some text lives in. `$within: {}` is the interior of the document itself — every block below the top level; a top-level block is therefore `$nor: [{ $within: {} }]`. #### Selection semantics A block predicate is evaluated against every block in the document, at every depth, always. Node operators select each matching block alone; tree operators (`$section`, `$quote`, `$list`) select the matching root together with every block below it. The result is a **forest**: the selected blocks arranged by ancestry — a selected block's parent is its nearest selected ancestor — in document order. Union over multiple matches is automatic: when a selector matches several headers, `$within` is the union of all their interiors. For sections, three selections line up on one predicate: | Selection | Predicate | Denotes | | --- | --- | --- | | the header alone | `$header: Goals` | one node — the heading line | | the section — header and contents | `$section: Goals` | the full tree | | the contents only | `$within: Goals` | the tree minus its root | Each consumer takes from the forest what it needs: membership (`$content` in filter) tests that it is non-empty; `$content` in projection renders it; `$blocks` flattens it to data; the unit update operators act on its roots; `$within` drops them. ### Block projection Three projection sources consume a block predicate. Each takes the predicate in an options mapping; the bare form is the empty-predicate shorthand. | Source | Projects | | --- | --- | | `{ $content: P }` | the document body narrowed to the selected blocks, rendered at their original depth — a string | | `$blocks` / `{ $blocks: P }` | one entry per selected block: `type`, `path` (enclosing section titles), `text` (own text) | | `{ $matches: REGEX }` | grep — one entry per matching line: `path` plus the full line as `text` | ```yaml project: key: $key toc: { $content: { $header: {} } } # headers-only rendering notes: { $content: { $section: Unreleased } } # one section, as it appears hits: { $blocks: { $within: Goals, $text: "Q3" } } # located blocks as data found: { $matches: "(?i)todo|fixme" } # grep lines ``` Bare `$content` is `{ $content: {} }` — every block, the full body. A matched document in which no block matches projects `""`. Multiple narrowed entries may coexist in one projection, each with its own predicate. **`$content` narrowing** renders each selected block's own content in document order, at its original depth, normalized. Nothing renders that was not selected; subtrees appear when the predicate selects them (`$section`, `$within`), never by implication. Re-assembly is structural: selected items whose list is not selected re-wrap into a list of the original type, and selected blocks inside a selected quote render inside the quote wrapper (`$within: { $quote: {} }` renders them without it). **`$blocks`** entries carry `type` (the operator name without the `$`), `path`, and `text` (the own text). A header's entry carries its heading text, not its section — use `$content` narrowing to read bodies. Every entry carries `text` — `""` for blocks without own text; a ref entry additionally carries `target`, its `text` holding the authored (piped) link text or `""` when there is none. The entry count is the number `$replaceText`'s `expect` pins; a tree selection lists every block of its trees flattened, so it over-counts unit-operator targets — list those by selecting the roots as nodes (a `$section: P` previews as `$header: P`). **`$matches`** applies the pattern to the own text of every block in scope and reports one entry per matching line, in document order — the full line, not the matched substring, and never the whole block. The argument is a regex scalar, or a mapping combining the pattern with a scope — bare `pattern` is the payload, `$`-keys select: `{ $matches: { pattern: "(?i)todo", $within: Goals } }`. In `find` markdown output, `$blocks` and `$matches` entries render one grep line each — `key › section path › text` — under the result's index line; a narrowed `$content` field switches to the fenced-block form like bare `$content`. See [`iwe find`](/docs/agentic/find/#block-projection). #### Top-level form `project` (not `addFields`) also accepts a block predicate directly in place of the field map — any `$`-key at the top level selects this reading, and the result carries the document's `key` plus a `content` field with the narrowed body: ```yaml project: { $header: {} } # the headers-only form of each document # exactly the same as project: key: $key content: { $content: { $header: {} } } ``` Top-level keys conjoin as in any block predicate — `project: { $header: {}, $within: Usage }` is the table of contents of one section. Mixing bare and `$`-keys is a parse-time error, `project: {}` keeps its v1 meaning (an explicit empty projection — each result carries no fields), and the projection sources are not predicate operators — `project: { $blocks: P }` is an unknown-operator error (the data reads keep their field names), and a top-level `$matches` reads as the block-predicate operator, never the grep source. ### Block update operators Block operators live directly in the `update` document, as siblings of `$set` and `$unset`. Every update operator is an *(address, payload)* pair: `$set` addresses by field path, the block operators address by block predicate. An operator's argument is one flat mapping — the `$`-keys form the predicate, the bare keys are the payload and the optional `expect` guard. The bare-key set per operator is closed; unknown bare keys are parse-time errors. Each operator appears at most once per update document; applying one operator to several independent selections requires several operations. | Operator | Payload | Effect per target | | --- | --- | --- | | `$replace` | `content` (markdown) | Replace the target as selected — a tree target wholesale, a node target as its own line(s); on a header node, a heading payload retitles the section | | `$replaceText` | `from` (optional), `to` | With `from`: replace that substring of the block's own text with `to`. Without `from`: `to` replaces the entire own text | | `$insertBefore` | `content` (markdown) | Insert sibling content before the target | | `$insertAfter` | `content` (markdown) | Insert sibling content after the target — after a header node: directly below the heading line, the first position of its section; after a `$section`: below the whole tree | | `$append` | `content` (markdown) | Append child content at the end of the block (containers only — a header: at the end of its section; an item: after its nested blocks; a list: after its last item; a quote: after its last block) | | `$delete` | — | Remove the target as selected — a `$section` with its contents; a bare `$header` dissolves into its enclosing section | ```yaml filter: $content: { $text: "Q3" } update: $set: { reviewed: true } $replaceText: $within: Goals $text: "Q3 Milestones" from: "Q3 Milestones" to: "Q3 2026 Milestones" expect: 1 $delete: $paragraph: { $references: archive/old-plan } ``` An argument with no `$`-keys carries the empty predicate and selects **all blocks**: `$delete: {}` clears the document body — title header included — bounded by the required `filter` and guardable with `expect`. #### Targets and coalescing The unit operators (`$replace`, `$insertBefore`, `$insertAfter`, `$append`, `$delete`) act on **targets**: the selection's forest roots, each taken *as selected* — a tree root (`$section`, `$quote`, `$list`) is the whole tree, a node root is the block alone, never its children. This is the same rule projection follows: `$content: { $header: Goals }` renders one heading line, and `$delete: { $header: Goals }` removes one heading line — the predicate means the same thing at every site. Targets **coalesce by extent**: a target lying inside another target's extent is absorbed by the outer one — selecting a section and a paragraph within it yields one target, the section, so `$delete: { $section: {} }` deletes each outermost section once rather than tripping over its own descendants. Node targets never absorb one another: own texts are pairwise disjoint, so two node targets always name disjoint lines. Operators apply once per target, in document order. Insertions are **anchored**: inserted content stays adjacent to its target, so when `$insertAfter` on a block and `$insertBefore` on its following sibling name the same gap, each lands hugging its anchor. **`$replaceText`** operates on own text and applies to **every selected block, un-coalesced** — own texts are pairwise disjoint, so nested selections cannot conflict. A selected header's `$replaceText` edits the heading text and nothing below it; a selected ref's edits its authored link text, and the target key never changes. `from` is optional. When present it must occur **exactly once** in each selected block's own text, matched **byte-exact** against the normalized form — case-sensitive, unlike `$text`; the workflow is to echo text read from `$blocks` output. When omitted, `to` replaces the block's **entire own text** — renaming a header is `{ $header: Goals, to: Aims }`; pair the `from`-less form with a text-bearing predicate or an `expect` guard, since it carries no proof the agent read what it overwrites. A selected block with no own text (a quote or list root, a horizontal rule, a ref without authored text) or a table is a validation error. **Payload markdown** (`content`) is parsed and normalized on write. Heading levels inside a supplied fragment define nesting within the fragment only; the fragment's root level is set by its attachment point — child of the target for `$append`, sibling of the target for the insertions, the target's own position for `$replace`. At a list attachment point — sibling of an item, child of a list — the fragment must parse as a single list; its items attach, and the target keeps its own list type. #### Headers: retitle, dissolve, remove, clear Every unit edit splices the target's lines in the normalized rendering; the document re-parses, and normalization re-derives nesting and header levels from the result. Headers are where the node/tree distinction matters — the three selections become three different operations: | Selection | `$delete` | `$replace: { content: C }` | | --- | --- | --- | | `$header: Goals` — the heading line | **Dissolve**: the heading line is removed; the former contents re-attach to the enclosing section and re-level | C splices at the heading line. A heading payload **retitles** — the former contents re-nest beneath it; any other payload dissolves the section around it | | `$section: Goals` — header and contents | **Remove**: the header and every block below it | C replaces the whole section | | `$within: Goals` — contents only | **Clear**: the header stays, its blocks go | C replaces each content target | Against ```markdown # Roadmap ## Goals Ship the editor integration ### Q3 Milestones Deliver block operations spec ``` `$delete: { $header: Goals }` yields ```markdown # Roadmap Ship the editor integration ## Q3 Milestones Deliver block operations spec ``` — the paragraph re-attaches to `Roadmap`, and `Q3 Milestones` re-levels from `###` to `##`. Re-leveling ripples through the whole former subtree: after a header edit, expect level changes below the splice point. `$replace: { $header: Goals, content: "## Aims" }` retitles — the same document with `## Aims` in place of `## Goals`, contents intact; the payload's own level is advisory, since normalization re-levels to the attachment point. The destructive whole-section forms require naming the tree: `$delete: { $section: Goals }`. The asymmetry of mistakes is deliberate: deleting a header node when the section was meant leaves visible, recoverable content behind; the reverse mistake under subtree semantics would silently destroy a section. The intuition is the text editor's — deleting a heading line does not delete the text under it. `$replace` is per-target, never "set the body to C": against a body with three top-level blocks, `$replace: { content: X }` yields `X` three times. Whole-body replacement is the CLI body-overwrite mode; a single-target `$replace` is guarded with `expect: 1`. #### Combining operators `$set`, `$unset`, and block operators combine freely in one update document and apply atomically per matched document. Block operator applications must be **pairwise disjoint by extent** after coalescing: no two applications may touch the same block, or a tree target and anything within it — a `$delete` taking the Goals *section* while a `$replaceText` edits a paragraph inside it is a validation error listing the pair, nothing written. A `$delete` of the Goals *header node* beside the same `$replaceText` is legal: the heading line and the paragraph are disjoint extents. Frontmatter operators apply to **every** filter-matched document regardless of block selections — membership is `filter`'s alone. To restrict a combined operation to documents where the block edit lands, conjoin `$content` into the filter with the same predicate: ```yaml filter: status: active $content: { $text: "Q3" } update: $set: { reviewed: true } $replaceText: { $text: "Q3", from: "Q3", to: "Q3 2026" } ``` Drop the `$content` line and `reviewed` is stamped on every `active` document while the text edit lands only where `Q3` occurs. ### `expect` guards Any block operator application may carry `expect: N` or `expect: { min: M, max: N }` as a bare key, asserting the number of applications it will make, counted across all mutated documents (post-`limit`): - The **unit operators** count **targets** — the coalesced roots: one section, however many blocks it holds, is one target, so `$delete: { $section: Goals, expect: 1 }` passes. - **`$replaceText`** counts selected **blocks**, un-coalesced, matching its per-block application. `update` and `delete` operation documents additionally take a **document-level** `expect` — a top-level clause, sibling of `filter` / `sort` / `limit` — asserting the number of matched documents the operation will write. In a `find` or `count` operation a top-level `expect` is a parse-time error. The two levels are independent quantities: a document-level `expect: 3` beside a block-level `expect: { min: 3 }` says three documents will be written with at least three blocks touched across them. On violation the whole operation fails before writing anything; the error reports the actual count plus each target as `key › section path › first line of own text` (documents as `key › title`). `expect: 1` is the single-target precision edit; `expect: 0` is legal and asserts an empty selection. ```yaml # Bounded cleanup: delete completed task paragraphs, refusing a runaway match filter: { type: meeting } update: $delete: $paragraph: { $matches: "^DONE " } expect: { max: 20 } ``` ### Validation and atomicity The operation validates fully before writing anything, anywhere. **Parse time:** unknown operators; unknown bare keys in an operator argument; invalid payload types; regex that fails to compile; a `$within` argument that cannot select content; block operators in a `find` / `count` / `delete` operation. **Evaluation time**, against the original normalized documents — every selection and anchor position is resolved before any edit applies — in order: 1. **Type compatibility** — `$append` on a non-container block; `$replaceText` selecting a block with no own text or a table; a `content` payload that cannot attach at its site. The error lists every offending block. 2. **`$replaceText` anchors** — a given `from` absent or occurring more than once in any selected block's own text. 3. **Disjointness** — block-operator applications with overlapping extents. The error lists the pairs. 4. **`expect`** — any violated guard, block-level or document-level. Any failure aborts the whole operation: frontmatter operators write nothing if a block operator fails validation, and no document is written if any document fails. Per document, frontmatter and block edits commit as one atomic rewrite. ### Strict mode The `expect` guards are optional in the language: a human at a terminal, with git behind the store, should not pay ceremony for a bulk edit they can see. Agents should. Strictness is therefore a property of the **surface**, not the grammar — the same operation document is valid everywhere; a strict surface refuses to *run* an unguarded mutation. - **CLI**: [`iwe update`](/docs/agentic/update/) and [`iwe delete`](/docs/agentic/delete/) take `--strict`. Under the flag, every mutating application must carry its guard — the operation's document-level `--expect` and each block operator's `expect`. A missing guard is an error before anything runs. - **MCP**: the [`iwe_query`](/docs/agentic/mcp/) tool is always strict, no opt-out; an unguarded mutation is refused with the missing guards named. `--dry-run` writes nothing and is exempt from strict mode — it is how the counts are learned: dry-run the mutation, read the matched documents, pin `expect` to what it shows, re-run. An agent that just located its targets with `$blocks` already knows the counts. ### Locate, then mutate The agent workflow is two operations sharing one predicate: ```yaml # 1. Locate (read) filter: $includedBy: { match: { $key: projects/roadmap }, maxDepth: 3 } $content: { $within: Goals, $text: "Q3" } project: key: $key hits: { $blocks: { $within: Goals, $text: "Q3" } } ``` ```yaml # 2. Mutate (same predicate, now inside the operator) filter: $includedBy: { match: { $key: projects/roadmap }, maxDepth: 3 } update: $replaceText: $within: Goals $text: "Q3 Milestones" from: "Q3 Milestones" to: "Q3 2026 Milestones" expect: 1 ``` ## CLI lowering On the CLI, structural anchor flags lower to graph operators. A `KEY[:DEPTH]` suffix sets `maxDepth` (or `maxDistance`) for that anchor; depth `0` is the unbounded sentinel. | CLI flag | Lowers to | | --- | --- | | `-k KEY` | `$key: KEY` (1 key = `$eq`; 2+ = `$in`) | | `--includes KEY` | `$includes: KEY` (scalar shorthand, depth 1) | | `--included-by KEY:5` | `$includedBy: { match: { $key: KEY }, maxDepth: 5 }` | | `--references KEY:0` | `$references: { match: { $key: KEY }, maxDistance: 0 }` (unbounded) | | `--referenced-by KEY` | `$referencedBy: KEY` | | `--max-depth N` | session default for `--includes` / `--included-by` (default 1) | | `--max-distance N` | session default for `--references` / `--referenced-by` (default 1) | | `--filter "EXPR"` | inline YAML filter document | | `--project "EXPR"` | `project: EXPR` — comma list (`title,author`, `body=$content`, bare `$blocks`) or inline YAML mapping (find, tree) | | `--add-fields "EXPR"` | `addFields: EXPR` — same grammar as `--project`, extends the defaults (find, tree) | | `--sort field:1` / `--sort field:-1` | `sort: { field: 1 / -1 }` (find only) | | `--sort field:DIR:binary` / `--sort field:DIR:sv-SE` | `sort: { field: { order: DIR, collation: binary / sv-SE } }` (find only) | | `-l, --limit N` | `limit: N` (find, count) | | `--blocks "PRED"` | `addFields: { blocks: { $blocks: PRED } }` (find only) | | `--matches PATTERN` | `filter: { $content: { $matches: PATTERN } }` **and** `addFields: { matches: { $matches: PATTERN } }` (find only) | | `--set FIELD=VALUE` | `$set: { FIELD: VALUE }` (update only; repeatable) | | `--unset FIELD` | `$unset: { FIELD: "" }` (update only; repeatable) | | `--replace`, `--replace-text`, `--insert-before`, `--insert-after`, `--append`, `--delete` `"ARG"` | one block-operator entry (`$replace`, `$replaceText`, …) in the `update` document (update only) | | `--expect VAL` | the document-level `expect` clause (update, delete) | | `--strict` | surface policy, not grammar: requires the `expect` guards on every mutating application (update, delete) | All filter flags AND together. For OR or NOR, write the composition inside `--filter`: ```bash iwe find --filter '$or: [{ status: draft }, { status: review }]' iwe find --filter '$nor: [{ status: archived }]' ``` `--matches` is a composite flag — it lowers to both a membership clause and a projection entry, because one-flag grep is the point. Each block-operator flag's argument is that operator's `{ $selector…, payload… }` mapping — the operator name is the flag, so the `$replaceText:` wrapper is dropped. Combining `-k KEY` with a `--filter` whose top level also contains `$key` is a parse-time error — pick one source, or use `-k a -k b` for multi-key match. --- # IWE Query Language Specification URL: https://iwe.md/docs/concepts/spec/ ## 1. Overview This document specifies the IWE query language: a YAML 1.2-based, MongoDB-style language for selecting, shaping, and mutating documents in an IWE workspace. It covers: - The **corpus model** — what a document is, the edge model, field names. - The four **operations** — find, count, update, delete — and the shape of an operation document. - The **filter language** — operators, types, composition. - **Graph operators** — identity and relational walk operators over inclusion and reference edges. - **Projection** — MongoDB-style output shaping with structural pseudo-field sources. - **Sort, limit, update operators** — ordering, capping, and mutation constructs. - The **CLI surface** — flags, lowering rules, deprecated aliases. - **Output formats** — markdown, keys, JSON, YAML, dot, csv, and mutation prose. - A **formal grammar** reference (Appendix A). ## 2. Corpus model ### 2.1 Documents A **document** is the parsed frontmatter of one note. Documents are mappings from string keys to YAML 1.2-typed values: strings, numbers, booleans, null, lists, mappings, dates, datetimes. Under YAML 1.2, bare `yes`, `no`, `on`, and `off` are plain strings, not booleans (unlike YAML 1.1). Notes with no frontmatter participate in the corpus as documents with an empty mapping (`{}`). They never match presence-style filters like `{status: draft}` but do match `{status: {$exists: false}}`. ### 2.2 The corpus The **corpus** is every document in the IWE workspace. ### 2.3 Field names the language cannot address The engine **preserves every frontmatter field**. Whatever a document carries on disk survives `find` output, `$frontmatter` projection, `update` writeback, `create --set` and template prepend, and is visible to schema validation and inference. The engine never deletes a user field. One name shape cannot be *addressed*: a field-path segment whose **first character is `$`**. `$`-prefixed names are the operator vocabulary everywhere in the language (`$eq`, `$set`, `$includedBy`, etc.), so a path segment beginning with `$` is a **parse-time error** wherever a path is parsed — filter keys, dotted paths, sort keys, projection sources, `$set` / `$unset` targets, and their CLI spellings. A `$`-named field is stored, returned and validated like any other field; it simply cannot be reached by a filter, sort, projection or update path. Every other leading character — letter, digit, `_`, `#`, `@`, hyphen, slash, parenthesis, etc. — is an ordinary field name: addressable, sortable, projectable, and listed by schema inference. Subsequent characters within a name are unconstrained per YAML rules, with one exception: a literal `.` is reserved as a path separator (§4.4) and cannot appear inside a single segment. Beyond the `$` and dot rules, a field-path segment used in a filter, projection, sort, or update path must be a **non-empty** string with **no Unicode whitespace** (leading, trailing, or embedded) and **no Unicode control characters**. An empty-string segment, a whitespace-only segment, or a segment containing control characters is a parse-time error. Other characters — digits, hyphens, slashes, parentheses, Unicode letters — are unrestricted. This rule is what makes the operator vocabulary safe: because the parser dispatches on the leading `$`, a `$`-prefixed key in a filter or update document is always read as an operator, never as a field name. ### 2.4 Edge model IWE's corpus graph contains two kinds of directed edges between documents: - **Inclusion edges** — structural transclusion links. When document A includes document B, B's content is rendered inline as part of A. - **Reference edges** — non-structural links, including inline mentions inside text. A document can reference another without including it. Both edge kinds form general directed graphs that may contain cycles, including self-loops. Walks over them terminate via visited-set tracking; see §5.5.1. Both edge kinds are directed. Direction-of-read convention for the relational operators (§5.2): | Operator | Reads as | This doc → anchor? | Anchor → this doc? | |---|---|---|---| | `$includes` | this doc includes an anchor | yes (outbound inclusion) | no | | `$includedBy` | this doc is included by an anchor | no | yes (inbound inclusion) | | `$references` | this doc references an anchor | yes (outbound reference) | no | | `$referencedBy` | this doc is referenced by an anchor | no | yes (inbound reference) | The "anchor" is one of the documents selected by the operator's argument. Relational operators take a `match` filter that resolves to an anchor set (§5.2); a relational predicate matches when this document stands in the named relation to at least one document in that set. ## 3. Operations and operation documents ### 3.1 Operations | Operation | Returns / effect | |---|---| | `find` | Returns matched documents (subject to `project`, §6). | | `count` | Returns the integer count of matched documents. | | `update` | Mutates each matched document by applying an update document (§9) — frontmatter operators, and the block operators of the block surface (§9.5). | | `delete` | Removes each matched document. | ### 3.2 Operation-document structure Every operation document is one YAML mapping. Top-level fields: | Field | Operations | Purpose | |---|---|---| | `filter` | all | Predicate document (§4). Required on `update` / `delete`. Graph operators that extend filter with cross-document selection are defined in §5. | | `search` | find | Relevance selection stage (§5A). Restricts membership to documents matching a `lexical` / `fuzzy` text query and supplies the default ordering. Semantics: [Search](/docs/concepts/query-language/#search-find-only). | | `project` | find | Projection (§6). Mutually exclusive with `addFields`. | | `addFields` | find | Additive projection (§6.3). Mutually exclusive with `project`. | | `sort` | all | §7. On `update` / `delete`, bounds iteration order before mutation. | | `limit` | all | §8. On `update` / `delete`, bounds the number of mutated / removed docs. | | `update` | update | Update document (§9). Required on `update`. | | `expect` | update, delete | Document-level guard: asserts the number of matched documents the operation will write (post-`limit`), refusing the whole operation on violation. Grammar: §A.7. Semantics: [`expect` guards](/docs/concepts/query-language/#expect-guards). | Operation-inappropriate fields are an error. The valid field set per operation: | Operation | Allowed fields | |---|---| | `find` | `filter`, `search`, `project`, `addFields`, `sort`, `limit` | | `count` | `filter`, `sort`, `limit` | | `update` | `filter` (required), `sort`, `limit`, `update` (required), `expect` | | `delete` | `filter` (required), `sort`, `limit`, `expect` | E.g. `project` in a `count` / `update` / `delete` operation, `update` in a `find` / `count` / `delete` operation, `search` in a `count` / `update` / `delete` operation, or `expect` in a `find` / `count` operation, are parse-time errors. `filter` is required on both `update` and `delete` to prevent accidental whole-corpus mutation. The empty filter `{}` matches all documents and must be passed explicitly. Example — a complete `find` operation document combining selection, projection, sort, and limit: ```yaml filter: $or: - $key: projects/alpha - $includedBy: { match: { $key: projects/alpha }, maxDepth: 5 } status: draft priority: { $gte: 5 } project: title: 1 modified_at: 1 sort: modified_at: -1 limit: 100 ``` Example — an `update` operation document: ```yaml filter: $or: - $key: projects/alpha - $includedBy: { match: { $key: projects/alpha }, maxDepth: 5 } status: draft priority: { $gte: 5 } sort: modified_at: -1 limit: 100 update: $set: flagged: true review_state: needs-review ``` Example — a `delete` operation document: ```yaml filter: $or: - $key: archive/2024 - $includedBy: { match: { $key: archive/2024 }, maxDepth: 5 } status: archived limit: 500 ``` ## 4. Filter language A filter document is a predicate evaluated against each document in the corpus. A document matches when every top-level key matches. Filter top-level keys are either user frontmatter field names (e.g. `status`, `priority`, `tags`) or `$`-prefixed operator names. The operator family includes the document-level logical operators (`$and`, `$or`, `$nor`; §4.6), the **graph operators** (`$key`, `$includes`, `$includedBy`, `$references`, `$referencedBy`) defined in §5, and the block-membership operator **`$content`** (grammar: §A.3; semantics: [Content membership](/docs/concepts/query-language/#content--content-membership)). All of these compose freely with frontmatter predicates under the same algebra. `$not` exists only as a **field-level** operator (§4.6); document-level negation is expressed via `$nor`. ### 4.1 Implicit equality (bare values) A bare value at a field key is an equality predicate: ```yaml filter: status: draft ``` Matches documents where `status` equals `"draft"`. The behavior of "equals" depends on the value type and the field type — see §4.5 for the full rule. The short version: | Predicate value | Field value | Matches when... | |---|---|---| | Scalar (string / number / bool / null / date) | Scalar | Values are deeply equal. | | Scalar | Array | Any element of the array deeply equals the scalar (membership). | | Array | Array | Arrays are deeply equal (same elements, same order). | | Mapping | Mapping | Mappings are deeply equal. | | Anything | Missing field | Never matches. | | Anything | Type mismatch | Never matches. | ### 4.2 Operator expressions A mapping value whose keys are all `$`-prefixed is an **operator expression**: ```yaml priority: { $gt: 3 } ``` This is unambiguous because user field names cannot begin with `$` (see §2.3). Any `$`-prefixed key in a filter is always an operator, never a field reference. Multiple operators in one expression are ANDed: ```yaml priority: { $gte: 3, $lte: 7 } # 3 ≤ priority ≤ 7 ``` A mapping with `$`-prefixed and bare keys at the same level behaves differently depending on **where** the mapping appears: - **At a document-matching position** — the filter root, every element of `$and` / `$or` / `$nor`, or the `match:` clause of a graph anchor — bare keys and operator keys may freely coexist. They are combined with **implicit AND** (consistent with §4.3 and §4.6). ```yaml type: tracker $or: - status: open - status: pending ``` is equivalent to ```yaml $and: - type: tracker - $or: - status: open - status: pending ``` - **At a field-value position** — inside `{ field: { ... } }` — the mapping's keys must be either *all* `$`-prefixed operators or *all* bare nested-field references. Mixing is an error because a bare key inside a field-value mapping is ambiguous: it could be a nested-field path or an argument to a sibling operator. ```yaml # OK — operator expression author: { $eq: alice } # OK — nested fields author: name: alice role: admin # ERROR — mixed (is `name` a nested field of `author`, or an argument to `$eq`?) author: $eq: alice name: alice ``` The same rule applies inside the body of a field-level `$not`: `{ field: { $not: { $gt: 5, x: 1 } } }` is rejected for the same reason. ### 4.3 Multiple keys are ANDed Multiple top-level keys in a filter combine with AND: ```yaml status: draft priority: { $gt: 3 } tags: rust ``` A document matches if every top-level key matches. To express OR, wrap with `$or` (§4.6). ### 4.4 Nested fields Nested fields can be addressed two ways. Both forms are equivalent: **Nested mapping:** ```yaml author: name: alice ``` **Dotted-key shorthand:** ```yaml author.name: alice ``` Mixing forms in a single filter is allowed: ```yaml status: draft author.name: alice review: reviewer: alice ``` Dots inside the key string always denote path separators. **Frontmatter fields whose name contains a literal `.` are not addressable**: neither the dotted shorthand nor the nested-mapping form can reference such a field, because path resolution always splits on `.` after YAML parsing. Quoting the dotted name in the source (`"foo.bar"`) does not change this — the parser still sees a string with a dot and splits it. Document authors should avoid creating field names that contain `.`. Operator expressions on a dotted key carry the same shape as on a nested key: ```yaml priority: { $gt: 3 } # top-level author.priority: { $gt: 3 } # nested via dotted shorthand author: { priority: { $gt: 3 } } # equivalent ``` #### Resolution rules When evaluating a nested-field predicate: - If any intermediate path component is missing, or is present but not a mapping, the leaf is treated as **missing** (never matches an equality / comparison; matches `$exists: false`). - If the intermediate path leads to a mapping, evaluation continues recursively. Example: filter `author.name: alice` against document `{ author: "alice" }` (where `author` is a string, not a mapping) — the leaf `author.name` is missing; the predicate does not match. ### 4.5 Equality, types, and missing fields These rules ground every operator in §4.6–§4.9. #### Deep equality Two values are equal when they are the same YAML type and deeply equal: - **Scalars** — strings match by codepoint sequence; numbers by numeric value (integer and float interoperate: `3` equals `3.0`); booleans by identity; null by identity; dates / datetimes by chronological identity. - **Arrays** — same length, element-wise deep equality, in order. - **Mappings** — same key set, value-wise deep equality. Cross-type comparisons are **always false** — there is no implicit coercion. `1` (number) does not equal `"1"` (string). `true` does not equal `"true"`. A YAML date does not equal a string of the same shape. #### Array membership exception When the predicate value is a **scalar** and the field's value is an **array**, equality tests membership: the scalar must deeply equal at least one element. This is the MongoDB convention. It applies to `$eq`, bare scalars, `$ne`, `$in`, `$nin`, and the comparison operators (`$gt`, etc.). To test whole-array equality, write the predicate as an array literal: ```yaml tags: [rust, async] # whole-array equality (length-2 array, in order) tags: rust # membership ("rust" is one of the tags) tags: { $eq: rust } # membership (same as bare scalar) ``` #### Null vs missing A frontmatter field with explicit value `null` is **present** with value `null`: - Matches `$eq: null` and bare `null`. - Matches `$exists: true` and `$type: "null"`. - Does NOT match `$exists: false`. A field absent from frontmatter is **missing**: - Does NOT match `$eq: null` (or any `$eq`). - Matches `$exists: false`. - Does NOT match `$type` of any kind (use `$exists: false` for absence). - Comparison operators (`$gt`, `$gte`, `$lt`, `$lte`) are always false against missing fields. - `$ne: x` and `$nin: [...]` match missing fields (consistent with MongoDB: "not equal to x" includes "doesn't exist"). #### Type bracketing for ordering `$gt`, `$gte`, `$lt`, `$lte` only compare values within a comparable type group: | Group | Members | Order | |---|---|---| | numeric | integer, float | numerical | | string | string | Unicode codepoint | | boolean | boolean | `false < true` | Cross-group comparison is always false (e.g. comparing a number with a string is false; a boolean with a number is false). Null is not orderable; ordering operators against null are always false. Use `$exists` / `$eq: null` to test for null explicitly. **Temporal values.** YAML date and datetime scalars are stored on the wire as strings — the engine's `Value` type does not carry a distinct temporal variant (§4.8 preserves the `date` / `datetime` names for `$type` matching only). Ordering operators on temporal-shaped values therefore reduce to the **string** group above: lexicographic Unicode-codepoint comparison. For ISO-8601 forms (`YYYY-MM-DD`, `YYYY-MM-DDTHH:MM:SS[Z|±HH:MM]`) lexicographic ordering is equivalent to chronological ordering, which is the only form documents and filters are expected to use. Mixing ISO-8601 with non-ISO date strings produces undefined ordering. #### Common YAML pitfalls Filter values are parsed by the YAML resolver before they reach the language. The resolver promotes bare scalars based on their lexical form, which can cause filters to silently never match documents whose stored values have a different type: | Filter source | Parses as | Document stores | Match? | |---|---|---|---| | `modified_at: 2026-01-01` | date scalar | string `"2026-01-01"` | no — date vs string is cross-group, always false | | `priority: "3"` | string `"3"` | integer `3` | no — string vs number | | `active: "true"` | string `"true"` | boolean `true` | no — string vs boolean | When in doubt, quote the value to force the string type, or leave it bare to accept YAML's auto-resolution. Equality is type-strict; there is no implicit coercion. Keep the document and the filter on the same side of the quoting boundary. ### 4.6 Logical operators Three document-level operators compose filters: `$and`, `$or`, `$nor`. They mirror MongoDB's logical operators and may appear at the filter root or as siblings of bare field keys (§4.2). A fourth operator, `$not`, exists only at the **field level** for negating a single field-value operator expression. #### `$and: [filter1, filter2, ...]` All listed filters must match. ```yaml $and: - status: draft - priority: { $gt: 3 } ``` - Every list element is a filter document. - A document matches if every sub-filter matches. - **Empty list** `$and: []` is a parse-time error. - `$and` is **implicit at the top level** — multiple top-level keys in a filter are already ANDed (§4.3). Use explicit `$and` when you need to wrap a sub-expression for use inside `$or` / `$nor`, or when you need to repeat a field name across multiple sub-filters (a YAML mapping cannot have duplicate keys). ```yaml # Two ranges on `priority` — needs $and to repeat the key $and: - priority: { $lt: 3 } - priority: { $gt: 0 } ``` #### `$or: [filter1, filter2, ...]` At least one of the listed filters must match. ```yaml $or: - status: draft - status: review ``` - Every list element is a filter document. - A document matches if at least one sub-filter matches. - **Empty list** `$or: []` is a parse-time error. - Sub-filters are independent — each is evaluated against the whole document. #### `$nor: [filter1, filter2, ...]` None of the listed filters may match. Use `$nor` for document-level negation: `$nor: [filter]` reads "documents where `filter` does not match". ```yaml $nor: - status: archived - status: deleted - tags: spam ``` - Every list element is a filter document. - A document matches if **every** sub-filter fails to match. - **Empty list** `$nor: []` is a parse-time error. - Sub-filters are independent — each is evaluated against the whole document. - **Missing-field interaction:** a sub-filter that fails because the field is missing counts as a non-match, contributing to a `$nor` match. To require presence and inequality, combine `$exists: true` with the negative predicate: `$nor: [{ reviewed: true }]` matches both "reviewed is false" and "reviewed is missing"; use `reviewed: { $exists: true, $ne: true }` to require the field be present. #### `$not: { $op: ... }` — field-level only `$not` is **field-level only** and wraps a single operator expression on the field: ```yaml priority: { $not: { $gt: 5 } } # priority is not > 5 (and is present) priority: { $not: { $gt: 5, $lt: 10 } } # NOT (5 < priority < 10) ``` - Body must be an operator expression — a mapping of `$`-prefixed comparison/element operators on the field. Bare nested-field references inside the body are rejected (§4.2). - Negates the inner field operator(s). - For document-level negation (negating a whole-document predicate, including a graph anchor or compound `$or`), use `$nor: [filter]` — there is no document-level `$not`. Writing `$not:` at the filter root is a parse-time error with a hint pointing to `$nor`. ### 4.7 Comparison operators #### `$eq: VALUE` Matches when the field's value equals VALUE. ```yaml status: { $eq: draft } ``` - Equivalent to bare value (`status: draft`); see §4.1. - Type-aware deep equality (§4.5). - Array membership rule applies when VALUE is scalar and the field is an array (§4.5). - Missing field never matches. #### `$ne: VALUE` Matches when the field's value does not equal VALUE. ```yaml status: { $ne: archived } ``` - Logical negation of `$eq`. - **Missing field matches** `$ne` (consistent with MongoDB). - For arrays with a scalar VALUE: `$ne: rust` matches arrays that do not contain `"rust"`. #### `$gt: VALUE` / `$gte: VALUE` / `$lt: VALUE` / `$lte: VALUE` Ordering comparisons. ```yaml priority: { $gt: 3 } modified_at: { $gte: 2026-01-01 } priority: { $gte: 3, $lte: 7 } # closed range [3, 7] ``` - `$gt` / `$lt` are exclusive; `$gte` / `$lte` are inclusive. - Type bracketing applies (§4.5): cross-group comparisons are always false. - Missing field is always false. - Arrays with scalar VALUE: matches if any element of the array satisfies the comparison. - Combining `$gt` and `$lt` (or `$gte` / `$lte`) in one operator expression yields a range; both endpoints must hold (operator expression is ANDed, §4.2). #### `$in: [v1, v2, ...]` Matches when the field's value equals any element of the list. ```yaml status: { $in: [draft, review] } tags: { $in: [rust, async] } # array → membership intersection ``` - Each list element is compared with the same equality rules as `$eq`. - The list elements may be of different types; each is tested independently. - Arrays with scalar list elements: matches if the field's array shares at least one element with the list (set intersection non-empty). - **Empty list** `$in: []` is a parse-time error. - Missing field never matches. #### `$nin: [v1, v2, ...]` Matches when the field's value is not in the list. ```yaml status: { $nin: [archived, deleted] } ``` - Negation of `$in`. - **Missing field matches** `$nin` (consistent with `$ne`). - **Empty list** `$nin: []` is a parse-time error. ### 4.8 Element operators #### `$exists: true | false` Tests presence vs. absence of the field. ```yaml reviewed_at: { $exists: true } draft_notes: { $exists: false } ``` - `$exists: true` matches when the field is present in the document. The value may be anything, including null. - `$exists: false` matches when the field is absent. - A field with explicit null is **present**: matches `$exists: true`. To distinguish, combine: `reviewed_at: { $exists: true, $ne: null }`. - For nested paths, the test is on the leaf. If any intermediate is missing or non-mapping, the leaf is treated as absent (§4.4). #### `$type: TYPE` or `$type: [TYPE, TYPE, ...]` Matches when the field's value has one of the given YAML types. ```yaml priority: { $type: number } ids: { $type: [string, number] } # accepts either type ``` Accepted type names: | Type | Matches | |---|---| | `string` | YAML strings (any encoding, any length, including the empty string). | | `number` | Integers and floats together. | | `boolean` | `true` / `false`. | | `null` | Explicit null value. | | `array` | Sequences (any length, any element type). | | `object` | Mappings. | | `date` | YAML date scalars (e.g. `2026-04-26`). | | `datetime` | YAML timestamp scalars (e.g. `2026-04-26T10:30:00Z`). | - A field with explicit null matches `$type: "null"` and no other type. - Missing field does not match any `$type`. Use `$exists: false` for absence. - The list form is OR over types: `$type: [string, number]` matches if the value is either. - **Empty list** `$type: []` is a parse-time error. Type names are matched as YAML strings. To test for the null type, write `$type: "null"` (quoted) — the bare YAML null literal `$type: null` is a parse-time error, because YAML resolves it to the null value rather than to a type-name string. The other names follow the same rule: `$type: number` is accepted because YAML resolves the bare word `number` to the string `"number"`; `$type: True` (which YAML resolves to a boolean) is a parse-time error. ### 4.9 Array operators These operators apply to fields whose value is an array. On non-array values (scalars, mappings, missing) they evaluate to **false** (no error). #### `$all: [v1, v2, ...]` Matches when the field's array contains every listed value. ```yaml tags: { $all: [rust, async] } ``` - Field must be an array. - Every element of the listed values must appear at least once in the field's array. Order is irrelevant; duplicates are irrelevant. - Element equality follows §4.5 (deep equality, type-strict). - **Empty list** `$all: []` is a parse-time error. #### `$size: count_pred` Matches when the length of the field's array satisfies a count predicate. ``` count_pred ::= non_neg_int | { count_op: non_neg_int, ... } count_op ::= $eq | $ne | $gt | $gte | $lt | $lte ``` ```yaml tags: { $size: 0 } # no tags (bare integer is $eq shorthand) authors: { $size: 1 } # exactly one author tags: { $size: { $gte: 3 } } # three or more tags tags: { $size: { $gte: 2, $lte: 5 } } # between 2 and 5, inclusive ``` - A bare integer is `$eq` shorthand. Each `count_op` value must be a non-negative integer (`$size: -1` is an error; `$size: 1.5` is an error). Multiple comparisons in the mapping AND together. - Field must be an array; non-arrays and missing fields → false. - The same `count_pred` grammar is reused by the relational operators' `$size` argument (§5.2.6). ### 4.10 Filter requirements (use-case checklist) The language MUST express the following queries directly: | Question | Filter | |---|---| | All drafts | `{status: draft}` | | Drafts modified this year | `{status: draft, modified_at: {$gte: 2026-01-01}}` | | Tagged either rust or async | `{tags: {$in: [rust, async]}}` | | Tagged with both rust and async | `{tags: {$all: [rust, async]}}` | | Has no tags | `{$or: [{tags: {$exists: false}}, {tags: {$size: 0}}]}` | | Reviewed but no reviewer | `{reviewed_at: {$exists: true}, reviewed_by: {$exists: false}}` | | Drafts not by alice | `{status: draft, author: {$ne: alice}}` | | Recent high-priority | `{$and: [{modified_at: {$gte: 2026-04-01}}, {$or: [{priority: {$gte: 8}}, {tags: urgent}]}]}` | ## 5. Graph operators Graph operators live inside filter documents alongside frontmatter predicates. They share the predicate algebra of filter: AND-composed at top level, composable under `$and` / `$or` / `$nor`, with the same operator-expression vocabulary as numeric frontmatter fields. Selection by graph relationship and selection by frontmatter content are written in the same filter document, distinguished only by whether the predicate key is a `$`-prefixed graph operator or a user frontmatter field name. The parser dispatch on the leading `$` (§2.3) makes this safe: a `$`-prefixed key in a filter is always an operator, so it never collides with a frontmatter field of the same name. | Category | Operator | Predicate over... | |---|---|---| | Identity (§5.1) | `$key` | the document's own key, by equality, membership or prefix | | Relational (§5.2) | `$includes` | the document's outbound inclusion relation to an anchor set | | Relational (§5.2) | `$includedBy` | the document's inbound inclusion relation to an anchor set | | Relational (§5.2) | `$references` | the document's outbound reference relation to an anchor set | | Relational (§5.2) | `$referencedBy` | the document's inbound reference relation to an anchor set | | Membership | `$content` | the document's blocks — at least one matching the block predicate (grammar: §A.3; semantics: [Content membership](/docs/concepts/query-language/#content--content-membership)) | Unknown `$`-prefixed operator names inside a filter are parse-time errors. Naming conventions: all operator names are camelCase, `$`-prefixed. The `$`-prefix is reserved for operators that evaluate; **walk parameters and payload field names inside operator arguments are bare-named** (`match`, `maxDepth`, `minDepth`, `maxDistance`, `minDistance`). They are configuration of the operator's walk, not operators in their own right. ### 5.1 Identity operator (`$key`) `$key` predicates the document's own key. #### 5.1.1 Argument shape `$key` accepts either a scalar key (implicit `$eq`) or an operator expression. ``` key_op ::= key | key_expr key_expr ::= { $eq: key } | { $ne: key } | { $in: [key, key, ...] } # non-empty array | { $nin: [key, key, ...] } # non-empty array | { $startsWith: key } # non-empty string ``` #### 5.1.2 Examples ```yaml filter: $key: notes/foo # implicit $eq $key: { $eq: notes/foo } # explicit $key: { $ne: drafts/scratch } # exclude one $key: { $in: [a, b, c] } # any of these $key: { $nin: [drafts/a, drafts/b] } # none of these $key: { $startsWith: notes/ } # everything under notes/ ``` #### 5.1.3 Constraints - `$key` accepts strings only. Operator expressions on `$key` use the comparison set above; `$gt` / `$gte` / `$lt` / `$lte` are parse-time errors (keys are identifiers, not ordered values). - A key expression carries exactly one operator. Two or more in the same mapping is a parse-time error. - Empty `$in: []` and `$nin: []` are parse-time errors. - Empty `$startsWith: ""` is a parse-time error: an empty prefix matches every document, which is never what the writer meant. - Every operand is normalized as a key before matching, so a trailing `.md` or `.dj` is stripped — `$startsWith: notes/foo.md` tests the prefix `notes/foo`. - `$startsWith` tests a **string** prefix of the key, not a path-segment prefix, and it is case-sensitive. `$startsWith: notes` matches `notes`, `notes/alpha` and `notes-archive/gamma` alike; `$startsWith: notes/` matches only documents under the directory. Selecting a subtree together with the hub note that names it is `$or: [{ $key: notes }, { $key: { $startsWith: notes/ } }]`. Document-level negation is `$nor`, as for every other filter clause. `$key` has only one role in the language: a top-level filter predicate over this document's own key. It also appears inside the `match` filter of relational operators (§5.2.2), but only because `match` is itself a filter document — there it carries the same semantics as any other filter-level `$key` predicate. ### 5.2 Relational operators The four relational operators predicate that the document being filtered stands in a graph relation to documents matching an anchor specification. The anchor specification is a `match` filter document — the full filter language, evaluated to select an anchor set. A **walk** is a BFS traversal from the anchor set over the operator's edge type, bounded by min/max depth (inclusion) or distance (reference). Each node appears at most once in a walk's result; cycles terminate via visited-set tracking. The detailed semantics — set vs path-multiset, BFS vs DFS, anchor exclusion, self-loops — are in §5.5.1. | Operator | True when this document... | Edge type | Walk parameters | |---|---|---|---| | `$includes` | has outbound inclusion edges to anchor docs within bounds | inclusion | `maxDepth`, `minDepth` | | `$includedBy` | has inbound inclusion edges from anchor docs within bounds | inclusion | `maxDepth`, `minDepth` | | `$references` | has outbound reference edges to anchor docs within bounds | reference | `maxDistance`, `minDistance` | | `$referencedBy` | has inbound reference edges from anchor docs within bounds | reference | `maxDistance`, `minDistance` | `$includes` and `$includedBy` walk only inclusion edges. `$references` and `$referencedBy` walk only reference edges. Walks only reach documents that exist in the workspace. A reference edge counts only for a link to a document (a `.md` or `.dj` file, or a path with no file extension); links to other local files such as `schema.yaml` or `image.png` are not edges. A link whose target document does not exist is skipped, so `$referencedBy` never returns a key that has no document. An anchor key that names no document, such as `$referencedBy: missing`, selects an empty anchor set, and the operator matches nothing. #### 5.2.1 Argument shape Each relational operator accepts either a scalar key (shorthand) or a mapping with an optional `match` field, optional walk parameters, and an optional `$size` count predicate: ``` relational_arg ::= key | relational_obj relational_obj ::= { match: filter (optional, default {} — any document) maxDepth: int >= 0 (inclusion ops only; optional, default 1; 0 = unbounded) minDepth: int >= 1 (inclusion ops only; optional, default 1) maxDistance: int >= 0 (reference ops only; optional, default 1; 0 = unbounded) minDistance: int >= 1 (reference ops only; optional, default 1) $size: count_pred (optional, default { $gte: 1 }; see §5.2.6) } ``` Walk-parameter field names inside `relational_obj` are bare-named — the `$`-prefix marks operators that evaluate. `$size` is the one exception: it is the operator that evaluates the relation's cardinality, so it keeps its `$`. The `match` field's value is a filter document; any `$`-prefixed names appearing inside it are filter-language operators, not walk configuration. A scalar key K is pure sugar for anchoring by identity: - For inclusion operators: `K` is equivalent to `{ match: { $key: K } }`. - For reference operators: `K` is equivalent to `{ match: { $key: K } }`. Under the default bounds (§5.2.3), both desugar to a direct-edge walk. Use the full mapping form to anchor by predicate, to widen the walk, to use range bounds, or to count with `$size`. Examples: ```yaml # Scalar shorthand — single-document anchor at direct edges (depth/distance 1) $includes: roadmap/q2 $includedBy: projects/alpha $references: people/alice $referencedBy: archive/index # Full form, match only — direct edges (maxDepth defaults to 1) $includedBy: { match: { $key: projects/alpha } } # Full form, unbounded — the whole tree $includedBy: { match: { $key: projects/alpha }, maxDepth: 0 } # Anchor by identity with explicit bounds $includes: { match: { $key: roadmap/q2 }, maxDepth: 2 } $includedBy: { match: { $key: projects/alpha }, maxDepth: 5 } # Anchor by frontmatter predicate $includes: { match: { status: draft }, maxDepth: 2 } $includedBy: { match: { status: active, type: project }, maxDepth: 5 } # Anchor by OR over predicates $includes: match: $or: - status: draft - tag: important maxDepth: 2 # Anchor by nested relational predicate $includes: match: $includedBy: { match: { $key: projects/alpha }, maxDepth: 5 } maxDepth: 2 # Range bounds $includedBy: { match: { $key: projects/alpha }, minDepth: 2, maxDepth: 5 } $referencedBy: { match: { $key: archive/index }, minDistance: 1, maxDistance: 3 } ``` #### 5.2.2 The `match` field `match` is a filter document. It accepts the full filter language: bare frontmatter fields, `$`-prefixed filter operators (`$key`, `$or`, `$and`, `$nor`, comparison operators, element operators, array operators), and **nested relational operators**. Nesting allows walks anchored at the result of another walk: ```yaml $includedBy: match: $includedBy: { match: { $key: projects/alpha }, maxDepth: 5 } maxDepth: 5 ``` `match` and the surrounding filter share one definition. Inside `match`, `$key` is the top-level identity operator from §5.1 — it accepts a scalar or any of the §5.1 key expressions (`$in`, `$nin`, `$eq`, `$ne`): ```yaml $includedBy: match: $key: { $in: [projects/alpha, projects/beta] } maxDepth: 5 ``` This subsumes what previous revisions of the spec called "OR-of-anchors" — write the OR inside `match`. #### 5.2.3 Walk parameters Walk parameters constrain how far the walk extends from the anchor set. Inclusion-edge operators (`$includes`, `$includedBy`) use `maxDepth` / `minDepth`: - `maxDepth: N` — walk includes levels 1 through N inclusive. - `minDepth: M` — walk excludes levels 1 through M-1; only levels ≥ M match. - Combining `minDepth: M, maxDepth: N` matches levels M through N inclusive (M ≤ N required; M > N is a parse-time error). Reference-edge operators (`$references`, `$referencedBy`) use `maxDistance` / `minDistance`: - `maxDistance: N` — walk includes hops 1 through N inclusive. - `minDistance: M` — walk excludes hops 1 through M-1; only hops ≥ M match. - Combining `minDistance: M, maxDistance: N` matches hops M through N inclusive (same M ≤ N constraint). Defaults in the full mapping form: - `maxDepth` / `maxDistance` absent → 1 (direct edges). - `minDepth` / `minDistance` absent → 1 (the walk starts at level / hop 1). - `maxDepth: 0` / `maxDistance: 0` → unbounded (the walk reaches every transitively related document). - Both bounds absent → a direct-edge walk over the relevant edge kind. Scalar-key shorthand desugars to `{ match: { $key: K } }` and inherits these defaults, so it too is a direct-edge walk; see §5.2.1. Wrong-category walk parameters (`maxDistance` / `minDistance` inside an inclusion-edge operator, or `maxDepth` / `minDepth` inside a reference-edge operator) are parse-time errors. Value constraints on walk parameters: - `maxDepth` / `maxDistance` are non-negative integers (≥ 0), where `0` is the unbounded sentinel. `minDepth` / `minDistance` are positive integers (≥ 1). Negatives, floats, strings, null, and operator expressions are parse-time errors. - `0` — meaningless as an edge count — is the single sentinel for "unconstrained", matching the CLI's `KEY:0` and the language's `limit: 0`. - A lone `minDepth`/`minDistance` ≥ 2 with no `maxDepth`/`maxDistance` is an inverted range (min > the defaulted max of 1) and is a parse-time error; set `maxDepth: 0` / `maxDistance: 0` to walk from that level to unbounded. Anchor exclusion: a relational operator never matches a document in its anchor set. `$includedBy: { match: { $key: K }, maxDepth: 5 }` matches the documents that K transitively includes within 5 levels but does not match K itself. More generally, a `match` that selects a set S contributes anchors S, and the walk's matches are documents reached *from* S — never S itself. To include the anchor set in the result, compose at the filter level: ```yaml $or: - $key: projects/alpha - $includedBy: { match: { $key: projects/alpha }, maxDepth: 5 } ``` #### 5.2.4 Composition A filter document may contain at most one occurrence of each top-level relational operator key (a YAML mapping cannot have duplicate keys). To express AND, OR, or NOT of multiple predicates using the same operator key, use the filter-level logical operators: ```yaml # AND of two $includedBy predicates with different bounds $and: - $includedBy: { match: { $key: projects/alpha }, maxDepth: 5 } - $includedBy: { match: { type: research, status: active }, maxDepth: 3 } ``` ```yaml # OR of two anchor sets — same edge type, different bounds $or: - $includedBy: { match: { $key: projects/alpha }, maxDepth: 5 } - $includedBy: { match: { $key: research/q2 }, maxDepth: 2 } ``` AND of multiple keyed anchors with the same bounds is also expressible by widening `match`: ```yaml $includedBy: match: $key: { $in: [projects/alpha, projects/beta] } maxDepth: 5 ``` #### 5.2.5 Empty argument and unknown fields The empty mapping `$includedBy: {}` is a parse-time error. `match` is optional — a mapping that omits it (for example `$includedBy: { $size: 0 }`) is well-formed and defaults `match` to `{}` (any document). The array form `$includedBy: []` is a parse-time error. The set of recognized keys inside a `relational_obj` is closed: `match`, `maxDepth`, `minDepth`, `maxDistance`, `minDistance`, `$size`. Any other key — including misspellings (`maxDepht`, `match_`) and other `$`-prefixed names — is a parse-time error. Implementations MUST reject unknown keys rather than silently ignoring them; this prevents typos from quietly widening or narrowing the walk. A `match` filter that selects no documents is well-formed but contributes an empty anchor set; a `$size`-free predicate then matches nothing, and a `$size` predicate counts zero related documents for every candidate. #### 5.2.6 Cardinality — `$size` The operator's argument denotes a **set**: the distinct documents standing in the named relation to the document under test, within `[min, max]` hops, matching `match`, always excluding the document itself (cycles included). - Without `$size`, the predicate holds iff that set is non-empty. This is the default `$size: { $gte: 1 }`; the two spellings are equivalent. - With `$size`, the predicate holds iff the set's cardinality satisfies the count predicate. `$size` reuses the `count_pred` grammar of §4.9: a bare non-negative integer (`$eq` shorthand) or a mapping of `$eq` / `$ne` / `$gt` / `$gte` / `$lt` / `$lte` comparisons that AND together. ```yaml $includedBy: { $size: 0 } # roots — nothing includes them $includes: { $size: 0 } # leaves — include nothing $referencedBy: { $size: 0 } # orphans — no inbound links $referencedBy: { $size: { $gte: 5 } } # hubs — 5+ direct referrers $includedBy: { $size: { $gte: 10 }, maxDepth: 0 } # 10+ transitive ancestors ``` `$size: 0` is bound-insensitive under the default `minDepth: 1`: any transitive relation implies a direct one, so zero direct edges means zero at every depth. ### 5.3 Graph operator composition with filter These operators participate in the filter language's predicate algebra exactly like other operators. Top-level AND — multiple top-level keys in a filter are AND-composed: ```yaml filter: $key: { $nin: [drafts/scratch, drafts/temp] } $includedBy: { match: { $key: projects/alpha }, maxDepth: 5 } status: draft ``` `$and` / `$or` / `$nor` — the logical operators wrap any filter document, including ones containing these operators: ```yaml filter: $or: - $key: archive/index - $includedBy: archive/index ``` Empty `filter: {}` matches every document. ### 5.4 Worked examples #### 5.4.1 Identity-based queries ```yaml # Direct lookup filter: $key: people/alice ``` ```yaml # Bulk fetch by key set filter: $key: { $in: [projects/alpha, projects/beta, projects/gamma] } ``` ```yaml # Anchor + descendants filter: $or: - $key: projects/alpha - $includedBy: { match: { $key: projects/alpha }, maxDepth: 5 } ``` ```yaml # Exclusion within a result set filter: $includedBy: { match: { $key: projects/alpha }, maxDepth: 5 } $key: { $ne: projects/alpha/private } ``` #### 5.4.2 Walk-based queries ```yaml # Documents directly under alpha — scalar shorthand fixes maxDepth: 1 filter: $includedBy: projects/alpha ``` ```yaml # Documents anywhere under alpha — full form, maxDepth omitted → unbounded filter: $includedBy: { match: { $key: projects/alpha } } ``` ```yaml # Documents under alpha within 10 levels filter: $includedBy: { match: { $key: projects/alpha }, maxDepth: 10 } ``` ```yaml # Documents at exactly depth 3 under alpha filter: $includedBy: { match: { $key: projects/alpha }, minDepth: 3, maxDepth: 3 } ``` ```yaml # Documents within 1 hop of alice filter: $references: people/alice ``` ```yaml # Documents 2 to 3 hops from the archive filter: $referencedBy: { match: { $key: archive/index }, minDistance: 2, maxDistance: 3 } ``` ```yaml # Documents under any active project filter: $includedBy: match: type: project status: active maxDepth: 5 ``` #### 5.4.3 Combined queries ```yaml # Documents under alpha that reference alice filter: $includedBy: { match: { $key: projects/alpha }, maxDepth: 5 } $references: people/alice ``` ```yaml # Documents under alpha, excluding the private namespace filter: $includedBy: { match: { $key: projects/alpha }, maxDepth: 10 } $key: { $nin: [projects/alpha/private] } ``` ### 5.5 Edge cases #### 5.5.1 Cycle handling IWE graphs (both inclusion and reference) are general directed graphs and may contain cycles, including self-loops. All walks — `$includes`, `$includedBy`, `$references`, `$referencedBy`, and any composition thereof — MUST track visited nodes and yield each node at most once per walk. A node already in the walk's visit set is skipped; its outgoing edges are not re-traversed. Self-edges (a node that includes or references itself) are degenerate cycles of length 1 and follow the same rule. Visited-set tracking is the primary termination mechanism. `maxDepth` / `maxDistance`, if specified, apply independently as an additional bound and do not substitute for cycle detection. Implementation requirements: - **Set semantics** — each node appears at most once per walk result. Path-multiset semantics (the same node appearing once per distinct path that reaches it) is not a valid implementation. - **Visit-set scope** — per-walk. Each relational operator evaluation maintains its own visit set; visit sets are not shared across nested or composed predicates. - **Traversal order** — BFS. Depth and distance bounds are well-defined under BFS and ambiguous under DFS. - **Depth / distance measure** — the shortest path from anchor to candidate (BFS-natural). - **Anchor self-results** — when a document has a self-edge, it does not appear in its own walk result. Anchor exclusion (§5.2.3) applies regardless of self-edges. **Existential vs cardinality evaluation.** A `$size`-free relational predicate (§5.2.6) is evaluated anchor-first: the `match` filter resolves to an anchor set, one walk runs from each anchor, and the reached documents are the result. A `$size` predicate is evaluated per-candidate: the `match` filter resolves once to a key set, then for each candidate document a walk runs from *that document* in the relation's direction, the distinct reached documents within `[min, max]` that are in the key set are counted (self always excluded), and the count is tested against the predicate. With the default `maxDepth`/`maxDistance` of 1, this is a direct-edge lookup and never traverses beyond one hop; `maxDepth: 0` counts full per-document reachability and is the expensive path, always visible in the query. #### 5.5.2 Other edge cases - **Empty corpus** — every relational predicate matches nothing. - **Empty anchor set** — a `match` filter that selects no documents (e.g. `match: { $key: typo }` against a corpus with no such key, or `match: { status: nonsense }`) contributes no anchors; the relational predicate matches nothing. Typos and stale references narrow the result rather than failing the operation. - **Disconnected graph** — walks operate per connected component; a walk anchored at K matches only documents reachable from K within bounds. - **Anchor exclusion** — a walk never matches a document in its anchor set. Use filter-level `$or` with `$key` (or with another predicate) to include the anchor set in the result. - **Default walk depth** — scalar-key shorthand fixes `maxDepth: 1` / `maxDistance: 1` (direct edges only). The full mapping form treats omitted `maxDepth` / `maxDistance` as unbounded; omitted `minDepth` / `minDistance` always default to 1. - **Operators inside `$nor`** — `$nor: [{ $includedBy: { match: { $key: K }, maxDepth: 5 } }]` matches documents that are *not* descendants of K within 5 levels. ## 5A. Search stage (find only) `search` is a top-level clause of a `find` operation (§3.2). It selects documents by relevance to a full-text query and, unless overridden by `sort`, supplies the result ordering. It is **not** a filter operator and **not** a sort variant: it both restricts membership and orders, jointly with `filter`. ```yaml search: lexical: "broken links cleanup" filter: { type: note } limit: 5 ``` The `search` document is a mapping with two optional keys: | Key | Ranker | |---|---| | `lexical` | BM25 full-text over title + body (§ [Search Ranking](/docs/architecture/search-ranking/)). | | `fuzzy` | Skim subsequence match over title + key. | At least one of `lexical` / `fuzzy` must be present; the empty mapping `search: {}` is a parse-time error. When both are present, their rankings are fused with Reciprocal Rank Fusion, exactly as `iwe find --fuzzy --lexical`. **Selection contract.** The result is defined set-theoretically, with no evaluation order implied: a document is in the result **iff** it matches the search **and** passes the `filter` (search matches ∩ filter matches). Candidates with no BM25 hit / no skim score are dropped — a text query is a filter here, not merely a sort. Over that joint set the default ordering is by relevance (ties by key ascending). No implementation may apply a top-K cutoff before the filter mask: `limit: 5` always means the 5 best documents that pass the filter, never the filtered survivors of a globally-truncated top list. **Scores use corpus-global statistics.** The BM25 index is fit to the whole workspace; IDF does not change under a filter (standard Lucene-style behavior, never re-fit per query). "Search within a filter" is therefore not identical to searching a corpus containing only the filtered documents. **`search` + `sort`.** Legal, and not redundant: `search` contributes membership and the default ordering; an explicit `sort` overrides the ordering only. This keeps "the 5 newest documents matching Q" expressible (`search` + `sort: { date: -1 }` + `limit: 5`). **No searchable terms.** A `lexical` query that reduces to nothing after stop-word removal and stemming matches nothing; the result carries a structured warning (surfaced in-band, like the truncation signal) so callers see why the result is empty. **Requires the index.** `search` requires the search-indexed graph; running it against a graph built without a search language is an execution-time error naming the missing index. Grammar: §A.4A. Full semantics: [Search](/docs/concepts/query-language/#search-find-only). ## 6. Projection Projection shapes *which fields* a `find` (or, by extension, `retrieve`) result carries, *under which output names*. It uses MongoDB-style `$project` semantics: the **left-hand side is the output field name**, the **right-hand side is the source**. The motivation: `find` returns metadata and `retrieve` returns content. MongoDB-style projection collapses the two into one query+shape pipeline AND lets the caller pick the output names. ### 6.1 Structural pseudo-field sources §2.3 reserves field names whose first character is `$`, `_`, `.`, `#`, or `@`. This section defines a concrete set of `$`-prefixed **pseudo-field source selectors** that are addressable as projection sources. They are not addressable in `sort` or `update`. On the filter side the single exception is `$content`, which doubles as the block-membership operator — it takes a block predicate and matches documents containing at least one matching block (grammar: §A.3; semantics: [Content membership](/docs/concepts/query-language/#content--content-membership)). The `$`-prefix is a **source-side marker**, not an output-side marker. It says "this name resolves against the engine, not against user frontmatter." Output names are always bare. | Source selector | Type | Meaning | |---|---|---| | `$key` | string | The document's key. | | `$title` | string | The document's title. | | `$titleSlug` | string | Slugified form of `$title`: lowercase, ASCII, whitespace and non-alphanumerics replaced with `-`, leading/trailing `-` trimmed. Derived deterministically from `$title`; no separate corpus storage. | | `$content` | string | Rendered markdown body, frontmatter stripped. | | `$frontmatter` | mapping | Full user frontmatter, exactly as the document carries it. | | `$includedBy` | `[EdgeRef]` | Inbound inclusion edges. | | `$includes` | `[EdgeRef]` | Outbound inclusion edges (= child documents). | | `$referencedBy` | `[EdgeRef]` | Inbound reference edges (= backlinks). | | `$references` | `[EdgeRef]` | Outbound reference edges. | `EdgeRef` is the shape `{ key, title, sectionPath: [string] }` (canonical definition: §13.2.2). `EdgeRef` sub-fields are unprefixed — they are produced by projection (engine output), not addressed as sources. These source selectors are reserved permanently. A `$`-prefixed key in a projection document is always read as a source selector, never as a field path (§2.3), so there is no collision risk between source names and user data. Consumers MUST tolerate unknown fields in result documents (per the schema-evolution rule in §13.1.3). **Every output name chosen by a projection is a bare identifier** — the left-hand side of `outputName: source`, an `addFields` name, and the default names of the `$`-selectors all carry no `$`. The `$`-prefix lives on the right-hand side (source selectors), not on the left. This applies recursively: `EdgeRef` sub-fields are bare (`key`, `title`, `sectionPath`), every level of the engine-authored result uses bare keys. User frontmatter keys are not projection-authored names: the default projection spreads them into the result document unchanged, so a document carrying `$foo` or `_private` yields a result document with those keys at the top level, beside `key` and `title`. The visual rule: `$X` in a projection document is a *reference* to engine-side data; bare keys are output names that ship to the consumer. ### 6.2 Projection document A projection document is a YAML mapping. Each entry has the shape: ``` <outputName>: <source> ``` Where: - **`outputName`** — a bare identifier. The key under which the field appears in the result document. MUST NOT start with `$` (reserved for source selectors). MUST NOT contain `.` (output is always flat at the top level — nesting is determined by the source's value type). Casing is preserved as written; the default projections (§6.2.2) use camelCase, but user projections may use any casing. - **`source`** — one of the forms below. #### 6.2.1 Source forms | RHS form | Meaning | |---|---| | `1` | Include a frontmatter field whose name equals `outputName`. Shorthand for `<outputName>: <outputName>`. `true` and YAML `null` are accepted as aliases of `1`. | | `$<selector>` | Include the named structural pseudo-field source. | | `path.to.fm.field` | Include a frontmatter value at a dotted path (per §4.4). | | `{ $<selector>: { <options> } }` | Options form, used by the block surface: `{ $content: P }` narrows the body to blocks matching the predicate `P`, and the `$blocks` / `$matches` sources take their predicate or pattern this way (grammar: §A.4; semantics: [Block projection](/docs/concepts/query-language/#block-projection)). | Examples: ```yaml project: status: 1 # frontmatter.status → status priority: metadata.priority # frontmatter.metadata.priority → priority body: $content # $content → body parents: $includedBy # $includedBy → parents links: $references # $references → links fm: $frontmatter # full user frontmatter mapping → fm ``` Result document for the projection above: ```yaml status: draft priority: 5 body: "# Doc One\n\n..." parents: - key: parent title: Parent Doc sectionPath: [Overview] links: [] fm: status: draft metadata: priority: 5 ``` `key` and `title` are absent because the projection does not select them. Add `key: $key` and `title: $title` (or rely on the default projection in §6.2.2) to include them. #### 6.2.2 Default projection When the operation document omits `project`, the command applies the **default projection** below. `find` and `retrieve` share the same default: ```yaml project: key: $key title: $title references: $references includes: $includes referencedBy: $referencedBy includedBy: $includedBy ``` The four edge fields cover both inclusion directions (`includes` / `includedBy`) and both reference directions (`references` / `referencedBy`). User frontmatter is **not** wrapped under a `frontmatter:` key in the default — it is flat-merged at the top level of the result per §13.2.3. Callers who want the whole frontmatter map under a single key can project it explicitly with `addFields: { fm: $frontmatter }` or `project: { ..., frontmatter: $frontmatter }`. `count` returns an integer and has no projection. **Frontmatter precedence on name collision.** A default projection entry that maps a structural source onto the output names `key` or `title` does not overwrite a user frontmatter field of the same name: if the document's frontmatter has `key` or `title`, the frontmatter value wins and the structural value is suppressed. The rule is limited to these two names — for the four edge fields (`references`, `includes`, `referencedBy`, `includedBy`) the structural (graph-derived) value is authoritative even when user frontmatter happens to use the same name. The rule also applies to `addFields` (which flat-merges, like the default), but **not** to explicit `project`: an explicit `project` document emits exactly the listed fields with their resolved sources and does not flat-merge user frontmatter at all. When `project` is set explicitly, it **replaces** the default — there is no merge. A user who wants to extend the default writes the full set: ```yaml project: key: $key title: $title references: $references includes: $includes referencedBy: $referencedBy includedBy: $includedBy body: $content # added ``` #### 6.2.3 Identity fields `key` and `title` are not implicit. They appear in the result only when the projection — default or explicit — selects them. The default projection in §6.2.2 includes both, so callers who do not pass `--project` see them without effort. A caller who passes `--project status,priority` gets exactly `status` and `priority`; `key` and `title` are absent unless added. A projection that maps `$key`, `$title`, or `$titleSlug` under a different output name (e.g. `slug: $titleSlug`, `heading: $title`) emits the alias only — there is no automatic duplicate `key` / `title` field. If the caller wants both the canonical name and an alias, both must be projected. Example: ```yaml project: slug: $titleSlug heading: $title # result: slug: doc-one heading: Doc One ``` #### 6.2.4 Conditional structural sources Some pseudo-field sources require auxiliary graph computation. On `find` (the supported path), projecting any of `$content`, `$includes`, `$includedBy`, `$references`, `$referencedBy` *implies* the corresponding compute — no flags needed. The implied depth for `$includes` is 1 (immediate children only); deeper traversal is currently only available on `retrieve`. `retrieve` exposes no projection flags (§6.4.1); its output follows the default projection, and the flag set gates which of these sources are computed. When a source's backing flag is not set, the field is still emitted, with its empty value (`[]`): - `includedBy` is always populated. - `referencedBy` without `-b` (default on) → `[]`. - `includes` without `--children` → `[]`. - `references` without a `references` expansion (`--expand-references N`, or the deprecated `-l`) → `[]`. The empty form preserves stable schema (§13.1.3). ### 6.3 `addFields` — additive projection For callers who want to *augment* the default projection rather than replace it, the operation document accepts an alternative key, `addFields`, alongside `project`. `addFields` follows the same grammar as `project` (§6.2.1). It does **not** replace the default — it extends it. The baseline that `addFields` augments is exactly the default projection from §6.2.2: ```yaml key: $key title: $title references: $references includes: $includes referencedBy: $referencedBy includedBy: $includedBy ``` Combine rule (projection layer — applied before evaluation): - Each entry in `addFields` is appended to the six default entries above. - Output names absent from the default are appended. - Output names that collide with a default-projection output name overwrite the default entry. For example, `addFields: { title: $key }` replaces the default `title: $title` *projection entry*; after this step the merged projection has `title: $key` in place of `title: $title`. `project` and `addFields` are mutually exclusive within a single operation document. Setting both is a parse-time error. The conditional-source rule (§6.2.4) applies identically. A structural pseudo-field appearing under `addFields` on `find` implies the corresponding compute, just as it would under `project`. **Frontmatter precedence at evaluation.** `addFields` flat-merges user frontmatter into the result (like the default projection — see §13.2.3), so the §6.2.2 frontmatter-precedence rule applies: for the output names `key` and `title`, a colliding user frontmatter value wins over the projected structural source. This means an `addFields` override of `key` or `title` only takes effect on documents whose frontmatter does *not* already define that name. With `addFields: { title: $key }` against a doc whose frontmatter has `title: "User Title"`, the result's `title` is `"User Title"` (frontmatter wins). Against a doc with no `title` in frontmatter, the result's `title` is the doc's key (the override applies). For other output names — including the four default-projection edge fields (`references`, `includes`, `referencedBy`, `includedBy`) — the projected value wins on collision. This rule does **not** apply under explicit `project`: that mode emits exactly the listed fields with their resolved sources and does not flat-merge user frontmatter, so there is no collision to resolve. **Tree carve-out.** On `iwe tree`, the output names `key`, `title`, and `children` are reserved by the recursive renderer and cannot be overwritten via `addFields` (or `project`). An `addFields` entry whose output name collides with one of those three is silently ignored on `tree` — the structural value is emitted instead. Other commands (`find`, `retrieve`) follow the unrestricted overwrite rule above. Example — `find` with the body added: ```yaml addFields: body: $content ``` Result document carries the full `find` default projection (§6.2.2), the document's user frontmatter flat-merged at the top level (per §13.2.3), plus the `body` field: ```yaml key: doc-1 title: Doc One references: [] includes: [] referencedBy: [] includedBy: [] status: draft # user frontmatter, flat-merged priority: 5 # user frontmatter, flat-merged body: "# Doc One\n\n..." ``` ### 6.4 Output shape under projection Projection shapes the per-document fields. The wire shape is **a flat array of projected documents** — no envelope. This matches §13.4.2 (`find`) and §13.4.3 (`retrieve`). ```yaml [<projected-doc>] ``` Each element is a projected document per §6.2.1. The shapes `FindResult` and `DocumentOutput` are the **default-projection** results for `find` and `retrieve` respectively; explicit projection produces whatever shape the projection document specifies. #### 6.4.1 Cross-command convergence Once projection is unified, `find` and `retrieve` differ only in **selection vocabulary**: `find` accepts text queries (`--fuzzy`, `--lexical`); `retrieve` accepts `-k KEY` (and graph-walk flags like `--expand-includes`, `--expand-included-by`, `--expand-references`). Default projection is the same on both (§6.2.2), and the wire shape is the same flat array. `retrieve` deliberately exposes no projection flags — every shaped read is `find`'s (`find -k KEY --project ...`). Under the shared default projection, `find` and `retrieve` produce the same per-document shape — and the same outer shape — for the same key set. ### 6.5 Markdown rendering under projection With explicit projection: - **`find` markdown:** emits a compact index — one `- [title](key)` line per result — unless the projection includes a `$content`-shaped field. When `$content` is projected, the whole invocation switches to one four-backtick fenced `markdown #<key>` block per result: the frontmatter inside the block contains the projected fields under their **output names**, with two omissions — `key` is hoisted to the fence info string (never duplicated inside frontmatter), and the `$content` field is rendered as the body rather than inside frontmatter. In index mode, edge fields render as `<-`/`->` arrows and other non-edge fields as ` · name: value` annotations. - **`retrieve` markdown:** the frontmatter block contains only the fields the projection requested, under their **output names**. Omitting a `$content` projection emits the frontmatter block with no body. The cross-format invariant in §13.1.1 still applies — markdown MUST NOT contradict JSON/YAML, but MAY abbreviate or omit fields for readability. ## 7. Sort ```yaml sort: modified_at: -1 ``` | Value | Meaning | |---|---| | `1` | Ascending | | `-1` | Descending | The sort direction is type-strict: integer `1` (ascending) or integer `-1` (descending). Floats (`1.0`), strings (`"1"`), booleans, and null are parse-time errors. (YAML `+1` resolves to the same integer as `1` and is accepted.) A `sort` mapping with two or more entries is a parse-time error. The value may also be a mapping with a required `order` (`1` or `-1`, same rules) and an optional `collation`: ```yaml sort: title: { order: 1, collation: binary } sort: title: { order: -1, collation: sv-SE } ``` `collation` is `binary` for codepoint order or a BCP-47 locale tag (`sv-SE`, not `sv_SE`) for that language's collation rules; a language with no collation data of its own uses the root rules without an error. Any other value, an unknown option name, or a mapping without `order` is a parse-time error. Without `collation`, strings compare with the host's collator (see below). Documents missing the sort key sort as if the value were null. Null sorts before all other values ascending, last descending. Sort applies to all four operations (on `update` / `delete` it bounds the iteration order before mutation). Ties — including the no-`sort` case — are broken by document key in ascending lexicographic order. The engine sorts the matched set by key first, then applies the user-provided sort with a stable algorithm; the result is deterministic given the same corpus and operation. The engine compares string values by codepoint unless the sort sets a `collation`. A host may supply a default locale-aware collator (the Unicode Collation Algorithm with the CLDR rules of a locale, numeric ordering on, emoji removed before comparing); the `iwe` CLI, the `iwes` LSP server and the `iwec` MCP server do so from `workspace.locale`, so there case and accents only break ties and `Note 2` precedes `Note 10`. Two strings whose emoji-free text collates equal fall back to the full string with emoji and then to codepoint order. Collation applies to `sort` only; the ordering operators in §4.7 always use codepoint comparison. ## 8. Limit A non-negative integer cap. ```yaml limit: 20 ``` `limit: 0` means no limit. Negative values are an error. Limit applies to all four operations; on `update` / `delete` it bounds the number of mutated / removed documents. ## 9. Update operators The `update` field of a mutation operation document specifies the mutations to apply to each matched document. It must contain at least one update operator at the top level. All operators in one update document apply atomically per matched document (§10). ### 9.1 Frontmatter operators Two frontmatter operators: | Operator | Effect | |---|---| | `$set` | Set fields to values | | `$unset` | Remove fields | #### `$set` ```yaml update: $set: reviewed: true audited_at: 2026-04-26 author: email: alice@example.com "review.reviewer": alice ``` Adds the field if absent, replaces it otherwise. Nested paths can be expressed via nested mappings or dotted-key shorthand (matching §4.4). Intermediate mappings are auto-created when a dotted path writes through a missing parent: `$set: { "a.b.c": 1 }` on a doc without `a` produces `a: { b: { c: 1 } }`. A dotted path that traverses a present-but-non-mapping intermediate **coerces the intermediate to a fresh mapping** holding the new leaf — `$set: { "a.b": 1 }` against `{ a: "scalar" }` produces `{ a: { b: 1 } }`. The previous scalar value is discarded; the user took explicit action by writing through that path. This is the symmetric write-side rule to §4.4, which on read treats a non-mapping intermediate as a missing leaf. Mapping values **replace wholesale**. `$set: { author: { name: alice } }` overwrites the existing `author` field with the literal mapping `{ name: alice }`; any pre-existing keys under `author` (e.g. `email`) are dropped. To merge into an existing mapping, address the inner fields with dotted shorthand (`$set: { "author.name": alice }`); per §4.4, that is equivalent in *path* to the nested-mapping form, but the dotted form only writes the named leaves and leaves siblings intact. `$set` requires at least one entry. Empty `$set: {}` is a parse-time error. #### `$unset` ```yaml update: $unset: draft_notes: "" temporary: "" ``` Values are ignored. Absent field → no-op. `$unset` requires at least one entry; empty `$unset: {}` is a parse-time error (same reason as `$set`). ### 9.2 Operator names are not update targets A path segment beginning with `$` cannot be addressed — see §2.3. On the mutation side, **operators that target a `$`-prefixed segment in any path are parse-time errors**. The check applies to every segment — top-level keys, dotted-shorthand segments, and nested-mapping keys at every depth — not only the leaf or the top-level segment. ```yaml # ERROR — top-level $-prefixed name update: $set: $hidden: 1 ``` ```yaml # ERROR — dotted segment with a $ prefix update: $set: "query.$includedBy": 1 ``` ```yaml # ERROR — $ prefix on a nested-mapping key (any depth) update: $set: query: $includedBy: 1 ``` ```yaml # ERROR — $ prefix on a leaf segment update: $set: "review.$user": alice ``` The error is detected during update-document validation, and makes the failure loud rather than letting the write resolve as a miss. Other leading characters are ordinary: `$set: { _hidden: 1 }`, `$set: { "#tag": foo }` and `$set: { "@user": bar }` all write the named field. ### 9.3 Combining operators Multiple operators in one update document apply atomically per matched document. `$set` and `$unset` paths are checked for **prefix overlap**: two paths conflict when, after canonicalizing nested-mapping form into dotted form per §4.4, one path is equal to or a prefix of the other. Conflicts are parse-time errors. The rule applies both across operators (`$set` vs `$unset`) and within a single operator (e.g. two `$set` entries that overlap after canonicalization). | Update document | Result | |---|---| | `$set: { "a.b": 1 }, $unset: { a: "" }` | error — `a` is a prefix of `a.b` | | `$set: { a: 1 }, $unset: { "a.b": "" }` | error — same prefix relation, opposite direction | | `$set: { author: { name: alice } }, $set: { "author.name": bob }` | error — both canonicalize to writes overlapping `author.name` | | `$set: { "a.b": 1 }, $unset: { "a.c": "" }` | OK — sibling paths, no overlap | | `$set: { a: 1 }, $unset: { b: "" }` | OK — disjoint top-level fields | ### 9.4 Update requirements (use-case checklist) The language MUST express the following mutations directly: | Operation | Update document | |---|---| | Mark all drafts reviewed | `$set: {reviewed: true}` | | Promote drafts to published | `$set: {status: published, published_at: 2026-04-26}, $unset: {draft_notes: ""}` | ### 9.5 Block operators The update document also accepts six **block operators** — `$replace`, `$replaceText`, `$insertBefore`, `$insertAfter`, `$append`, `$delete` — as siblings of `$set` / `$unset`. Each addresses blocks inside the matched documents through a block predicate carried in its argument: the `$`-prefixed keys select, the bare keys are the payload and the optional `expect` guard. Block operators are valid only in `update` operations, combine freely with the frontmatter operators, apply atomically with them per matched document (§10.1), and never conflict with `$set` / `$unset` paths — they mutate different data. Grammar: §A.7 (operators) and §A.9 (block predicates). Semantics — targets and coalescing, header splice behavior, `$replaceText` anchor rules, extent disjointness, guards — are specified in [Block update operators](/docs/concepts/query-language/#block-update-operators). ## 10. Atomicity ### 10.1 Per-document All operators in one update document apply atomically per matched document: either every operator succeeds and the engine emits a single rewritten document — frontmatter and body — for that document, or no replacement is emitted for it. There is no half-applied rewrite. `$set` and `$unset` have no evaluation-time failure modes — their invalid forms are rejected at parse time (`$set` / `$unset` conflict, `$`-prefixed path segments, etc.) before any matching runs. The block operators (§9.5) do have evaluation-time failure modes — type compatibility, `$replaceText` anchors, extent disjointness, `expect` guards; the operation validates fully against the original documents before anything is emitted, so a failure in any matched document means no document is rewritten and the frontmatter operators write nothing (semantics: [Validation and atomicity](/docs/concepts/query-language/#validation-and-atomicity)). ### 10.2 Across-document Across-document atomicity is **not** provided. The engine itself is a pure function: given an `update` operation it returns `changes` — a list of `(key, new markdown)` pairs the host should write. A `delete` operation returns the list of keys to remove. The host applies these effects to its storage; how it sequences writes, recovers from partial application, or surfaces partial success is host-defined. Because the engine never writes itself, a "preview-only mode" requires no special flag: the host simply consumes the outcome without applying it. Engine output contains everything needed to render the post-operation state in memory. ## 11. Composition order Within one operation, predicates compose in this order — each step intersects with the previous: 1. **Filter** (`filter`) — narrows by per-document predicate. Includes both frontmatter predicates (§4) and graph operators (§5). *(all four operations)* 2. **Search** (`search`, §5A) — on `find` only: intersects the filter survivors with the search matches and supplies the default ordering. Selection is joint with step 1 (defined set-theoretically, no order implied). Skipped when `search` is absent. After selection: 3. **Sort** (§7) orders the matched set. On `find` with `search` but no `sort`, the relevance order from step 2 is the ordering; an explicit `sort` overrides it. 4. **Limit** (§8) caps the matched set. 5. **Action**: `find` projects (§6) and returns matches; `count` returns the integer; `update` applies the update operators (§9) atomically per document and returns the rendered patch (§10); `delete` returns the keys to remove. For mutating actions the host applies the returned effects to its storage. ## 12. CLI surface This section specifies the `iwe` CLI surface for the four query operations. It covers the flag set that maps each spec operator to a CLI flag, the `--filter` inline expression form, legacy aliases, and how each command lowers its flags into a spec operation document. ### 12.1 Subcommands | Subcommand | Spec operation | Notes | |---|---|---| | `iwe find [QUERY]` | `find` | Combines a text query — `--fuzzy` (title/key) or `--lexical` (BM25 on title and body); the bare positional `QUERY` is a deprecated alias of `--fuzzy` (§12.6) — with filter flags via AND. Supports `--project`, `--sort`, `--limit`, `--blocks`, `--matches`. | | `iwe count` | `count` | Prints integer matches to stdout. Supports `--limit`. | | `iwe update` | `update` (mutation mode) | Two modes: body overwrite (`-k -c`) or mutation (`--filter`/`-k` + `--set`/`--unset` and the block-operator flags, §12.4). Modes are mutually exclusive. | | `iwe delete [KEY]` | `delete` | Positional `KEY` is sugar for `$key: K`. Combine with `--filter` to widen. Either `KEY` or `--filter` is required. | | `iwe tree`, `retrieve`, `export` | (selection only) | Reuse the same filter flag set to narrow what they operate on. They are not spec operations. | ### 12.2 Filter flags Each flag mirrors the spec operator name (camelCase → kebab-case). All flags are AND-composed at the top level. ``` --filter "EXPR" inline YAML; wrapped in `{}` if not already a mapping -k, --key KEY $key match. 1 key = $eq, 2+ = $in. --includes KEY[:DEPTH] $includes anchor; DEPTH defaults to --max-depth (1) --included-by KEY[:DEPTH] $includedBy anchor; DEPTH defaults to --max-depth (1) --references KEY[:DIST] $references anchor; DIST defaults to --max-distance (1) --referenced-by KEY[:DIST] $referencedBy anchor; DIST defaults to --max-distance (1) --max-depth N default maxDepth applied to inclusion anchor flags without a colon-suffix. Default 1. --max-distance N default maxDistance applied to reference anchor flags without a colon-suffix. Default 1. ``` #### 12.2.1 `--filter` lowering The argument to `--filter` is parsed as a YAML value. If the parsed value is a mapping, it is used directly as a filter document. Otherwise the engine wraps the input in `{` and `}` and re-parses. This lets users write either form: ``` --filter 'status: draft' # block-style mapping (preferred) --filter '{status: draft, priority: 5}' # flow-style mapping --filter '$key: notes/foo' # graph operator at top level ``` The resulting filter document is parsed by the same builder that handles full operation documents, so all errors defined in §4 (mixed `$`/bare keys in field-value mappings, top-level `$not`, etc.) are surfaced verbatim. #### 12.2.2 Anchor depth syntax Inclusion anchors (`--includes`, `--included-by`) accept `KEY[:DEPTH]` where DEPTH is a non-negative integer that becomes `maxDepth`. Reference anchors (`--references`, `--referenced-by`) use the same `KEY[:DIST]` syntax, lowered to `maxDistance`. **DEPTH / DIST `0` is the unbounded sentinel** — see below. **Default values.** The CLI carries two session-level defaults, both starting at 1: - `--max-depth N` — applied to inclusion anchor flags (`--includes`, `--included-by`) that omit a per-flag value. - `--max-distance N` — applied to reference anchor flags (`--references`, `--referenced-by`) that omit a per-flag value. `0` is the **unbounded sentinel** for both flags and the colon-suffix: passing `--max-depth 0`, `--max-distance 0`, or `KEY:0` lowers to the full form with `maxDepth` / `maxDistance` omitted (the language's "unbounded" form, §5.2.3). This mirrors `limit: 0` in the language (§8). Positive integers behave as today. A colon-suffix on a single anchor (`--includes KEY:5`) overrides the session default for that anchor only. The lowered shape depends on the effective depth: - When the effective depth is 1 (the default, with no `--max-depth` / `--max-distance` and no colon-suffix, or an explicit `:1`), a bare `--includes KEY` lowers to **scalar shorthand** `$includes: KEY` — the language defines this as `{ match: { $key: KEY }, maxDepth: 1 }` (§5.2.1). - When the effective depth is `0` (unbounded sentinel), the lowering is the full form **without** a `maxDepth` key: `$includes: { match: { $key: KEY } }`. - When the effective depth is any other positive integer N, a bare `--includes KEY` lowers to the **full form** `$includes: { match: { $key: KEY }, maxDepth: N }`. The session default appears explicitly in the lowered document; scalar shorthand is reserved for the depth-1 case. - A per-flag colon-suffix always wins over the session default. Lowering examples without `--max-depth` / `--max-distance` (defaults at 1): ``` --includes roadmap/q2 → $includes: roadmap/q2 (scalar shorthand; expands to depth 1 by language rule) --includes roadmap/q2:2 → $includes: { match: { $key: roadmap/q2 }, maxDepth: 2 } --included-by projects/alpha:5 → $includedBy: { match: { $key: projects/alpha }, maxDepth: 5 } --references people/alice → $references: people/alice (scalar shorthand; expands to distance 1 by language rule) --referenced-by archive/index:2 → $referencedBy: { match: { $key: archive/index }, maxDistance: 2 } ``` Lowering examples with `--max-depth 3 --max-distance 2`: ``` --max-depth 3 --includes roadmap/q2 → $includes: { match: { $key: roadmap/q2 }, maxDepth: 3 } --max-depth 3 --includes roadmap/q2:1 → $includes: { match: { $key: roadmap/q2 }, maxDepth: 1 } (per-flag colon wins over the session default) --max-distance 2 --references people/alice → $references: { match: { $key: people/alice }, maxDistance: 2 } ``` Lowering examples with the `0` (unbounded) sentinel: ``` --includes roadmap/q2:0 → $includes: { match: { $key: roadmap/q2 } } (full form, maxDepth omitted → unbounded) --max-depth 0 --includes roadmap/q2 → $includes: { match: { $key: roadmap/q2 } } (session default 0 → unbounded) --max-depth 0 --includes roadmap/q2:3 → $includes: { match: { $key: roadmap/q2 }, maxDepth: 3 } (per-flag colon wins over the session default) --references people/alice:0 → $references: { match: { $key: people/alice } } (full form, maxDistance omitted → unbounded) ``` For range bounds (`minDepth` / `maxDepth`, `minDistance` / `maxDistance`), anchoring by frontmatter predicate (`match: { status: draft }`), or any combination not expressible as a single keyed anchor, use `--filter` directly. ### 12.3 Shape flags #### 12.3.1 Format flags matrix | Subcommand | `-f` / `--format` accepted values | Default | |---|---|---| | `iwe find` | `markdown`, `keys`, `json`, `yaml` | `markdown` | | `iwe retrieve` | `markdown`, `keys`, `json`, `yaml` | `markdown` | | `iwe tree` | `markdown`, `keys`, `json`, `yaml` | `markdown` | | `iwe export` | `dot` | `dot` | | `iwe count` | (no format flag — output is always a single integer) | n/a | | `iwe delete` | `markdown`, `keys` | `markdown` | | `iwe rename`, `extract`, `inline` | `markdown`, `keys` | `markdown` | Read-side commands (`find`, `retrieve`, `tree`) share one format set so a query written for one renders the same way under another; `iwe export` emits DOT only (§13.5.2). Mutation commands return a status report and only need `markdown` (human) or `keys` (machine) modes. `count`'s output is the integer match count and admits no format choice. #### 12.3.2 Projection and sort flags | Flag | Lowers to | Operations | |---|---|---| | `--project f1,f2[,f3]` | `project: { f1: 1, f2: 1, f3: 1 }` | `find`, `tree` | | `--add-fields f1,f2[,f3]` | `addFields: { f1: 1, f2: 1, f3: 1 }` | `find`, `tree` | | `--sort field:1`, `--sort field:-1` | `sort: { field: 1 }` / `sort: { field: -1 }` | `find` only | | `-l, --limit N` | `limit: N` (0 = unlimited, matching §8) | `find`, `count` | | `--blocks "PRED"` | `addFields: { blocks: { $blocks: PRED } }` | `find` only | | `--matches PATTERN` | `filter: { $content: { $matches: PATTERN } }` **and** `addFields: { matches: { $matches: PATTERN } }` — a composite lowering: one-flag grep | `find` only | `--project` and `--add-fields` accept two argument forms: - **Comma list:** `--add-fields body=$content,parents=$includedBy` - **Inline YAML mapping:** `--add-fields 'body: $content'` or `--add-fields '{body: $content, parents: $includedBy}'` The argument is parsed as YAML first; if it is a mapping, it is used as the projection document directly — this form carries the full §6 grammar, including the block sources and, on `--project`, the top-level block-predicate form (§A.4). Otherwise it is treated as a comma list. This mirrors the `--filter` lowering in §12.2.1. **Comma-list form.** Each `ITEM` lowers to a single `<outputName>: <source>` entry: | ITEM form | Lowered entry | Notes | |---|---|---| | `name` | `name: 1` | Frontmatter field, output as `name`. | | `name=path.to.fm` | `name: path.to.fm` | Frontmatter at dotted path, output as `name`. | | `name=$selector` | `name: $selector` | Pseudo-field source, output as `name`. | | `$selector` | `selector: $selector` | Pseudo-field, output name = selector minus `$`. Convenience form. | | `$blocks` / `name=$blocks` | `blocks: { $blocks: {} }` / `name: { $blocks: {} }` | Block source with the empty predicate — every block. A parameterized predicate (or the `$content` / `$matches` block sources) requires the mapping form. | `--project` and `--add-fields` are mutually exclusive on a single invocation. Passing both is a CLI parse error, mirroring the document-level rule in §6.3. `--sort` accepts exactly one `field:DIR` pair, matching the single sort key rule (§7). **Shell quoting.** The `$`-prefix in source selectors triggers shell variable expansion in unquoted form. Quote `--project` and `--add-fields` arguments with single quotes: `--add-fields 'body=$content,parents=$includedBy'` or `--add-fields 'body: $content'`. Bash, zsh, fish, and PowerShell all preserve `$` inside single quotes. #### 12.3.3 `iwe retrieve` flags `retrieve` assembles reading context around a set of documents. It runs as **two internal queries** when a search flag is present: a **seed query** (a `find` with `search` and `limit` over the resolved candidate set) followed by the **expansion read** over the ordered seeds. Without a search flag, the seed set is `-k` / `--filter` / anchors directly. **Expansion (`--expand-*`).** Expansion directions use the standard edge vocabulary — `includes`, `includedBy`, `references`, `referencedBy` — one direct flag each: `--expand-includes`, `--expand-included-by`, `--expand-references`, `--expand-referenced-by`. Each takes an optional depth: ``` --expand-references 1 --expand-included-by 1 --expand-includes 2 --expand-includes # bare flag = 1 level ``` A bare flag means depth `1`; an explicit value is an integer depth with **`0` = unbounded**; an omitted flag is not followed. A negative or non-integer value is an error. Each direction pulls documents into the result set: `includes` = inclusion descendants, `includedBy` = inclusion ancestors, `references` = outbound reference walk, `referencedBy` = inbound reference walk. **Expansion is doc-only by default** — with no `--expand-*` flag and no legacy alias, `retrieve` returns the requested document(s) with no expansion. **Edge-list output toggles.** Separate from expansion; they populate the output edge arrays and never add documents. `includedBy` is always populated; the others follow their flag (§6.2.4). When a flag is omitted the field is still emitted with its empty value (`[]`). | Flag | Effect | |---|---| | `-k, --key KEY` | Repeatable. The set of root keys to retrieve (candidate restriction when a search flag is present). | | `--expand-includes [N]` / `--expand-included-by [N]` / `--expand-references [N]` / `--expand-referenced-by [N]` | Expansion depth per direction (bare = `1`, `0` = unbounded, omitted = not followed). | | `--lexical Q` | Seed search: BM25 full-text query. Switches `retrieve` to the search-indexed loader and runs the seed query. | | `--fuzzy Q` | Seed search: fuzzy query on title + key. `--lexical` + `--fuzzy` fuse with RRF. | | `--limit N` | Cap the seed count **before** expansion — top-N by relevance when searching, the first N of the selection otherwise (`0` = unlimited). | | `--max-documents N` | Cap the document count **after** expansion, trimming periphery documents first (`0` = unlimited). | | `-b, --backlinks` | Populate `referencedBy` on each `DocumentOutput` (default true). Edge-list toggle, not expansion. | | `--children` | Populate `includes` on each `DocumentOutput`. Edge-list toggle, not expansion. | | `-e, --exclude KEY` | Repeatable. Skip these keys when assembling the result. | | `--filter`, `--includes`, `--included-by`, `--references`, `--referenced-by`, `--max-depth`, `--max-distance` | Same selection-side filter flags documented in §12.2; restrict the candidate set. With a search flag, search runs **within** these candidates, so listed `-k` keys that do not match the query drop from the output. | **Output order.** Seeds first, in relevance order, then expansion; seeds survive token-budget trimming first (§12.3.4). Because search restricts, `-k a -k b -k c --lexical Q` searches within those keys — `-k` is candidate restriction, not a guaranteed read. #### 12.3.4 Token budgets (`iwe find`, `iwe retrieve`) Two output-budget flags cap how much rendered content the command emits. They are host-side output shaping, not part of the operation document: they do not affect which documents match, only how much of each body is printed. | Flag | Effect | |---|---| | `--max-tokens N` | Cap total content tokens across all results (`0` = unlimited). | | `--max-document-tokens N` | Cap content tokens per document (`0` = unlimited). | When a budget clips output, the command prints a truncation warning to stderr naming the limits that apply. `$blocks` / `$matches` entries are exempt from both budgets; a narrowed `$content` field is counted and capped like a full body (§13.6.1). ### 12.4 Update flags (`iwe update` mutation mode) | Flag | Lowers to | |---|---| | `--set FIELD=VALUE` | `$set: { FIELD: VALUE }` (repeatable) | | `--unset FIELD` | `$unset: { FIELD: "" }` (repeatable) | | `--replace "ARG"`, `--replace-text "ARG"`, `--insert-before "ARG"`, `--insert-after "ARG"`, `--append "ARG"`, `--delete "ARG"` | one block-operator entry (`$replace`, `$replaceText`, `$insertBefore`, `$insertAfter`, `$append`, `$delete`) in the update document; `ARG` is the operator's `{ <selector>, <payload> }` mapping (§A.7), the operator-name wrapper dropped. Each flag appears at most once. | | `--expect VAL` | the document-level `expect` clause (§3.2) | | `--strict` | surface policy, not grammar: refuses to run unless every mutating application carries its `expect` guard (document-level `--expect` plus each block operator's `expect`); exempt under `--dry-run` | | `--filter "EXPR"` | required if `-k` is omitted | | `--dry-run` | preview only; print the would-be changes per doc and exit | | `--quiet` | suppress progress output | `--set FIELD=VALUE` parses VALUE as a YAML scalar. `5` is an integer, `true` is a bool, `draft` is a string, `[a, b]` is a list. To force a string, quote it as YAML: `--set 'count="5"'`. `iwe update` does not prompt for confirmation. The caller is responsible for passing the right `--filter` / `-k` selector; use `--dry-run` to inspect the matched set before applying. This matches the engine contract in §10.2 — the engine returns the patch and the host writes it without further interaction. Body-overwrite mode (`-k KEY -c CONTENT`) is the existing single-doc body rewrite. It does not touch frontmatter and is not a spec `update` operation. Body and mutation flags cannot be combined in one invocation. `--dry-run` applies to both modes. ### 12.5 Delete flags (`iwe delete`) | Flag | Lowers to | |---|---| | Positional `KEY` | `$key: K` (sugar) | | `--filter "EXPR"` | inline filter | | `--expect VAL` | the document-level `expect` clause (§3.2) | | `--strict` | requires `--expect`; refuses to run without it; exempt under `--dry-run` | | `--dry-run` | preview | | `-f, --format markdown\|keys` | output format (default `markdown`); `keys` prints affected document keys, suppresses progress | | `--quiet` | suppress progress | The deprecated alias `--keys` (also accepted on `rename`, `extract`, `inline`) lowers to `-f keys`; see §12.6. Either `KEY` or `--filter` (or both) must be present, matching the spec's required-filter rule (§3.2). When both are given, the union is deleted. `iwe delete` does not prompt; use `--dry-run` to preview before applying. Reference cleanup runs once over the whole matched set. `-f keys` returns *affected* keys (the deleted target plus every doc whose references were rewritten) rather than *matched* keys. The same `-f markdown|keys` selector is also available on `iwe rename`, `iwe extract`, and `iwe inline` with identical semantics. ### 12.6 Deprecated aliases These flags predate the language and remain accepted on the commands they originally appeared on. Selector aliases (`--in`, `--refs-to`, etc.) print a one-line `warning: --X is deprecated; use --Y` to stderr **each time the deprecated flag appears in a parsed command**. For one-shot CLI invocations this is one warning per run. For long-running hosts (LSP, MCP), the warning fires on every operation that uses the alias — making the deprecation visible across many requests instead of being suppressed after the first. Mutation-output aliases (`--keys`) are silent — `--keys` and `-f keys` behave identically and produce the same output. | Deprecated | Lowers to | |---|---| | bare positional `QUERY` (on `find`) | `--fuzzy QUERY` (prints a deprecation warning; use `--lexical QUERY` for BM25 full-text) | | `--in KEY[:N]` | `--included-by KEY[:N]` | | `--in-any K1 --in-any K2` | `$or: [{ $includedBy: K1 }, { $includedBy: K2 }]` (scalar shorthand for each) | | `--not-in KEY` | `$nor: [{ $includedBy: KEY }]` (scalar shorthand) | | `--refs-to KEY` | `$or: [{ $includes: KEY }, { $references: KEY }]` (scalar shorthand; legacy mixed-edge) | | `--refs-from KEY` | `$or: [{ $includedBy: KEY }, { $referencedBy: KEY }]` (scalar shorthand; legacy mixed-edge) | | `--keys` (on `delete`, `rename`, `extract`, `inline`) | `-f keys` | | `-d, --depth N` (on `retrieve`) | `--expand-includes N` (legacy `0` = off, not unbounded) | | `-c, --context N` (on `retrieve`) | `--expand-included-by N` (legacy `0` = off) | | `-l, --links` (on `retrieve`) | `--expand-references 1` | The mixed-edge lowering of `--refs-to` / `--refs-from` preserves their pre-spec semantics (matching either inclusion or reference edges to the target). New code should pick the spec operators directly. On `retrieve`, `-d` / `-c` / `-l` keep their legacy **`0` = off** meaning (they never expressed unbounded), whereas the `--expand-*` flags treat `0` as the unbounded sentinel. Passing a legacy alias together with the `--expand-*` flag for the same direction is an error. `--in`, `--in-any`, and `--not-in` defaulted to depth 1 before the spec, matching the new `--included-by` default (§12.2.2); the lowering above is behavior-preserving. Use `--max-depth N` or a per-flag colon-suffix to widen. ### 12.7 Composition rules Within a single command: 1. All filter flags at the top level are AND-composed. The text query (`--fuzzy` / `--lexical` on `iwe find`, or the deprecated bare positional) is also ANDed: the result is the **set intersection** of the text-match set and the filter-match set. Order of evaluation is implementation-defined (typically the more selective predicate is applied first for performance), but the result set is order-independent. 2. `--filter "EXPR"` contributes its top-level filter document to the same AND. 3. `-k` / positional `KEY` participate in the AND like any other clause. 4. `--sort`, `--limit`, `--project` apply after filtering, in the order defined by §11. **`-k` / `$key` collision.** Combining `-k KEY` with a `--filter` whose top level contains a `$key` predicate is a CLI parse-time error: both clauses contribute to the document's key predicate, and silently AND-composing them would either produce a YAML mapping with two `$key` keys or quietly match the empty set. The error message points users at OR-composition (`--filter '$or: [{$key: a}, {$key: b}]'`) when they wanted a multi-key match, or at picking one source when they didn't. Multi-key match via `-k a -k b` (which lowers to `$key: { $in: [a, b] }`) remains valid. For OR or NOR compositions, write the filter inside `--filter`: ``` --filter '$or: [{ status: draft }, { status: review }]' --filter '$nor: [{ status: archived }]' ``` ### 12.8 CLI examples #### 12.8.1 Find ``` iwe find --fuzzy rust # fuzzy on "rust" (title/key) iwe find --lexical "borrow checker" # BM25 full-text on title and body iwe find --filter 'status: draft' # all drafts iwe find --fuzzy rust --filter 'status: draft' # fuzzy AND status==draft iwe find --included-by projects/alpha:5 # descendants within 5 levels iwe find --included-by projects/alpha:0 # all descendants of alpha (unbounded) iwe find --references people/alice # docs that reference alice iwe find --filter 'priority: { $gt: 3 }' --sort modified_at:-1 iwe find --project title,modified_at -f json # only project two fields iwe find --project title,modified_at -f yaml # same, as YAML iwe find --filter 'status: draft' --add-fields body=$content iwe find --add-fields 'body=$content,parents=$includedBy' -f json iwe find --filter 'status: draft' --project '$content,$includedBy' iwe find --filter 'status: draft' --project 'body=$content,parents=$includedBy,status' iwe find --filter 'status: draft' --project 'key=$key,title=$title,body=$content' iwe find --filter 'status: draft' --add-fields 'body: $content' iwe find --add-fields '{body: $content, parents: $includedBy}' -f json iwe find --project '{key: $key, title: $title, body: $content}' ``` #### 12.8.2 Count ``` iwe count # total documents iwe count --filter 'status: draft' # count drafts iwe count --included-by projects/alpha:10 # count descendants of alpha ``` #### 12.8.3 Update ``` # Body overwrite (existing behavior) iwe update -k notes/draft -c "# New body" cat new.md | iwe update -k notes/draft -c - # Single-doc frontmatter mutation iwe update -k notes/draft --set status=published # Bulk frontmatter mutation iwe update --filter 'status: draft' --set 'reviewed=true' iwe update --filter 'status: archived' --unset draft_notes # Preview only — no writeback iwe update --filter 'status: draft' --set status=published --dry-run # Block mutation — retitle one section, guarded to a single target iwe update -k projects/roadmap --replace '{ $header: Goals, content: "## Goals 2026", expect: 1 }' # Strict bulk edit: every mutating application must carry its guard iwe update --filter 'status: draft' --strict --expect 3 --set status=published ``` #### 12.8.4 Delete ``` iwe delete document-key # single doc iwe delete --filter 'status: archived' # bulk delete by filter iwe delete --filter 'status: draft' --strict --expect '{ max: 5 }' # guarded bulk delete iwe delete --filter '$key: drafts/scratch' --dry-run # preview a deletion by filter ``` ## 13. Output formats ### 13.1 Format invariants #### 13.1.1 Cross-format invariant A given query MUST encode the same logical result data in JSON and YAML. JSON and YAML are isomorphic at the field level. Markdown is a human projection of the same data and may omit detail for readability, but MUST NOT contradict JSON/YAML. `keys` is a strict projection: one key per line, no header. #### 13.1.2 Field-name convention | Surface | Convention | |---|---| | JSON keys | `camelCase` | | YAML keys (top-level output and frontmatter inside markdown) | `camelCase` (same as JSON) | | Markdown body text (headers, prose) | sentence case as authored | A single name per concept across all surfaces avoids triple-naming. Pick one external name per field and use it everywhere. #### 13.1.3 Stable-schema rule Every field this spec lists for JSON/YAML output MUST be present in every emission of that command, with its declared type. Empty values are encoded explicitly: | Type | Empty encoding | |---|---| | array | `[]` | | mapping | `{}` | | string | `""` | | nullable scalar | `null` | A consumer MUST NOT need to test for key presence to handle the empty case. This applies to `find` (`includedBy: []`), `retrieve` (`includedBy: []`, `includes: []`, `referencedBy: []`), `tree` (`children: []`), and all structured output. Note that with `--project`, only the listed fields are emitted — the stable-schema rule applies to fields the spec declares for the command, not to fields the user excluded by projection. Markdown is a human surface and is exempt — see §13.3.8. ### 13.2 Common shapes #### 13.2.1 `KeyTitleRef` ```yaml { key: string, title: string } ``` Used wherever one document references another *by identity* without carrying any positional context. #### 13.2.2 `EdgeRef` ```yaml { key: string, title: string, sectionPath: [string] } ``` `KeyTitleRef` extended with `sectionPath`: the chain of section header texts (root-to-leaf) under which the edge appears in the source document. Empty array when the edge sits at the document root. Used for every inclusion or reference edge surfaced in structured output: `includedBy`, `includes`, `referencedBy`. #### 13.2.3 User frontmatter merging `find` and `tree` flatten user frontmatter into each result/node alongside system fields (`key`, `title`, etc.) — there is no nested `frontmatter` object. Reserved-prefix entries (`_`, `$`, `.`, `#`, `@` per §2.3) are stripped before merging and MUST NOT appear in any output. On collision between a user frontmatter field named `key` or `title` and the corresponding system field, the user frontmatter value wins (§6.2.2). For the four edge fields (`references`, `includes`, `referencedBy`, `includedBy`), the structural value is authoritative even on name collision. `--project f1,f2,...` restricts the result to the listed fields, in the listed order; system fields and user frontmatter fields are projectable interchangeably. Without `--project`, every system field plus every user frontmatter field is flat-merged into the result. With `--project`, no flat-merge happens — only the listed fields appear, each resolved from the source on its right-hand side. `--add-fields` flat-merges like the default; the `key`/`title` precedence rule above applies to it as well. ### 13.3 Markdown and keys #### 13.3.1 Format set Commands that accept `-f markdown` or `-f keys`: | Command | `markdown` | `keys` | Default | |---|---|---|---| | `iwe find` | ✓ | ✓ | `markdown` | | `iwe retrieve` | ✓ | ✓ | `markdown` | | `iwe tree` | ✓ | ✓ | `markdown` | | `iwe stats` (aggregate) | ✓ | — | `markdown` | | `iwe delete`, `rename`, `extract`, `inline` | ✓ | ✓ | `markdown` | Markdown is a human surface and may omit detail (e.g. counts, edge metadata) for readability — but it MUST NOT contradict the structured form for the same query. `keys` is a strict projection: one key per line, no header, no trailing blank. #### 13.3.2 Common markdown frontmatter shapes `EdgeRef` inside markdown frontmatter: - Always emits `key`, `title`. - Emits `sectionPath` only when non-empty (the human surface tolerates omission; JSON/YAML always include it). User frontmatter inside markdown documents is emitted as the document carries it — no field is dropped, whatever its name starts with. #### 13.3.3 `iwe find` (markdown / keys) **Markdown:** `find` markdown is a compact index — one `- [title](key)` line per result, meant to be scanned. No header line. No `(showing M)` / `for "Q"` annotation — counts and query metadata live in `-f json|yaml`. Result order matches the find result order (filter / sort / limit applied upstream). Each line is `- [title](key)`, where the link text falls back to the key string when `title` is not projected. Non-empty edge fields render after the link as arrows keyed on direction — incoming (`includedBy`, `referencedBy`) as `<- [Title](key)`, outgoing (`includes`, `references`) as `-> [Title](key)`, multiple targets comma-joined after one arrow, empty edges omitted. Other non-edge fields render as ` · name: value`; arrays of scalars join with `, `, and richer values fall back to compact inline YAML. Projecting a `$content`-shaped field switches the whole invocation to the fenced-block form that `retrieve` emits (byte-identical for the matched key set): each result is a four-backtick fenced block with info string `markdown #<key>`, body and frontmatter as defined in §13.3.4. This is the only way `find` markdown emits document bodies — see §6.5. When the result set is empty, stdout is empty. **Keys:** One key per line, no header, no blank lines, no trailing blank. #### 13.3.4 `iwe retrieve` (markdown / keys) **Markdown:** Each returned document is wrapped in a **four-backtick fenced code block** with the info string `markdown #<key>`. Inside the fence: 1. A YAML frontmatter block — flat, no wrapper key. The structure mirrors the JSON `DocumentOutput` shape (§13.4.3), minus the fields lifted out: `key` lives in the fence info string, and the body lives below the frontmatter, not as a `content:` field. 2. A blank line. 3. The rendered markdown content. The fence info string carries the key prefixed with `#` (e.g. `markdown #child`). If the embedded content contains a four-or-more-backtick fence, use one more backtick for the outer fence so the inner fence cannot terminate it. Frontmatter fields (rendered in this order): | Location | Field | Required? | Source | |---|---|---|---| | fence info | `key` | always | `DocumentOutput.key` | | frontmatter | `title` | always | `DocumentOutput.title` | | frontmatter | `references` | omit when empty | `EdgeRef` list | | frontmatter | `includes` | omit when empty | `EdgeRef` list | | frontmatter | `referencedBy` | omit when empty | `EdgeRef` list | | frontmatter | `includedBy` | omit when empty | `EdgeRef` list | The order matches the unified default projection (§6.2.2). When `find` markdown is in fenced-block form (a `$content`-shaped field is projected), it produces byte-identical frontmatter to `retrieve -f markdown` for the same key; otherwise `find` renders the compact index (§13.3.3). `EdgeRef` inside frontmatter omits `sectionPath` when empty. There is no `document:` wrapper map; the frontmatter is always flat. Field naming matches JSON (no `parents` / `back-links` aliases). ````` ````markdown #child --- title: Child Document includedBy: - key: parent title: Parent Document sectionPath: - Overview --- # Child Document Child content. ```` ````` A multi-document stream concatenates blocks separated by **exactly one blank line** between the closing fence of one block and the opening fence of the next; no trailing blank line after the final block: ````` ````markdown #doc-a --- title: Doc A --- # Doc A Body. ```` ````markdown #doc-b --- title: Doc B --- # Doc B Body. ```` ````` **Keys:** One key per line, in the order documents appear in the envelope. No header. #### 13.3.5 `iwe tree` (markdown / keys) **Markdown:** Nested unordered list, two-space indent per depth level, each entry as a markdown link `[<title>](<key>)`: ``` - [AI Agent Memory](ai-memory) - [Post One](post-1) - [Post Two](post-2) ``` **Keys:** One key per line, **tab-indented** by depth (root has zero tabs). Order matches a depth-first walk of the tree. ``` ai-memory post-1 post-2 ``` #### 13.3.6 `iwe stats` (markdown) Aggregate `markdown` is a human-readable report of corpus-level statistics. Per-document stats (`-k KEY`) does not produce markdown — see §13.4.5 for the per-doc shape. #### 13.3.7 Mutation commands (markdown / keys) `delete`, `rename`, `extract`, `inline` produce a status report and accept `-f markdown|keys`. **Markdown — status report:** A human-prose report of what changed: ``` <verb-ing> '<source-key>'[ to '<target-key>'] Updated N document(s) ``` Or, with `--dry-run`: ``` Would <verb> '<source-key>'[ to '<target-key>'] Would update N document(s) <key> <key> ``` `--quiet` suppresses both forms. **Keys — affected-keys list:** One key per line, no header. The list contains every key that was modified — the operation's primary target plus every document whose references were rewritten. With `--dry-run`, the list is the keys that *would* be modified. #### 13.3.8 Stable-schema exemption for markdown The stable-schema rule in §13.1.3 requires every declared field to appear in every emission. Markdown is a human surface and is exempt: - Renderer MAY omit `includedBy` / `includes` / `referencedBy` from frontmatter when empty. - `EdgeRef` inside frontmatter MAY omit `sectionPath` when empty. - `key` is always carried in the fence info string; `title` is always in frontmatter. `keys` output is a strict projection — one key per line, no envelope — so the rule does not apply. ### 13.4 JSON and YAML JSON and YAML are isomorphic — same keys, same values, same nesting; only the surface syntax differs. They are the authoritative wire shape: when markdown disagrees, JSON/YAML wins. #### 13.4.1 Per-command format matrix | Command | `json` | `yaml` | Default | |---|---|---|---| | `iwe find` | ✓ | ✓ | `markdown` | | `iwe retrieve` | ✓ | ✓ | `markdown` | | `iwe tree` | ✓ | ✓ | `markdown` | | `iwe stats` | ✓ | ✓ | `markdown` | `iwe count` and the mutation commands have no `-f json|yaml` form (see §13.5). #### 13.4.2 `iwe find` (JSON / YAML) The top-level value is an **array** of `FindResult` — no envelope. ```yaml [FindResult] ``` A `FindResult` is a flat mapping. There is no nested `frontmatter` object: user frontmatter fields are siblings of `key`, `title`, and the system-derived edge arrays. Without `--project` the result carries the full set of system fields plus all user frontmatter fields. With `--project` the result carries only the listed fields, in the listed order. System fields (the full set emitted without `--project`): ```yaml FindResult: key: string title: string references: [EdgeRef] # always present; [] when none includes: [EdgeRef] # always present; [] when none referencedBy: [EdgeRef] # always present; [] when none includedBy: [EdgeRef] # always present; [] when none ``` Reserved-prefix entries are stripped from user frontmatter before merging and MUST NOT appear in output. On collision between a system field and a user frontmatter field of the same name, the user frontmatter value wins. **JSON** — without `--project`, system fields and the document's user frontmatter are flat-merged at the top level (`status`, `priority` below come from frontmatter): ```json [ { "key": "doc1", "title": "Document One", "references": [], "includes": [], "referencedBy": [], "includedBy": [], "status": "draft", "priority": 5 } ] ``` Trailing newline after the closing `]`. **YAML:** ```yaml - key: doc1 title: Document One references: [] includes: [] referencedBy: [] includedBy: [] status: draft priority: 5 ``` #### 13.4.3 `iwe retrieve` (JSON / YAML) The top-level value is an **array** of `DocumentOutput` — no envelope. ```yaml DocumentOutput: key: string title: string content: string # rendered markdown body only — source-file YAML frontmatter is always stripped references: [EdgeRef] # always present; populated when --links; [] otherwise includes: [EdgeRef] # always present; populated when --children; [] otherwise referencedBy: [EdgeRef] # always present; populated when --backlinks; [] otherwise includedBy: [EdgeRef] # always present; [] when none ``` **JSON:** ```json [ { "key": "test-doc", "title": "Test Document", "content": "# Test Document\n\nContent here.\n", "references": [], "includes": [], "referencedBy": [], "includedBy": [] } ] ``` **YAML:** ```yaml - key: test-doc title: Test Document content: | # Test Document Content here. references: [] includes: [] referencedBy: [] includedBy: [] ``` #### 13.4.4 `iwe tree` (JSON / YAML) The top-level value is an **array** of root nodes (not wrapped in an object): ```yaml TreeNode: key: string title: string children: [TreeNode] # always present in spec; [] when leaf ``` Each `TreeNode` is a flat mapping with the same projection semantics as a `FindResult` (§13.4.2). Without `--project`, only the system fields (`key`, `title`, `children`) are emitted. With `--project`, the listed user frontmatter fields are added as siblings of `key` and `title`, in projection order; `children` is always present regardless of projection so the tree shape remains traversable. **JSON:** ```json [ { "key": "ai-memory", "title": "AI Agent Memory", "children": [ { "key": "post-1", "title": "Post One", "children": [] } ] } ] ``` **YAML:** ```yaml - key: ai-memory title: AI Agent Memory children: - key: post-1 title: Post One children: [] ``` #### 13.4.5 `iwe stats` (JSON / YAML) Two modes: aggregate (no `-k`) and per-document (`-k KEY`). **Aggregate** — `json` and `yaml` serialize the `GraphStatistics` struct directly. Aggregate `markdown` is in §13.3.6; aggregate `csv` is in §13.5.3. **Per-document** — `-k KEY` emits a single object/document. With `-f yaml`, YAML; with `-f json`, JSON. Per-doc format is restricted to `json|yaml`. ### 13.5 Other formats #### 13.5.1 `iwe count` — bare integer ``` 25 ``` A single integer followed by a newline. No format flag, no envelope. Empty corpus returns `0`. Stderr carries errors as usual; stdout is exactly the integer plus newline. #### 13.5.2 `iwe export` — `dot` `-f dot` only. The output is a graphviz DOT document; its internal grammar is owned by graphviz. The envelope is whatever `dot_exporter::export_dot` produces. `--include-headers` switches to a denser variant (`dot_details_exporter::export_dot_with_headers`) but stays valid DOT. This spec does not pin the internal DOT grammar. Consumers should pass the output to a DOT-aware tool unmodified. #### 13.5.3 `iwe stats` — `csv` Aggregate mode (no `-k`) emits one row per document with `GraphStatistics::export_csv` headers. Per-document mode (`-k KEY`) does not produce csv — it falls through to JSON in the current implementation; §13.4.5 proposes restricting per-doc output to `json|yaml` at parse time. #### 13.5.4 Mutation commands — prose status `update`, `attach`, `new`, `init`, `normalize`, `squash` (and the rest of the create-family) emit a fixed prose status line. They have no format flag because the operation has nothing structured to report. ``` Updated '<key>' Updated N document(s) Created '<key>' Renamed '<old>' to '<new>' ``` `--quiet` suppresses the status line. `--dry-run` (where supported) prefixes with `Would `. ### 13.6 Flag effects on output shape Flags that change the *shape* (not just the selection) of output. Selection-only flags (filter, sort, limit, anchors) are in §12.2. #### 13.6.1 `iwe find` | Flag | Effect on shape | |---|---| | `--project f1,f2,...` | `markdown` renders the compact index (one `- [title](key)` line per result) unless a `$content`-shaped field is projected, in which case it switches to one fenced block per result — the frontmatter carries the projected fields under their output names, with `key` lifted to the fence info string and the `$content` field rendered as the body. `keys` is unaffected. JSON/YAML: each `FindResult` carries only the listed fields, in the listed order. | | `--add-fields f1,f2,...` | Same as `--project` — additive over the default projection in structured output (§6.3); `keys` ignores it. JSON/YAML: each `FindResult` carries the default projection plus the listed fields. | | `--blocks "PRED"` / a projected `$blocks` field | `markdown`: one grep line per entry — `key › section path › text` — under the result's index line; never a ` · name: value` annotation. JSON/YAML: an array of entry mappings — `type`, `path`, and `text` on every entry (`""` when the block has no own text), plus `target` on every `ref` entry — per the stable-schema rule (§13.1.3). Entries are exempt from `--max-tokens` / `--max-document-tokens`. | | `--matches PATTERN` / a projected `$matches` field | Same rendering, one entry per matching line (`path`, `text`). | A narrowed `{ $content: PREDICATE }` field is `$content`-shaped for the fenced-block rule in the `--project` row: it switches the invocation to the fenced-block form with the narrowed rendering as the body, and it is counted and capped by the token budgets like a full body. #### 13.6.2 `iwe retrieve` The flags below gate which structural sources are populated (per the conditional-source rule in §6.2.4). When a backing flag is omitted, the corresponding field is still emitted with its empty value. | Flag | Effect on shape | |---|---| | `--children` | `includes` populated with `EdgeRef` entries for child documents. | | `-b`, `--backlinks` | `referencedBy` populated with `EdgeRef` entries for inbound reference edges (default on). | | `--expand-includes N` | Adds N levels of inclusion descendants to the top-level array (selection, not shape). | | `--expand-included-by N` | Same, for inclusion ancestors. | | `--expand-references N` | Adds outbound reference targets within N hops, and populates `references` with their `EdgeRef` entries. | | `--expand-referenced-by N` | Adds inbound reference sources within N hops (selection, not shape). | #### 13.6.3 `iwe tree` | Flag | Effect on shape | |---|---| | `--project EXPR` | JSON/YAML only: each `TreeNode` carries the projected fields alongside the system fields (`key`, `title`, `children`). `EXPR` takes the full `--project` grammar (§12.3.2) — frontmatter fields, `$`-selectors, block sources. `children` is always present regardless of projection. With no `--project`: only system fields. `markdown` and `keys` ignore projection. | | `--add-fields EXPR` | Same grammar; extends each node's default fields instead of replacing. Mutually exclusive with `--project`. | #### 13.6.4 Mutation commands | Flag | Effect on shape | |---|---| | `-f keys` | Switches from prose status to one-key-per-line. | | `--dry-run` | Prefixes prose with `Would …`; for `keys`, lists the keys that *would* be affected. Suppresses writeback. | | `--quiet` | Suppresses prose-form output. Has no effect on `-f keys`. | ### 13.7 Error output All commands write errors and progress to stderr. Stdout carries only the format-determined payload. An error path MUST NOT print partial output to stdout — if the command fails before producing a complete result, stdout is empty and the process exits non-zero. Error message form is free-text, prefixed `Error: ` (or `error: ` for parse-stage failures). ## Appendix A. Formal grammar (BNF) This appendix collects the full BNF grammar for the IWE query language. The semantic rules — type coercion, missing-field behavior, equality, edge model, walk semantics — live in §§2–11. This appendix is the syntactic source of truth. ### Notation - `::=` defines a production. - `|` separates alternatives. - `[X, ...]` is a YAML sequence of `X`. Empty sequences are noted as parse-time errors where they apply. - `{ K: V, ... }` is a YAML mapping. Required vs optional entries are annotated inline. - All literal `$`-prefixed names are operator keywords; user frontmatter field names cannot begin with `$` (§2.3). ### A.1 Operation documents ``` operation ::= find_op | count_op | update_op | delete_op find_op ::= { filter: filter (optional, default {}) search: search_spec (optional; §A.4A) project: projection (optional, mutually exclusive with addFields) addFields: projection (optional, mutually exclusive with project) sort: sort (optional) limit: limit (optional) } count_op ::= { filter: filter (optional, default {}) sort: sort (optional) limit: limit (optional) } update_op ::= { filter: filter (required) sort: sort (optional) limit: limit (optional) update: update_doc (required) expect: expect_val (optional; matched-document guard, §3.2; production in §A.7) } delete_op ::= { filter: filter (required) sort: sort (optional) limit: limit (optional) expect: expect_val (optional; matched-document guard, §3.2; production in §A.7) } ``` Operation-inappropriate fields are parse-time errors (e.g. `project` outside `find`, `update` outside `update`, `expect` outside `update` / `delete`). `project` and `addFields` cannot both be set in a single `find_op` (§6.3). ### A.2 Filter ``` filter ::= { (filter_entry)* } # entries AND-composed at top level filter_entry ::= field_path : field_predicate | logical_op | graph_op field_predicate ::= value # implicit $eq (§4.1) | operator_expr | nested_filter operator_expr ::= { ($_field_op : V)+ } # all keys $-prefixed; multiple keys ANDed nested_filter ::= { (sub_field_entry)+ } # all keys non-$-prefixed sub_field_entry ::= field_path : field_predicate # A mapping that mixes $-prefixed and non-$-prefixed keys at the same level is a parse-time error. ``` #### A.2.1 Logical operators ``` logical_op ::= $and : [filter, ...] # non-empty | $or : [filter, ...] # non-empty | $nor : [filter, ...] # non-empty ``` `$not` is field-level only; see `$_field_op` below. #### A.2.2 Field operators ``` $_field_op ::= comparison_op | element_op | array_op | $not : operator_expr # per-field negation comparison_op ::= $eq: value | $ne: value | $gt: value | $gte: value | $lt: value | $lte: value | $in: [value, ...] # non-empty | $nin: [value, ...] # non-empty element_op ::= $exists: bool | $type: type_name | [type_name, ...] # non-empty list array_op ::= $all: [value, ...] # non-empty | $size: count_pred count_pred ::= non_neg_int | { count_op: non_neg_int, ... } # bare int is $eq; ops AND together count_op ::= $eq | $ne | $gt | $gte | $lt | $lte type_name ::= "string" | "number" | "boolean" | "null" | "array" | "object" | "date" | "datetime" # Type names are YAML strings only. The bare YAML null literal ($type: null) is a # parse-time error — write $type: "null" to test for the null type. ``` #### A.2.3 Field paths ``` field_path ::= segment ("." segment)* # dotted shorthand segment ::= identifier # §2.3 # non-empty; no whitespace; no control chars; no `.`; # first char not in $, _, ., #, @ ``` A nested mapping (`author: { name: ... }`) is equivalent to the dotted form (`author.name: ...`). Field names containing a literal `.` are not addressable. ### A.3 Graph operators ``` graph_op ::= $key : key_op | $includes : relational_arg | $includedBy : relational_arg | $references : relational_arg | $referencedBy : relational_arg | $content : block_predicate # block membership: at least one matching block (§A.9) ``` The `filter` production used inside relational operators (`match` field, §A.3.2) is the same `filter` production from §A.2 — the grammar is mutually recursive. #### A.3.1 Identity ``` key_op ::= key | key_expr key_expr ::= { $eq: key } | { $ne: key } | { $in: [key, ...] } # non-empty | { $nin: [key, ...] } # non-empty | { $startsWith: key } # non-empty # Exactly one operator per key expression. # $gt / $gte / $lt / $lte on $key are parse-time errors. ``` #### A.3.2 Relational operators ``` relational_arg ::= key | relational_obj relational_obj ::= { match: filter (optional; absent = {} — any document) maxDepth: int >= 0 (inclusion ops, optional; default 1; 0 = unbounded) minDepth: int >= 1 (inclusion ops, optional; default 1) maxDistance: int >= 0 (reference ops, optional; default 1; 0 = unbounded) minDistance: int >= 1 (reference ops, optional; default 1) $size: count_pred (optional; default { $gte: 1 }; §5.2.6) } # Scalar `key` shorthand expands to { match: { $key: KEY } } for every operator, # inheriting the default bounds (direct edges). # Inclusion-edge ops accept maxDepth / minDepth only; # maxDistance / minDistance are parse-time errors. # Reference-edge ops accept maxDistance / minDistance only; # maxDepth / minDepth are parse-time errors. # match is optional; an object without match anchors on any document. # Empty mapping {} is a parse-time error. The array form [...] is a parse-time error. # maxDepth / maxDistance are >= 0 (0 = unbounded); minDepth / minDistance are >= 1. # 0 is the single unbounded sentinel; a lone minDepth/minDistance >= 2 with no max is # an inverted range and a parse-time error (§5.2.3). # minDepth > maxDepth (and minDistance > maxDistance) is a parse-time error (§5.2.3). # Walk-parameter names inside relational_obj are bare; $size keeps its $ as an evaluating operator. # The recognized key set is closed: any key other than match / maxDepth / minDepth / # maxDistance / minDistance / $size is a parse-time error (unknown keys are not silently ignored). # The filter inside `match` is the §A.2 filter production — the grammar is mutually recursive. # Because that filter accepts any §A.2 / §A.3 production, $key is allowed inside `match`: # { match: { $key: K }, maxDepth: 5 } # { match: { $key: { $in: [a, b] } }, maxDepth: 5 } # { match: { $or: [{ $key: a }, { tag: x }] } } ``` ### A.4 Projection ``` projection ::= { (project_entry)* } # empty mapping = explicit empty projection: # each result carries no fields project_entry ::= field_path : source source ::= include_marker # include frontmatter[outputName] | "$" pseudo_field # include the named structural pseudo-field source | dotted_path # include frontmatter at the dotted path | block_source # block surface (§A.9 for block_predicate) include_marker ::= 1 | true | null # all three mean "include frontmatter[outputName]"; # type-strict: integer 1, bool true, or YAML null # 0, false, "1", "true", "null", 1.0 → parse-time error pseudo_field ::= "key" | "title" | "titleSlug" | "content" | "frontmatter" | "includedBy" | "includes" | "referencedBy" | "references" # closed set; see §6.1 block_source ::= { $content: block_predicate } # body narrowed to matching blocks — a string | $blocks | { $blocks: block_predicate } # located blocks as entries; bare form = { $blocks: {} } | { $matches: regex } # grep — one entry per matching line | { $matches: { pattern: regex, (block_pred_entry)* } } # scoped grep: bare `pattern` is the payload, $-keys select dotted_path ::= segment ("." segment)* # §A.2.3 segment rules ``` The `project` clause of a `find_op` (not `addFields`) additionally accepts a **block predicate** in place of the field map — any `$`-key at the top level of `project` selects this reading: ``` project_clause ::= projection # v1 field map (bare output names) | block_predicate # §A.9; lowers to # { key: $key, content: { $content: P } }; # mixing bare and $ keys = parse-time error; # {} keeps its v1 empty-projection meaning ``` ### A.4A Search (find only) ``` search_spec ::= { lexical: string (optional) fuzzy: string (optional) } # At least one of lexical / fuzzy is required; the empty mapping {} is a parse-time error. # Both present → RRF fusion of the two rankings. # The recognized key set is closed: any key other than lexical / fuzzy is a parse-time error. # search is valid only in a find operation; appearing in count / update / delete is a parse-time error. # search selects and orders (§5A): membership = filter ∩ search-matches, default order = relevance. # A lexical query with no searchable terms (stop words only) matches nothing and carries a warning. ``` ### A.5 Sort ``` sort ::= { field_path : sort_dir } # exactly one entry sort_dir ::= 1 | -1 # type-strict integer; YAML +1 normalizes to 1 and is accepted; # 1.0, "1", true, null → parse-time error ``` ### A.6 Limit ``` limit ::= non_neg_int # 0 = no limit ``` ### A.7 Update document ``` update_doc ::= { (update_op_entry)+ } # at least one operator update_op_entry ::= $set: { (field_path : value)+ } # body must be non-empty | $unset: { (field_path : any_value)+ } # body must be non-empty; values ignored | $replace: replace_arg # block operators (§9.5); block_pred_entry in §A.9 | $replaceText: replace_text_arg | $insertBefore: insert_arg | $insertAfter: insert_arg | $append: insert_arg | $delete: delete_arg replace_arg ::= { (block_pred_entry)*, content: markdown, expect?: expect_val } replace_text_arg ::= { (block_pred_entry)*, from?: string, to: string, expect?: expect_val } # from omitted: to replaces the entire own text insert_arg ::= { (block_pred_entry)*, content: markdown, expect?: expect_val } delete_arg ::= { (block_pred_entry)*, expect?: expect_val } expect_val ::= non_neg_int | { min?: non_neg_int, max?: non_neg_int } # also the document-level expect of update_op / delete_op (§A.1) # Within a block operator's argument, $-prefixed keys form the block predicate # and bare keys are the payload. The bare-key set per operator is closed; # unknown bare keys are parse-time errors. # An argument with no $-keys carries the empty predicate and selects all blocks. # Block operators are valid only in update documents; appearing in a find / # count / delete operation is a parse-time error. # Unit operators act on the selection's coalesced roots; their expect counts # targets. $replaceText applies per selected block, un-coalesced; its expect # counts blocks. Semantics: query-language.md#block-update-operators. # Empty $set: {} / $unset: {} is a parse-time error (grammar requires +). # Targeting a $-prefixed segment ($ as first character of any segment in any path — # top-level, dotted, or nested mapping key, recursively) is a parse-time error. # Two paths in $set / $unset conflict when, after canonicalizing nested-mapping form # to dotted form per §4.4, one path equals or is a prefix of the other. Conflicts are # parse-time errors. The check applies across operators ($set vs $unset) and within # a single operator (two $set entries). # A dotted $set path that traverses a present-but-non-mapping intermediate coerces the # intermediate to a fresh mapping holding the new leaf (per §9.1, $set). # Not a parse-time error and not a runtime failure. # Mapping values in $set replace wholesale; use dotted shorthand to write subset leaves. ``` ### A.8 Primitives ``` key ::= string # document key (relative path without .md) identifier ::= YAML name; non-empty; no whitespace; no control chars; no `.`; first char not in $, _, ., #, @ value ::= scalar | array | mapping | null scalar ::= string | number | boolean | date | datetime array ::= [value, ...] mapping ::= { (string : value)+ } bool ::= true | false non_neg_int ::= integer ≥ 0 pos_int ::= integer ≥ 1 any_value ::= value # placeholder; ignored by $unset regex ::= string # Rust regex syntax; compile failure = parse-time error markdown ::= string # parsed and normalized on write ``` ### A.9 Block predicates One grammar, consumed at three sites: `$content` in filter (§A.3), the block projection sources and the top-level `project` form (§A.4), and the block operators' selectors (§A.7). Semantics — selection forests, own text, normalized-form matching — are specified in the [Query Language reference](/docs/concepts/query-language/#block-predicates). ``` block_predicate ::= { (block_pred_entry)* } # empty mapping matches every block block_pred_entry ::= $text: string | { $eq: string } # case-insensitive; bare = substring, $eq = whole own text | $matches: regex # case-sensitive; (?i) for case-insensitive | $within: string | block_predicate # interior of the argument's forest; # scalar T = { $section: { $text: { $eq: T } } }; # {} = the interior of the document; # a mapping argument must select content: # hold a tree_selector ($and: some branch, # $or: every branch) — else parse-time error | $contains: block_predicate # descendant axis: has such a block below it | tree_selector # in block position: all blocks of the trees | type_op # per-type operators | $references: key # own content links to the key | $and: [block_predicate, ...] # non-empty | $or: [block_predicate, ...] # non-empty | $nor: [block_predicate, ...] # non-empty type_op ::= $header: type_arg | $paragraph: type_arg | $item: type_arg | $code: type_arg | $table: type_arg | $ref: block_predicate # scalar = parse-time error; own text = authored (piped) link text | $hr: block_predicate # no own text: scalar or direct $text / $matches = parse-time error type_arg ::= string | block_predicate # scalar T = { $text: { $eq: T } }; {} = any block of the type tree_selector ::= { $section: string | block_predicate } # trees rooted at headers matching the argument | { $quote: block_predicate } # empty head; scalar or direct $text / $matches = parse-time error | { $list: block_predicate } # empty head; scalar or direct $text / $matches = parse-time error ``` Every key in a block predicate is a `$`-prefixed operator; unknown `$`-names and bare keys are parse-time errors. Top-level keys AND together, as everywhere in the language. --- # Document Schema URL: https://iwe.md/docs/concepts/document-schema/ A document schema declares the required shape of a page: which frontmatter fields it carries, which sections it contains and in what order, how headers are written, how deep the heading tree may nest, and how large each part may grow. Schemas are checked by [`iwe schema validate`](/docs/cli/schema/), so a store's conventions become machine-checked policy in the loop *write → validate → fix*. The language is JSON-Schema-aligned. Frontmatter is validated by literal JSON Schema (draft 2020-12). The body schema mirrors the document's own structure — a document has `sections`, a section has a `header` and its own nested `sections` — and keyword names and semantics come from JSON Schema wherever the concept maps: `pattern`, `const`, `enum`, `minLength`, `maxLength`, `minContains`, `maxContains`, `additionalSections` (after `additionalProperties`), `description`. A complete schema: ``` yaml $schema: https://document-schema.org/draft/2026-06/schema frontmatter: type: object required: [status] properties: status: { enum: [draft, published] } maxTokens: 1200 sections: - header: { pattern: "^[A-Z]", maxTokens: 12 } maxContains: 1 sections: - header: { const: Summary } maxContains: 1 description: every note opens with a summary - header: { const: Tasks } maxContains: 1 additionalSections: false ``` This reads: the frontmatter declares a `status`; the page body stays under 1200 tokens; there is exactly one top-level section, its header capitalized and at most 12 tokens; it contains a `Summary` section and then a `Tasks` section, in that order; no other top-level sections are allowed. ## 1. Schema documents Schemas live in `.iwe/schemas/<name>.yaml`, one schema per file. Files are YAML 1.2, so JSON content is equally valid. The optional `$schema` key names the dialect; only `https://document-schema.org/draft/2026-06/schema` is accepted. A schema binds to documents through the `[schemas]` section of `.iwe/config.toml` (see [Configuration](/docs/configuration/)): ``` toml [schemas.note] match = "notes/**" [schemas.session] match = ["journal/*", "meetings/**"] ``` The entry name is the schema name — `[schemas.note]` resolves to `.iwe/schemas/note.yaml`. `match` is a glob, or a list of globs, matched against the document key: `*` stays within a path segment, `**` crosses segments, and a leading `/` is optional. A `!` prefix negates a pattern — within one list patterns apply in order and the last matching one decides, gitignore-style, so `["notes/**", "!notes/index"]` binds everything under `notes/` except the index. Bindings compose order-free — a document is validated against **every** schema whose `match` hits, so overlapping bindings compose (as JSON Schema `allOf` does). A document matching no entry is unvalidated. Every keyword in a schema is optional, and an absent keyword constrains nothing. An empty schema (`{}`) passes every document. ## 2. What is validated - **The section tree.** Sections at their structural depth after normalization (`1` for `#`, `2` for `##`). A section's subsections are its `sections`, in document order. Non-section content — paragraphs, lists, code blocks, tables — is invisible to the schema: it is never matched and never violates. - **Header text.** The rendered plain text of a heading, inline markup stripped. - **Token counts.** The same counting as the retrieve budgets, over rendered text, frontmatter excluded. - **The frontmatter mapping.** `{}` when the page has no frontmatter. Every field is presented to the validator, at every depth, whatever its name starts with — a stored query keyed on `$includedBy` validates like any other mapping. YAML dates and datetimes are presented to the validator as ISO-8601 / RFC 3339 strings. ## 3. Document schema The top level of a schema file: | Keyword | Value | Meaning | | -------------------- | ------------------------------ | ---------------------------------------------------------------------- | | `$schema` | string | optional dialect id | | `description` | string | default hint for document-level violations | | `frontmatter` | JSON Schema | validates the frontmatter mapping | | `maxTokens` | integer | budget for the whole rendered body | | `maxDepth` | integer | maximum heading nesting (`3` allows `###`, forbids `####`) | | `allSections` | reduced section schema (§6) | applies to every section at every depth | | `sections` | array of section schemas | ordered shapes for the top-level sections | | `additionalSections` | bool or reduced section schema | policy for top-level sections matching no entry; default `true` (open) | | `blocks` | array of block schemas (§8) | ordered shapes for the content above the first heading | | `additionalBlocks` | bool or reduced block schema | policy for that content matching no entry; default `true` | | `allBlocks` | reduced block schema (§8) | applies to every block at every depth | The `frontmatter` value is standard JSON Schema, draft 2020-12, with `format` assertions enabled. Only in-document references (`#/...`) are allowed; external and remote `$ref` are rejected. ## 4. Section schema An item in a `sections` array is always a section. | Keyword | Value | Meaning | | -------------------- | ------------------------------ | ------------------------------------------------------------------------ | | `header` | header schema (§5) | constrains the header text; also decides binding (§7) | | `maxTokens` | integer | budget for this section's subtree, header included | | `maxDepth` | integer | maximum nesting below this section (`1` allows children, forbids deeper) | | `minContains` | integer, default `1` | minimum occurrences of this shape; `0` makes it optional | | `maxContains` | integer, default unbounded | maximum occurrences | | `description` | string | the violation hint for anything failing in this entry | | `allSections` | reduced section schema | applies to every section below this one | | `sections` | array of section schemas | ordered shapes for the subsections | | `additionalSections` | bool or reduced section schema | policy for subsections matching no entry; default `true` | | `blocks` | array of block schemas (§8) | ordered shapes for this section's content blocks | | `additionalBlocks` | bool or reduced block schema | policy for content blocks matching no entry; default `true` | | `allBlocks` | reduced block schema (§8) | applies to every block below this section | The occurrence defaults are JSON Schema's `contains` defaults: a listed shape is required at least once and unbounded above. The recipes: | Intent | Spelling | | ----------------------- | -------------------------------- | | one or more (default) | nothing | | optional | `minContains: 0` | | exactly one | `maxContains: 1` | | at most one | `minContains: 0, maxContains: 1` | | n or more | `minContains: n` | | exactly n | `minContains: n, maxContains: n` | ## 5. Header schema Applies to the header's plain text. The string keywords carry JSON Schema semantics exactly; `maxTokens` is the one extension. | Keyword | Meaning | | ------------- | ------------------------------------------------------------------------------------------ | | `pattern` | regex the text must match; unanchored, as in JSON Schema — write `^...$` for a full match | | `const` | the text equals this string | | `enum` | the text equals one of these strings | | `minLength` | minimum length in characters | | `maxLength` | maximum length in characters | | `maxTokens` | maximum tokens in the header text | | `description` | hint override for header violations | `const` is in principle an anchored, regex-escaped `pattern` (`const: Tasks` ≡ `pattern: "^Tasks$"` with metacharacters escaped), but it is a distinct keyword for three reasons: JSON Schema alignment; escaping safety for literal headers containing regex metacharacters (a header like `C++ (Draft)` needs no escaping under `const`); and error naming — a missing-section message takes the section's name from `const` directly. Mind the asymmetry: `pattern` is unanchored, so `pattern: Tasks` matches any header *containing* "Tasks", while `const: Tasks` matches exactly. `enum` is a disjunction of consts. `enum` and `const` cannot be combined. > **Quoting regexes in YAML.** A backslash class like `\d` is an invalid > escape inside a YAML double-quoted string. Write patterns in **single > quotes** — `pattern: '^\d{4}$'` — or double the backslashes. ## 6. Reduced section schemas `allSections` and a schema-valued `additionalSections` take a **reduced** section schema: `header`, `maxTokens`, `maxDepth`, and `description` only. Occurrence keywords are meaningless there — `allSections` applies to every section, `additionalSections` applies per leftover section — and structural keywords (`sections`, `additionalSections`, `allSections`) are not allowed inside them. When several `allSections` are in scope (the document's plus enclosing sections'), all of them apply. `additionalSections` is **boolean or schema**, matching JSON Schema's own value form for array extras (`items` after `prefixItems`, draft-07's `additionalItems`, `unevaluatedItems` — all schema-or-boolean, all open by default). `true` allows leftover sections unconstrained, a schema validates each leftover against it, `false` makes each leftover a violation. Semantically it is closest to `unevaluatedItems`: it governs the sections no listed shape claimed. ## 7. Matching semantics For each node — the document, then each bound section, recursively — the node's sections are matched against its `sections` entries. Matching is **ordered, sequential, and greedy, without backtracking**: 1. Walk the instance sections in document order, holding a pointer into the entry list, starting at the first entry. 2. For each section, find the first entry — at the pointer or later — whose **`header` schema** the section's header text satisfies (an entry with no `header` matches any section). Bind the section to that entry and advance the pointer to it. Entries before the pointer are closed and never bind again. 3. A section that satisfies no entry at or after the pointer — including one that would only match an already-closed entry, i.e. out of order — is **additional** and is handled by `additionalSections`. 4. After the walk, every entry's bound count is checked against `minContains` and `maxContains`. An entry bound fewer than `minContains` times reports a missing required section, named by its `const`, else `enum`, else `pattern`, else its position. 5. Each bound section is then validated against the rest of its entry: `maxTokens`, `maxDepth`, `allSections`, and the nested `sections` matching, recursively. Consequences: - **Binding is decided by `header` alone.** A `Tasks` section missing its required subsections still binds to the `Tasks` entry and reports the missing pieces — it does not fall through to `additionalSections`. - Occurrences are counted in **total**, not per consecutive run. A repeated entry stays open and keeps binding matching sections until the pointer advances past it — which happens only when a section binds to a *later* entry. An **additional** section (one matching no open entry) does not advance the pointer, so a matching section after it rejoins the same entry. Hence `date, date, other, date` counts as three dates: it satisfies `minContains: 3` and violates `maxContains: 2`. `minContains` / `maxContains` bound the total, not the length of an unbroken run. - A headerless (wildcard) entry greedily absorbs every remaining section, so any entry after it can never bind — a wildcard must be the last entry. Placing one earlier is rejected as a schema error (§10); `maxContains` does not rescue it, since the wildcard still binds every match first. - There is no backtracking: matching is deterministic and errors are explainable. Order entries specific-first. ## 8. Blocks The schema also reaches below the section, to a section's **content** — paragraphs, lists, code, quotes, tables, rules. The vocabulary mirrors the section level one step down: where a section has `sections`, it also has `blocks`; where a `header` constrains a section's title, `text` constrains a block's content; the `all*` / `additional*` / occurrence machinery and the [matching algorithm](#7-matching-semantics) are reused verbatim. ### 8.1 What a block is Every piece of non-section content is a **block**: a `type`, plain `text` (empty for containers), a token count, and — for containers — child blocks. The seven types map onto the document model: | `type` | content | its `text` | type-only keywords | | -------------- | ----------------------------- | ------------------ | ----------------------------------------- | | `paragraph` | a paragraph | the paragraph text | — | | `bullet-list` | a `-`/`*`/`+` list | (empty) | `items`, `minItems`, `maxItems` | | `ordered-list` | a `1.`/`1)` list | (empty) | `items`, `minItems`, `maxItems` | | `code` | a fenced / raw code block | the code body | `lang` | | `quote` | a block quote | (empty) | `blocks`, `additionalBlocks`, `allBlocks` | | `table` | a table | the cell text | — | | `rule` | a horizontal rule | (empty) | — | An inclusion link (a lone reference to another document) is not a distinct type; it is matched as a `paragraph` whose `text` is the link text. Bullet and ordered lists are **distinct types** — there is no `ordered` flag. A list's elements are its **items**; an item is itself a container (its own `text` plus nested `blocks`), shaped through `items` (§8.4). A section's children are its content blocks **followed by** its subsections — subsections never interleave. The two arrays partition the children cleanly: **`blocks` matches the leading content, `sections` matches the subsections.** The document's top level works the same way — `blocks` there is the content above the first heading. ### 8.2 Block keywords on the document and every section | Keyword | Value | Meaning | | ------------------ | ---------------------------- | ----------------------------------------------------------- | | `blocks` | array of block schemas | ordered shapes for this node's content blocks | | `additionalBlocks` | bool or reduced block schema | policy for content blocks matching no entry; default `true` | | `allBlocks` | reduced block schema | applies to every block at every depth below this node | `allBlocks` is to blocks what `allSections` is to sections; when several are in scope (document, enclosing sections, enclosing containers) all apply. ### 8.3 Block schema — an entry in a `blocks` array | Keyword | Value | Meaning | Types | | ------------------ | ---------------------------- | -------------------------------------------------------------- | ------- | | `type` | one of the seven, or a list | the block kind(s); part of the binding identity (§8.5) | all | | `text` | text schema (§5) | constrains the block's plain text; part of the binding identity | all | | `maxTokens` | integer | budget for this block's whole subtree | all | | `minContains` | integer, default `1` | minimum occurrences of this shape | all | | `maxContains` | integer, default unbounded | maximum occurrences | all | | `description` | string | violation hint for anything failing in this entry | all | | `lang` | text schema (§5) | constrains the code language; part of the binding identity | `code` | | `items` | item schema (§8.4) | schema applied to every list item | lists | | `minItems` | integer | minimum number of list items | lists | | `maxItems` | integer | maximum number of list items | lists | | `blocks` | array of block schemas | ordered shapes for the quote's child blocks | `quote` | | `additionalBlocks` | bool or reduced block schema | policy for the quote's child blocks matching no entry | `quote` | | `allBlocks` | reduced block schema | applies to every block inside the quote | `quote` | `text` and `lang` reuse the [header schema shape](#5-header-schema): `pattern`, `const`, `enum`, `minLength`, `maxLength`, `maxTokens`, `description`. A keyword used on the wrong type — `lang` on a `paragraph`, `items` on a `code`, `blocks` on a `table` — is a load error (§10), as is an unknown `type` value. `type` may also be a **list** of type names — `type: [bullet-list, ordered-list]` binds a block whose kind is any one of them, the block analogue of a header `enum`. A type-specific keyword is then allowed only when every listed type accepts it (`items` on `[bullet-list, ordered-list]` is fine; `items` on `[bullet-list, code]` is a load error). Two token budgets, mirroring `header.maxTokens` vs a section's `maxTokens`: `text: { maxTokens }` bounds the block's **own** text, block `maxTokens` bounds its whole subtree. For a paragraph they coincide; for a list or quote the subtree includes the children. ### 8.4 Lists and items A list (`bullet-list` / `ordered-list`) bounds its length with `minItems` / `maxItems` and shapes each item with `items` — one schema applied to **every** item. Repetition uses this pair, not repeated entries: one-to-ten items is `{ type: bullet-list, minItems: 1, maxItems: 10 }`. > **`minItems` is not `minContains`.** Both `minItems` and `maxItems` default > to *unbounded* — omitting `minItems` means **no minimum** (an empty list > passes). This differs from `minContains`, which defaults to `1`. So a > `bullet-list` entry with nothing set is required to appear (via > `minContains: 1`) but may itself be empty until you add `minItems`. An **item schema** is a container without `type` or occurrence: `text` (the item's own text), `maxTokens` (the item's subtree), `description`, and its own `blocks` / `additionalBlocks` / `allBlocks` for the item's nested content. Containers nest without limit — a quote holding a list whose items hold paragraphs is fully expressible, and an `allBlocks` in scope at any container reaches every block beneath it. ### 8.5 Block matching Within a node, its content blocks are matched against its `blocks` entries by the **same ordered, greedy, no-backtracking algorithm as sections** ([§7](#7-matching-semantics)) — the only difference is the binding identity. A block binds to the first entry at or after the pointer whose **`type`** matches (an entry with no `type` matches any block; a list `type` matches any of its kinds) **and** whose `text` / `lang` identity (`const`, `enum`, `pattern`), if present, the block satisfies. Blocks matching no open entry are **additional**, governed by `additionalBlocks`. After the walk, `minContains` / `maxContains` are tallied, each bound block is validated against the rest of its entry, and every `allBlocks` in scope applies to every block throughout. A missing required block is named by its `text` `const` / `enum` when present, else its `type`, else its position. Every §7 consequence carries over — in particular, **two entries with the same identity are rejected at load (§10): the second could never bind.** Write "two to four lead paragraphs" as `{ type: paragraph, minContains: 2, maxContains: 4 }`, never as repeated `{ type: paragraph }` entries. ### 8.6 Reduced block schema `allBlocks` and a schema-valued `additionalBlocks` take a **reduced** block schema — `text`, `maxTokens`, `description` only. Type-specific keywords (`type`, `lang`, `items`), structural keywords (`blocks`, `additionalBlocks`, `allBlocks`), and occurrence keywords (`minContains`, `maxContains`, `minItems`, `maxItems`) are rejected there, mirroring the [reduced section schema](#6-reduced-section-schemas). ### 8.7 Examples Every line short — no paragraph, list item, table cell, or code body over 40 tokens, anywhere. `text` is the block's *own* text, so this never trips on a long list's total: ``` yaml allBlocks: text: { maxTokens: 40 } ``` A hub page — one lead paragraph, then a bulleted index of short links, nothing else: ``` yaml sections: - header: { pattern: ".+" } blocks: - type: paragraph maxContains: 1 - type: bullet-list minItems: 1 items: text: { maxTokens: 40 } additionalBlocks: false ``` A code-sample section — exactly one fenced block in an allowed language: ``` yaml blocks: - type: code lang: { enum: [rust, toml, bash] } maxContains: 1 ``` ## 9. Violations `iwe schema validate` reports one line per violation, `<key> › <breadcrumb>: <message>` (or `<key>: <message>` when the breadcrumb is empty). The breadcrumb is built from the matched header texts — a position like `sections[2]` where no header text is available, `blocks[1]` / `items[3]` for content blocks and list items, a frontmatter path like `frontmatter › status`. A `hint:` line follows when a hint is present: ``` text journal/2026-01-05 › Tasks: header is 18 tokens (limit 12) hint: keep section headers short notes/intro: required section 'Summary' missing hint: every note opens with a summary notes/intro › frontmatter › status: not one of 'draft', 'published' ``` The hint is the nearest `description` walking outward from the failing keyword — header schema, then entry, then enclosing entries, then the document schema. Without one, no hint line is shown. `-f json` output is an array of `{ key, schema, violations }` objects; each violation additionally carries the machine paths `schemaPath` (a JSON Pointer into the schema file, e.g. `/sections/0/sections/1/header`) and the failing `keyword`. A document bound to several schemas yields one report per schema. The command exits `1` when any document has a violation, `0` when the store is clean. ## 10. Schema errors These are configuration errors — `iwe schema validate` prints them to stderr and exits `2` before validating any document, rather than reporting them as violations: - a `[schemas]` entry naming a schema file that does not exist; - an invalid glob in a `[schemas]` entry's `match`; - a `frontmatter` subschema that fails the 2020-12 meta-schema, or contains an external or remote `$ref`; - an unknown keyword anywhere outside `frontmatter` — unlike JSON Schema, unknown keywords are rejected, so a typo cannot silently validate nothing (`ordered` is not a keyword, so it reports as unknown); - an unknown block `type`, an empty `type` list, or a type-specific block keyword on the wrong type (`lang` off `code`, `items` off a list, `blocks` off `quote` — for a list `type`, every listed type must accept the keyword); - an invalid `pattern` regex, a negative count, `minContains` greater than `maxContains`, `minItems` greater than `maxItems`, or `enum` and `const` together; - occurrence or structural keywords inside a reduced section or block schema; - an **unreachable entry** in a `sections` or `blocks` array — a wildcard entry (one with no identity constraint) that is not last, since it absorbs every match and starves the entries after it; or an entry whose identity exactly duplicates an earlier one, since the earlier always binds first. ## 11. Examples Header discipline for a whole store — every header capitalized and short, every section within budget, nothing deeper than `###`: ``` yaml maxDepth: 3 allSections: header: { pattern: "^[A-Z]", maxLength: 60 } maxTokens: 400 ``` A log page — at least three dated entries, each small, extra sections allowed but budgeted: ``` yaml sections: - header: { pattern: '^\d{4}-\d{2}-\d{2}$' } minContains: 3 maxTokens: 150 additionalSections: maxTokens: 300 ``` A docs page — Installation and Usage required in order, Configuration optional: ``` yaml sections: - header: { pattern: ".+" } maxContains: 1 sections: - header: { const: Installation } - header: { const: Usage } - header: { const: Configuration } minContains: 0 ``` Frontmatter only — the body left free: ``` yaml frontmatter: type: object required: [type, date] properties: type: { const: post } date: { type: string, format: date } tags: type: array items: { type: string, pattern: "^[a-z][a-z0-9-]*$" } ``` ## See also - [`iwe schema validate`](/docs/cli/schema/) — the command that checks schemas, its output and exit codes. - [Configuration](/docs/configuration/) — the `[schemas]` section and glob binding. - [Query Language](/docs/concepts/spec/) — the corpus model and the field names the language cannot address. --- # Compare IWE URL: https://iwe.md/docs/compare/ These pages answer one question each: which tool fits the way you write notes. Each one compares features, says where IWE is weaker, and carries the date its facts were checked, because other tools change and a comparison goes stale. ## How these pages are written - **Dated.** Every page states when its facts were checked. Competitor facts come from each tool's documentation, source and releases. - **Honest.** Every page lists where the other tool is ahead of IWE. - **Open to corrections.** If something is wrong or out of date, [open an issue](https://github.com/iwe-org/iwe/issues) and it will be fixed. The [full comparison](/docs/concepts/comparison/) is the hub with every tool and its sources. To try IWE, start with [get started](/get-started/). --- # IWE vs markdown-oxide vs Marksman: Markdown LSPs compared URL: https://iwe.md/docs/compare/iwe-vs-markdown-oxide-vs-marksman/ *Checked on 2026-11-07 for project status and stars. Features checked on 2026-09-03 against [marksman's feature documentation](https://github.com/artempyanykh/marksman/blob/main/docs/features.md) and its 2026-02-08 release, [oxide.md](https://oxide.md/) and markdown-oxide release `v0.25.12`, and IWE `0.23.0`.* ## markdown-oxide vs Marksman: the short answer **Pick Marksman** if you want correct links, headings and references in almost any editor with the least setup. It covers the most editors, including Vim, Emacs, Sublime Text and BBEdit, and Helix ships it as a default. **Pick markdown-oxide** if you are coming from Obsidian and want its editing model in your editor: daily notes with natural-language dates, block references (`[[doc#^block]]`) and tags. **Pick IWE** if you also want to query your notes, restructure them, and let an AI agent work on them through a CLI and an MCP server, not only from the editor. One thing to know before you choose: Marksman's last push was on 2026-02-08, and it has had no release since. It remains the most-starred of the three, with about 3,390 GitHub stars on 2026-11-07. markdown-oxide is actively maintained, with about 2,340 stars on the same date. ## Feature comparison | Capability | Marksman | markdown-oxide | IWE | | --- | --- | --- | --- | | **Aimed at** | Correct links, headings and references in any editor | Obsidian's editing model inside a text editor | Notes as a knowledge graph, with an editor, a CLI and AI agents on the same files | | **Link syntax** | Markdown inline and reference links, wiki links | Obsidian-style wiki links | Markdown links, `[[wiki]]`, `[[wiki\|piped]]` and [inclusion links](/docs/concepts/inclusion-links/) | | **Link targets** | Documents and headings | Documents, headings and blocks (`[[doc#^block]]`) | Documents only; `#fragment` is dropped when resolving | | **Diagnostics** | Live wiki-link diagnostics | Unresolved-reference diagnostics | None from the language server; `iwe stats` reports broken links in batch | | **Reference counts** | Per heading, via code lens | Per heading or block, via code lens (needs enabling in some clients) | One backlink count per document, as an [inlay hint](/docs/features/inlay-hints/) | | **Daily notes** | None | `:Today`, `:Daily next monday`, relative `prev` / `next` | Template-based via [attach](/docs/code-actions/attach/); no date resolution | | **Tags** | – | Tag completion and tag references | Frontmatter `tags` are queryable; no inline `#tag` index | | **Table of contents** | Code action generates and updates one | – | Not supported | | **Search ranking** | Subsequence matching, no relevance ranking | No ranking of its own; the editor sorts completions | Fuzzy and BM25 fused with [RRF](/docs/architecture/search-ranking/) | | **Restructuring** | Not a goal | Linking and rename | [Extract](/docs/code-actions/extract/) and [inline](/docs/code-actions/inline/) sections, section to list, attach, squash | | **Formatting** | – | Basic formatting | [Full normalization](/docs/features/normalization/) of headers, lists, link titles and wrapping | | **Query language** | None | None | [MongoDB-style YAML](/docs/concepts/query-language/) over frontmatter and graph edges | | **CLI** | None (a check command is on the roadmap) | Small CLI for daily notes and config files | [Full CLI](/docs/cli/): find, retrieve, tree, stats, schema, normalize, bulk update and delete | | **AI agents (MCP)** | None | None | [MCP server](/docs/agentic/mcp/) with 14 tools, prompts and resources | | **Editors** | VS Code, Neovim, Vim, Emacs, Helix, Kakoune, Sublime Text, BBEdit, Zed | Neovim, VS Code, Zed, Helix, Kakoune | VS Code, Neovim, Zed, Helix; any LSP client by hand | | **Written in** | F#/.NET | Rust | Rust | | **License** | MIT | Apache-2.0 | Open source | A dash means the sources did not cover that capability for that tool. The table compresses a lot. The [full comparison](/docs/concepts/comparison/) has a separate table for each tool, with links to the source behind each claim. ## When to pick each ### Marksman Marksman is the long-standing general-purpose Markdown language server. Its job ends at making links, headings and references work correctly in the editor, and it does that job well. Choose it if: - you use Vim, Emacs, Sublime Text or BBEdit, which neither markdown-oxide nor IWE lists - you want live diagnostics for broken links as you type - you link to headings, not only to whole notes - you want a generated table of contents ### markdown-oxide markdown-oxide brings Obsidian's model into a text editor. On Obsidian compatibility it is ahead of both other tools. Choose it if: - you keep daily notes and want to open them by date, including phrases like "next monday" - you link to individual blocks without moving them - you use inline `#tags` and want them completed and indexed - you are moving from an Obsidian vault and want its conventions to keep working markdown-oxide parses with regular expressions, a deliberate tradeoff for indexing speed. IWE builds a full syntax tree. Both work; they fail on different edge cases. The [full comparison](/docs/concepts/comparison/#on-parsing) explains the difference. ### IWE IWE treats the notes as a graph, and the language server is one view of it. The [`iwe` CLI](/docs/cli/) and the [MCP server](/docs/agentic/mcp/) are two more views of the same model, so an AI agent such as Claude Code or Codex can search, read and edit the notes you write in your editor. Choose it if: - you want to split a long note into several, or merge notes back, with links rewritten for you - you want to query notes by frontmatter and by links, from the editor or the command line - you want an AI agent to work on your notes with structured, guarded edits - you want bulk updates across the whole folder, filtered by a query ## Where IWE is weaker IWE is not the best pick for everyone. Compared with these two: - **No live diagnostics.** Broken links show up only in batch, through `iwe stats`. - **No heading or block link targets.** IWE resolves links to whole documents. Its answer is to [extract](/docs/code-actions/extract/) a section into its own note, which is a different answer, not a strictly better one. - **Weaker daily notes.** Templates only, with no natural-language dates and no weekly, monthly or yearly periods. - **One reference count per document,** not per heading. - **No inline `#tag` index** and no aliases. - **Fewer documented editors.** Marksman also covers Emacs, Vim, Sublime Text, BBEdit and Kakoune; markdown-oxide also covers Kakoune. - **No table of contents** action. ## Set up IWE in your editor Each guide covers install, shortcuts and troubleshooting: - [VS Code](/docs/editors/vscode/): extension from the Marketplace - [Neovim](/docs/editors/neovim/): the iwe.nvim plugin with Telescope - [Zed](/docs/editors/zed/): extension from Zed Extensions - [Helix](/docs/editors/helix/): plain LSP configuration, no plugin Any other editor with Language Server Protocol support works too: run `iwes` as the language server for Markdown files. ## Next steps - [Get started](/get-started/) to install IWE - [How IWE compares](/docs/concepts/comparison/) for every tool, including zk, telekasten.nvim and mdbase - [Obsidian vs your editor](/docs/compare/obsidian-vs-editor/) if you are also weighing Obsidian --- # Obsidian vs Zed, Neovim or VS Code for Markdown notes URL: https://iwe.md/docs/compare/obsidian-vs-editor/ *Checked on 2026-09-03 for Obsidian pricing and platform support, and on 2026-09-11 for its plugin marketplace. Obsidian feature rows were last checked in May 2026. IWE facts checked against IWE `0.23.0`.* Obsidian and a code editor can open the same folder of Markdown files. The question is which one you want to write in. This page is for you if you already spend your day in Zed, Neovim, VS Code or Helix and wonder whether your notes belong there too. ## The short answer **Pick Obsidian** if you want one application that does everything, with mobile apps, a graph view, canvas, a web clipper and thousands of community plugins. If you do not already live in a text editor, Obsidian is very likely the right choice. **Pick your code editor with IWE** if you already write code in Zed, Neovim, VS Code or Helix and want your notes to work like code: follow links, find what links here, rename with every reference updated, and restructure notes with code actions. Your AI agent works on the same folder through the `iwe` CLI and MCP server. You do not have to choose once. Your notes stay a folder of `.md` files, so Obsidian, your editor and IWE can all open the same folder. ## Obsidian vs Zed In Zed, the notes features come from the [IWE extension for Zed](/docs/editors/zed/): - `F12` follows a link and `Shift+F12` lists every note that links to the current one. - `Cmd+T` finds a note by heading. - `Cmd+.` opens code actions that extract a section into its own note or inline it back. - `F2` renames a note and updates every link to it. - Format on save keeps titles and link text consistent. The extension installs from Zed Extensions and downloads the `iwes` language server on first start. It supports macOS and Linux; Windows support is planned. Obsidian, by contrast, has its own editor, mobile apps, and a graph view. ## Obsidian vs Neovim, VS Code and Helix The same language server runs in each editor, so the features are the same and only the keys differ: | Editor | How IWE installs | Follow link | Backlinks | | --- | --- | --- | --- | | [VS Code](/docs/editors/vscode/) | Extension from the Marketplace | `F12` | `Shift+F12` | | [Neovim](/docs/editors/neovim/) | iwe.nvim plugin, with Telescope pickers | `gd` | `gr` | | [Zed](/docs/editors/zed/) | Extension from Zed Extensions | `F12` | `Shift+F12` | | [Helix](/docs/editors/helix/) | Plain LSP configuration, no plugin | `gd` | `gr` | Any other editor with Language Server Protocol support works too, configured by hand. ## Feature comparison | Feature | Obsidian | Code editor with IWE | | --- | --- | --- | | **Where you write** | Obsidian's own editor | The editor you already use | | **Cost** | Free for personal use; Sync $4–5/month, Publish $8–10/month, commercial use $50/user/year | Free and open source | | **Plugins** | 4,000+ community plugins | No plugin ecosystem; extend through LSP features, the CLI and MCP tools | | **Preview** | Built in | Not part of IWE; whatever your editor or its plugins provide | | **Graph view** | Interactive, with custom styling | Dot export from the CLI | | **Canvas** | Native infinite canvas | Not supported | | **Web clipper** | Native browser extension | Not supported | | **Mobile** | Native apps with sync | Desktop and terminal only | | **Sync** | Obsidian Sync (paid) or your own Git setup | Git, with any hosting | | **Restructuring** | Manual linking and organization | Extract and inline sections, section to list, attach, with links rewritten | | **Queries** | Bases (typed database views) and the Dataview plugin | [MongoDB-style YAML queries](/docs/concepts/query-language/) over frontmatter and links | | **Bulk edits** | No batch operations | `iwe update` and `iwe delete` across the folder, filtered by a query | | **Formatting** | Basic; plugins for more | [Normalization](/docs/features/normalization/) of headers, lists and link titles on save | | **AI agents** | Through third-party plugins | Built-in [MCP server](/docs/agentic/mcp/) and CLI for Claude Code, Codex and other agents | | **Performance** | Electron app; can be slower with large vaults | Rust; [20,000 files in under a second](/docs/architecture/benchmark/) | | **Learning curve** | GUI, works out of the box | Needs a terminal, a binary and editor setup | ## What the editor setup gains - **Your editor's keys, themes and splits.** Notes open in the same tool as your code, with the same shortcuts. - **Notes that refactor like code.** A code action moves a section into its own note and leaves a link behind, or merges a linked note back in. - **One folder for you and your agent.** Claude Code, Codex or any MCP client searches the notes, pulls a note with its surrounding structure into context, and makes guarded edits. Agent tooling ships with IWE, not through community plugins. - **Queries and bulk edits from the command line.** The same query language works in the CLI and the MCP server. - **No subscription.** Sync is whatever Git hosting you already use. ## What it lacks next to Obsidian Be clear about what you give up: - **No plugin ecosystem.** Obsidian has thousands of community plugins. IWE has none of its own; it extends through LSP features, the CLI and MCP tools. - **No preview from IWE.** IWE does not render Markdown. You get whatever preview your editor offers, or none; in Neovim, the guide suggests [render-markdown.nvim](https://github.com/MeanderingProgrammer/render-markdown.nvim) for in-buffer rendering. - **No mobile apps, canvas, web clipper or interactive graph view.** Obsidian wins these outright. - **Slower first run.** IWE is a binary, an editor configuration and a folder of Markdown. If you have never used a language server, Obsidian will be working before IWE is configured. - **Some Obsidian syntax resolves differently.** IWE reads `[[wiki]]` and `[[wiki|piped]]` links, but resolves them to whole notes: links to a heading or a `^block` point at the note, not the section. Inline `#tags` are not indexed, and notes have no aliases. - **Weaker daily notes.** Templates through [attach](/docs/code-actions/attach/), with no natural-language dates. ## Bring your Obsidian vault An Obsidian vault is a folder of `.md` files, so it already works as an IWE workspace. Run `iwe init` in the folder to write `.iwe/config.toml`, fitted to the conventions the notes already use. Nothing is imported or converted, and Obsidian keeps working on the same files. ## Want an app, not an editor? [iWe for Mac](/app/) is a free native app for the same kind of folder Obsidian uses. It has a graph view, full-text search, search pages that filter notes by properties and links, and Claude Code or Codex working inside the document. It needs macOS 15 or later, and your editor and the `iwe` CLI keep reading the same files. ## Next steps - [Get started](/get-started/) to install IWE - [Editor setup](/docs/editors/) for VS Code, Neovim, Zed and Helix - [How IWE compares](/docs/concepts/comparison/) for the full, sourced comparison with Obsidian and other tools - [IWE vs markdown-oxide vs Marksman](/docs/compare/iwe-vs-markdown-oxide-vs-marksman/) if you are choosing a Markdown language server --- # Use cases URL: https://iwe.md/docs/use-cases/ Each guide here takes one job and walks you through it in a scratch workspace, with the real commands and their output. Start with an empty folder, follow the steps, and you end up with something you can reuse on your own notes. --- # Markdown database: query and update Markdown files with IWE URL: https://iwe.md/docs/use-cases/markdown-database/ A Markdown database is a folder of `.md` files that you query like a table. Each file is a row, its path is the primary key, the YAML frontmatter holds the columns, and links between files are the relations. IWE reads the folder directly, so there is no import step and no second copy of your data: `iwe find`, `iwe count`, `iwe update` and `iwe delete` run filters over the files, and `iwe schema validate` checks them against a schema. This guide builds a small task database from scratch and runs every operation on it. For the long-form reasoning behind the idea, read [Markdown as a database](/blog/markdown-as-a-database/). For an overview of the query language and what it can do, see [Your notes become a database](/database/). ## How database concepts map to Markdown | Database concept | In a Markdown database | Example | | --- | --- | --- | | Table row | One Markdown file | `tasks/analytics.md` | | Primary key | The [key](/docs/architecture/keys/): the file path relative to the workspace root, without `.md` | `tasks/analytics` | | Columns | Frontmatter fields | `status: review`, `priority: 5` | | Containment relation | An [inclusion link](/docs/concepts/inclusion-links/): a link alone on its own line | `[Set up analytics](../tasks/analytics)` | | Reference relation | An inline link inside a sentence | `Draft the copy with [Alice](../people/alice).` | | Schema | A [document schema](/docs/concepts/document-schema/) in `.iwe/schemas/`, bound to keys in `.iwe/config.toml` | `.iwe/schemas/task.yaml` | | Query and CRUD | One filter language shared by `find`, `count`, `update` and `delete` | `iwe find --filter 'status: draft'` | Folders only group files on disk. A document belongs to a project because the project includes it, not because of the folder it sits in. ## Prerequisites - IWE installed and on your `PATH` (see [Installation](/docs/getting-started/installation/)). The output below comes from `iwe` 0.25.0. - A terminal. Everything happens in a new, empty folder, so you can follow along without touching your notes. ## Step 1: create the workspace Make a folder and initialize it: ```bash mkdir tasks-db && cd tasks-db iwe init ``` ```text 0 files here initialized .iwe/config.toml try: iwe normalize --dry-run · iwe find --fuzzy <query> ``` ## Step 2: add a few rows Create five files. Three tasks share the same columns, a project includes them, and a person is referenced from one task. `projects/launch.md`: ```markdown --- type: project status: active --- # Website launch [Write the landing page](../tasks/landing-page) [Set up analytics](../tasks/analytics) [Publish the docs](../tasks/docs) ``` `tasks/landing-page.md`: ```markdown --- type: task status: draft priority: 8 tags: [design, copy] --- # Write the landing page Draft the hero copy with [Alice](../people/alice). ``` `tasks/analytics.md`: ```markdown --- type: task status: review priority: 5 tags: [ops] --- # Set up analytics Pick a privacy-friendly tracker. ``` `tasks/docs.md`: ```markdown --- type: task status: done priority: 3 tags: [copy] --- # Publish the docs Sync the reference pages. ``` `people/alice.md`: ```markdown --- type: person role: designer --- # Alice ``` The three links in `projects/launch.md` each sit alone on a line, so they are inclusion links: the tasks belong to the project. The link to Alice sits inside a sentence, so it is a reference. ## Step 3: list every row `iwe find` with no arguments returns every document as a one-line index. Arrows show the relations and the dots list the columns: ```bash iwe find ``` ```text - [Alice](people/alice) <- [Write the landing page](tasks/landing-page) · type: person · role: designer - [Set up analytics](tasks/analytics) <- [Website launch](projects/launch) · type: task · status: review · priority: 5 · tags: ops - [Publish the docs](tasks/docs) <- [Website launch](projects/launch) · type: task · status: done · priority: 3 · tags: copy - [Write the landing page](tasks/landing-page) -> [Alice](people/alice) <- [Website launch](projects/launch) · type: task · status: draft · priority: 8 · tags: design, copy - [Website launch](projects/launch) -> [Set up analytics](tasks/analytics), [Publish the docs](tasks/docs), [Write the landing page](tasks/landing-page) · type: project · status: active ``` To see the columns your files actually use, run `iwe schema`. It scans the frontmatter and reports types, coverage and values: ```bash iwe schema --filter 'type: task' ``` ```text | Field | Types | Coverage | Distinct | Values | | -------- | ------------- | -------- | -------- | --- | | priority | number (100%) | 3 (100%) | 3 | 3 (1), 5 (1), 8 (1) | | status | string (100%) | 3 (100%) | 3 | done (1), draft (1), review (1) | | tags | array (100%) | 3 (100%) | 0 | --- | | type | string (100%) | 3 (100%) | 1 | task (3) | ``` ## Step 4: query by frontmatter `--filter` takes a YAML filter in the style of MongoDB. Top-level fields are combined with AND, and `$`-prefixed keys are operators. This query returns tasks that are still in draft or review: ```bash iwe find --filter 'type: task, status: { $in: [draft, review] }' ``` ```text - [Set up analytics](tasks/analytics) <- [Website launch](projects/launch) · type: task · status: review · priority: 5 · tags: ops - [Write the landing page](tasks/landing-page) -> [Alice](people/alice) <- [Website launch](projects/launch) · type: task · status: draft · priority: 8 · tags: design, copy ``` To choose the columns and the order, add `--project` and `--sort`. `$title` is the document title, and `priority:-1` sorts in descending order: ```bash iwe find --filter 'type: task' --project 'title=$title,status,priority' --sort priority:-1 ``` ```text - [Write the landing page](tasks/landing-page) · status: draft · priority: 8 - [Set up analytics](tasks/analytics) · status: review · priority: 5 - [Publish the docs](tasks/docs) · status: done · priority: 3 ``` For scripts and agents, switch the output to JSON with `-f json`: ```bash iwe find --filter 'priority: { $gte: 5 }' --project 'key=$key,status,priority' -f json ``` ```json [ { "key": "tasks/analytics", "status": "review", "priority": 5 }, { "key": "tasks/landing-page", "status": "draft", "priority": 8 } ] ``` The [query language reference](/docs/concepts/query-language/) lists every operator, including `$ne`, `$nin`, `$exists`, `$or` and `$nor`. ## Step 5: query by relations Relations work like joins. `--included-by` follows inclusion links down from a document, and you can combine it with a frontmatter filter. These are the open tasks in the launch project: ```bash iwe find --included-by projects/launch --filter 'status: { $ne: done }' ``` ```text - [Set up analytics](tasks/analytics) <- [Website launch](projects/launch) · type: task · status: review · priority: 5 · tags: ops - [Write the landing page](tasks/landing-page) -> [Alice](people/alice) <- [Website launch](projects/launch) · type: task · status: draft · priority: 8 · tags: design, copy ``` `--references` follows inline links. These are the documents that mention Alice: ```bash iwe find --references people/alice ``` ```text - [Write the landing page](tasks/landing-page) -> [Alice](people/alice) <- [Website launch](projects/launch) · type: task · status: draft · priority: 8 · tags: design, copy ``` Add `:0` to an anchor, as in `--included-by projects/launch:0`, to walk the whole tree instead of direct children only. See [`iwe find`](/docs/agentic/find/) for all the flags. ## Step 6: count rows `iwe count` takes the same filters and prints a single number: ```bash iwe count iwe count --filter 'type: task, status: { $ne: done }' ``` ```text 5 2 ``` Because the output is just an integer, `iwe count` fits checks in scripts and CI. See [`iwe count`](/docs/agentic/count/). ## Step 7: update rows `iwe update --set` writes frontmatter fields on every document the filter matches, and `--unset` removes them. Preview first with `--dry-run`: ```bash iwe update --filter 'status: review' --set owner=alice --set due=2026-12-01 --dry-run ``` ```text Would update 1 document(s) ``` Then run it for real: ```bash iwe update --filter 'status: review' --set owner=alice --set due=2026-12-01 ``` ```text Updated 1 document(s) ``` The new columns are now in the file: ```markdown --- type: task status: review priority: 5 tags: - ops owner: alice due: 2026-12-01 --- # Set up analytics Pick a privacy-friendly tracker. ``` To protect against a filter that matches more than you meant, add `--expect` with the number of documents you expect to change. If the count is wrong, nothing is written: ```bash iwe update --filter 'type: task, priority: { $gte: 5 }' --set status=review --expect 1 ``` ```text error: update expects 1 document, matched 2 tasks/analytics › Set up analytics tasks/landing-page › Write the landing page hint: adjust the filter or raise expect ``` `update` and `delete` refuse to run without a filter or key, so you cannot change the whole folder by accident. See [`iwe update`](/docs/agentic/update/) for block edits and body replacement. ## Step 8: delete rows `iwe delete` removes the matched files and cleans up links to them. The dry run shows which documents would be deleted and which would be edited: ```bash iwe delete --filter 'status: done' --dry-run ``` ```text Would delete 'tasks/docs' Would update 1 document(s) projects/launch ``` ```bash iwe delete --filter 'status: done' --expect 1 ``` ```text Deleting 'tasks/docs' Updated 1 document(s) ``` The inclusion link in the project is gone, so no broken links remain: ```markdown --- type: project status: active --- # Website launch [Write the landing page](../tasks/landing-page) [Set up analytics](../tasks/analytics) ``` See [`iwe delete`](/docs/agentic/delete/) for how inline references are turned into plain text. ## Step 9: validate with a schema Frontmatter is schemaless until you add a schema. Create `.iwe/schemas/task.yaml`. The `frontmatter` part is standard JSON Schema: ```yaml frontmatter: type: object required: [status, priority] properties: status: { enum: [draft, review, done] } priority: { type: integer, minimum: 1, maximum: 10 } ``` Bind it to every document under `tasks/` by adding this to `.iwe/config.toml`, after the existing `[schemas]` line: ```toml [schemas.task] match = "tasks/**" ``` Run the validator. When every document passes, it prints nothing and exits with code 0: ```bash iwe schema validate ``` Now break a row on purpose and validate again: ```bash iwe update -k tasks/analytics --set status=blocked iwe schema validate ``` ```text Updated 1 document(s) tasks/analytics › frontmatter › status: "blocked" is not one of "draft", "review" or "done" ``` The validator exits with code 1, so you can run it in CI or have an agent fix what it reports. Add `-f json` for a machine-readable report. Schemas can also check sections, headers and size; see [Document schema](/docs/concepts/document-schema/). ## Summary You built a Markdown database without a database server: files as rows, keys as primary keys, frontmatter as columns, and links as relations. One filter language reads, counts, updates and deletes them, and a schema keeps the columns consistent. Your files stay plain Markdown, so any editor can open them and `git diff` shows exactly what each command changed. ## Next steps - [Query language](/docs/concepts/query-language/): every filter operator, graph operator and projection. - [`iwe find`](/docs/agentic/find/): search, filters, sorting and output formats. - [`iwe update`](/docs/agentic/update/): frontmatter and block edits with guards. - [Document schema](/docs/concepts/document-schema/): validate frontmatter and document structure. - [Keys](/docs/architecture/keys/): how file paths become document keys. - [Inclusion links](/docs/concepts/inclusion-links/): how a link on its own line builds the hierarchy. --- # Architecture URL: https://iwe.md/docs/architecture/ IWE's data model is built around a **graph-based representation** of markdown documents, where each structural element becomes a node in an interconnected graph. This design enables sophisticated document operations, cross-references, and transformations that go far beyond traditional markdown processing. --- # Data Model URL: https://iwe.md/docs/architecture/data-model/ ## Graph-Based Document Representation Unlike traditional parsers that work with document trees, IWE represents text as a **directed graph** where every **header**, **paragraph**, **list**, **list item**, **code block**, **table**, and **reference** becomes a **node**. Each node can have up to two primary relationships: - **next-element** (the `next` field in code): Points to the sibling node at the same hierarchical level - **child-element** (the `child` field in code): Points to the first child node (for container elements) This creates a hybrid tree-graph structure that preserves both document hierarchy and enables complex cross-document relationships. ![](documents-as-binary-tree.png) Structure and text content are stored separately: nodes capture the document hierarchy and relationships, while the text itself lives in a separate content store. This keeps graph operations fast and lets structural changes happen without copying text around. --- # Node Types and Structure URL: https://iwe.md/docs/architecture/note-types/ ## GraphNode Enumeration IWE defines 9 distinct node types, each optimized for specific markdown elements: ``` rust pub enum GraphNode { Empty, // Deleted/placeholder nodes Document(Document), // Root document container Section(Section), // Headers (h1-h6) Quote(Quote), // Blockquotes BulletList(BulletList), // Unordered lists OrderedList(OrderedList), // Numbered lists Leaf(Leaf), // Paragraphs and simple blocks Raw(RawLeaf), // Code blocks and raw content HorizontalRule(HorizontalRule), // Horizontal rules Reference(Reference), // Inclusion links to other documents Table(Table), // Markdown tables } ``` ## Node Relationships and Navigation Each node (except Document and Empty) contains: - **id**: Unique identifier within the graph - **prev**: Reference to previous sibling or parent - **next**: Optional reference to next sibling - **child**: Optional reference to first child (container nodes only) **Navigation patterns:** - **Siblings**: Follow `next` pointers horizontally - **Children**: Follow `child` pointer then `next` for all children - **Parent**: Use `prev` pointer and traverse up ## Content Storage Separation Text content is stored separately from structure in `Line` objects: ``` rust pub struct Line { id: LineId, inlines: GraphInlines, // Vector of inline elements (text, links, formatting) } ``` This separation enables: - **Structure reuse**: Multiple nodes can reference same content - **Efficient updates**: Content changes don't affect structure - **Memory optimization**: Structure and content cached independently --- # Document Processing Pipeline URL: https://iwe.md/docs/architecture/documents-processing/ ## 1. Markdown Parsing (DocumentBlock Creation) Raw markdown is first parsed into intermediate `DocumentBlock` representations: ``` rust pub enum DocumentBlock { Plain(Plain), // Plain text paragraphs Para(Para), // Regular paragraphs CodeBlock(CodeBlock), // Fenced code blocks Header(Header), // Headers with level and content BulletList(BulletList), // List containers Table(Table), // Table structures // ... additional block types } ``` ## 2. Graph Construction (DocumentBlock → GraphNode) The `SectionsBuilder` transforms `DocumentBlock` elements into graph nodes: ``` rust // High-level transformation process DocumentBlock::Header(header) → GraphNode::Section(section) DocumentBlock::Para(para) → GraphNode::Leaf(leaf) DocumentBlock::BulletList(list) → GraphNode::BulletList(bulletlist) ``` **Key transformations:** - **Headers become Sections**: With child relationships to content - **Lists become containers**: With children for each list item - **Paragraphs become Leaves**: Terminal nodes with text content - **Code blocks become Raw nodes**: With language and content metadata ## 3. Reference Resolution and Indexing After graph construction, the `RefIndex` system processes all references: ``` rust pub struct RefIndex { block_references: HashMap<Key, HashSet<NodeId>>, // [[note]] references inline_references: HashMap<Key, HashSet<NodeId>>, // [link](note) references } ``` --- # Keys and Cross-References URL: https://iwe.md/docs/architecture/keys/ ## Document Identification Each document is identified by a key — its path relative to the project root, without the `.md` extension (e.g. `folder/document`). **Key features:** - **Path-based**: Hierarchical organization support - **Extension handling**: Automatic `.md` extension management - **Relative linking**: Support for `../parent/document` syntax ## Named Keys vs Filters A key passed to a command either **names a document** or **filters a set**. - **Named keys** must exist. The command fails before doing anything and lists every missing key: `Document 'a' not found`, or `Documents not found: 'a', 'b'`. This covers `-k` / `keys` on `retrieve`, `tree`, `export`, and `update`, the `key` of single-document commands such as `delete`, `rename`, `extract`, `inline`, `squash`, `stats`, and `attach`, and the `review` / `refactor` MCP prompts. - **Filters** narrow a set and may match nothing, which returns an empty result. This covers `find` / `count` `-k`, `--filter` and `$key`, structural anchors (`--included-by`, `--references`, …), `--exclude`, and `-k` / `keys` combined with `--lexical` / `--fuzzy` / `search`, where they are the candidate set searched within. When named keys are combined with a filter, the keys are checked first and the filter then narrows them. ## Reference Types IWE supports three reference types: 1. **Regular markdown links**: `[text](document.md)` 2. **Wiki-style links**: `[[document]]` 3. **Piped wiki links**: `[[document|display text]]` Each reference type is preserved and can be normalized or converted as needed. ## Link Resolution Every link is resolved to a `Key` when a document is read. How the link text maps to a key depends on the reference type. ### Markdown links Markdown links resolve **relative to the document that contains them**. A link `[text](../shared/topic.md)` inside `projects/plan` resolves to the key `shared/topic`. The `.md` extension is dropped, and `..` segments walk up from the containing document's directory. A link that starts with `/` resolves **from the workspace root** instead, regardless of where the linking document lives: `[text](/shared/topic.md)` resolves to the key `shared/topic` whether it is written from the root or from a nested document. A `#section` fragment is dropped before the key is computed, so `[text](/shared/topic.md#usage)` resolves to the same key as `[text](/shared/topic.md)`. Set `refs_path = "absolute"` in `[markdown]` to have write-time normalization emit every markdown link in this root-absolute form. ### Wiki links Wiki links resolve **by path-suffix across the whole document set**, independent of where the link appears. The link text is matched against every document key, and a key matches when its path *ends with* the segments you wrote: - `[[topic]]` matches any document whose file name is `topic`, regardless of directory. - `[[shared/topic]]` matches a document whose path ends with `shared/topic`, e.g. `projects/shared/topic`. This means you can write the short form `[[topic]]` from anywhere and it finds the right document, without tracking relative paths. When more than one document matches the suffix, resolution is deterministic: 1. The key with the **fewest path segments** wins. 2. Ties break **lexicographically** by path. Given documents `area-one/note` and `area-two/note`, the link `[[note]]` is ambiguous on its own — to point at a specific one you write a longer suffix, `[[area-one/note]]` or `[[area-two/note]]`. The `.md` extension is optional in wiki links and is stripped if present. ## Write-Time Normalization When IWE writes a document, it normalizes every link so the stored form is canonical. This happens on formatting, normalization, rename, and any action that rewrites a document. ### Wiki link paths How the path inside a wiki link (`[[…]]`) is written on normalization is controlled by `wiki_link_path` under `[markdown]`. It takes one of three values: ``` toml [markdown] wiki_link_path = "preserve" # "preserve" | "full" | "short" ``` - `"preserve"` (default) — links are kept exactly as written. IWE does not rewrite the path; `[[topic]]` and `[[clippings/topic]]` are both left unchanged. - `"full"` — every link is rewritten to its target's full key path, so `[[topic]]` becomes `[[clippings/topic]]`. - `"short"` — every link is rewritten to the **shortest path suffix that still resolves to its target uniquely**. With `"short"`, IWE starts from the bare file name and grows the suffix only as far as needed to disambiguate: - A link to `clippings/topic` is written `[[topic]]` when no other document is named `topic`. - With both `area-one/note` and `area-two/note` present, links are written `[[area-one/note]]` and `[[area-two/note]]` — the bare name `note` would be ambiguous, so one extra segment is added. - Segments are added one at a time only as far as needed; the full path is the upper bound, used when no shorter suffix is unique. A shortened link only ever resolves back to the exact document it was shortened from — if a link's target is not one of the indexed documents, it is left at its full path rather than collapsed onto an unrelated document that happens to share the file name. Whichever value you choose, write-time normalization, completion, and the link actions all follow it. Resolution of existing links is unaffected — short, full, and as-typed links all resolve regardless of the setting. ### Markdown links and the reference extension By default markdown links are written as paths **relative** to the containing document — a link from `guide/intro` to `reference/api` is written `../reference/api`. Set `refs_path = "absolute"` in `[markdown]` to instead write every markdown link as a root-absolute path from the workspace root, so the same link becomes `/reference/api`. This only changes how links are *written*; resolution is unaffected, and a leading `/` is always resolved from the workspace root regardless of the setting. The two settings compose. `refs_extension` (for example `".md"`) appends an extension to written markdown links, so with `refs_path = "absolute"` and `refs_extension = ".md"` the link is written `/reference/api.md`. Wiki links never receive an extension. Fragment anchors (`#section`) are preserved. See [Configuration](/docs/configuration/) for `refs_extension`, `refs_path`, `wiki_link_path`, and the completion `link_format` option that controls which link style new links are created in. ## Caveats With `wiki_link_path = "short"`, the shortest form of a wiki link depends on the whole document set, so a few behaviors are worth knowing: - **Adding or removing a document can change the canonical form of links in other files.** Creating a second `note` makes the bare `[[note]]` ambiguous; existing `[[note]]` links elsewhere are only rewritten to the disambiguated form the next time those files are normalized. - **A bare link that has become ambiguous still resolves without error** — it picks the match with the fewest path segments, breaking ties lexicographically. Adding a document that sorts ahead of the previous winner can therefore change which document an un-normalized bare link points to. Re-normalizing the referencing files settles them on explicit suffixes. - **Round-tripping holds only for a fixed document set.** Within one snapshot, shortening and resolution are inverses; across edits to the corpus the shortest form can change. Keep the default `wiki_link_path = "preserve"` (or use `"full"`) if you prefer links that never change shape. --- # Graph Operations and Algorithms URL: https://iwe.md/docs/architecture/graph/ ## Tree Collection Converting graph sections to tree structures for processing: ``` rust // Collect a complete tree starting from a node let tree = graph_node_pointer.collect_tree(); // Tree provides hierarchical access to content for child in tree.children() { process_content(child); } ``` ## Squashing (Content Extraction) Extract content at limited depth with proper hierarchy flattening: ``` rust // Extract content up to depth 2 let squashed = graph.squash(&document_key, 2); // Headers are flattened: h1 → h2, h2 → h3, etc. // Content preserved with adjusted hierarchy ``` ## Path Generation Generate navigable paths through the document graph: ``` rust pub struct NodePath { ids: Vec<NodeId>, // Sequence of nodes forming path target: NodeId, // Final destination node } // Paths enable: // - Search result ranking // - Navigation breadcrumbs // - Content organization ``` --- # Advanced Features URL: https://iwe.md/docs/architecture/graph-operations/ ## Notes Embedding ![](notes-embedding.png) IWE can embed referenced documents while preserving structure: Embedding process: 1. Identify reference nodes 2. Load target document graph 3. Adjust header levels for context 4. Insert content maintaining hierarchy **Header level adjustment:** - Embedded under h2 → all headers +2 levels - h1 becomes h3, h2 becomes h4, etc. - Maintains document structure integrity ## Graph Transformations All document operations are implemented as graph transformations: - **Normalization**: Graph → normalized graph → markdown - **Reference resolution**: Update reference targets across graph - **Content extraction**: Subgraph extraction with proper boundaries - **Document merging**: Graph composition with conflict resolution ## Parallel Processing IWE leverages Rayon for parallel operations: ``` rust // Parallel document processing let documents: Vec<(Key, Document)> = content .par_iter() .map(|(k, v)| (Key::name(k), reader.document(v))) .collect(); // Parallel search path generation let search_paths = self.paths() .par_iter() .map(|path| generate_search_path(path)) .collect(); ``` This architecture enables IWE to handle large document collections efficiently while maintaining real-time responsiveness for editor integration. --- # Indexing and Reference Systems URL: https://iwe.md/docs/architecture/indexing/ ## Reference Index Structure The `RefIndex` maintains bidirectional reference mappings: ``` rust impl RefIndex { // Find all nodes that reference a specific document pub fn get_block_references_to(&self, key: &Key) -> Vec<NodeId> // Find all inline references (links) to a document pub fn get_inline_references_to(&self, key: &Key) -> Vec<NodeId> // Recursively index a node and all its children pub fn index_node(&mut self, graph: &Graph, node_id: NodeId) } ``` **Indexing process:** 1. **Graph traversal**: Depth-first traversal of all nodes 2. **Reference extraction**: Parse inline content for links 3. **Bidirectional mapping**: Build forward and reverse reference maps 4. **Incremental updates**: Re-index only changed portions ## Search Path Generation Search paths provide hierarchical navigation: ``` rust pub struct SearchPath { pub search_text: String, // Concatenated plain text for matching pub node_rank: usize, // Importance ranking pub key: Key, // Source document pub root: bool, // Is document root pub line: u32, // Line number in source pub path: NodePath, // Complete navigation path } ``` **Ranking algorithm:** - **Content depth**: Deeper content ranked lower - **Reference count**: More referenced content ranked higher - **Document position**: Earlier content ranked higher - **Search relevance**: Fuzzy match score integration --- # Data Flow Architecture URL: https://iwe.md/docs/architecture/data-flow/ ## CLI Operations CLI commands operate directly on the graph: ``` rust // Normalization: Rewrite all documents with consistent formatting fn normalize() { graph.export() → filesystem } // Export: Generate visualization formats (DOT, etc.) fn export() { graph → GraphData → DOTExporter } // Tree: Display document hierarchy fn tree() { graph.paths() → tree structure → markdown/keys/json } // Squash: Extract partial content at specified depth fn squash() { graph.squash(key, depth) → markdown } ``` ## LSP Server Integration The LSP server maintains a live `Database` wrapper around the graph: ``` rust pub struct Database { graph: Graph, // Core graph structure content: HashMap<Key, Content>, // Original markdown content paths: Vec<SearchPath>, // Pre-computed search paths } ``` **Real-time operations:** - **Document updates**: Incremental graph rebuilding - **Reference resolution**: Live link validation - **Search**: Fuzzy matching against pre-computed paths - **Completion**: Context-aware suggestions based on graph structure ## Memory and Performance Characteristics **Graph construction:** - **Parallel processing**: Rayon integration for multi-document parsing - **Incremental updates**: Only affected nodes rebuilt on changes - **Memory efficiency**: Arena pattern minimizes allocation overhead **Search performance:** - **Pre-computed paths**: Search index built once, queried repeatedly - **Fuzzy matching**: SkimMatcher for intelligent search ranking - **Parallel search**: Multi-threaded query processing --- # Design Principles URL: https://iwe.md/docs/architecture/design-principles/ IWE is text graph management assistant. Any text graph. The graph structure is not imposed. - Do not assume any particular graph structure or file naming convention. It is up to the user to decide on these details. - The goal is to minimize routine operations. Keeping the graph consistent should require the least possible amount of effort. The text formatting needs to be automated. - Focus on building blocks, not specific features. All simple operations as a "daily note" can be implemented at the editor level. IWE is merely a tool for navigating and changing text graphs. --- # Benchmark URL: https://iwe.md/docs/architecture/benchmark/ How we measure IWE's performance, what the harness looks like, and the numbers we have today. ## What we benchmark The `iwe` CLI rebuilds the document graph on every invocation, then runs a query. Each invocation pays both costs in sequence, so we measure them separately: 1. **Load** — walk the project directory, read every `.md` file, parse it, build the in-memory `Graph` (arena + indexes). Paid once per invocation. The LSP server (`iwes`) and MCP server (`iwec`) pay this cost only once at startup and reuse the graph for the lifetime of the session. The load bench measures this both without (`plain`) and with (`indexed`) the BM25 search index — see [Search Ranking](/docs/architecture/search-ranking/). 2. **Query** — evaluate filters / structural anchors / projections against an already-loaded `Graph`. Paid per query inside an LSP session, or amortized in CLI commands that fan out (find, count, tree, retrieve, stats). We don't benchmark `iwes` (the LSP server) end-to-end, the MCP server, or mutating subcommands (`update`, `delete`, `extract`, `inline`, etc.) — they write to disk and aren't in the hot read path. ## Harness We use [`criterion`](https://docs.rs/criterion) — the de facto Rust benchmarking framework. Two bench targets live under [`crates/iwe/benches/`](../crates/iwe/benches/): - `crates/iwe/benches/load.rs` — Stage 1, load only. - `crates/iwe/benches/query.rs` — Stage 2, query only. Both are wired in [`crates/iwe/Cargo.toml`](../crates/iwe/Cargo.toml) under `[[bench]] harness = false`. Shared test helpers (corpus generator, graph loader) live in `crates/iwe/benches/common.rs`. Per-bench config: `sample_size(10)` for both stages (criterion's minimum), `measurement_time(30s)` for load and `measurement_time(7.5s)` for query. Trades narrower confidence intervals for shorter wall-clock — full runs land at ~5 minutes per stage. ## Corpus Synthetic, deterministic, generated into a `TempDir` at bench setup. The generator (`crates/iwe/benches/common.rs::generate_corpus`) uses `StdRng::seed_from_u64(42)` so the corpus is byte-identical across runs. **Per doc** (`doc-NNNNN.md`): ```markdown --- title: "Doc 00001" type: post category: <one of alpha | beta | gamma | delta> status: <one of draft | published | scheduled> created: 2026-01-01 --- [Doc 00742](doc-00742) [Doc 00193](doc-00193) [Doc 00518](doc-00518) ## Section 1 Some paragraph text mentioning [Doc 00077](doc-00077) in passing. Lorem ipsum dolor sit amet. ... (3 paragraphs per section, 2 sections per doc) ``` | Element | Count | Purpose | |---|---|---| | Frontmatter fields | 5 | exercise `Filter::Field` queries | | Inclusion links (own line) | 3 per doc | inclusion edges for descendants/ancestors traversal | | Headers (`##`) | 2 per doc | section structure for parser | | Paragraphs | 6 per doc | parser load | | Inline links | 6 per doc | reference edges for `referenced_by` / `references` | Plus one `hub.md` with `n_docs / 10` inclusion links — anchor for `included_by_d1` and `included_by_unbounded` benches. **Sizes used**: `5_000`, `10_000`, `20_000` for both stages. Total file counts on disk (each ~700 bytes): 5_001 / 10_001 / 20_001. ## Running ```bash # Both stages cargo bench -p iwe # One stage at a time cargo bench -p iwe --bench load cargo bench -p iwe --bench query # Filter to specific benches (substring match) cargo bench -p iwe --bench load -- 'load/indexed/10000' cargo bench -p iwe --bench query -- 'query/filter/included_by_unbounded' cargo bench -p iwe -- '/20000$' # all 20k variants (plain + indexed) # Quick mode (~10 samples, no statistical inference — for iteration) cargo bench -p iwe -- --quick # Save / compare baselines cargo bench -p iwe -- --save-baseline before # ... make changes ... cargo bench -p iwe -- --baseline before ``` Criterion writes HTML reports + JSON to `target/criterion/`. Open `target/criterion/report/index.html` for the overview. ## Reading criterion output Each bench prints: ``` load/indexed/10000 time: [low_95% median high_95%] change: [low% median% high%] (p = 0.NN < 0.05) Performance has improved. ``` The three numbers are a **95% confidence interval** for per-iteration time, derived from criterion's bootstrap of the 10 samples it took. Tight intervals (low ≈ high, e.g. ±1%) mean the measurement is stable. Wide intervals (±20%+) mean noise — re-run the affected benches in isolation: ```bash cargo bench -p iwe --bench load -- '/20000$' ``` The `change:` line compares to the previous run criterion has saved to disk (or to a named baseline if `--baseline NAME` was passed). Outliers like `Found 2 outliers among 10 measurements (high severe)` flag samples that fell far outside the typical distribution — usually system noise (GC pause, disk hiccup). They don't invalidate the run but suggest re-running for a clean signal. ## Numbers All times are per-iteration on an Apple M3 Pro laptop. Stage 2 (query) was measured in May 2026; Stage 1 (load) was re-measured in July 2026 when the BM25 search index landed. ### Stage 1 — load The load bench runs two variants per corpus size: `plain` builds the graph without the search index; `indexed` builds it with BM25 full-text indexing enabled — what `iwe find` and the LSP / MCP servers use. | Bench | Time | | -------------------- | ------ | | `load/plain/5000` | 144 ms | | `load/indexed/5000` | 204 ms | | `load/plain/10000` | 329 ms | | `load/indexed/10000` | 435 ms | | `load/plain/20000` | 705 ms | | `load/indexed/20000` | 958 ms | **Search index overhead.** Building the BM25 index adds roughly a third to a cold load, scaling linearly at ~12 µs per document: | Corpus | plain | indexed | added | overhead | | ------ | ------ | ------- | ------- | -------- | | 5,000 | 144 ms | 204 ms | +60 ms | +42% | | 10,000 | 329 ms | 435 ms | +106 ms | +32% | | 20,000 | 705 ms | 958 ms | +253 ms | +36% | The cost is dominated by **tokenization and Snowball stemming**: the default tokenizer normalizes, lowercases, splits on unicode word boundaries, drops stop words, and stems every remaining token — and stemming is CPU-bound. Corpus extraction (walking each document to plain text) and embedding both run in parallel across documents above a 128-document threshold; only the final scorer population is serial, because it writes a shared inverted index. The CLI pays this once per `find` run; the LSP and MCP servers pay it once at startup, then only the per-edit cost of re-indexing a single changed document. See [Search Ranking](/docs/architecture/search-ranking/) for the indexing logic. ### Stage 2 — query | Bench | n=5k | n=10k | n=20k | | -------------------------------------- | -------- | -------- | -------- | | `query/filter/field_eq` | 2.7 ms | 5.7 ms | 12.1 ms | | `query/filter/compound_and` | 3.0 ms | 6.3 ms | 12.9 ms | | `query/filter/in_many_keys` | 1.4 ms | 2.9 ms | 5.9 ms | | `query/filter/included_by_d1` | 410 µs | 920 µs | 2.3 ms | | `query/filter/included_by_unbounded` | 9.2 ms | 21 ms | 48 ms | | `query/filter/referenced_by` | 121 µs | 251 µs | 499 µs | | `query/filter/roots` | 3.3 ms | 13.3 ms | 60 ms | | `query/op/find_full` | 1.8 ms | 4.7 ms | 15.0 ms | | `query/op/count` | 2.2 ms | 5.8 ms | 17.6 ms | | `query/op/retrieve_backlinks` | 1.7 ms | 3.5 ms | 7.4 ms | | `query/op/stats_all` | 5.5 ms | 13.0 ms | 30.4 ms | Notable scaling: - `query/filter/included_by_unbounded` is the slowest filter case — at n=20k the hub reaches ~94% of the corpus through 4-7 hops. - `query/filter/roots` is super-linear in n. The `--roots` implementation at `crates/iwe/src/main.rs:1260` calls `graph.get_inclusion_edges_to(k).is_empty()` for every key, and that lookup is linear in total inclusion edges. - `query/op/retrieve_backlinks` is super-linear for the same reason (inbound reference walk uses the same index). ## Iteration counts Criterion runs each bench `(measurement_time / per_iter)` times, organized into 10 samples for the bootstrap. Concrete iteration counts from the current load run: | Bench | per-iter | iters/sample | total iters | | -------------------- | -------- | ------------ | ----------- | | `load/plain/5000` | 144 ms | 22 | 220 | | `load/indexed/5000` | 204 ms | 16 | 165 | | `load/plain/10000` | 329 ms | 11 | 110 | | `load/indexed/10000` | 435 ms | 11 | 110 | | `load/plain/20000` | 705 ms | 5 | 55 | | `load/indexed/20000` | 958 ms | 5 | 55 | At n=20 k the bench runs ~55 iterations per measurement — solid signal. For tighter intervals at scale bump `sample_size(20)` and `measurement_time(60s)`, accepting roughly 2× wall-clock. ## Profiling For finding bottlenecks in the load path, `xctrace` (Apple's Instruments CLI) works without sudo and produces a trace viewable interactively in Instruments.app: ```bash # Build the bench binary with debug symbols CARGO_PROFILE_BENCH_DEBUG=true cargo bench -p iwe --bench load --no-run # Find the binary path (hash will differ) ls target/release/deps/load-* # Record a 20-second profile of one bench xctrace record --template "Time Profiler" \ --output /tmp/load.trace \ --launch -- target/release/deps/load-XXXXX \ --profile-time 20 'load/indexed/20000' # Open in Instruments open /tmp/load.trace ``` The bench binary's `--profile-time SECONDS BENCH_NAME` flag (provided by criterion) runs the iter closure in a tight loop without statistical analysis, so you get clean profiling samples. For flamegraph SVG output, `cargo flamegraph` works but needs `sudo` because `dtrace` is restricted by SIP on macOS: ```bash # cache sudo credentials, then: sudo cargo flamegraph --bench load -- --profile-time 20 'load/indexed/20000' ``` ## Out of scope - `iwes` (LSP server) — separate harness; not currently benched. - `iwec` MCP server — separate harness; not currently benched. - Mutating CLI subcommands — not on the hot read path. - Real-world corpus runs against `../marketing/` (102 files) — covered by the synthetic generator, which produces realistic frontmatter shapes. - `iwe` binary as a subprocess — would re-pay the load on every invocation, defeating the load/query split. --- # Search Ranking URL: https://iwe.md/docs/architecture/search-ranking/ How IWE ranks search results. This document covers the internals: what gets indexed, how relevance is scored, how the index stays in sync with edits, and how the search surfaces differ. For the user-facing overview see [Notes Search](/docs/features/search/). The fuzzy/lexical split and RRF fusion described here are implemented across the CLI, MCP, and LSP surfaces. ## Two rankers IWE ranks with **two independent rankers**: - **Fuzzy** — skim subsequence matching over a document's short `title + key` text. Tolerant of partial words and dropped characters (`auth` matches `Authentication`); it matches a subsequence of characters, not substitutions, so it does not correct a mistyped letter. Ranks by skim's match score. - **Lexical** — [BM25](https://en.wikipedia.org/wiki/Okapi_BM25) full-text relevance over `title + body`. Stemmed exact tokens, ranked by term frequency, rarity, and length normalization. When both are used for the same search, results are combined with **Reciprocal Rank Fusion (RRF)** — see [Fusion](#fusion-rrf). The BM25 index is built and maintained inside the document ingestion pipeline, so every surface reads the same, always-current relevance data. ## Surfaces | Surface | Query inputs | Default | | --- | --- | --- | | CLI [`iwe find`](/docs/agentic/find/) | positional (fuzzy) · `--fuzzy` · `--lexical` | fuzzy (positional is **deprecated** — use the flags) | | MCP `iwe_find` | `fuzzy` · `lexical` | none — the caller picks a ranker explicitly | | Query [`search`](/docs/concepts/query-language/#search-find-only) stage on `find` (CLI `iwe find`, MCP `iwe_query`) | `lexical` · `fuzzy` | none — a `search` clause names its rankers | | [`iwe retrieve`](/docs/agentic/retrieve/) seed query | `--lexical` · `--fuzzy` (`--limit` caps the seeds) | none — search runs only when a flag is given | | LSP workspace symbols | one query string | always fuzzy **and** lexical, fused | The query-language `search` stage and the `retrieve` seed query reuse this same ranking machinery — the `search` stage restricts membership and orders by relevance (see [Search](/docs/concepts/query-language/#search-find-only)), and `retrieve` runs `search` over its candidate set to pick seeds before expanding the graph around them. On the CLI, supplying both `--fuzzy` and `--lexical` fuses the two; the bare positional query stays fuzzy for now but prints a deprecation warning and will be removed. The MCP tool exposes only the explicit `fuzzy` / `lexical` parameters (an agent issues precise queries; there is no implicit default). The LSP always runs both rankers and fuses them. ## The BM25 index Ranking uses [BM25](https://en.wikipedia.org/wiki/Okapi_BM25), the standard full-text relevance function. It scores a document against a query from three signals: how often the query terms appear in the document (term frequency), how rare those terms are across the whole corpus (inverse document frequency), and the document's length relative to the average (so a term in a short note counts for more than the same term in a long one). The index is an in-memory sparse-vector store: each document becomes a sparse embedding keyed by token, plus an inverted index from token to the documents that contain it. A query is embedded the same way, and only documents sharing at least one token are scored — matching is sub-linear in corpus size. ### Corpus Each document is indexed as plain text, not markup: ``` {title} {body} ``` - **title** — the frontmatter title field if configured, otherwise the document's first `# ` header. - **body** — the document rendered to plain text: headings lose their `#`, emphasis/strong/etc. unwrap to their text, link **display text is kept but the URL is dropped**, code block **content** is kept (searchable), and table header and cell text is included. Frontmatter is dropped. Block-reference (inclusion) links are **not** expanded, so transcluded content is not double-counted. The document **key (path) is deliberately excluded** from the indexed text — ranking is by content, not filename. ### Tokenization The default tokenizer normalizes unicode, lowercases, splits on unicode word boundaries, removes stop words, and applies [Snowball](https://snowballstem.org/) stemming for the configured language. Stemming is why `deploying`, `deployed`, and `deployment` all match a search for `deploy` — they share the stem `deploy`. ## Where the index lives The index is a field on the `Graph`, built and kept current by the same three mutators that own all document content: - **build** — a full graph load (`from_state` / `from_path`) builds the index once, after titles are resolved. Corpus extraction and embedding run in parallel (above a 128-document threshold); the scorer is populated serially. - **insert / update** — re-indexes the single changed document. An update removes the document's previous vector before inserting the new one, so terms that were edited out do not linger. - **remove** — deletes the document's embedding and its inverted-index entries, so a deleted document can never appear in later results. Because these three mutators are the only choke points for content, every consumer (CLI, MCP tools, the MCP file watcher, LSP save/change/delete handlers) keeps the index in sync with no extra wiring. Indexing is opt-in per graph. A graph built without a search language carries no index and its mutators do nothing search-related, so commands that never search pay nothing. The CLI enables the index only for `find`; the long-running MCP and LSP servers always enable it. ## Fusion (RRF) When a search runs both rankers, their results are combined with **Reciprocal Rank Fusion**. Each ranker produces its own ordered list; a document's fused score sums a rank-based contribution from every list it appears in: ``` RRF(d) = Σ over each ranker where d appears: 1 / (k + rank_d) rank is 1-based, k = 60 ``` RRF fuses on **rank position only**, never on the raw scores. This is deliberate: skim's fuzzy scores (arbitrary positive integers) and BM25's tf·idf floats live on incomparable scales, so normalizing and averaging them is fragile. Rank-based fusion sidesteps that and is the standard method for hybrid lexical retrieval. The match set is the **union** — a document surfaced by either ranker appears; documents ranked high in *both* float to the top, so a doc strong in title-fuzzy *and* body-lexical outranks docs strong in only one. Ties break deterministically (by key on the CLI/MCP; by path length, page rank, key, line on the LSP). ## CLI and MCP ranking [`iwe find`](/docs/agentic/find/) and the MCP `iwe_find` tool share one code path. A query first narrows the corpus to candidates via the structural filter (`--filter`, `--includes`, `--references`, …). Then, within those candidates: - one ranker set (`--fuzzy` only, or `--lexical` only) ranks by that ranker; - both set (`--fuzzy` *and* `--lexical`) ranks by RRF fusion of the two. A text query is a **filter** here (unlike the LSP): only matching documents are returned — the union of the fuzzy and lexical matches. A no-query browse (filter only) falls back to the popularity ordering (inbound reference + inclusion edge count). With `--sort`, the query acts purely as a membership filter (the same union) and the survivors are ordered by the requested frontmatter field instead of by score. ## LSP fusion The LSP workspace-symbol index is **section-grain** (one entry per header, carrying the section's fuzzy-matchable path text), while the BM25 index is **document-grain**. The LSP always runs both rankers and fuses with RRF: 1. Rank sections by fuzzy score against each section's path text. 2. Rank sections by **their document's** BM25 score for the query. 3. Fuse the two rankings with RRF and sort; sections matching neither ranker fall to the tail (still returned, up to 100). An empty query keeps the page-rank ordering unchanged. So a section can rank on either signal — a body term lifts a document even when the header does not fuzzy-match, and fuzzy matching still surfaces near-miss header text that BM25's exact-token matching would miss. Link completion is unaffected — it stays alphabetical and client-filtered. ## Configuration Set the stemming language in `.iwe/config.toml`: ```toml [search] language = "english" ``` Default is `english`. Supported: `arabic`, `danish`, `dutch`, `english`, `french`, `german`, `greek`, `hungarian`, `italian`, `norwegian`, `portuguese`, `romanian`, `russian`, `spanish`, `swedish`, `tamil`, `turkish`. An unknown value falls back to `english`. See [Configuration](/docs/configuration/). ## Performance Building the index adds **roughly a third** (~30–40%) to a cold graph load and scales linearly at about **12 µs per document**. The cost is dominated by **tokenization and Snowball stemming** — stemming every token is CPU-bound. Corpus extraction and embedding run in parallel across documents (above a 128-document threshold); the final scorer population is serial because it writes a shared inverted index. The CLI pays this once per `find` invocation; the LSP and MCP servers pay it once at startup and then only the per-edit cost of re-indexing a single document. See [Benchmark](/docs/architecture/benchmark/#stage-1--load) for the numbers. ## Caveats - **Lexical is exact-token** — BM25 matches stemmed exact tokens, so `--lexical auth` will not match `authentication` (their stems differ). Reach for the fuzzy ranker (the CLI default, `--fuzzy`, or the LSP) when you want partial-word and subsequence tolerance; fuse both when you want relevance *and* tolerance. - **avgdl drift** — the average document length used for length normalization is fixed when the index is built. Over a very long editing session the incrementally-maintained index (LSP, MCP) drifts slightly from the true average, mildly degrading scores. The CLI rebuilds per run and never drifts. - **CJK** — languages without whitespace word boundaries are not tokenized by the default tokenizer; a custom (script-aware) tokenizer would be required. Tracked separately. --- # Configuration URL: https://iwe.md/docs/configuration/ IWE projects are configured through a `.iwe/config.toml` file in your project root. Below are all available configuration options. ## Basic Configuration ``` toml [markdown] refs_extension = "" refs_path = "relative" refs_text = "preserve" date_format = "%b %d, %Y" time_format = "%b %d, %Y %H:%M" locale = "de_DE" wiki_link_path = "preserve" [markdown.formatting] emphasis_token = "_" strong_token = "__" list_token = "-" ordered_list_token = "." code_block_token = "`" code_block_token_count = 3 increment_ordered_list_bullets = true ordered_list_content_indent = 4 bullet_list_content_indent = 4 rule_token = "-" rule_token_count = 72 [workspace] path = "" date_format = "%Y-%m-%d" time_format = "%Y-%m-%d %H:%M" locale = "en_US" frontmatter_document_title = "title" [completion] link_format = "markdown" min_prefix_length = 0 trigger_characters = ["["] ``` ### Markdown Settings - `refs_extension`: File extension for markdown references (default: empty, uses `.md`) - `refs_path`: How the path inside a regular markdown link (`[…](…)`) is written (default: `"relative"`). One of `"relative"` or `"absolute"`: `"relative"` writes each link relative to the linking document's directory, and `"absolute"` writes every link as a root-absolute path from the workspace root (`/dir/note.md`). Affects normalization, formatting, and link completion. Resolution is unaffected — a link with a leading `/` always resolves from the workspace root regardless of this setting, and a `#section` fragment is dropped before the target key is computed. See [Keys and Cross-References](/docs/architecture/keys/) for details. - `refs_text`: How the text of a regular markdown link (`[text](…)`) is written when documents are written (default: `"preserve"`). One of `"preserve"` or `"normalize"`: `"preserve"` keeps the link text exactly as typed, and `"normalize"` rewrites each link's text to the linked document's title (its frontmatter title when configured, otherwise its first header). Affects normalization and formatting; wiki links are unaffected. See [Keys and Cross-References](/docs/architecture/keys/) for details. - `date_format`: Date format for markdown content display and the `{{today}}` variable (default: `"%b %d, %Y"`, e.g., "Jan 15, 2024") - `time_format`: Format for the `{{now}}` variable in document content (default: falls back to `date_format`). Use this to include time components like `%H`, `%M`, `%S` in `{{now}}` while keeping `{{today}}` date-only. - `locale`: Locale for date formatting in document content (default: system locale). Allows different localization for content than for file keys. - `wiki_link_path`: How the path inside a wiki link (`[[…]]`) is written when documents are written (default: `"preserve"`). One of `"preserve"`, `"full"`, or `"short"`: `"preserve"` keeps each link exactly as typed, `"full"` rewrites every link to its full key path, and `"short"` rewrites every link to its shortest unambiguous path suffix. Affects normalization, completion, and link actions; resolution of existing links is unaffected. See [Keys and Cross-References](/docs/architecture/keys/) for details. ### Formatting Settings Control how IWE renders markdown when normalizing or formatting documents. All fields are optional and use sensible defaults. Invalid values fall back to defaults. ``` toml [markdown.formatting] emphasis_token = "_" strong_token = "__" list_token = "-" ``` - `emphasis_token`: Character for italic text (default: `"*"`). Options: `"*"` or `"_"` - `strong_token`: Characters for bold text (default: `"**"`). Options: `"**"` or `"__"` - `list_token`: Character for unordered list items (default: `"-"`). Options: `"-"`, `"*"`, or `"+"` - `ordered_list_token`: Character after numbers in ordered lists (default: `"."`). Options: `"."` (`1. item`) or `")"` (`1) item`) - `code_block_token`: Character for code block fences (default: `` "`" ``). Options: `` "`" `` or `"~"` - `code_block_token_count`: Minimum number of fence characters (default: `3`) - `increment_ordered_list_bullets`: Whether to increment ordered list numbers (default: `true`). When `true`, items use `1.`, `2.`, `3.`; when `false`, all items use `1.` - `ordered_list_content_indent`: Minimum column where ordered list item content and continuation lines start (default: unset, content aligns one space after the marker, e.g. `1. item` with 3-space continuation). Accepts `2`–`4`; values outside the range are ignored. Set to `4` for MkDocs-style alignment (`1. item` with 4-space continuation). The marker width is always respected, so wider markers (e.g. `10.`) use their natural width when it exceeds the configured value. - `bullet_list_content_indent`: Minimum column where unordered list item content and continuation lines start (default: unset, content aligns one space after the marker, e.g. `- item` with 2-space continuation). Accepts `2`–`4`; values outside the range are ignored. Set to `4` for MkDocs-style alignment (`- item` with 4-space continuation). - `rule_token`: Character for horizontal rules (default: `"-"`). Options: `"-"`, `"*"`, or `"_"` - `rule_token_count`: Number of characters in horizontal rules (default: `72`) - `wrap_column`: Wrap paragraphs at this column (default: unset, no wrapping). Minimum effective value is `20`; lower values are ignored. Wrapping splits at word boundaries; inline code, wiki links, math, and link/image URLs stay atomic; inline-link and image text wraps at spaces with the closing `](url)` glued to the last word. List and blockquote indents are subtracted from the effective width so wrapped lines inside `- ` items respect `wrap_column`. Tokens longer than the limit sit on their own line. Preserved breaks segment the wrap: with `preserve_line_breaks` or `preserve_newlines` on, each line between them is wrapped on its own rather than the paragraph being reflowed as one block. - `preserve_line_breaks`: Keep hard line breaks instead of dropping them (default: `false`). Recognizes two trailing spaces (` \n`) and backslash (`\\\n`) in the source and emits them in the configured `line_break_style` on output. - `line_break_style`: How preserved hard breaks are emitted (default: `"backslash"`). Options: `"backslash"` (`\\\n`, visible and survives whitespace-trimming editors), `"spaces"` (` \n`, invisible CommonMark default). Only takes effect when `preserve_line_breaks = true`. - `preserve_newlines`: Keep soft line breaks inside a paragraph instead of joining the lines (default: `false`). With this on, a paragraph written one sentence per line ([semantic line breaks](https://sembr.org/)) keeps its line layout through normalization instead of being reflowed onto a single line. Composes with `wrap_column`: a preserved line longer than the limit is wrapped on its own, and short lines are left alone instead of being joined. ### Workspace Settings - `path`: Subdirectory for markdown files relative to project root (default: empty, uses root) - `date_format`: Date format for file key generation and the `{{today}}` variable (default: `"%Y-%m-%d"`, e.g., "2024-01-15") - `time_format`: Format for the `{{now}}` variable in file key generation (default: falls back to `date_format`). Use this to include time components in keys, e.g., `"%Y-%m-%d-%H%M"` for sortable keys with time. - `locale`: Locale for date formatting and for sorting (default: auto-detected from system). Affects day and month names when using `%A`, `%B`, etc., and picks the language rules used to order titles and other text in query `sort`, the sort action and completion. - `frontmatter_document_title`: YAML frontmatter field to use as document title (default: none, uses first header) ### Completion Settings - `link_format`: Format for auto-completed links (default: `"markdown"`). Overridden by a typed `[` or `[[` prefix at the cursor. - `"markdown"`: Creates `[title](key)` style links - `"wiki"`: Creates `[[key]]` style WikiLinks - `min_prefix_length`: Minimum number of characters typed before completions appear (default: `0`). Measured against the search query after any leading `[` or `[[` is stripped. Raise to `3` (or higher) to suppress the popup until the user has typed a few characters. - `trigger_characters`: Characters that open the completion popup (default: `["["]`). Typing any listed character makes the editor request completions from the LSP server. Word characters trigger completion via editor heuristics regardless of this list. ### Date Format Patterns Date formats use [chrono format specifiers](https://docs.rs/chrono/latest/chrono/format/strftime/index.html): **Date specifiers:** - `%Y`: 4-digit year (2024) - `%y`: 2-digit year (24) - `%m`: Month as number (01-12) - `%b`: Abbreviated month name (Jan) - `%B`: Full month name (January) - `%d`: Day of month (01-31) - `%A`: Full weekday name (Monday) - `%a`: Abbreviated weekday name (Mon) **Time specifiers:** - `%H`: Hour in 24-hour format (00-23) - `%M`: Minute (00-59) - `%S`: Second (00-59) **Combined examples:** - `"%Y-%m-%d %H:%M"` → "2024-01-15 14:30" - `"%b %d, %Y %H:%M:%S"` → "Jan 15, 2024 14:30:45" - `"%Y%m%d%H%M"` → "202401151430" (useful for sortable file keys) Textual specifiers (`%A`, `%a`, `%B`, `%b`) are localized based on the `locale` setting. For example, with `locale = "de_DE"` and `date_format = "%A, %d. %B %Y"`, dates display as "Freitag, 27. März 2026". ### Locale Settings IWE supports separate locales for file keys and document content. By default, both use your system locale independently. - **`workspace.locale`**: Controls the language for file key generation (e.g., `journal/Friday-March-27`) - **`markdown.locale`**: Controls the language for document content (e.g., `# Freitag, 27. März 2026`) ``` toml [workspace] date_format = "%A-%B-%d" locale = "en_US" [markdown] date_format = "%A, %d. %B %Y" locale = "de_DE" ``` With this configuration: - File keys use English day/month names: `journal/Friday-March-27` - Document content uses German: `# Freitag, 27. März 2026` `workspace.locale` also selects the collation rules for sorting. Text is ordered the way the language expects (case and accents only break ties, digit runs compare by value so `Note 2` comes before `Note 10`, and emoji are skipped unless the remaining text is identical). Set it in a shared workspace so every machine sorts the same way; without it the system locale is used, and English is the fallback. The locale accepts both POSIX format (`de_DE`) and BCP47 format (`de-DE`). Encoding suffixes like `.UTF-8` are automatically stripped. ### Frontmatter Document Title By default, IWE uses the first header in a document as its title for links, autocomplete suggestions, and search results. You can override this behavior by specifying a YAML frontmatter field to use instead: ``` toml [workspace] frontmatter_document_title = "title" ``` With this configuration, a document like: ``` markdown --- title: My Custom Title --- # Header (ignored for title) Document content... ``` Will use "My Custom Title" as the document title instead of "Header (ignored for title)". This affects: - Link text in auto-completed links: `[My Custom Title](document-key)` - Link text normalization when references are updated - Document titles in search results and workspace symbols If the configured frontmatter field is missing or the document has no frontmatter, IWE falls back to using the first header as the title. ## Commands Define CLI commands for text transformation actions. Commands receive input via stdin and output transformed content to stdout: ``` toml [commands.claude] run = "claude -p" timeout_seconds = 120 [commands.uppercase] run = "tr '[:lower:]' '[:upper:]'" timeout_seconds = 5 [commands.custom_script] run = "/path/to/my-script.sh" timeout_seconds = 60 ``` Each command requires: - `run`: Command to execute (by default runs via `sh -c`) Optional parameters: - `args`: Array of arguments when using direct execution (only used when `shell = false`) - `cwd`: Working directory for command execution - `env`: Environment variables as key-value pairs (supports `$VAR` or `${VAR}` expansion from parent environment) - `shell`: Execute via shell (`true`, default) or directly (`false`) - `timeout_seconds`: Maximum execution time in seconds (default: 120) Commands are executed with the processed input template piped to stdin. The command's stdout becomes the replacement content. ### Example Commands **Using Claude CLI:** ``` toml [commands.claude] run = "claude -p" timeout_seconds = 120 ``` **Using a custom script:** ``` toml [commands.rewriter] run = "python ~/scripts/rewrite.py" timeout_seconds = 30 ``` **Simple text transformation:** ``` toml [commands.uppercase] run = "tr '[:lower:]' '[:upper:]'" timeout_seconds = 5 ``` **Direct execution with arguments (no shell):** ``` toml [commands.claude_direct] run = "claude" args = ["-p", "--model", "sonnet"] shell = false timeout_seconds = 120 ``` **With environment variables:** ``` toml [commands.custom_api] run = "my-api-tool" env = { API_KEY = "$MY_API_KEY", DEBUG = "true" } timeout_seconds = 60 ``` **With custom working directory:** ``` toml [commands.project_script] run = "./scripts/process.sh" cwd = "/path/to/project" timeout_seconds = 30 ``` ## Transform Actions Transform actions modify text content in-place using configured commands: ``` toml [actions.rewrite] type = "transform" title = "Rewrite" command = "claude" input_template = """ Here's a text that I'm going to ask you to edit. The text is marked with {{context_start}}{{context_end}} tag. The part you'll need to update is marked with {{update_start}}{{update_end}}. {{context_start}} {{context}} {{context_end}} Rewrite the given text to improve clarity and readability. """ ``` Transform action parameters: - `type`: Must be `"transform"` - `title`: Display name in editor - `command`: Reference to command configuration - `input_template`: Template for preparing stdin input ### Attach Actions Link content under cursor to another file, creating daily notes or collections: ``` toml [actions.today] type = "attach" title = "Add to Today" key_template = "{{today}}" document_template = "# {{today}}\n\n{{content}}\n" [actions.weekly_review] type = "attach" title = "Add to Weekly Review" key_template = "weekly-{{today}}" document_template = "# Weekly Review - {{today}}\n\n## Notes\n\n{{content}}\n\n## Action Items\n\n- [ ] \n" ``` Attach action parameters: - `type`: Must be `"attach"` - `title`: Display name in editor code actions - `key_template`: Template for target file key (supports `{{today}}` variable) - `document_template`: Template for new document content (supports `{{today}}` and `{{content}}` variables) ### Template Variables **Attach Actions** support: - `{{today}}`: Current date formatted using `workspace.date_format` (for keys) or `markdown.date_format` (for content). Intended for date-only formatting. - `{{now}}`: Current date/time formatted using `workspace.time_format` (for keys) or `markdown.time_format` (for content). Falls back to `date_format` if `time_format` is not set. Intended for date+time formatting. - `{{content}}`: The content being attached **Transform Actions** support: - `{{context}}`: Document context with the target block marked - `{{context_start}}`, `{{context_end}}`: Context delimiters - `{{update_start}}`, `{{update_end}}`: Update region delimiters ### Examples **Daily Note Creation** ``` toml [actions.daily] type = "attach" title = "Add to Daily Note" key_template = "daily/{{today}}" document_template = """# Daily Note - {{today}} ## Today's Focus {{content}} ## Tasks - [ ] ## Notes """ ``` **Project Collection** ``` toml [actions.project_ideas] type = "attach" title = "Add to Project Ideas" key_template = "projects/ideas" document_template = "# Project Ideas\n\n{{content}}\n" ``` **Text Transformation with Claude CLI** ``` toml [commands.claude] run = "claude -p" timeout_seconds = 120 [actions.expand] type = "transform" title = "Expand" command = "claude" input_template = """ Here's a text that I'm going to ask you to edit. The text is marked with {{context_start}}{{context_end}} tag. The part you'll need to update is marked with {{update_start}}{{update_end}}. {{context_start}} {{context}} {{context_end}} Expand the text you need to update, generate a couple paragraphs. """ ``` **Simple Text Transformation** ``` toml [commands.uppercase] run = "tr '[:lower:]' '[:upper:]'" timeout_seconds = 5 [actions.uppercase] type = "transform" title = "UPPERCASE" command = "uppercase" input_template = "{{context}}" ``` ## Schemas Bind [document schemas](/docs/concepts/document-schema/) to documents. Each entry under `[schemas]` names a schema file in `.iwe/schemas/` and a glob (or list of globs) that selects which document keys it applies to: ``` toml [schemas.person] match = "people/**" [schemas.session] match = ["journal/*", "meetings/**"] [schemas.note] match = ["notes/**", "!notes/index"] ``` - The entry name is the schema name: `[schemas.person]` resolves to `.iwe/schemas/person.yaml`. - `match` is required and accepts a single glob or a list of globs, matched against the document key (the relative path without the file extension). - Globs follow gitignore/globset syntax: `*` matches within a single path segment and stops at `/`, `**` crosses segments. A leading `/` is optional — patterns are always anchored at the workspace root. - A `!` prefix negates a pattern, gitignore-style: within one `match` list patterns apply in order and the last matching one decides, so `["notes/**", "!notes/index"]` binds everything under `notes/` except `notes/index`, and a later pattern can re-include what an earlier `!` removed. Escape a literal leading `!` as `\!`. - Bindings compose order-free: a document is validated against **every** schema whose `match` hits, so overlapping entries compose. A document that matches no entry is unvalidated. Run [`iwe schema validate`](/docs/cli/schema/) to check the store against these bindings. ## Migration from Version 2 If you're upgrading from a configuration using the old `[models]` section, IWE will automatically migrate your configuration to version 3. The migration: 1. Renames `[models]` section to `[commands]` with empty `run` values 2. Renames `model` field to `command` in transform actions 3. Renames `prompt_template` field to `input_template` in transform actions 4. Removes the `context` field from transform actions After migration, you'll need to manually update the `run` field in each command to specify the actual CLI command to execute. **Before (version 2):** ``` toml version = 2 [models.default] api_key_env = "OPENAI_API_KEY" base_url = "https://api.openai.com" name = "gpt-4o" [actions.rewrite] type = "transform" title = "Rewrite" model = "default" prompt_template = "..." context = "Document" ``` **After (version 3):** ``` toml version = 3 [commands.default] run = "claude -p" # Update this to your preferred CLI command timeout_seconds = 120 [actions.rewrite] type = "transform" title = "Rewrite" command = "default" input_template = "..." ``` --- # PKM Demo URL: https://iwe.md/docs/examples/pkm-demo/ A personal knowledge management graph showcasing IWE's capabilities for managing interconnected notes, tasks, and knowledge using markdown files with inclusion links. ## Overview This graph demonstrates how IWE organizes a personal knowledge base as a hierarchical graph. Each section is a separate document tree, connected through inclusion links that allow notes to appear in multiple contexts. The graph includes **200+ documents** covering common PKM workflows: - **Journal** — Daily entries organized by week and month - **Zettelkasten** — Atomic notes with unique IDs, linked by topic - **GTD** — Task management with inbox, projects, and contexts - **Writing** — Drafts and outlines with linked research notes - **Books** — Reading notes and highlights - **Quotes** — Collected quotes linked to source materials - **Timeline** — Chronological view across all sections - **Knowledge Base Structure** — Meta-documentation of the graph itself ## How it works The graph uses IWE's inclusion links to create a polyhierarchical structure. A note about a book can appear under both "Books" and "Zettelkasten" without duplication. The journal links to notes created that day, creating a timeline view alongside the topical organization. ## Links - [Browse the graph](https://iwe.pub/pkm-demo/) - [GitHub repository](https://github.com/iwe-org/pkm-demo) --- # Seventeen Centuries URL: https://iwe.md/docs/examples/seventeen-centuries/ A knowledge graph connecting three philosophical texts across seventeen centuries—from Marcus Aurelius (170 AD) to Machiavelli (1513) to Nietzsche (1886)—exploring how ideas of virtue, power, and morality evolved through time. ## Overview This project transforms philosophical texts into an interconnected knowledge graph using IWE. The graph is **hierarchical** with **polyhierarchy support**—documents can belong to multiple parent documents simultaneously. A concept like "virtue" can appear under both a book index and a thematic category, reflecting how knowledge naturally connects across domains. The graph contains **1,200+ documents** including: - **838 text fragments** from three books - **340+ concept files** linking philosophical ideas, historical figures, and themes - **Category documents** organizing concepts into navigable groupings ## Source texts All source texts are public domain editions from [Standard Ebooks](https://standardebooks.org/): | Book | Author | Fragments | |------|--------|-----------| | [Beyond Good and Evil](https://standardebooks.org/ebooks/friedrich-nietzsche/beyond-good-and-evil/helen-zimmern) | Friedrich Nietzsche | 296 | | [Meditations](https://standardebooks.org/ebooks/marcus-aurelius/meditations/george-long) | Marcus Aurelius | 515 | | [The Prince](https://standardebooks.org/ebooks/niccolo-machiavelli/the-prince/w-k-marriott) | Niccolò Machiavelli | 27 | ## Highlights Some concept files reveal how the same idea carries different meanings across philosophers. **[Virtue](https://iwe.pub/seventeen-centuries/virtue/)** — For Marcus Aurelius, virtue is acting according to nature and reason, serving the common good as naturally as the eye sees. Machiavelli inverts this: a prince who acts entirely virtuously will be ruined among so much evil. Nietzsche warns against becoming enslaved to one's own virtues, noting that every virtue inclines toward stupidity. ## Links - [Browse the graph](https://iwe.pub/seventeen-centuries/) - [GitHub repository](https://github.com/iwe-org/seventeen-centuries) --- # How to Use in Command Line URL: https://iwe.md/docs/cli/ IWE provides a powerful command-line interface for managing markdown-based knowledge graphs. The CLI enables you to initialize projects, normalize documents, explore connections, export visualizations, and create consolidated documents. ## Quick Start 1. **Initialize a project**: `iwe init` 2. **Create a new document**: `iwe create --template default --var title="My Note"` 3. **Retrieve a document with context**: `iwe retrieve -k my-note` 4. **Find and search documents**: `iwe find "search term"` 5. **Count matching documents**: `iwe count --filter 'status: draft'` 6. **Normalize all documents**: `iwe normalize` 7. **View document hierarchy**: `iwe tree` 8. **Analyze your knowledge base**: `iwe stats` 9. **Export graph visualization**: `iwe export -f dot` 10. **Rename a document**: `iwe rename old-key new-key` 11. **Delete a document**: `iwe delete document-key` 12. **Bulk delete by filter**: `iwe delete --filter 'status: archived'` 13. **Extract a section**: `iwe extract document --section "Title"` 14. **Inline a reference**: `iwe inline document --reference "other-doc"` 15. **Overwrite a document body**: `iwe update -k document-key -c "new content"` 16. **Mutate frontmatter**: `iwe update --filter 'status: draft' --set reviewed=true` 17. **Attach via configured action**: `iwe attach --to today -k document-key` ## Installation & Setup Before using the CLI, ensure IWE is installed and available in your PATH. Initialize any directory as an IWE project: ``` bash cd your-notes-directory iwe init ``` This creates a `.iwe/` directory with configuration files. ## Global Usage ``` bash iwe [OPTIONS] <COMMAND> ``` ### Global Options - `-V`, `--version`: Display version information - `-v`, `--verbose <LEVEL>`: Set verbosity level (default: 0) - `1`: Minimal output (INFO level messages to stderr) - `2` or higher: Debug-level information to stderr - `-h`, `--help`: Show help information ## Configuration Commands respect settings in `.iwe/config.toml`: ``` toml [workspace] path = "" # Subdirectory containing markdown files [markdown] refs_extension = "" refs_path = "relative" date_format = "%b %d, %Y" ``` See [Configuration](/docs/configuration/) for all available options. ## Querying `find`, `count`, `update`, and `delete` accept the same YAML-based filter language. Read-only commands (`retrieve`, `tree`, `export`) accept the filter flags as a selector to narrow what they operate on. The two entry points: - `--filter "EXPR"` — inline YAML filter document. - Structural anchor flags — `-k`, `--includes`, `--included-by`, `--references`, `--referenced-by` (all repeatable, ANDed at the top level), with optional `KEY[:DEPTH]` / `KEY[:DIST]` colon-suffix. See the [Query Language](/docs/concepts/query-language/) reference for the YAML syntax, operators, and examples. ## Command Categories ### Document Management | Command | Description | Documentation | | ----------- | ------------------------------------------------------------ | --------------------------------- | | `init` | Initialize a new IWE project | [IWE Init](/docs/cli/init/) | | `create` | Create a document from complete content, or from a template | [IWE Create](/docs/cli/create/) | | `new` | Create a new document from a title | [IWE New](/docs/cli/new/) | | `update` | Overwrite a document body, or mutate frontmatter via filter | [IWE Update](/docs/agentic/update/) | | `normalize` | Normalize all documents | [IWE Normalize](/docs/cli/normalize/) | ### Document Retrieval | Command | Description | Documentation | | ---------- | ---------------------------------------- | ------------------------------- | | `retrieve` | Retrieve document with context | [IWE Retrieve](/docs/agentic/retrieve/) | | `find` | Search and discover documents | [IWE Find](/docs/agentic/find/) | | `count` | Count documents matching a filter | [IWE Count](/docs/agentic/count/) | | `tree` | Display document hierarchy | [IWE Tree](/docs/agentic/tree/) | ### Refactoring Operations | Command | Description | Documentation | | --------- | ----------------------------------------- | ----------------------------- | | `rename` | Rename a document and update references | [IWE Rename](/docs/agentic/rename/) | | `delete` | Delete a document and clean up references | [IWE Delete](/docs/agentic/delete/) | | `extract` | Extract a section to a new document | [IWE Extract](/docs/agentic/extract/) | | `inline` | Inline a referenced document | [IWE Inline](/docs/agentic/inline/) | ### Analysis & Export | Command | Description | Documentation | | -------- | --------------------------------- | --------------------------- | | `schema` | Infer and display frontmatter schema | [IWE Schema](/docs/cli/schema/) | | `stats` | Analyze knowledge base statistics | [IWE Stats](/docs/agentic/stats/) | | `export` | Export graph visualization | [IWE Export](/docs/cli/export/) | | `squash` | Squash documents | [IWE Squash](/docs/cli/squash/) | ## Exit Codes | Code | Meaning | | ---- | ---------------------------------------------------------- | | `0` | Success - command completed without errors | | `1` | Error - invalid arguments, missing files, operation failed | All commands return exit code `0` on success. On error, commands print a message to stderr and return exit code `1`. ``` bash # Check exit code iwe find "nonexistent-query" echo $? # Returns 0 (empty result is not an error) iwe retrieve --key "missing-doc" echo $? # Returns 1 (document not found is an error) ``` --- # IWE Init URL: https://iwe.md/docs/cli/init/ Initializes the current directory as an IWE project. ## Usage ``` bash iwe init ``` ## What It Creates Running `iwe init` creates: ``` .iwe/ └── config.toml # Project configuration ``` The `.iwe/` directory marks the project root and stores configuration files. ## Default Configuration The generated `config.toml` contains: ``` toml version = 3 [markdown] refs_extension = "" refs_path = "relative" date_format = "%b %d, %Y" wiki_link_path = "preserve" [workspace] path = "" date_format = "%Y-%m-%d" [completion] [commands] [commands.default] run = "claude -p" timeout_seconds = 120 [actions] [actions.extract] type = "extract" title = "Extract" link_type = "markdown" key_template = "{{id}}" [actions.inline_section] type = "inline" title = "Inline section" inline_type = "section" keep_target = false [actions.sort] type = "sort" title = "Sort A-Z" reverse = false [templates] [templates.default] key_template = "{{slug}}" document_template = "# {{title}}\n\n{{content}}" ``` ## Configuration Options ### Workspace Section | Option | Default | Description | | ------------------ | ---------- | ------------------------------------------------------------- | | `path` | `""` | Subdirectory containing markdown files (empty = project root) | | `date_format` | `%Y-%m-%d` | Format for dates in document keys | | `default_template` | `null` | Template name `iwe new` uses when `-t` is not given | ### Markdown Section | Option | Default | Description | | -------------------- | ----------- | ----------------------------------------------------------------- | | `refs_extension` | `""` | File extension to append to references (e.g., `.md`) | | `refs_path` | `relative` | How markdown link paths are written: `relative` or `absolute` | | `date_format` | `%b %d, %Y` | Format for dates displayed in documents | | `wiki_link_path` | `preserve` | How wiki link paths are written: `preserve`, `full`, or `short` | ### Completion Section | Option | Default | Description | | -------------------- | ------- | --------------------------------------------------------------------------------- | | `link_format` | `null` | Link style for completions: `markdown` or `wiki` (overridden by a typed `[`/`[[`) | | `min_prefix_length` | `0` | Characters required in the search query before completions appear | | `trigger_characters` | `["["]` | Characters that open the completion popup in the editor | ## Example ``` bash cd ~/my-notes iwe init # Output: Creates .iwe/config.toml # Verify initialization ls -la .iwe/ # Output: config.toml ``` ## Customization After initialization, edit `.iwe/config.toml` to: - Store markdown files in a subdirectory: set `workspace.path = "docs"` - Use wiki-style links: set `completion.link_format = "wiki"` - Change completion trigger characters: set `completion.trigger_characters = ["[", "+"]` - Define custom templates for document creation - Configure AI-powered actions with custom commands ## Re-initialization Running `iwe init` in an already-initialized project is safe - it will not overwrite an existing `config.toml`. --- # IWE New URL: https://iwe.md/docs/cli/new/ Creates a new document from a title — the quick-capture form of document creation. The title is the positional argument, `-c` or piped stdin fills the template's content slot, and `workspace.default_template` picks the template when `-t` is not given. For composing from named templates with typed variables, or for writing a complete document verbatim, see [`iwe create`](/docs/cli/create/). The commands overlap but differ: `iwe create` requires an explicit template name (it never reads `workspace.default_template`), takes no positional title, and its template mode never reads stdin. | `iwe new` | `iwe create` equivalent | | ---------------------------------- | --------------------------------------------------- | | `iwe new "My Note"` | `iwe create --template default --var title="My Note"` | | `iwe new "Sync" -t meeting` | `iwe create -t meeting --var title="Sync"` | | `iwe new "Ada" --key people/ada` | `iwe create people/ada -t default --var title="Ada"` | | `iwe new "Note" -c "body text"` | `iwe create -t default --var title="Note" --var body="body text"` | ## Usage ``` bash iwe new <TITLE> [OPTIONS] ``` ## Arguments - `<TITLE>`: Title for the new document (required) ## Options - `-t, --template <NAME>`: Template name from config (default: `workspace.default_template`, or "default" when unset) - `-c, --content <CONTENT>`: Initial content for the document - `-k, --key <KEY>`: Explicit document key, bypassing the template's key derivation. Subdirectory keys are allowed (e.g. `people/ada`). Omit the file extension. - `-i, --if-exists <MODE>`: Behavior when file already exists (default: "suffix", or "fail" when `--key` is given) - `suffix`: Append `-1`, `-2`, etc. to filename until unique - `override`: Overwrite existing file - `skip`: Do nothing, exit successfully without output - `fail`: Report an error and exit with a non-zero status - `-e, --edit`: Open created file in `$EDITOR` after creation Frontmatter is not available on `iwe new` — use [`iwe create`](/docs/cli/create/)'s `--set`. ## Explicit keys By default the filename is derived from the title (slugified) through the template's `key_template`. Pass `--key` to set the document key yourself and skip that derivation — the title still fills the document body. Use this when the key is a stable identifier drawn from metadata (an entity name, a session date) rather than the title wording. Because an explicit key asserts an identity, `--key` defaults `--if-exists` to `fail`: creating a document whose key already exists reports an error instead of silently appending a `-1` suffix. Pass `--if-exists skip` or `--if-exists override` to opt into idempotent or forced re-creation. ## What it does - Creates a new markdown file using the specified template - Generates filename from the title (slugified) - Supports content from command-line argument or stdin pipe - Prints the absolute path of the created file to stdout - Optionally opens the file in your configured editor ## Template Variables Templates support the following variables: - `{{title}}`: The provided title argument - `{{slug}}`: Slugified title (kebab-case) - `{{today}}`: Current date (uses `workspace.date_format` for key, `markdown.date_format` for content). Intended for date-only formatting. - `{{now}}`: Current date/time (uses `workspace.time_format` for key, `markdown.time_format` for content). Falls back to `date_format` if `time_format` is not set. Supports both date specifiers (`%Y`, `%m`, `%d`) and time specifiers (`%H`, `%M`, `%S`). - `{{id}}`: Random 8-character alphanumeric ID - `{{content}}`: Content from `-c` option or stdin ## Examples ``` bash # Create a new note with default template iwe new "My New Note" # Creates: my-new-note.md (or my-new-note-1.md if exists) # Create with content iwe new "Meeting Notes" --content "Discussed project timeline" # Pipe content from clipboard (macOS) pbpaste | iwe new "Clipboard Note" # Create and open in editor iwe new "Quick Idea" --edit # Use a custom template iwe new "Daily Journal" --template journal # Overwrite existing file iwe new "My Note" --if-exists override # Skip if file exists (useful in scripts) iwe new "My Note" --if-exists skip # Create at an explicit key (fails if people/ada already exists) iwe new "Ada Lovelace" --key people/ada # Idempotent create at an explicit key iwe new "Ada Lovelace" --key people/ada --if-exists skip ``` ## Configuration Templates are defined in `.iwe/config.toml`: ``` toml [workspace] default_template = "default" # Optional: set default template [templates.default] key_template = "{{slug}}" document_template = "# {{title}}\n\n{{content}}" [templates.journal] key_template = "journal/{{today}}" document_template = "# {{today}}\n\n{{content}}" ``` --- # IWE Normalize URL: https://iwe.md/docs/cli/normalize/ Performs comprehensive document normalization across all markdown files in your knowledge base. ## Usage ``` bash iwe normalize ``` ## Operations Performed | Operation | Description | | ------------------------ | ----------------------------------------------------------------- | | Link title sync | Rewrites link text to the target document's title (opt-in, off by default; enable with `refs_text = "normalize"`) | | Link path rewriting | Writes each link path per the `refs_path` setting | | Header leveling | Adjusts header levels for consistent hierarchy | | List renumbering | Fixes ordered list numbering (1, 2, 3...) | | Whitespace normalization | Standardizes newlines and indentation | | List formatting | Ensures consistent list item formatting | | Structure cleanup | Removes redundant empty lines | ## Before/After Examples ### Link Title Sync By default link text is preserved exactly as written. Set `refs_text = "normalize"` in `[markdown]` to rewrite each link's text to the linked document's title. Before: ``` markdown See the [old title](project-docs) for details. ``` After, with `refs_text = "normalize"` (and `project-docs.md` has header `# Project Documentation`): ``` markdown See the [Project Documentation](project-docs) for details. ``` ### Link Path Rewriting Every markdown link is rewritten according to the `refs_path` setting (see below). With `refs_path = "absolute"`, a link written relative to the current document is rewritten as a root-absolute path from the workspace root. Before (in `guide/intro.md`, with `refs_path = "absolute"`): ``` markdown See the [API](../reference/api) for details. ``` After: ``` markdown See the [API](/reference/api) for details. ``` Regardless of the setting, a link that already starts with `/` is resolved from the workspace root, and any `#section` fragment is preserved. ### List Renumbering Before: ``` markdown 1. First item 1. Second item 5. Third item ``` After: ``` markdown 1. First item 2. Second item 3. Third item ``` ### Whitespace Normalization Before: ``` markdown # Title Some paragraph. Another paragraph. ``` After: ``` markdown # Title Some paragraph. Another paragraph. ``` ## Examples ``` bash # Basic normalization iwe normalize # With INFO level logging iwe -v 1 normalize # With DEBUG level logging iwe -v 2 normalize ``` ## Safety Warning **Important:** The normalize command modifies files in place. Always ensure you have a backup or version control before running: ``` bash # Recommended: commit changes before normalizing git add -A && git commit -m "Before normalization" # Run normalization iwe normalize # Review changes git diff ``` ## Configuration Normalization behavior is controlled by `.iwe/config.toml`: ``` toml [markdown] refs_extension = "" # Extension for reference links refs_path = "relative" # Link path form: "relative" or "absolute" refs_text = "preserve" # Link text: "preserve" or "normalize" ``` ## Idempotency Running `normalize` multiple times produces the same result - once files are normalized, subsequent runs make no changes. --- # IWE Paths URL: https://iwe.md/docs/cli/paths/ Displays all possible navigation paths in your document graph. ## Usage ``` bash iwe paths [OPTIONS] ``` ## Options - `-d, --depth <DEPTH>`: Maximum path depth to explore (default: 4) - `-v, --verbose <LEVEL>`: Verbosity level ## Output format Shows hierarchical paths through your documents, revealing connection patterns and document relationships. ## Examples ``` bash # Show paths up to depth 4 iwe paths # Show deeper paths iwe paths --depth 6 # With debug output iwe paths -v 2 --depth 3 ``` --- # IWE Contents URL: https://iwe.md/docs/cli/contents/ Lists root documents (notes without parent references). ## Usage ``` bash iwe contents ``` ## Purpose Identifies entry points in your knowledge graph - documents that aren't referenced by others, potentially serving as main topics or starting points. ## Example ``` bash iwe contents ``` --- # IWE Squash URL: https://iwe.md/docs/cli/squash/ Creates consolidated documents by combining linked content into a single markdown file. ## Usage ``` bash iwe squash <KEY> [OPTIONS] ``` ## Arguments | Argument | Description | | -------- | ---------------------- | | `<KEY>` | Document key to squash | ## Options | Option | Default | Description | | ----------------------- | ------- | -------------------------- | | `-d, --depth <DEPTH>` | `2` | How deep to traverse links | | `-v, --verbose <LEVEL>` | `0` | Verbosity level | ## What It Does 1. Starts from the specified document 2. Traverses [Inclusion Links](/docs/concepts/inclusion-links/) up to specified depth 3. Combines content into a single markdown document 4. Converts linked sections to inline sections 5. Adjusts header levels to maintain hierarchy ## Output Format Given this document structure: `project-overview.md`: ``` markdown # Project Overview Introduction to the project. - [Goals](goals) - [Architecture](/docs/architecture/) ``` `goals.md`: ``` markdown # Goals Our main objectives are: - Improve performance - Add new features ``` Running `iwe squash project-overview` outputs: ``` markdown # Project Overview Introduction to the project. ## Goals Our main objectives are: - Improve performance - Add new features ## Architecture ... ``` Note that linked document headers become sub-headers (# → ##) to preserve hierarchy. ## Examples ``` bash # Squash starting from document "project-overview" iwe squash project-overview # Squash with greater depth iwe squash main-topic --depth 4 # Save output to file iwe squash research-notes --depth 3 > output.md # With debug output iwe squash research-notes --depth 3 -v 2 ``` ## Header Level Adjustment When documents are inlined, their header levels are adjusted: | Original Level | Becomes | Reason | | -------------- | ----------- | ------------------------------- | | `#` (h1) | `##` (h2) | Linked document becomes section | | `##` (h2) | `###` (h3) | Preserves relative hierarchy | | `###` (h3) | `####` (h4) | And so on... | This ensures the squashed document maintains a logical structure with the root document as the top-level heading. ## Use Cases - **Export to PDF**: Create a single document for typesetting with tools like Typst - **Sharing**: Generate a standalone file from interconnected notes - **Backup**: Consolidate project knowledge into a portable format - **Context building**: Create comprehensive documents for review ## AI Agent Tips - Use `squash` to build comprehensive context from related documents - Combine with `retrieve` for maximum context: squash provides structure, retrieve provides backlinks - Adjust depth based on scope: shallow (2) for focused topics, deep (4+) for comprehensive overviews - Squashed output is ideal for sending to language models as context Example [PDF](https://github.com/iwe-org/iwe/blob/master/docs/book.pdf) generated using `squash` command and Typst. --- # IWE Export URL: https://iwe.md/docs/cli/export/ Exports graph structure in various formats for visualization and analysis. ## Usage ``` bash iwe export -f <FORMAT> [OPTIONS] ``` ## Available Formats | Format | Description | | ------ | ------------------------------------------- | | `dot` | Graphviz DOT format for graph visualization | ## Options | Option | Default | Description | | ------------------------------- | --------- | ------------------------------------------------------------------------------------ | | `-f, --format <FORMAT>` | `dot` | Output format. Currently `dot` only. | | `-d, --depth <DEPTH>` | `0` | Maximum depth to include (0 = unlimited). | | `--include-headers` | false | Include section headers and create detailed subgraphs. | | `--filter <EXPR>` | - | Inline YAML filter expression. See [Query Language](/docs/concepts/query-language/). | | `-k, --key <KEY>` | all roots | Filter to specific document(s). Repeatable; 1 key = `$eq`, 2+ = `$in`. Unknown keys are an error. | | `--includes <KEY[:DEPTH]>` | - | `$includes` anchor. Repeatable; anchors are ANDed. | | `--included-by <KEY[:DEPTH]>` | - | `$includedBy` anchor. Repeatable; anchors are ANDed. | | `--references <KEY[:DIST]>` | - | `$references` anchor. Repeatable; anchors are ANDed. | | `--referenced-by <KEY[:DIST]>` | - | `$referencedBy` anchor. Repeatable; anchors are ANDed. | | `--max-depth <N>` | `1` | Session default for inclusion anchor flags without a colon-suffix. `0` = unbounded. | | `--max-distance <N>` | `1` | Session default for reference anchor flags without a colon-suffix. `0` = unbounded. | > **Breaking change:** in earlier versions, `iwe export <FORMAT>` accepted the format as a positional argument. The format is now the `-f / --format` flag with a default of `dot`. The previous `iwe export dot` becomes `iwe export -f dot` (or simply `iwe export`). ## DOT Output Format The DOT format produces Graphviz-compatible output: ``` dot digraph G { rankdir="LR" fontname="Verdana" fontsize="13" nodesep="0.7" splines="polyline" pad="0.5,0.2" ranksep="1.2" overlap="false" 0 [label="Project Overview" fillcolor="#e8f0e8" fontsize="24" fontname="Verdana" color="#b3b3b3" penwidth="1.5" shape="note" style="filled"] 1 [label="Goals" fillcolor="#e8f0e8" fontsize="16" fontname="Verdana" color="#b3b3b3" penwidth="1.5" shape="note" style="filled"] 2 [label="Architecture" fillcolor="#e8f0e8" fontsize="16" fontname="Verdana" color="#b3b3b3" penwidth="1.5" shape="note" style="filled"] 0 -> 1 [color="#38546c66" arrowhead="normal" penwidth="1.2"] 0 -> 2 [color="#38546c66" arrowhead="normal" penwidth="1.2"] } ``` Nodes represent documents, edges represent links between them. ## Examples ``` bash # Export entire graph iwe export # Export specific document and connections iwe export --key project-main # Include section headers for detailed view iwe export --include-headers # Export with depth limit iwe export --key research --depth 3 # Export with headers and depth limit iwe export --key research --depth 3 --include-headers # Restrict to documents under a hub (unbounded) iwe export --included-by projects/alpha:0 # Restrict by frontmatter iwe export --filter 'status: active' ``` ## Generating Images ``` bash # Generate PNG visualization iwe export > graph.dot dot -Tpng graph.dot -o graph.png # Generate SVG for web use iwe export --include-headers > detailed.dot dot -Tsvg detailed.dot -o detailed.svg # Direct to PNG (one-liner) iwe export | dot -Tpng -o graph.png # Interactive visualization in browser iwe export | dot -Tsvg > graph.svg && open graph.svg ``` ## Depth Behavior | Depth | Behavior | | ----- | ------------------------------------------- | | `0` | Unlimited - include all reachable documents | | `1` | Only the specified document | | `2` | Document and its direct links | | `3+` | Document and N-1 levels of connections | ## With vs Without Headers | Mode | Use Case | | --------------------------- | --------------------------------------- | | Without `--include-headers` | High-level document relationships | | With `--include-headers` | Detailed view showing internal sections | ## AI Agent Tips - Use `export` to analyze document relationship topology - Generate visualizations to identify disconnected clusters - Use `--depth` to focus on specific neighborhoods in large graphs - Combine with `--key`, `--included-by`, or `--filter` to visualize a single topic and its context - The graph structure reveals how knowledge is organized and connected ## Deprecated aliases The following flags pre-date the query language and remain accepted for backward compatibility. Each invocation prints a one-line `warning: ... is deprecated` to stderr. | Deprecated | Use instead | | ------------------ | --------------------------------------------------------------------------- | | `--in KEY[:N]` | `--included-by KEY[:N]` | | `--in-any K1 K2` | `--filter '$or: [{ $includedBy: K1 }, { $includedBy: K2 }]'` | | `--not-in KEY` | `--filter '$nor: [{ $includedBy: KEY }]'` | | `--refs-to KEY` | `--references KEY` (legacy semantics: ORs `$includes` and `$references`) | | `--refs-from KEY` | `--referenced-by KEY` (legacy semantics: ORs `$includedBy` and `$referencedBy`) | --- # IWE Attach URL: https://iwe.md/docs/cli/attach/ Attach a document as a block reference under one or more configured **attach actions**. ## Usage ``` bash iwe attach --list iwe attach --to <NAME> [--to <NAME> ...] -k <KEY> ``` ## Options | Flag | Description | Default | | ---------------------- | -------------------------------------------------------------------- | ------- | | `--to <NAME>` | Configured attach action to attach to. Repeatable for multiple targets. | (required when not in list mode) | | `-k, --key <KEY>` | Source document key to attach | (required when not in list mode) | | `--list` | List configured attach actions and the resolved target keys, then exit | false | | `--dry-run` | Preview without writing | false | | `--quiet` | Suppress progress output | false | ## How it works For each `--to <NAME>`: 1. Look up `<NAME>` in `[actions]` of `.iwe/config.toml`. The action must be of type `attach`. 2. Render the action's `key_template` to compute the target document key (e.g. `daily/{{today}}` becomes `daily/2026-04-25`). 3. If the target document doesn't exist yet, it is created with the action's `title` as the H1 and the new block reference as the body. 4. If the target exists, the new block reference is appended. 5. If the source is **already attached** in the target, that target is silently skipped — no error, no warning, no duplicate write. The reference text on the new block is the source document's title. ## Configuration Define an attach action in `.iwe/config.toml`: ``` toml [actions.today] type = "attach" title = "{{today}}" key_template = "daily/{{today}}" ``` ## Examples ``` bash # Discover available attach actions iwe attach --list # Attach a document under one configured target iwe attach --to today -k meetings/standup # Attach the same document under multiple targets at once iwe attach --to today --to weekly -k meetings/standup # Preview without writing iwe attach --to today --to weekly -k meetings/standup --dry-run ``` ## Relationship to MCP This command mirrors the `iwe_attach` MCP tool. The MCP tool's `to` field accepts an array of action names with the same semantics. --- # IWE Schema URL: https://iwe.md/docs/cli/schema/ Bare `iwe schema` infers and displays the frontmatter schema across your workspace — it scans all documents (or a filtered subset) and reports field names, type distributions, coverage, distinct enumerable-value counts, and value breakdowns. The `iwe schema validate` subcommand instead checks documents against the [document schemas](/docs/concepts/document-schema/) bound to them. ## Usage ``` bash iwe schema [OPTIONS] iwe schema validate [OPTIONS] ``` ## Options | Flag | Description | Default | | ------------------------------- | ----------------------------------------------------------------------- | ---------- | | `-f, --format <FORMAT>` | Output format: `markdown`, `json`, `yaml` | `markdown` | | `--field <NAME>` | Restrict output to a specific field and its children | none | | `--filter <EXPR>` | Inline YAML filter expression. See [Query Language](/docs/concepts/query-language/). | none | | `-k, --key <KEY>` | Match by document key. Repeatable. | none | | `--includes <KEY[:DEPTH]>` | `$includes` anchor. Repeatable; anchors are ANDed. | none | | `--included-by <KEY[:DEPTH]>` | `$includedBy` anchor. Repeatable; anchors are ANDed. | none | | `--references <KEY[:DIST]>` | `$references` anchor. Repeatable; anchors are ANDed. | none | | `--referenced-by <KEY[:DIST]>` | `$referencedBy` anchor. Repeatable; anchors are ANDed. | none | | `--roots` | Only documents with no incoming inclusion edges | false | The `--project` and `--add-fields` flags accepted by `find` and `tree` do not apply here -- schema operates on raw frontmatter. ## What it shows For each frontmatter field found across the scanned documents: - **Field** — dot-notation for nested fields (e.g., `engagement.upvotes`) - **Types** — which YAML types appear and their percentage breakdown (string, number, boolean, null, date, datetime, array, object) - **Coverage** — how many documents contain this field (count and percentage of total) - **Distinct** — number of distinct enumerable values (see below) - **Values** — value distribution for fields with at most 100 distinct enumerable values; otherwise empty ### Enumerable values Only enumerable values participate in the `Distinct` count and the `Values` listing: - `null`, booleans, and numbers - strings made up entirely of `[A-Za-z0-9_.-/]` Free-text strings (titles, prose, URLs containing `:`) are counted in `Coverage` and `Types` but do not appear in `Values`, and they do not increment `Distinct`. A field with only such values shows `Distinct: 0` and `Values: ---`. ## Examples ``` bash # Full workspace schema iwe schema # Schema for posts only iwe schema --filter 'type: post' # Drill into a specific field iwe schema --field status # JSON output for scripting iwe schema -f json # Schema for a subtree iwe schema --included-by ai-memory:0 # Nested field inspection iwe schema --field engagement -f json ``` ## Sample Markdown Output ``` markdown | Field | Types | Coverage | Distinct | Values | | ------ | ------------------------ | -------- | -------- | --- | | status | string (100%) | 3 (100%) | 2 | draft (2), published (1) | | type | string (100%) | 3 (100%) | 3 | external (1), hub (1), post (1) | | url | null (50%), string (50%) | 2 (67%) | 1 | null (1) | ``` Fields with no enumerable values, or with more than 100 distinct enumerable values, show `---` in the `Values` column. ## JSON Format JSON output is a top-level array of field objects: ``` json [ { "field": "status", "types": [ { "type": "string", "count": 3, "percentage": 100.0 } ], "coverage": { "count": 3, "percentage": 100.0 }, "distinct": 2, "values": [ { "value": "draft", "count": 2 }, { "value": "published", "count": 1 } ] } ] ``` YAML output has the same shape. ## Validate `iwe schema validate` checks documents against the [document schemas](/docs/concepts/document-schema/) bound to them in the `[schemas]` section of `.iwe/config.toml`. Each `[schemas]` entry names a schema file in `.iwe/schemas/` and a glob that binds it to document keys; a document is validated against every schema whose glob matches it. ``` bash iwe schema validate [OPTIONS] ``` | Flag | Description | Default | | ----------------------- | ---------------------------------------------------------- | ------- | | `-f, --format <FORMAT>` | Output format: `text`, `json` | `text` | | `--schema-file <PATH>` | Validate against this schema file directly, ignoring the `[schemas]` bindings. | none | | `--filter <EXPR>` | Inline YAML filter to scope which documents are validated. | none | | `-k, --key <KEY>` | Match by document key. Repeatable. | none | The same structural anchor flags as `iwe schema` (`--includes`, `--included-by`, `--references`, `--referenced-by`, `--roots`) also apply; with no selector, every document is checked. `--schema-file` runs an ad-hoc check: the selected documents are validated against that one file rather than the schemas bound to them in config, so a schema can be tried against a document without wiring up a `[schemas]` binding. Pair it with `-k` to check a single document — `iwe schema validate -k notes/intro --schema-file draft.yaml`. In `json` output the report's `schema` field is the file's stem (`draft.yaml` → `draft`). ### Output `text` (default) reports one line per violation, `<key> › <breadcrumb>: <message>`, with an indented `hint:` line when the schema supplies one: ``` text notes/intro: required section 'Summary' missing hint: every note opens with a summary notes/intro › Tasks: header is 18 tokens (limit 12) ``` `json` emits an array of `{ key, schema, violations }` objects, one per `(document, matching schema)` pair with violations. Each violation carries `breadcrumb`, `message`, `hint`, `schemaPath` (a JSON Pointer into the schema file), and `keyword`: ``` json [ { "key": "notes/intro", "schema": "note", "violations": [ { "breadcrumb": [], "message": "required section 'Summary' missing", "hint": "every note opens with a summary", "schemaPath": "/sections/0/minContains", "keyword": "minContains" } ] } ] ``` Clean documents produce no output. ### Exit codes | Code | Meaning | | ---- | ------------------------------------------------------------------- | | `0` | every validated document is clean (or no schema binds any document) | | `1` | at least one document has a violation | | `2` | a configuration or schema-file error (bad glob, missing or uncompilable schema file) — printed to stderr before any document is validated | ## See also - [`iwe find`](/docs/agentic/find/) — search documents using the same filter language. - [`iwe stats`](/docs/agentic/stats/) — structural statistics (sections, words, edges). --- # CLI Workflow Examples URL: https://iwe.md/docs/cli/workflows/ Practical examples of using IWE CLI commands for common tasks. ## Daily Maintenance ``` bash iwe normalize iwe tree --depth 5 ``` ## Content Analysis ``` bash iwe tree iwe tree -f keys | grep api iwe export -f dot --key "machine-learning" --include-headers > ml.dot dot -Tpng ml.dot -o ml-graph.png ``` ## Document Consolidation ``` bash # Create comprehensive document from research notes iwe squash research-index --depth 4 > consolidated-research.md # Generate presentation material iwe squash project-summary --depth 2 > project-overview.md ``` ## Large Workspace Management ``` bash iwe normalize -v 2 iwe tree --depth 8 -v 2 iwe export -f dot --include-headers --depth 5 > full-graph.dot ``` --- # CLI Troubleshooting URL: https://iwe.md/docs/cli/troubleshooting/ Best practices and solutions to common issues when using the IWE CLI. ## Best practices 1. **Start small**: Test commands on a few files before processing large libraries 2. **Backup first**: Always backup before running `normalize` or other bulk operations 3. **Use debug mode**: Add `-v 2` to see detailed debug information 4. **Iterate gradually**: Use increasing depth values to explore graph complexity 5. **Visualize regularly**: Export graphs to understand document relationships 6. **Monitor root documents**: Use `tree` to track entry points as your workspace grows ## Common issues | Issue | Solution | | -------------------------- | -------------------------------------------------- | | No changes after normalize | Check that files are properly formatted markdown | | Export produces no output | Verify documents contain links and references | | Squash fails | Ensure the specified key exists and is accessible | | Command not found | Ensure IWE is installed and available in your PATH | | Permission denied | Check file permissions in your project directory | ## Debugging When encountering issues, use verbose mode to get more information: ``` bash # INFO level logging iwe -v 1 <command> # DEBUG level logging iwe -v 2 <command> ``` ## Edge Cases ### Empty Knowledge Base When no markdown files exist: | Command | Behavior | | ------------ | -------------------- | | `tree` | No output | | `find` | No matches found | | `stats` | Shows zero counts | | `export` | Produces empty graph | ### Missing Referenced Documents When a document links to a non-existent file: - **Normalize**: Updates link title to empty string - **Retrieve**: Skips missing references in expansion - **Squash**: Skips missing linked documents - **Export**: Excludes edges to missing documents ### Circular References IWE handles circular references gracefully: - **Retrieve**: Expands each document once, avoiding infinite loops - **Squash**: Includes each document once at first encounter - **Tree**: Use `-k` to start from any document in a cycle - **Export**: Renders cycles as valid graph edges ### Large Knowledge Bases For repositories with thousands of files: - Use `--depth` limits to constrain exploration - Use `find` with filters for targeted searches - Use `tree -k` to explore specific subtrees - Consider exporting subgraphs with `--key` filter ## Getting help For any command, use the `--help` flag to see available options: ``` bash iwe --help iwe <command> --help ``` --- # IWE Create URL: https://iwe.md/docs/cli/create/ Creates a document in one of two explicit modes. | Mode | Selected by | Who composes the document | Key | | ------------ | ---------------------- | ----------------------------------- | -------------------------------------- | | **Content** | default | you — `--content` is the whole file | required | | **Template** | `--template` / `-t` | iwe — a named template renders your variables | derived from `key_template`, or explicit | The mode is never inferred from the shape of the input. `--content` and `--template` are mutually exclusive. ## Usage ``` bash # Content mode -- the document is written exactly as passed iwe create <KEY> --content '<DOCUMENT>' cat doc.md | iwe create <KEY> # Template mode iwe create --template NAME --var title="..." [--var NAME=VALUE ...] iwe create --template NAME --vars-yaml '<YAML mapping>' iwe create --template NAME --vars-json '<JSON object>' iwe create <KEY> --template NAME --set FIELD=VALUE [--set FIELD=VALUE ...] ``` ## Options | Flag | Description | Mode | | ------------------------- | ---------------------------------------------------------------------------------------- | -------- | | `<KEY>` | Document key. Required in content mode; optional in template mode. | both | | `-c, --content <STR>` | The complete document, written verbatim. `-` reads it from stdin. | content | | `-t, --template <NAME>` | Compose from the named template. The name is always required. | template | | `--vars-yaml <YAML>` | YAML mapping of template variables, values keeping their types. | template | | `--vars-json <JSON>` | JSON object of template variables, values keeping their types. | template | | `--var <NAME=VALUE>` | Set a single template variable to a verbatim string. Repeatable; overrides the bulk forms. | template | | `--set <FIELD=VALUE>` | Set a single frontmatter field, written above the rendered document. Repeatable. | template | | `-i, --if-exists <MODE>` | `fail` / `skip`, plus `suffix` / `override` in template mode. | both | | `--strict` | Validate against the configured document schema before writing. | both | | `-e, --edit` | Open the created file in `$EDITOR`. | both | On success the absolute path of the created file is printed. `--if-exists skip` on an existing document prints nothing and exits successfully. ## Content mode `--content` is the **complete document**: the YAML frontmatter block first (when there is one), then the markdown, normally starting with the title heading. The bytes are written to disk as given — nothing is added, nothing is moved. ``` bash iwe create people/ada --content '--- type: person tags: [pioneer] --- # Ada Lovelace First English computer programmer.' ``` ``` markdown --- type: person tags: [pioneer] --- # Ada Lovelace First English computer programmer. ``` Piped input works with or without `--content -`: ``` bash cat draft.md | iwe create projects/overview cat draft.md | iwe create projects/overview --content - ``` `--content -` is an explicit request for stdin and is honoured at a terminal too — type the document and finish with Ctrl-D. Bare piped input is only read when stdin is not a terminal. Because the key is explicit it *is* the document's identity, so `--if-exists` accepts only `fail` (default) and `skip`. There is no `suffix` — returning a different key than the one you asked for would be wrong — and no `override`; replacing an existing document is [`iwe update`](/docs/agentic/update/)'s job. `--var`, `--vars-yaml`, `--vars-json` and `--set` all require `--template` — passing one without it is an argument error. In content mode the document you pass is already composed and already carries its own frontmatter, so there is nothing for them to do. ## Template mode `--template NAME` composes the document from a template in `.iwe/config.toml`. The name is always required — nothing about template mode is implicit: `--var`, `--vars-yaml`, `--vars-json` and `--set` all require `--template`, and `--template` in turn requires a name. (`workspace.default_template` applies to [`iwe new`](/docs/cli/new/) only.) ``` bash # The stock template shipped in every configuration iwe create --template default --var title="Standup notes" # Named template, explicit key iwe create meetings/2026-07-28 -t meeting --var title="Sync" ``` ### Variables `--var NAME=VALUE` sets one variable and the VALUE is used **verbatim, as a string** — never parsed. Markdown, punctuation and quotes all reach the template exactly as typed. ``` bash iwe create -t meeting --var title=Sync --var body='## Notes' ``` Typed values — booleans, numbers, lists, nested maps — come from the bulk forms instead: `--vars-yaml '<YAML mapping>'` or `--vars-json '<JSON object>'`, at most one per command. ``` bash iwe create -t meeting --vars-yaml 'title: Sync attendees: [ada, alan] draft: false' iwe create -t meeting --vars-json '{"title": "Sync", "attendees": ["ada", "alan"], "draft": false}' ``` Those two produce identical documents: `draft` is a boolean for `{% if draft %}`, `attendees` a list for `{% for %}`. The bulk mapping is applied first and every `--var` overrides it, **wherever the flags sit on the command line** — `--var title=A --vars-yaml 'title: B'` still yields `A`. Among `--var` flags alone the last one for a name wins. Values are replaced wholesale; there is no deep merge. Because `--var` values are strings, `--var draft=false` is the **string** `"false"`, which `{% if draft %}` reads as true. Write `draft: false` in `--vars-yaml` when you mean the boolean. Shell quoting is stripped before iwe sees the argument, so a value-typing rule on `--var` could only ever guess; strings-only removes the guess. - `body` is the prose slot by convention. `{{content}}` is a legacy alias for `{{body}}` in templates — `content` names the same variable, and setting both is an error. - Template mode never reads stdin — piped input belongs to content mode. Every variable arrives through a flag; a variable no flag sets renders as empty. - `slug`, `today`, `now` and `id` are computed by iwe and cannot be set as variables, in any of the three forms. - The key derivation and `{{slug}}` read the `title` variable. `title` and `body` are ordinary variables by convention, not privileged parameters — a template that uses neither simply ignores them. - `--vars-yaml ''` and `--vars-json ''` are errors; `--vars-yaml '{}'` is a valid no-op. A `--var` argument without `=` that looks like a mapping points you at `--vars-yaml`. - A null variable value (`body: null`, `body: ~`, or a bare `body:`) is rejected — the template would render it as the text `none`. Pass `''` for an empty value. #### Passing multiline prose `--var` values are verbatim, so a quoted multiline argument works as-is. In a `--vars-yaml` mapping, use a block scalar: ``` bash iwe create -t note --vars-yaml 'title: Release body: | ## Notes Shipped.' ``` ### Frontmatter `--set FIELD=VALUE` sets one frontmatter field, written **above** the rendered document. This is metadata prepended to the render, as distinct from the variable flags, which feed the render. Unlike `--var`, `--set` values *are* parsed as YAML — variables are render text, frontmatter is data. There is deliberately no bulk frontmatter flag. Per-field flags (`--set`, `--var`) are the CLI's way of naming one thing; bulk structured input only comes in the format-named forms (`--vars-yaml`, `--vars-json`), and frontmatter has no such pair — `iwe update --set` doesn't either. A whole mapping is repeated `--set`. ``` bash iwe create people/ada --template default \ --var title="Ada Lovelace" \ --set type=person \ --set tags='[pioneer]' \ --set status=draft ``` ``` markdown --- type: person tags: - pioneer status: draft --- # Ada Lovelace ``` - `--set` is repeatable, its VALUE is parsed as YAML, and the last `--set` for a field wins. YAML typing means `--set version=1.10` stores the number `1.1` and `--set note=~` stores null; quote to force a string, `--set version='"1.10"'`. - Fields are applied in command-line order. A new field is appended; a repeated field is replaced in place, keeping its position. Values are replaced wholesale — there is no deep merge. - When the template already emits its own frontmatter, passing `--set` is an error. Drop the flag, or drop the frontmatter from the template. ### Mode boundaries Variable values are trusted as passed — template mode does not inspect them. One check keeps composition honest: `--set` against a template that emits its own frontmatter is rejected, since the document would carry two frontmatter blocks. If you already hold the complete document — frontmatter, title heading and all — pass it to `--content` and skip composition entirely. ### Keys Without a `<KEY>` the key comes from the template's `key_template`. With a `<KEY>` that derivation is skipped and the title still fills the document body. `--if-exists` defaults to `suffix` for a derived key (quick capture appends `-1`, `-2`, …) and to `fail` for an explicit key, since an explicit key asserts an identity. ## Template variables | Variable | Value | | ------------- | --------------------------------------------------------------------------------------------- | | `{{title}}` | The `title` variable | | `{{body}}` | The `body` variable. `{{content}}` is a legacy alias | | `{{slug}}` | URL-safe form of the `title` variable | | `{{today}}` | Current date — `workspace.date_format` for the key, `markdown.date_format` for the document | | `{{now}}` | Current date and time — `workspace.time_format` / `markdown.time_format`, each falling back to the matching `date_format` | | `{{id}}` | Random 8-character alphanumeric ID | Any other name passed with `--var`, `--vars-yaml` or `--vars-json` is available under that name. ## Configuration Templates are defined in `.iwe/config.toml`: ``` toml [templates.default] key_template = "{{slug}}" document_template = "# {{title}}\n\n{{body}}" [templates.journal] key_template = "journal/{{today}}" document_template = "# {{today}}\n\n{{body}}" ``` ## Examples ``` bash # Write a document you already have iwe create projects/overview --content "$(cat overview.md)" # Idempotent create for scripts and agents cat note.md | iwe create notes/inbox --if-exists skip # Quick capture through the stock template iwe create --template default --var title="Random idea" # Named template with structured variables iwe create -t meeting --vars-json '{"title": "Sync", "attendees": ["ada", "alan"]}' # Reject the write when it breaks the document schema iwe create docs/one --content "$(cat one.md)" --strict ``` ## See also - [`iwe update`](/docs/agentic/update/) — replace or mutate an existing document - [`iwe new`](/docs/cli/new/) — title-first quick capture with `workspace.default_template`