IWE Retrieve

Retrieves document content with graph expansion and relationship context. Designed for AI agents to navigate and understand the knowledge graph.

Usage

iwe retrieve [OPTIONS]

Without -k, reads the document key from stdin (for piping).

Options

Filter and structural anchors

When filter or anchor flags are provided, retrieve reads the documents that match. With both -k and a filter expression, the result is the intersection — -k clauses AND with the filter at the top level.

# Retrieve every doc under projects/alpha (unbounded)
iwe retrieve --included-by projects/alpha:0 --depth 0

# Sub-documents of two anchors (intersection)
iwe retrieve --included-by projects/alpha --included-by people/dmytro --depth 0

# Restrict explicit keys to those inside an anchor's subtree
iwe retrieve -k x -k y -k z --included-by projects/alpha

# Filter by frontmatter
iwe retrieve --filter 'status: draft' --depth 0

See Query Language for the full filter syntax.

How It Works

The retrieve command collects documents in a specific order:

1. Main document - The document specified by -k
2. Children (-d N) - Documents included via Inclusion Links, expanded N levels deep
3. Context (-c N) - Parent documents of the main document, up N levels
4. Sub-document parents - Parents of direct sub-documents (only when both -d > 0 and -c > 0)
5. Links (-l) - Documents referenced inline within the main document

Documents are deduplicated - each document appears only once in the output.

Depth Expansion (-d)

Controls how deep to follow children (sub-documents):

• -d 0: Only the main document, no expansion
• -d 1 (default): Main document + direct sub-documents
• -d 2: Main + sub-documents + their sub-documents
• -d N: Follow children up to N levels deep

Context Expansion (-c)

Controls how many levels of parent documents to include:

• -c 0: No parent context documents included
• -c 1 (default): Include direct parents of main document and direct parents of immediate sub-documents
• -c N: Include parent documents up to N levels up

Sub-document Parent Context

When you retrieve a document with both depth (-d) and context (-c) enabled, the command also fetches parent documents of any sub-documents. This provides important context: the parent document often defines what type of thing the sub-document is.

Example: A component used in multiple projects

Consider a button component embedded in two different projects. When you retrieve mobile-app:

iwe retrieve -k mobile-app -d 1 -c 1

The output for the button sub-document shows its other parents:

# Button

[Button](button) <- [Web Dashboard](web-dashboard)

A reusable button component.

The <- notation lists other documents where button is embedded. Since we’re already viewing button from within mobile-app, only web-dashboard is shown — revealing that this component is shared across projects.

Result includes:

1. mobile-app — the main document
2. button — sub-document (via -d 1)
3. web-dashboard — another parent of button (via -c 1)

The web-dashboard document is fetched because it’s a parent of button. This context helps understand what the component is and where else it’s used.

Note: Only parents of direct sub-documents are included.

Links (-l)

When enabled, includes documents that are referenced inline (not as inclusion links):

# Main Document

See [Related Topic](related-topic) for more details.

With -l, related-topic would be included in the output.

Document Relationships

Parent Documents vs Back Links

Parent Documents - Inclusion links (document embedded in another):

# Parent Document

## Overview

[child](child)   <- Inclusion link creates parent-child relationship

The child document shows: **Parent documents:** [Parent Document](parent) > Overview

Back Links - Inline references (links within text):

# Some Document

See [target](target) for more details.   <- Inline reference creates backlink

The target document shows: **Back links:** [Some Document](some-document)

Output Format

Markdown Format (default)

Documents are rendered with YAML frontmatter containing metadata, followed by the document content:

---
document:
  key: my-document
  title: My Document
  includedBy:
  - key: index
    title: Index
  referencedBy:
  - key: related-doc
    title: Related Doc
---

# My Document

Original document content preserved exactly as written.

Each document in the result set includes:

• YAML frontmatter with key, title, includedBy, and referencedBy
• Document content with original headers preserved
• Two empty lines after each document for easy parsing

Multiple documents are separated by their frontmatter delimiters (---), no horizontal rules.

Keys Format (-f keys)

my-document
child-document
parent-document

One key per line, suitable for piping to other commands or building exclude lists.

JSON Format (-f json)

[
  {
    "key": "my-document",
    "title": "My Document",
    "content": "# My Document\n\nContent here...",
    "includedBy": [
      {
        "key": "index",
        "title": "Index",
        "sectionPath": ["Topics", "My Topic"]
      }
    ],
    "includes": [],
    "referencedBy": [
      {
        "key": "related-doc",
        "title": "Related Doc",
        "sectionPath": []
      }
    ]
  }
]

The top-level value is a bare array of document objects. Each carries key, title, content, and three edge arrays — includedBy, includes, referencedBy — using the same EdgeRef { key, title, sectionPath } shape. Empty arrays are emitted explicitly.

includes is populated only when --children is passed; --no-content blanks content independently.

Dry Run (--dry-run)

Default (markdown / no -f): two-line prose summary.

$ iwe retrieve -k my-document --dry-run
documents: 5
lines: 234

With -f json or -f yaml, dry-run emits a structured summary instead:

$ iwe retrieve -k my-document --dry-run -f json
{
  "documents": 5,
  "lines": 234
}

Useful for checking how much content would be retrieved before fetching it.

Examples

# Retrieve single document with defaults (depth=1, context=1)
iwe retrieve -k my-document

# Retrieve multiple documents at once
iwe retrieve -k doc1 -k doc2 -k doc3

# Retrieve only the main document, no expansion
iwe retrieve -k my-document -d 0 -c 0

# Retrieve with deep expansion (3 levels of sub-documents)
iwe retrieve -k my-document -d 3

# Include more parent context
iwe retrieve -k my-document -c 2

# Include inline referenced documents
iwe retrieve -k my-document -l

# Exclude documents which you don't need
iwe retrieve -k my-document -e already-loaded -e another-loaded

# Get metadata only, no content
iwe retrieve -k my-document --no-content

# Populate the `includes` edge array (independent of --no-content)
iwe retrieve -k my-document --children

# Check size before retrieving
iwe retrieve -k my-document -d 2 -c 1 --dry-run

# Get JSON output for programmatic processing
iwe retrieve -k my-document -f json

# Or YAML
iwe retrieve -k my-document -f yaml

# Pipe keys from stdin (one per line)
echo -e "doc1\ndoc2\ndoc3" | iwe retrieve

# Without backlinks (cleaner output)
iwe retrieve -k my-document -b false

Use Cases

AI Agent Context Building

Build rich context for AI agents navigating the knowledge base:

# Get document with immediate context
iwe retrieve -k authentication

# Check context size first
iwe retrieve -k authentication -d 2 --dry-run

# Get broader context if needed
iwe retrieve -k authentication -d 2 -c 2

Understanding Document Relationships

# See where a document is used (parent documents in output)
iwe retrieve -k my-topic -d 0 -c 0

# See the full context including inline references
iwe retrieve -k my-topic -l

Exploring Knowledge Base Structure

# Start from an entry point and expand
iwe retrieve -k index -d 2

# Get JSON for analysis
iwe retrieve -k project-overview -d 1 -f json

Chaining Retrievals

Use the keys format to chain retrieval operations and exclude already-fetched documents:

# Get keys from first retrieval
KEYS_A=$(iwe retrieve -k topic-a -f keys)

# Retrieve for topic-b, excluding keys from topic-a
iwe retrieve -k topic-b -e $KEYS_A

# Or using command substitution with xargs
iwe retrieve -k topic-b $(iwe retrieve -k topic-a -f keys | xargs -I {} echo "-e {}")

Deprecated aliases

The following flags pre-date the query language and remain accepted for backward compatibility. Each invocation prints a one-line warning: ... is deprecated to stderr.

Technical Notes

• Documents use YAML frontmatter for metadata, content follows with original formatting
• Empty includedBy or referencedBy fields are omitted from markdown frontmatter (JSON/YAML output always emits them as [])
• Original document headers are preserved (no level shifting)
• Two empty lines separate documents for easier parsing
• Duplicate documents are automatically filtered out
• Sub-document parent context only includes parents of direct (first-level) sub-documents, not nested ones