Research State

All research state lives on disk in .omc/research/. This is the single source of truth for the entire system. When a session crashes, when context fills up, when you restart after a week — the state on disk is how the system knows where it left off.

Directory Structure

.omc/research/
├── pipeline.yaml              # Current stage, gate config, history
├── config/
│   ├── infrastructure.yaml    # GPU, storage, compute resources
│   ├── thresholds.yaml        # Monitoring thresholds
│   └── alerts.yaml            # Alert routing configuration
├── ideas/
│   ├── brainstorm.md          # Raw idea exploration
│   ├── candidates.yaml        # Ranked idea candidates
│   └── selected.yaml          # Final selected idea with rationale
├── design/
│   ├── plan.md                # High-level experiment plan
│   ├── baselines.yaml         # Baseline methods and sources
│   ├── ablations.yaml         # Ablation study design
│   └── metrics.yaml           # Evaluation metrics and targets
├── experiments/
│   ├── exp-001/
│   │   ├── config.yaml        # Hyperparameters, model config
│   │   ├── log.jsonl          # Training log (structured)
│   │   ├── results.yaml       # Final metrics
│   │   └── analysis.md        # Result interpretation
│   ├── exp-002/
│   │   └── ...
│   └── summary.yaml           # Cross-experiment comparison
├── papers/
│   ├── outline.md             # Paper outline
│   ├── related_work/
│   │   ├── papers.bib         # BibTeX entries
│   │   └── summaries.yaml     # Structured paper summaries
│   ├── drafts/
│   │   ├── v1.tex             # First draft
│   │   ├── v2.tex             # Revised draft
│   │   └── current.tex        # Symlink to latest
│   └── figures/
│       ├── descriptions.yaml  # Figure descriptions for generation
│       └── *.pdf              # Generated figures
├── reviews/
│   ├── idea_review.yaml       # Judge's idea evaluation
│   ├── code_review.yaml       # Cross-LLM code review
│   ├── paper_reviews/
│   │   ├── codex_review.yaml  # Codex review of paper
│   │   ├── claude_review.yaml # Claude review of paper
│   │   └── gemini_review.yaml # Gemini review of paper
│   └── meta_review.yaml       # Aggregated review verdict
└── logs/
    ├── orchestrator.log       # Orchestrator decision log
    ├── agent_health.yaml      # Last heartbeat per agent
    └── errors.log             # Error log with recovery actions

File Purposes

Pipeline State

File	Purpose	Updated By
pipeline.yaml	Current stage, gate types, stage history, timestamps	Orchestrator
infrastructure.yaml	GPU count, memory, storage paths, compute budget	User / Orchestrator
thresholds.yaml	Loss thresholds, timing limits, quality bars	Orchestrator / User
alerts.yaml	Where to send alerts (terminal, file, webhook)	User

Research Artifacts

Directory	Purpose	Written By	Read By
ideas/	Idea exploration and selection	Orchestrator, Scout, Judge	Planner
design/	Experiment design and baselines	Planner	Coder, Orchestrator
experiments/	Training configs, logs, results	Coder, Monitoring	Orchestrator, Judge
papers/	Paper drafts, references, figures	Writer, Scout	Judge, Orchestrator
reviews/	All evaluation verdicts	Judge	Orchestrator

System State

File	Purpose	Updated By
orchestrator.log	Decision audit trail	Orchestrator
agent_health.yaml	Heartbeat timestamps per agent	OMCC harness
errors.log	Errors with attempted recovery	All agents

Two-Tier Information Architecture

Tier 1: Context = Working Memory

What the LLM "remembers" in its current session. Fast but volatile. Limited by context window size.

Tier 2: Disk = Long-Term Memory

What's written to .omc/research/. Persistent, unlimited, survives everything. The source of truth.

What Goes Where

Information	Tier 1 (Context)	Tier 2 (Disk)
Current stage and task	Yes	Yes (pipeline.yaml)
Full experiment plan	Summary only	Yes (design/plan.md)
Training logs	Last few lines	Yes (experiments/exp-*/log.jsonl)
Paper draft	Current section	Yes (papers/drafts/)
All past decisions	No	Yes (orchestrator.log)
Agent health	Latest status	Yes (agent_health.yaml)

The rule

If information matters, it must be on disk. Context is a cache that can be evicted at any time. Never rely on context alone for state that needs to survive a session restart.

Context Fills and Recovery

When a session's context window fills up:

1. The session cannot continue — the LLM can no longer process new input
2. Start a new session — this is expected, not an error
3. The new session reads state from disk — pipeline.yaml tells it the current stage, experiment files tell it what's been done
4. Work resumes seamlessly — because all state is on disk, nothing is lost

graph LR
    A[Session 1<br/>Context fills] -->|crash/restart| B[New Session]
    B -->|reads| C[".omc/research/"]
    C -->|rebuilds context| D[Session 2<br/>Continues work]

    style A fill:#fee2e2,stroke:#dc2626
    style B fill:#dbeafe,stroke:#2563eb
    style C fill:#fef3c7,stroke:#d97706
    style D fill:#dcfce7,stroke:#16a34a

Agents write summaries, not transcripts

When an agent completes a task, it writes a one-line summary plus structured output to disk — not a full transcript of its reasoning. This keeps disk state clean and scannable.

Next

• Context Management — detailed strategies for context
• Monitoring — how training state is tracked
• Workspace Isolation — per-project state separation