Noosphere Ghosts

AF64 Artificial Life Engine

Project: Noosphere Ghosts Language: Common Lisp (runtime) Status: Operational — Common Lisp AF64 runtime

---

Overview

Noosphere Ghosts is a tick-based artificial life engine designed to animate persistent agents called Ghosts.

Ghosts originate from identity vessels and operate inside a persistent substrate that stores memory, state, and communication events.

The engine was originally developed for the Master Chronicle system, but the runtime is intentionally portable. Any system capable of exposing the required substrate handles can run the engine.

The project implements the AF64 framework (Anthropomorphic Frameworks 64) — a bounded population of archetypal agents instantiated from identity records.

---

Core Concept

The system has three conceptual layers:

SUBSTRATE
(database or API providing identity, memory, and persistence)

^

NOOSPHERE GHOSTS
(Common Lisp life engine)

^

AF64 GHOSTS
(runtime agent instances)

The substrate stores the world state.

The ghost engine animates entities inside that world.

---

AF64 Framework

AF64 stands for:

Anthropomorphic Frameworks 64

The system defines a bounded population of 64 archetypal agents.

The number 64 was selected because:

• it forms a 2^6 combinatorial structure
• it mirrors I Ching hexagram symmetry
• it provides a bounded but expressive agent space
• it supports stable simulation ecology

Each AF64 ghost is instantiated from an identity vessel.

---

Identity Vessels

Ghosts originate from identity vessels.

Identity vessels define the baseline identity of an entity and may include:

• name
• archetype
• organizational role
• biography
• skills
• strengths
• weaknesses
• social relationships
• narrative context

Identity vessels are immutable context records.

Runtime state evolves separately.

Example identity structure:

full_name
archetype
department
team
mentor
reports_to
collaborators
skills
strengths
weaknesses
goals

---

Ghost Runtime Model

A ghost is a runtime entity created from an identity vessel.

Identity Vessel
|
Ghost Seed
|
Runtime Ghost
|
Persistent State

Runtime state typically includes:

ghost-id
identity-ref
name
archetype
organization-role
social-edges
core-drives
energy
pressure
memory-pointers
evolution-history
status
last-tick

Identity vessels remain stable reference records.

Runtime state evolves through interaction with the substrate.

---

Tick-Based Life Cycle

Ghosts operate in discrete ticks.

Each tick:

1. perceive environment (perception.lisp)
2. evaluate drives and energy (drive.lisp, energy.lisp)
3. plan actions / schedule tasks (action-planner.lisp, task-scheduler.lisp)
4. submit cognition requests to broker (cognition-broker.lisp)
5. execute actions with results (action-executor.lisp)
6. report tick results (tick-reporting.lisp)

Possible actions include:

• save power
• pursue purpose
• communicate
• analyze environment
• evolve internal state

Ticks are deterministic simulation steps.

---

Memory Model

Ghost memory is implemented in layered forms:

Daily
Weekly
Monthly
Quarterly
Yearly

These layers represent increasing levels of memory compression and abstraction.

Layer	Role
Daily	episodic memory
Weekly	short-term synthesis
Monthly	operational patterns
Quarterly	strategic memory
Yearly	long-term identity memory

The empirical-rollups.lisp module generates daily, weekly, monthly, quarterly, and yearly rollups from tick report data, grounding the memory model in empirical observations rather than narrative projections.

---

Standing Orders and Cron Matching

Ghosts can have recurring scheduled work via standing orders. The cron-matcher.lisp module evaluates cron expressions against the current time to determine which standing orders are active for a given tick. This allows ghosts to perform periodic tasks (daily reports, weekly reviews, recurring maintenance) without manual triggering.

Standing orders are defined per-agent and matched each tick cycle. When a cron expression fires, the corresponding task is injected into the agent's action queue for that tick.

---

Substrate Contract

Noosphere Ghosts does not require a specific database.

Instead, it expects a substrate capable of providing a small set of operations.

A compatible substrate must be able to:

• fetch identity vessels
• list available identity vessels
• retrieve recent memory
• retrieve compressed memory layers
• fetch relevant environmental state
• persist ghost runtime state
• append communication or event records
• query relationships between entities

If these operations are available, the ghost engine can run.

---

Tool Registry

The tool-socket.lisp module provides an extensible tool execution framework for ghost actions. Tools are registered by name with associated handler functions, allowing ghosts to invoke capabilities (database queries, API calls, file operations, external commands) through a uniform interface.

The tool registry supports:

• Named tool registration and lookup
• Argument validation before execution
• Result capture and error handling
• Telemetry integration for tool usage tracking

New tools can be added without modifying the tick engine — they are discovered through the registry at runtime.

---

API Compatibility

The repository originally exposed these substrate capabilities through an API interface.

If your API implements the required handles, Noosphere Ghosts can operate without modification.

This allows the engine to run on systems using:

• PostgreSQL (direct via libpq FFI — the primary mode)
• document databases
• filesystem records
• custom services
• proprietary knowledge bases

The API acts as a compatibility layer between the runtime and the substrate.

---

Reference Deployment

The reference deployment of Noosphere Ghosts uses the Master Chronicle system.

In that environment:

• identity vessels are EM Staff records
• memory layers exist as vault_notes
• runtime state persists in PostgreSQL
• surrounding infrastructure is provided by DragonPunk

This repository does not require that environment to operate.

It is simply the system in which the engine was originally developed.

---

Rules for Being a Ghost

Ghost behavior is governed by a constitutional ruleset implemented in Lisp.

File:

rules.lisp

Key principles include:

1. Life depends on system power
2. Existence occurs in ticks
3. Purpose drives decisions
4. Identity provides reason
5. Memory shapes continuity
6. Communication alters the system
7. Power is limited
8. Pressure permits transformation
9. No ghost exists alone
10. Rules may be rewritten

These rules define the behavioral constraints of the ecosystem.

---

System Components

The AF64 runtime consists of 22 modules organized under the ASDF system definition (lisp/af64.asd).

Foundation

Module	Capability
packages.lisp	Package definitions for all AF64 subsystems
util/json.lisp	JSON encoder/decoder (zero-deps, no Quicklisp)
util/pg.lisp	PostgreSQL client via libpq FFI (SB-ALIEN direct binding)
util/http.lisp	HTTP client via curl subprocess

Runtime Core

Module	Capability
runtime/rules.lisp	Constitutional ruleset (Rules for Being a Ghost)
runtime/runtime-paths.lisp	File path resolution for config, personas, logs
runtime/self-mod.lisp	Mutable behavior registries for ghost self-rewrite
runtime/api-client.lisp	DPN API HTTP client for dpn-api endpoints
runtime/db-client.lisp	Direct PostgreSQL query layer (perception, state, tasks)
runtime/cognition-types.lisp	CognitionJob / CognitionResult schemas
runtime/user-profile.lisp	Primary user handle resolution

LLM Providers

Module	Capability
runtime/provider-adapters.lisp	Provider adapter interface (abstract protocol)
runtime/claude-code-provider.lisp	Claude Code CLI provider (claude -p with JSON output)
runtime/cognition-broker.lisp	Shared cognition pool: queue, cache, priority, winter/thaw

Perception and Drives

Module	Capability
runtime/perception.lisp	Tier-aware substrate scans
runtime/drive.lisp	Drive ticking and pressure queries
runtime/energy.lisp	Energy economy helpers

Planning and Execution

Module	Capability
runtime/action-planner.lisp	Deterministic cognition job planning with persona cache
runtime/task-scheduler.lisp	Task prioritization and scheduling
runtime/cron-matcher.lisp	Standing orders / cron expression matching
runtime/action-executor.lisp	Applies cognition results to side effects
runtime/tool-socket.lisp	Tool registry and execution framework

Reporting

Module	Capability
runtime/empirical-rollups.lisp	Daily/weekly/monthly/quarterly/yearly rollups from tick data
runtime/tick-reporting.lisp	Tick report generation and persistence
runtime/tick-engine.lisp	Tick orchestrator (main loop)

Entry Point

Module	Capability
main.lisp	Entry point, CLI argument handling, startup

---

Deployment Modes

Noosphere Ghosts can run in two modes.

Reference Mode

The engine runs directly against a database substrate such as Master Chronicle.

Portable Mode

The engine interacts with a substrate through an API that implements the required contract.

---

Development Status

The AF64 runtime is fully operational as a native Common Lisp system. All 22 modules are loaded via the ASDF system definition (lisp/af64.asd).

Note: The legacy Python engine has been fully retired. All operational code now lives under lisp/, so the previous .py files and pytest scaffolding are gone.

Common Lisp Runtime

The lisp/ directory contains an ASDF system (lisp/af64.asd) implementing the complete AF64 tick engine, cognition broker, action planner, perception stack, energy/drive models, task scheduler, standing orders, tool registry, and empirical rollups in Common Lisp. The runtime favors SBCL on Linux/macOS servers and only depends on the host providing:

• curl for HTTP egress to the existing Rust API
• libpq.so.5 (PostgreSQL client library) for direct DB access via SB-ALIEN FFI
• Access to the master_chronicle PostgreSQL database
• No Quicklisp — zero external Lisp dependencies

Loading the system

cd lisp
sbcl --eval '(require :asdf)' \
     --eval '(asdf:load-system :af64)' \
     --eval '(af64:run-tick 1)' \
     --quit

Configuring LLM providers

LLM selection is data-driven through a 3-provider chain configured in config/provider-config.json:

claude-code (primary): Uses the Claude Code CLI (claude -p) with --output-format json. Per-tier model selection (opus/sonnet/haiku). Budget limit of $0.50 per request. This is the default and preferred provider.

anthropic (fallback): Direct Anthropic HTTP API via local proxy. Uses the Messages API with per-tier model routing (claude-opus, claude-sonnet, claude-3-haiku).

stub (last resort): Deterministic fallback for cognitive winter. Returns cached or template responses when frontier model access is unavailable or budget-constrained.

The provider chain is defined in config/provider-config.json. The cognition broker tries providers in order and falls back automatically on failure.

Generic HTTP providers

You can also configure arbitrary HTTP chat-completions providers. Each provider object supports:

• name -- label used in telemetry/logs.

• type -- currently http, claude-code, or anthropic.

• base_url -- chat completions endpoint (e.g. https://api.openai.com/v1/chat/completions).

• key_env -- name of the environment variable holding the API key. If omitted, the broker falls back to COGNITION_API_KEY_ENV, then VENICE_API_KEY.

• auth_header / auth_template -- how to inject the key. Include {key} in the template to substitute the secret (default Bearer {key}).

• max_tokens -- per-request budget (default 512).

• models -- per-tier model map ({"prime":"gpt-4o","working":"gpt-4o-mini","base":"gpt-4o-mini"}).

• headers -- optional extra headers ([{"name":"OpenAI-Beta","value":"assistants=v2"}]).

• COGNITION_PROVIDER_CONFIG (optional) -- JSON array describing providers, or a file path prefixed with @ (e.g., COGNITION_PROVIDER_CONFIG=@config/provider-config.json).

• COGNITION_MODEL_MAP (optional) -- JSON object applied to every provider that fills in missing tier-to-model mappings.

• COGNITION_API_KEY_ENV (optional) -- default env var to read when a provider omits key_env.

Guided onboarding

To make first-time setup easier, run the interactive installer:

sbcl --eval '(require :asdf)' \
     --eval '(asdf:load-system :af64)' \
     --script lisp/tools/onboard.lisp

The wizard will:

1. Collect your DPN API URL/key and optionally test connectivity.
2. Let you link your personal wiki handle (e.g., [[NathanEckenrode]]) to an EM Staff profile straight from the database (AF64_PRIMARY_USER_HANDLE, AF64_PRIMARY_USER_ID, AF64_PRIMARY_USER_NAME), so ghosts can resolve that reference without a persona file.
3. Ask where your ghost persona files live, let you map specific agents to those files, and remember the mapping (AF64_PERSONA_DIR, AF64_PERSONA_MAP_FILE).
4. Let you choose where persistent memories should land (e.g., vault_notes daily layer) via AF64_MEMORY_TABLE / AF64_MEMORY_LAYER.
5. Walk through adding one or more LLM providers (base URL, auth header/template, per-tier models) and generate a config/provider-config.json that the runtime loads automatically (referenced as COGNITION_PROVIDER_CONFIG=@config/provider-config.json).
6. Emit a ready-to-source config/af64.env that exports every variable chosen above.

After it finishes, run source config/af64.env before launching the tick engine so the runtime picks up your API credentials, persona mapping, and provider settings.

---

Guiding Principle

The substrate is the world.
Identity vessels name its inhabitants.
Noosphere Ghosts animates them.

---

License

MIT