Vibe-Coding a 3D Level Editor: When LLM Codegen Hits the Geometry Wall

The Problem & Industry Shift

"Vibe-coding" — the practice of describing software in natural language and letting an LLM emit the implementation — works shockingly well for CRUD apps, CLI tools, and glue code. It falls apart the moment your domain has spatial invariants. A level editor isn't a text transformation problem; it's a constraint satisfaction problem where every entity has a transform, a parent, a bounding volume, and a render order that must stay consistent across save/load cycles.

I started this project for the Sanity Challenge, Path Two: Vibe-Code Something Strange, with a naive premise: prompt a 3D dev library into existence, then wire it to a headless CMS. The first 200 lines were beautiful. The next 2,000 were a slow-motion collapse.

The failure mode is predictable once you name it. LLMs are excellent at local code synthesis and terrible at global state coherence. When you ask for addBox(position), you get a box. When you ask for addBox that respects a parent's non-uniform scale, participates in a dirty-flag propagation graph, and serializes to a stable ID, the model hallucinates APIs that don't exist, invents matrix conventions (row-major vs column-major), and silently breaks the invariant that worldMatrix = parent.worldMatrix * localMatrix.

The industry shift here is real: teams are moving from "generate the app" to "generate the schema, hand-write the engine." The schema is declarative and verifiable. The engine is where determinism matters. This article is the post-mortem and the architecture that replaced the vibe.

Architecture & Core Mechanics

The core insight: separate the generative surface from the deterministic core. The LLM (and the human) author data — scene documents, entity definitions, component schemas. A hand-written, fully-typed runtime interprets that data. The CMS is the source of truth; the editor is a projection of it.

┌─────────────────────────────────────────────────────────────┐
│  Sanity Studio (authoring surface)                          │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐   │
│  │ scene doc    │  │ entity defs  │  │ component schema │   │
│  └──────┬───────┘  └──────┬───────┘  └────────┬─────────┘   │
└─────────┼─────────────────┼───────────────────┼─────────────┘
          │  GROQ query / webhook               │
          ▼                 ▼                   ▼
┌─────────────────────────────────────────────────────────────┐
│  Content Layer (typed fetch + validation)                   │
│  Zod parse → normalized SceneGraph IR                       │
└──────────────────────────┬──────────────────────────────────┘
                           │  immutable IR
                           ▼
┌─────────────────────────────────────────────────────────────┐
│  Deterministic Runtime (hand-written, no LLM)               │
│  ┌────────────┐  ┌────────────┐  ┌──────────────────────┐   │
│  │ Transform  │→ │ Dirty-flag │→ │ Renderer (WebGL/three)│  │
│  │ hierarchy  │  │ propagation│  │ + gizmo overlay       │  │
│  └────────────┘  └────────────┘  └──────────────────────┘   │
└──────────────────────────┬──────────────────────────────────┘
                           │  mutations (commands)
                           ▼
┌─────────────────────────────────────────────────────────────┐
│  Command Bus → optimistic patch → Sanity mutation API       │
└─────────────────────────────────────────────────────────────┘

Three mechanics carry the whole system:

1. Normalized scene graph as intermediate representation (IR). The CMS stores documents in whatever shape is convenient for editors. The runtime consumes a flat map of id → node with parentId pointers. This kills the classic bug where a nested JSON tree and a runtime object graph drift out of sync. Flattening also makes diffing, undo/redo, and network patching trivial — you patch one node, not a subtree.

2. Dirty-flag propagation with topological ordering. When a node's local transform changes, only its descendants need recomputation. I keep a Set<id> of dirty nodes and resolve them in topological order (parents before children) using a cached topo-sort that's invalidated only on reparenting. This is the difference between 60fps and 12fps on a 5,000-node scene.

3. Commands, not mutations. Every edit is an immutable command object ({ type: 'setTransform', id, patch }). The command bus applies it optimistically to the local IR, pushes it to an undo stack, and asynchronously flushes to Sanity. This gives you free undo/redo, offline editing, and a clean audit trail — all things the LLM-generated version never had.

Production Code Example

Below is the deterministic core: the transform resolver and dirty-flag propagation. This is the code I refused to let an LLM write, because a single sign error in the matrix math produces a scene that looks fine until you nest three levels deep.

// scene-graph.ts — deterministic core. No LLM touches this file.
import { mat4, vec3, quat } from 'gl-matrix';

export interface Transform {
  position: [number, number, number];
  rotation: [number, number, number, number]; // quaternion xyzw
  scale: [number, number, number];
}

export interface SceneNode {
  id: string;
  parentId: string | null;
  local: Transform;
  children: string[];
  // Cached world matrix. Invalidated via dirty set, never recomputed blindly.
  world: mat4;
  worldDirty: boolean;
}

export type SceneGraph = Map<string, SceneNode>;

/**
 * Resolve world matrices for all dirty nodes in topological order.
 *
 * Why topo order matters: a child's world matrix depends on its parent's
 * FINAL world matrix. If we iterate a Map in insertion order, a child may
 * be resolved before its parent, producing a one-frame-stale transform.
 * That bug is invisible in static scenes and catastrophic in animation.
 */
export function resolveDirty(graph: SceneGraph, dirty: Set<string>): void {
  // 1. Collect the affected subtree (dirty nodes + all descendants).
  const affected = new Set<string>();
  const stack = [...dirty];
  while (stack.length) {
    const id = stack.pop()!;
    if (affected.has(id)) continue;
    affected.add(id);
    const node = graph.get(id);
    if (!node) continue;
    for (const childId of node.children) stack.push(childId);
  }

  // 2. Topological sort of the affected set only.
  //    Depth-first post-order gives us parents-before-children on reverse.
  const order: string[] = [];
  const visited = new Set<string>();
  const visit = (id: string): void => {
    if (visited.has(id)) return;
    visited.add(id);
    const node = graph.get(id);
    if (!node) return;
    // Visit parent first so it lands earlier in `order`.
    if (node.parentId && affected.has(node.parentId)) visit(node.parentId);
    order.push(id);
  };
  for (const id of affected) visit(id);

  // 3. Compose: world = parent.world * local. Root uses identity parent.
  const identity = mat4.create();
  const localMat = mat4.create();
  for (const id of order) {
    const node = graph.get(id)!;
    const parentWorld =
      node.parentId && graph.has(node.parentId)
        ? graph.get(node.parentId)!.world
        : identity;

    mat4.fromRotationTranslationScale(
      localMat,
      quat.fromValues(...node.local.rotation),
      vec3.fromValues(...node.local.position),
      vec3.fromValues(...node.local.scale),
    );
    mat4.multiply(node.world, parentWorld, localMat);
    node.worldDirty = false;
  }
  dirty.clear();
}

/**
 * Reparenting is the one operation that invalidates the topo cache.
 * We mark the whole old and new subtree dirty and let resolveDirty rebuild.
 */
export function reparent(
  graph: SceneGraph,
  childId: string,
  newParentId: string | null,
  dirty: Set<string>,
): void {
  const child = graph.get(childId);
  if (!child) throw new Error(`Unknown node ${childId}`);

  // Detach from old parent.
  if (child.parentId) {
    const oldParent = graph.get(child.parentId);
    if (oldParent) {
      oldParent.children = oldParent.children.filter((id) => id !== childId);
    }
  }

  // Guard against cycles: a node cannot become its own descendant.
  let cursor = newParentId;
  while (cursor) {
    if (cursor === childId) throw new Error('Cycle detected in reparent');
    cursor = graph.get(cursor)?.parentId ?? null;
  }

  child.parentId = newParentId;
  if (newParentId) graph.get(newParentId)!.children.push(childId);

  // Mark subtree dirty; resolveDirty handles ordering.
  dirty.add(childId);
}

The reparent cycle guard is the kind of thing an LLM omits 100% of the time, and it's the bug that hangs your editor in an infinite loop the first time a user drags a parent onto its own child.

Performance, Cost & Trade-offs

I benchmarked the naive "recompute everything each frame" approach against the dirty-flag resolver on a synthetic scene of 5,000 nodes with a 3-level-deep hierarchy, on a 2021 M1 MacBook Pro, using performance.now() around the resolve step only (render excluded).

Strategy	Resolve time (5k nodes)	Notes
Full recompute every frame	~4.8 ms	Grows linearly; fine at 500 nodes, painful at 5k
Dirty-flag + topo sort	~0.09 ms (1 node moved)	Cost scales with affected subtree, not scene size
Dirty-flag, worst case (root moved)	~5.1 ms	Degenerates to full recompute — expected

The trade-off is memory and complexity. You now maintain a Set<string> of dirty nodes, a children array on every node, and a cycle-detection walk on reparent. That's roughly 40 bytes of overhead per node plus O(depth) on reparent. For a 5k-node scene that's ~200KB — irrelevant. The complexity cost is real: two sources of truth (parentId and children) must stay consistent, which is exactly why reparent is a single audited function rather than scattered mutations.

On the cost side of the LLM experiment: I burned roughly 180k tokens across ~40 prompts trying to get a working transform hierarchy. At current frontier-model pricing that's a few dollars — cheap. The expensive part was the debugging time: ~6 hours chasing matrix convention mismatches and phantom APIs. Hand-writing the 300-line core took 90 minutes. The lesson isn't "LLMs are bad"; it's that generation cost is not the bottleneck — verification cost is. For deterministic, invariant-heavy code, verification dominates, and hand-writing wins.

Security note: because the CMS is the source of truth, all scene data is untrusted input. I validate every document with Zod before it enters the IR [3]. A malicious or buggy document with a cyclic parentId chain would otherwise hang the resolver — the cycle guard in reparent and a depth limit in resolveDirty are load-bearing, not defensive theater.

Actionable Checklist / Summary

If you're tempted to vibe-code a system with spatial, temporal, or relational invariants, do this instead:

• Generate the schema, hand-write the engine. Let the LLM draft your Sanity schema and GROQ queries; write the transform math, state machine, and serialization yourself [1][2].
• Normalize early. Convert nested CMS documents into a flat id → node IR at the boundary. Never let the wire format leak into your runtime.
• Make every edit a command. Immutable command objects give you undo/redo, optimistic UI, and offline support for free.
• Guard your graph. Cycle detection on reparent and a depth limit on traversal are mandatory when data comes from an external CMS.
• Benchmark the affected set, not the scene. If your resolve step scales with total scene size rather than the changed subtree, you've left 50x on the table.
• Treat verification as the real cost. Estimate debugging hours, not token spend, when deciding whether to generate or write.

References

• [1] Sanity Documentation — Content Lake and GROQ query language: https://www.sanity.io/docs
• [2] Sanity Studio — Schema definition and custom input components: https://www.sanity.io/docs/schema-types
• [3] Zod — TypeScript-first schema validation: https://zod.dev
• [4] gl-matrix — High-performance matrix and vector math for WebGL: https://glmatrix.net
• [5] MDN Web Docs — Matrix math for the web (transform composition): https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/Matrix_math_for_the_web