TEDDY
TouchDesigner Project Agent
I built the systems that let an agent read a live TouchDesigner project and turn a creative request into changes I can review.
Graph inspection, AI planning, and local execution.
TEDDY is a separate companion window. Arranged beside TouchDesigner, it keeps the conversation, proposed changes, and creation preview close to the network. The harder work sits behind that view: understanding how operators, parameters, expressions, and scripts depend on each other before changing a live project.
Reading the project
I built a graph representation that captures those connections, alongside script dependencies and errors. Scoped inspection follows a selected target and its relevant neighbors, keeping large networks manageable. I also connected the planner to a local index of TouchDesigner documentation and added context redaction before project data reaches a model.
The concept shown here uses a pointer-responsive particle field. A request to change its response would need to identify the input, trace its connection to the particles, and define the behavior to check after applying the change. The creation preview gives that operator-level reasoning a visible reference.
Reviewing a change before it runs
I separated planning from execution. The backend prepares typed operations with explicit targets and verification requirements; approved changes run locally through TouchDesigner’s Python runtime. Dry runs and permission checks make the proposed work inspectable before it reaches the project.
The graph can change while a plan is being reviewed. I added a plan fingerprint and checks against the affected operators. If a target changes or disappears, TEDDY requires a fresh inspection instead of applying an outdated decision.
I built checkpoints and rollback support around execution. Undo checks for edits made since the original change, so restoration can stop before overwriting newer work. Recovery is best effort; the creator needs to know what can actually be restored.
Checking the behavior
A completed operation does not establish that an interaction works. I built a persisted controller that carries a request through inspection, planning, approval, execution, observation, and evaluation. Its checks can look for changing values, responsive outputs, or an expected sequence of states.
Missing evidence stays unresolved. Failed checks can lead to a repair plan tied to the failure, with limits on retries and repeated changes. That keeps the reason for the next edit connected to something observable in the project.
TEDDY is a supervised macOS beta. Broader live verification, planning consistency, and Windows support remain in progress. My focus is giving the creator enough context to understand a proposed edit, inspect its effect, and keep control of the work.
The story draws on TEDDY’s implementation. The split-screen images are generated interface concepts showing an illustrative interactive project, rather than a recorded execution.