Open Source ★ Featured Open Source

Temporal Terraform Orchestrator

Impact Summary

Built a workflow orchestration system using Temporal for managing complex multi-workspace Terraform deployments with automatic dependency resolution, output propagation, and AI integration via MCP.

Role

Creator & Maintainer

Timeline

2025–Present

Scale

  • Multi-workspace
  • DAG-based orchestration
  • AI-integrated

Links

Decision Summary

Problem
Managing Terraform deployments across multiple workspaces with interdependencies required manual coordination, brittle scripting, and lacked durability guarantees. Failed deployments required manual intervention and re-execution.
Constraints
  • Must handle workspace dependencies automatically (DAG resolution)
  • Output values from one workspace must propagate to dependent workspaces
  • Deployments must survive process restarts (durability requirement)
  • Should support AI-assisted operations for conversational infrastructure management
Tradeoffs Considered
Temporal Workflows Chosen
Pros
  • + Built-in durability and automatic retries
  • + Native support for long-running operations
  • + Event sourcing for full auditability
Cons
  • Additional infrastructure dependency
  • Learning curve for Temporal patterns
Custom State Machine
Pros
  • + Full control over implementation
  • + No external dependencies
Cons
  • Must build durability from scratch
  • Complex failure handling
CI/CD Pipeline (GitHub Actions)
Pros
  • + Familiar tooling
  • + Built-in UI
Cons
  • Poor support for dynamic dependencies
  • Limited durability guarantees

Problem

Managing Terraform deployments across multiple workspaces with interdependencies is challenging:

  • Manual dependency tracking: Teams must manually determine which workspaces depend on others and execute them in the correct order.
  • Output propagation: Passing outputs from one workspace as inputs to another requires manual intervention or brittle scripting.
  • No parallelization: Without proper orchestration, independent workspaces run sequentially, wasting time.
  • Lack of durability: Failed deployments require manual recovery and re-execution.

Approach

I built a Temporal-based orchestration system that treats multi-workspace Terraform deployments as durable workflows with automatic dependency resolution.

Architecture

The system consists of two main workflow types:

  • ParentWorkflow: Orchestrates the overall deployment by building a directed acyclic graph (DAG) of workspace dependencies, validating configuration, and managing completion signals.
  • TerraformWorkflow: Executes individual workspace operations (init, validate, plan, apply) and captures outputs for downstream consumption.

Key Features

  • Automatic dependency resolution Workspaces declare their dependencies, and the system builds a DAG to determine execution order. Independent workspaces run concurrently for faster deployments.

  • Output propagation Terraform outputs from completed workspaces automatically map to input variables in downstream workspaces through configurable input mappings.

  • Temporal durability Workflows survive worker restarts and failures. Temporal handles automatic retries with exponential backoff, making deployments resilient.

  • MCP server integration AI agents can trigger deployments and monitor execution status through standardized Model Context Protocol tools:

    • list_workflows: Returns configured workspaces and relationships
    • execute_workflow: Initiates deployment with provided configuration
    • get_workflow_status: Checks execution status by workflow ID

Outcomes

  • Faster deployments: Independent workspaces run in parallel, reducing overall deployment time.
  • Reliable infrastructure changes: Temporal’s durability guarantees mean deployments complete even through failures.
  • AI-assisted operations: MCP integration enables AI agents to manage infrastructure deployments conversationally.
  • Clear dependency visualization: The DAG representation makes workspace relationships explicit and auditable.

Key Contributions

  • Designed and implemented a Temporal workflow architecture for multi-workspace Terraform orchestration.
  • Built automatic dependency resolution using DAG construction with cycle detection.
  • Implemented output-to-input mapping for seamless variable propagation between workspaces.
  • Created an MCP server enabling AI-driven infrastructure management.
  • Documented deployment patterns and configuration schemas for production use.

Related Projects