Task Orchestration

What is Task Orchestration?

Task orchestration in Spry is a system that automatically manages the execution order of your tasks based on their dependencies. Think of it like a smart to-do list that knows which tasks must happen before others.

Simple Example:

You can't deploy an app before you've built it.
You can't build it before you've installed dependencies.

Spry handles this automatically using a Directed Acyclic Graph (DAG).

Core Concept: The DAG

What is a DAG?

A Directed Acyclic Graph is a fancy term for a one-way dependency chain with no loops.

Directed: Tasks flow in one direction (A → B → C)
Acyclic: No circular dependencies (A can’t depend on C if C depends on A)
Graph: Multiple tasks can connect in complex ways

Visual Example:

setup-db ──────┐
               ├──→ process-data ──→ deploy
fetch-api ─────┘

Both setup-db and fetch-api run in parallel, then process-data runs when both complete, finally deploy runs.

Basic Task Dependencies

Simple Chain

Tasks run one after another:

```bash install-deps --descr "Install dependencies"
npm install
```

```bash build --dep install-deps --descr "Build application"
npm run build
```

```bash test --dep build --descr "Run tests"
npm test
```

```bash deploy --dep test --descr "Deploy to production"
./deploy.sh
```

Execution Order:

install-deps → build → test → deploy

Multiple Dependencies

A task can depend on multiple other tasks:

```bash setup-database --descr "Initialize database"
sqlite3 app.db < schema.sql
```

```bash fetch-users --descr "Fetch user data"
curl https://api.example.com/users > users.json
```

```bash fetch-products --descr "Fetch product data"
curl https://api.example.com/products > products.json
```

```bash import-data --dep setup-database --dep fetch-users --dep fetch-products
deno run import.ts
```

Execution Flow:

setup-database ─┐
                │
fetch-users ────┼──→ import-data
                │
fetch-products ─┘

setup-database, fetch-users, and fetch-products all run in parallel (they’re independent). Once ALL three complete, import-data runs.

Parallel Execution Explained

When Tasks Run in Parallel

Spry automatically runs tasks in parallel when they:

Have no dependencies, OR
All their dependencies are satisfied

Example:

```bash download-images --descr "Download images"
wget https://example.com/images.zip
```

```bash download-videos --descr "Download videos"
wget https://example.com/videos.zip
```

```bash download-fonts --descr "Download fonts"
wget https://example.com/fonts.zip
```

```bash unzip-all --dep download-images --dep download-videos --dep download-fonts
unzip '*.zip'
```

Timeline:

Time 0s:  download-images, download-videos, download-fonts (all start simultaneously)
Time 10s: All downloads complete
Time 11s: unzip-all runs

Mixed Dependencies

Complex real-world scenario:

```bash install-deps
npm install
```

```bash compile-ts --dep install-deps
tsc
```

```bash compile-sass --dep install-deps
sass styles.scss styles.css
```

```bash run-tests --dep compile-ts
npm test
```

```bash build-bundle --dep compile-ts --dep compile-sass
webpack
```

```bash deploy --dep run-tests --dep build-bundle
./deploy.sh
```

Execution Visualization:

                  ┌──→ compile-ts ───┬──→ run-tests ──┐
install-deps ─────┤                  │                ├──→ deploy
                  └──→ compile-sass ─┴──→ build-bundle┘

Timeline:

install-deps runs first
compile-ts and compile-sass run in parallel
run-tests runs after compile-ts
build-bundle runs after both compile-ts and compile-sass
deploy runs after both run-tests and build-bundle

Running Tasks

Run a Specific Task

Execute one task and all its dependencies:

./spry.ts task deploy

What happens:

Spry analyzes the DAG
Identifies all dependencies of deploy
Executes them in the correct order
Finally runs deploy

Run All Tasks (Runbook)

Execute every task in the Spryfile:

./spry.ts runbook

Useful for CI/CD pipelines or complete project setup.

Dry Run

See what would execute without running:

./spry.ts task deploy --dry-run

Advanced Patterns

Conditional Dependencies

Use environment variables to control flow:

```bash build-dev --descr "Development build"
npm run build:dev
```

```bash build-prod --descr "Production build"
npm run build:prod
```

```bash deploy-dev --dep build-dev
deploy-to-dev.sh
```

```bash deploy-prod --dep build-prod
deploy-to-prod.sh
```

Then run specific tasks based on environment:

# Development
./spry.ts task deploy-dev

# Production
./spry.ts task deploy-prod

Database Migration Pattern

Common pattern for database work:

```bash db-backup --descr "Backup current database"
sqlite3 app.db ".backup backup-$(date +%Y%m%d).db"
```

```bash db-migrate --dep db-backup --descr "Run migrations"
sqlite3 app.db < migrations/001_add_users.sql
sqlite3 app.db < migrations/002_add_products.sql
```

```bash db-seed --dep db-migrate --descr "Seed test data"
sqlite3 app.db < seed.sql
```

```bash db-verify --dep db-seed --descr "Verify database"
sqlite3 app.db "SELECT COUNT(*) FROM users;"
```

Safety: Backup happens first, ensuring you can rollback if needed.

Multi-Environment Deployment

```bash build --descr "Build application"
npm run build
```

```bash test --dep build --descr "Run tests"
npm test
```

```bash deploy-staging --dep test --descr "Deploy to staging"
./deploy.sh staging
```

```bash smoke-test --dep deploy-staging --descr "Run smoke tests"
curl https://staging.example.com/health
```

```bash deploy-production --dep smoke-test --descr "Deploy to production"
./deploy.sh production
```

Each step gates the next, ensuring quality at every stage.

Error Handling

Task Failure Behavior

When a task fails, Spry stops the execution chain:

Example:

```bash step1
echo "Step 1 complete"
```

```bash step2 --dep step1
exit 1  # This task fails
```

```bash step3 --dep step2
echo "Step 3 complete"  # Never runs
```

Output:

✓ step1 completed
✗ step2 failed with exit code 1
⊘ step3 skipped (dependency failed)

Retry Logic

Implement retry logic in your task:

```bash flaky-api-call --descr "Call flaky API with retries"
#!/bin/bash
for i in {1..3}; do
  curl https://api.example.com/data && break
  echo "Retry $i/3..."
  sleep 5
done
```

Cleanup Tasks

Always-run cleanup regardless of success/failure:

```bash setup-test-env
docker-compose up -d
```

```bash run-tests --dep setup-test-env
npm test
```

```bash cleanup --dep run-tests
docker-compose down
```

For more robust cleanup, use a wrapper script:

#!/bin/bash
trap 'docker-compose down' EXIT
./spry.ts task run-tests

Common Pitfalls

Circular Dependencies

WRONG:

```bash task-a --dep task-b
echo "A"
```

```bash task-b --dep task-a
echo "B"
```

Error:

Error: Circular dependency detected: task-a → task-b → task-a

Missing Dependencies

WRONG:

```bash deploy --dep build
./deploy.sh
```
# Oops! No "build" task defined!

Error:

Error: Task "build" referenced by "deploy" does not exist

❌ Typo in Dependency Name

WRONG:

```bash build-app
npm run build
```

```bash deploy --dep biuld-app
./deploy.sh
```

Error:

Error: Task "biuld-app" not found (did you mean "build-app"?)

Real-World Example: Full Stack App

---
title: Full Stack Deployment
---

# E-Commerce Application Deployment

## Environment Setup

```bash check-tools --descr "Verify required tools"
#!/bin/bash
command -v docker >/dev/null || { echo "Docker required"; exit 1; }
command -v node >/dev/null || { echo "Node.js required"; exit 1; }
```

## Database Tasks

```bash db-start --dep check-tools --descr "Start database container"
docker-compose up -d postgres
sleep 5  # Wait for startup
```

```bash db-migrate --dep db-start --descr "Run database migrations"
npm run db:migrate
```

```bash db-seed --dep db-migrate --descr "Seed initial data"
npm run db:seed
```

## Backend Tasks

```bash backend-install --dep check-tools
cd backend && npm install
```

```bash backend-build --dep backend-install
cd backend && npm run build
```

```bash backend-test --dep backend-build --dep db-seed
cd backend && npm test
```

## Frontend Tasks

```bash frontend-install --dep check-tools
cd frontend && npm install
```

```bash frontend-build --dep frontend-install
cd frontend && npm run build
```

```bash frontend-test --dep frontend-build
cd frontend && npm test
```

## Integration Tests

```bash integration-test --dep backend-test --dep frontend-test --dep db-seed
npm run test:integration
```

## Deployment

```bash deploy-backend --dep integration-test
./deploy-backend.sh
```

```bash deploy-frontend --dep integration-test
./deploy-frontend.sh
```

```bash deploy-complete --dep deploy-backend --dep deploy-frontend
echo "🚀 Deployment complete!"
curl https://api.example.com/health
```

## Cleanup

```bash cleanup --dep deploy-complete
docker-compose down
```

Execution Graph:

                                      ┌──→ backend-build ──→ backend-test ─┐
                     ┌─→ backend-install                                    │
                     │                                                      │
check-tools ─────────┼─→ db-start ──→ db-migrate ──→ db-seed ──────────────┼──→ integration-test ──┬──→ deploy-backend ──┐
                     │                                                      │                       │                     │
                     └─→ frontend-install ──→ frontend-build ──→ frontend-test                      └──→ deploy-frontend ─┴──→ deploy-complete ──→ cleanup

Performance Tips

1. Maximize Parallelism

Before (Sequential):

```bash task1
slow-operation-1
```

```bash task2 --dep task1
slow-operation-2
```

```bash task3 --dep task2
slow-operation-3
```

Total time: 30 seconds (10s + 10s + 10s)

After (Parallel):

```bash task1
slow-operation-1
```

```bash task2
slow-operation-2
```

```bash task3
slow-operation-3
```

```bash combine --dep task1 --dep task2 --dep task3
echo "All done!"
```

Total time: 10 seconds (all run simultaneously)

2. Cache Expensive Operations

```bash download-deps --descr "Download dependencies (cached)"
#!/bin/bash
if [ ! -d "node_modules" ]; then
  npm install
else
  echo "Using cached node_modules"
fi
```

3. Fail Fast

Put quick validation tasks early:

```bash validate-config --descr "Validate configuration"
# Fast check (runs first)
if [ ! -f "config.json" ]; then
  echo "config.json missing!"
  exit 1
fi
```

```bash expensive-build --dep validate-config
# Slow build (only runs if config valid)
npm run build
```

Debugging Task Execution

Verbose Output

./spry.ts task deploy --verbose

Shows:

Task execution order
Dependencies resolved
Timing information
Detailed error messages

Visualize DAG

./spry.ts task deploy --show-dag

Outputs ASCII art of the dependency graph.

Best Practices

✅ DO

Use descriptive task names

```bash setup-production-database --dep backup-existing-data
```

Add descriptions

```bash deploy --descr "Deploy to production with health checks"
```

Keep tasks focused
- One task = one responsibility
- Easier to reuse and test

Document complex dependencies

<!-- Migration must run before seeding -->
```bash db-seed --dep db-migrate
```

❌ DON’T

Create circular dependencies
Make tasks too granular (100 tasks for simple app)
Hide important logic in scripts (keep it visible in the task)
Forget error handling (tasks should exit with non-zero on failure)

Summary

Concept	Description
DAG	Directed Acyclic Graph - one-way dependency chain
—dep	Flag to declare dependencies
Parallel	Independent tasks run simultaneously
Topological Sort	Algorithm Spry uses to order tasks
Fail Fast	Failed tasks stop dependent tasks
Runbook	Execute all tasks in order

Key Takeaway: Spry automatically figures out the optimal execution order. You just declare what depends on what, and Spry handles the rest—including parallel execution for maximum speed!