Anyone who has spent time debugging a CI/CD pipeline knows the frustration: the pipeline fails, but you can’t reproduce it locally. You push a fix, wait 5 minutes for CI to run, it fails again on the next line, repeat. YAML pipelines (GitHub Actions, GitLab CI, Jenkins) are declarative and portable in theory, but in practice they’re brittle, hard to test locally, and tightly coupled to the CI platform.
Dagger is an attempt to fix this. Instead of YAML, your CI/CD pipeline is code — Go, Python, TypeScript, or PHP. It runs identically on your laptop and in CI, because everything runs inside containers using the Docker engine.
What Dagger Is
Dagger is a programmable CI/CD engine that runs your pipeline in containers via a GraphQL API to a container runtime (Docker or compatible). Your pipeline code calls the Dagger SDK, which translates your code into container operations.
The key insight: your CI runner becomes just another computer running Docker. The pipeline logic lives in your code, not in YAML on a CI platform. You run the exact same code locally and in CI.
Your Go/Python code → Dagger SDK → Dagger Engine → Docker containers
The Problem Dagger Solves
Consider a typical GitHub Actions workflow:
# .github/workflows/ci.yml
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: '1.24'
- name: Run tests
run: go test ./...
- name: Build
run: go build -o myapp ./cmd/myapp
To debug this locally, you’d need to simulate the GitHub Actions environment, understand which step failed, and run commands manually. You can’t just run ./ci.sh and have it work identically.
With Dagger in Go:
// ci/main.go
package main
import (
"context"
"os"
"dagger.io/dagger"
)
func main() {
ctx := context.Background()
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stdout))
if err != nil {
panic(err)
}
defer client.Close()
// Source code from current directory
src := client.Host().Directory(".")
// Build and test container
goContainer := client.Container().
From("golang:1.24-alpine").
WithDirectory("/app", src).
WithWorkdir("/app").
WithExec([]string{"go", "test", "./..."}).
WithExec([]string{"go", "build", "-o", "/app/myapp", "./cmd/myapp"})
// Get test output
_, err = goContainer.Stdout(ctx)
if err != nil {
panic(err)
}
// Export the built binary
_, err = goContainer.File("/app/myapp").Export(ctx, "./myapp")
if err != nil {
panic(err)
}
}
Run this with go run ./ci/ and it runs exactly the same in CI:
# .github/workflows/ci.yml
jobs:
ci:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run Dagger pipeline
run: go run ./ci/
env:
DAGGER_CLOUD_TOKEN: $
A More Complete Pipeline
Here’s a realistic Dagger pipeline for a Go application — build, test, lint, and publish a Docker image:
package main
import (
"context"
"fmt"
"os"
"dagger.io/dagger"
)
func main() {
ctx := context.Background()
client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stdout))
if err != nil {
panic(err)
}
defer client.Close()
src := client.Host().Directory(".", dagger.HostDirectoryOpts{
Exclude: []string{"ci/", ".git/", "*.md"},
})
// Shared Go cache for performance
goCache := client.CacheVolume("go-modules")
goContainer := client.Container().
From("golang:1.24-alpine").
WithMountedCache("/root/go/pkg/mod", goCache).
WithDirectory("/app", src).
WithWorkdir("/app")
// Lint
lintOutput, err := goContainer.
WithExec([]string{"go", "install", "github.com/golangci/golangci-lint/cmd/golangci-lint@latest"}).
WithExec([]string{"golangci-lint", "run", "--timeout", "3m"}).
Stdout(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "Lint failed: %s\n", lintOutput)
os.Exit(1)
}
// Test
testOutput, err := goContainer.
WithExec([]string{"go", "test", "-race", "-cover", "./..."}).
Stdout(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "Tests failed: %s\n", testOutput)
os.Exit(1)
}
// Build final image
appImage := client.Container().
From("gcr.io/distroless/static:nonroot").
WithFile("/app", goContainer.
WithExec([]string{"go", "build", "-ldflags", "-s -w", "-o", "/app", "./cmd/myapp"}).
File("/app")).
WithEntrypoint([]string{"/app"})
// Publish if we have credentials
if token := os.Getenv("REGISTRY_TOKEN"); token != "" {
registry := os.Getenv("REGISTRY_URL")
imageTag := fmt.Sprintf("%s/myapp:latest", registry)
_, err = appImage.
WithRegistryAuth(registry, "token", client.SetSecret("registry-token", token)).
Publish(ctx, imageTag)
if err != nil {
fmt.Fprintf(os.Stderr, "Publish failed: %v\n", err)
os.Exit(1)
}
fmt.Printf("Published: %s\n", imageTag)
}
}
Using Dagger Modules
Dagger has a module ecosystem — reusable components for common tasks. Think npm packages but for CI/CD operations:
# Browse and install modules
dagger search golang
dagger install github.com/kpenfound/dagger-modules/[email protected]
// Use the golang module
package main
import (
"context"
"golang" // Dagger module
)
type MyCi struct{}
func (m *MyCi) Test(ctx context.Context, src *dagger.Directory) (string, error) {
return dag.Golang().
WithSource(src).
Test(ctx, golang.GolangTestOpts{
Race: true,
})
}
Python and TypeScript SDKs
If Go isn’t your language, Dagger has SDKs for Python and TypeScript as well:
# ci/main.py
import sys
import anyio
import dagger
async def main():
async with dagger.Connection(dagger.Config(log_output=sys.stderr)) as client:
src = await client.host().directory(".").id()
python_container = (
client.container()
.from_("python:3.12-slim")
.with_directory("/app", client.host().directory("."))
.with_workdir("/app")
.with_exec(["pip", "install", "-r", "requirements.txt"])
.with_exec(["pytest", "-v"])
)
output = await python_container.stdout()
print(output)
anyio.run(main)
Dagger vs GitHub Actions: When to Use Each
Dagger is not a replacement for your CI platform — you still need something to trigger builds (GitHub Actions, GitLab CI, Jenkins). Dagger replaces the execution logic inside those triggers:
| Scenario | Use GitHub Actions | Use Dagger |
|---|---|---|
| Simple projects | ✓ Less complexity | |
| Complex multi-step pipelines | ✓ More debuggable | |
| Local development feedback | ✓ Runs identically | |
| Reusing pipeline logic across services | ✓ It’s just code | |
| Team unfamiliar with Docker | ✓ Lower barrier | |
| Security scanning, custom tooling | ✓ Full control |
Caching and Performance
Dagger’s caching is one of its best features. Since everything is container-based, you get fine-grained caching:
// Go module cache persists across runs
goCache := client.CacheVolume("go-modules")
// npm cache for a Node.js project
npmCache := client.CacheVolume("npm-cache")
nodeContainer := client.Container().
From("node:20-alpine").
WithMountedCache("/root/.npm", npmCache).
WithDirectory("/app", src).
WithWorkdir("/app").
WithExec([]string{"npm", "ci"}).
WithExec([]string{"npm", "test"})
Dependencies are cached between runs, making subsequent pipeline runs significantly faster than starting from scratch each time.
Conclusion
Dagger solves a real problem: pipelines that work in CI but fail locally, or that are impossible to test without pushing commits. By writing pipelines in real code with a consistent container-based runtime, you get debuggable, reproducible builds that run the same everywhere.
It’s not the right tool for every project — if your CI is simple, the added complexity isn’t worth it. But for complex pipelines with many steps, custom tooling requirements, or multiple services sharing pipeline logic, Dagger is worth a serious evaluation.
Start with the Go or Python SDK, migrate one pipeline, and run it locally. The experience of running dagger run go run ./ci/ and seeing your full CI pipeline execute on your laptop is genuinely useful.
