Troubleshooting Guide¶

This guide covers common issues with the DDEV Coder template and their solutions.

Table of Contents¶

Template Deployment Issues
Workspace Startup Failures
Docker Daemon Issues
DDEV Container Issues
Permission and File Ownership
Networking and Port Forwarding
VS Code for Web Issues
Performance Problems
Debugging Tools

Template Deployment Issues¶

Template Push Fails¶

Symptom: coder templates push fails with validation error

Check:

# Validate Terraform syntax
cd freeform
terraform init
terraform validate

# Check for syntax errors
terraform fmt -check

Common causes:

Invalid HCL syntax in template.tf
Missing required variables
Invalid Docker image reference
Registry authentication issues

Solution:

# Fix syntax errors
terraform fmt freeform/template.tf

# Test locally
cd freeform
terraform init
terraform plan

# Push with verbose output
coder templates push --directory freeform freeform --yes --verbose

Template Not Visible to Users¶

Symptom: Users don’t see template in Coder UI

Check:

# List templates
coder templates list

# Check template organization
coder templates list -o json | jq '.[] | select(.name=="freeform") | .organization_name'

# Check user's organization
coder users show <username> --json | grep organization

Solution:

Ensure template is deployed to user’s organization
Verify user has “Member” role or higher
Check template access restrictions (Enterprise only)

Docker Image Pull Fails¶

Symptom: Workspace creation fails with “image pull error”

Check:

# Test image pull locally
docker pull ddev/coder-ddev:v0.1

# Check registry authentication
docker login

# Check template image reference
grep workspace_image_registry freeform/template.tf

Solution:

# For private registries, configure template variables:
coder create --template freeform my-workspace \
  --parameter registry_username=myuser \
  --parameter registry_password=mypass \
  --yes

# Or set default credentials in template.tf

Workspace Startup Failures¶

Workspace Stuck in “Starting” State¶

Symptom: Workspace creation hangs, never reaches “Running”

Check:

# View workspace logs
coder logs my-workspace

# Check Coder agent logs
docker logs coder-<workspace-id>

# SSH into workspace (if agent started)
coder ssh my-workspace -- journalctl -u coder-agent -f

Common causes:

Startup script failure - Check Coder agent logs: coder logs my-workspace
Docker daemon not starting - Check /tmp/dockerd.log
Sysbox runtime missing - Verify host has Sysbox installed
Resource exhaustion - Check host has available CPU/RAM

Solution:

# Check startup logs (output goes to Coder agent logs)
coder logs my-workspace

# Check Docker daemon logs
coder ssh my-workspace -- cat /tmp/dockerd.log

# Check Sysbox on host
sysbox-runc --version

# Check resource availability
docker stats
df -h /coder-workspaces/

Startup Script Errors¶

Symptom: Startup script fails with permission or command errors

Check startup script: inline in the startup_script field of freeform/template.tf (the coder_agent resource)

Common issues:

Issue 1: chown fails¶

# Error: chown: cannot access '/home/coder': Permission denied

Solution: Check volume mount in template.tf:

volume {
  container_path = "/home/coder"
  host_path     = "/coder-workspaces/${data.coder_workspace_owner.me.name}-${data.coder_workspace.me.name}"
}

Issue 2: Docker daemon won’t start¶

# Error: failed to start dockerd

Solution: Check Sysbox runtime and security options:

runtime = "sysbox-runc"
security_opt = ["apparmor:unconfined", "seccomp:unconfined"]

Issue 3: DDEV not found¶

# Error: ddev: command not found

Solution: Rebuild Docker image with DDEV installed:

cd image
docker build --no-cache -t ddev/coder-ddev:v0.1 .
docker push ddev/coder-ddev:v0.1

Workspace Won’t Start After Stop¶

Symptom: Previously working workspace fails to restart

Check:

# Check workspace status
coder show my-workspace

# View logs
coder logs my-workspace

# Check for stale Docker processes
docker ps -a | grep coder

Solution:

# Delete and recreate workspace
coder delete my-workspace --yes
coder create --template freeform my-workspace --yes

# Or force restart
docker restart coder-<workspace-id>

Docker Daemon Issues¶

Docker Daemon Not Starting¶

Symptom: docker ps fails inside workspace with “Cannot connect to Docker daemon”

Check:

# Inside workspace
coder ssh my-workspace

# Check dockerd process
ps aux | grep dockerd

# Check Docker socket
ls -la /var/run/docker.sock

# Check dockerd logs
cat /tmp/dockerd.log

# Check systemd service
systemctl status docker

Common causes:

Cause 1: Sysbox not installed on host¶

# On Coder host
sysbox-runc --version

# Install if missing (no apt repo; download .deb from releases page)
# https://github.com/nestybox/sysbox/releases
SYSBOX_VERSION=0.6.7
wget https://downloads.nestybox.com/sysbox/releases/v${SYSBOX_VERSION}/sysbox-ce_${SYSBOX_VERSION}-0.linux_amd64.deb
apt-get install -y jq ./sysbox-ce_${SYSBOX_VERSION}-0.linux_amd64.deb
# Note: installer automatically restarts Docker

Cause 2: Container not using Sysbox runtime¶

# Check template.tf
grep runtime freeform/template.tf
# Should be: runtime = "sysbox-runc"

Cause 3: Missing security profiles¶

# Check template.tf
grep security_opt freeform/template.tf
# Should include:
# security_opt = ["apparmor:unconfined", "seccomp:unconfined"]

Cause 4: Docker volume not mounted¶

# Check volume mount
docker inspect coder-<workspace-id> | grep "Destination.*docker"
# Should show: /var/lib/docker mounted

# Check volume exists
docker volume ls | grep dind-cache

Solution:

# Fix template.tf and redeploy
coder templates push --directory freeform freeform --yes

# Recreate workspace
coder delete my-workspace --yes
coder create --template freeform my-workspace --yes

Docker Daemon Crashes¶

Symptom: Docker works initially but stops responding

Check:

# Inside workspace
journalctl -u docker -f

# Check disk space
df -h /var/lib/docker

# Check for OOM kills
dmesg | grep oom

# Check Docker daemon logs
cat /tmp/dockerd.log

Solution:

# To increase resources, delete and recreate workspace with higher memory
# (Back up data first: coder ssh my-workspace -- tar -czf ~/backup.tar.gz ~/projects)
coder delete my-workspace --yes
coder create --template freeform my-workspace --parameter memory=16 --yes

# Clean up Docker resources
docker system prune -a --volumes -f

# Restart Docker daemon
sudo systemctl restart docker

Docker Permission Errors¶

Symptom: permission denied while trying to connect to Docker daemon

Check:

# Inside workspace
groups  # Should include "docker"
ls -la /var/run/docker.sock
getent group docker  # Check GID

Solution:

# Check docker_gid in template.tf matches host
# Default is 988

# Add user to docker group (should be done in startup script)
sudo usermod -aG docker $USER

# Or check group_add in template.tf:
# group_add = ["988"]

DDEV Container Issues¶

DDEV Containers Won’t Start¶

Symptom: ddev start fails or containers exit immediately

Check:

# Inside workspace
ddev start --debug

# Check Docker daemon
docker ps
docker info

# Check DDEV version
ddev version

# Check for port conflicts
docker ps -a | grep ddev

Common causes:

Cause 1: Docker daemon not running¶

docker ps
# If fails: systemctl status docker

Cause 2: Insufficient resources¶

docker stats
free -h
# Increase workspace memory/CPU if needed

Cause 3: Port conflicts¶

ddev describe
# Check for conflicting projects
ddev list
ddev stop --all

Cause 4: Corrupt DDEV project¶

ddev delete --omit-snapshot
rm -rf .ddev
ddev config --auto
ddev start

Solution:

# Clean restart
ddev poweroff
ddev start

# Or rebuild
ddev delete --omit-snapshot
ddev config --auto
ddev start

DDEV Database Import Fails¶

Symptom: ddev import-db fails with memory or timeout errors

Check:

# Check workspace resources
docker stats

# Check database container logs
ddev logs db

# Check disk space
df -h

Solution:

# To increase memory, delete and recreate workspace with higher memory
# (Back up data first: ddev export-db --file=dump.sql.gz)
coder delete my-workspace --yes
coder create --template freeform my-workspace --parameter memory=16 --yes

# Split large imports
gunzip < dump.sql.gz | split -l 50000 - split_
for file in split_*; do ddev import-db --file=$file; done

# Or use mysql directly
ddev mysql < dump.sql

DDEV URLs Not Accessible¶

Symptom: DDEV project URLs don’t work

Check:

# Check DDEV status
ddev describe

# Check port forwarding in Coder UI
# Should see ports 80, 443, 8080, 8443, 8025, 8026

# Test locally in workspace
curl localhost:80

Solution:

Access via Coder port forwarding UI (not direct URLs)
Click on port links in Coder dashboard under “Apps”
DDEV URLs shown by ddev describe won’t work directly
Use Coder’s proxied URLs instead

Permission and File Ownership¶

File Ownership Issues¶

Symptom: Files owned by root or wrong UID

Check:

# Inside workspace
ls -la /home/coder
id  # Should be UID 1000

# Check volume ownership on host
ls -la /coder-workspaces/<owner>-<workspace>

Solution:

# Startup script should fix this automatically
# If not, run manually:
sudo chown -R coder:coder /home/coder

# Or on host:
chown -R 1000:1000 /coder-workspaces/<owner>-<workspace>

Sudo Doesn’t Work¶

Symptom: sudo: command not found or permission denied

Check:

# Inside workspace
which sudo
cat /etc/sudoers.d/coder
groups

Solution: Rebuild image with sudo configured:

# In image/Dockerfile
RUN echo "coder ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/coder

Can’t Write to Home Directory¶

Symptom: Permission denied writing to /home/coder

Check:

ls -la /home/coder
df /home/coder  # Check if volume mounted

Solution:

# Fix ownership
sudo chown -R coder:coder /home/coder

# Check volume mount in template.tf
# Should have:
# volume {
#   container_path = "/home/coder"
#   host_path      = "/coder-workspaces/${...}"
# }

Networking and Port Forwarding¶

Can’t Access DDEV Project¶

Symptom: Coder port forwarding shows ports but URLs don’t work

Check:

# Inside workspace
ddev describe
curl localhost:80
docker ps | grep ddev

# Check Coder port forwarding UI
# Should see forwarded ports

Solution:

Use Coder’s port forwarding links (not DDEV’s URLs)
Ensure DDEV is configured for Coder environment
Check host_webserver_port in .ddev/global_config.yaml

Port Conflicts¶

Symptom: Port already in use errors

Check:

# Inside workspace
ddev list
docker ps

# Check for conflicting projects
netstat -tlnp | grep :80

Solution:

# Stop all DDEV projects
ddev stop --all

# Or configure different ports in .ddev/config.yaml:
router_http_port: "8080"
router_https_port: "8443"

VS Code Port Forwarding Not Working¶

Symptom: VS Code can’t forward ports from workspace

Check:

# Verify service is listening
netstat -tlnp | grep <port>

# Test locally
curl localhost:<port>

Solution:

Use Coder’s port forwarding (not VS Code’s)
Configure ports in template.tf coder_app resources
VS Code port forwarding may not work with Docker-in-Docker

VS Code for Web Issues¶

VS Code Won’t Load¶

Symptom: VS Code app shows blank page or infinite loading

Check:

# Check workspace is running
coder list

# Check agent is running
coder ssh my-workspace -- echo "Agent OK"

# Check browser console for errors

Solution:

Refresh browser
Clear browser cache and cookies
Try different browser
Restart workspace: coder restart my-workspace

VS Code Extensions Won’t Install¶

Symptom: Extensions fail to install or don’t work

Check:

# Check disk space
coder ssh my-workspace -- df -h

# Check VS Code version
# (View in VS Code: Help → About)

Solution:

Some extensions require desktop VS Code
Use SSH connection with desktop VS Code:

coder config-ssh
# Then connect via VS Code Remote-SSH

VS Code Terminal Issues¶

Symptom: Terminal in VS Code doesn’t work or has wrong shell

Check:

# Inside workspace terminal
echo $SHELL
which bash zsh

# Check VS Code terminal settings

Solution:

Default shell is bash
Change in VS Code settings: Terminal › Integrated › Default Profile: Linux
Restart VS Code

Performance Problems¶

Workspace is Slow¶

Check:

# Check resource usage
docker stats coder-<workspace-id>

# Inside workspace
top
df -h
docker stats

Causes:

Insufficient resources - Increase CPU/memory
Disk I/O - Check Docker volume performance
Too many DDEV projects running - Stop unused projects

Solution:

# Increase resources
coder update my-workspace \
  --parameter cpu=8 \
  --parameter memory=16

# Stop unused DDEV projects
ddev stop --all
ddev start  # Only current project

# Clean Docker cache
docker system prune -a -f

Docker Build is Slow¶

Symptom: docker build or DDEV builds take very long

Check:

# Check Docker info
docker info | grep "Storage Driver"

# Check disk performance
dd if=/dev/zero of=/tmp/test bs=1M count=1024
rm /tmp/test

Solution:

Increase workspace resources
Use Docker BuildKit: DOCKER_BUILDKIT=1 docker build ...
Use multi-stage builds to reduce layer count
Host may need SSD for /var/lib/docker volumes

DDEV Performance Issues¶

Symptom: DDEV project is slow (page loads, composer, npm)

Check:

ddev describe
docker stats $(docker ps -q --filter name=ddev)

Solution:

# To increase resources, delete and recreate workspace
# (Back up data first: coder ssh my-workspace -- tar -czf ~/backup.tar.gz ~/projects)
coder delete my-workspace --yes
coder create --template freeform my-workspace --parameter memory=16 --yes

# Use NFS for file sharing (if Mutagen is slow)
# Edit .ddev/config.yaml:
# nfs_mount_enabled: true

# Disable xdebug when not needed
ddev xdebug off

Debugging Tools¶

Viewing Logs¶

Workspace agent logs:

# Via Coder CLI
coder logs my-workspace

# Via Docker
docker logs coder-<workspace-id>

# Inside workspace
journalctl -u coder-agent -f

Startup script logs (output goes to Coder agent logs):

coder logs my-workspace

Docker daemon logs:

coder ssh my-workspace -- cat /tmp/dockerd.log
coder ssh my-workspace -- journalctl -u docker -f

DDEV logs:

coder ssh my-workspace -- ddev logs
coder ssh my-workspace -- ddev logs db
coder ssh my-workspace -- ddev logs web

Interactive Debugging¶

SSH into workspace:

# Open shell
coder ssh my-workspace

# Run command
coder ssh my-workspace -- docker ps

# View startup logs (written to Coder agent logs)
coder logs my-workspace

Check workspace state:

# Show workspace details
coder show my-workspace

# Check container
docker inspect coder-<workspace-id>

# Check volumes
docker volume ls | grep coder
docker volume inspect coder-<owner>-<workspace>-dind-cache

Test Docker inside workspace:

coder ssh my-workspace -- docker ps
coder ssh my-workspace -- docker info
coder ssh my-workspace -- docker run hello-world

Test DDEV:

coder ssh my-workspace -- ddev version
coder ssh my-workspace -- ddev list
coder ssh my-workspace -- ddev describe

Common Error Messages¶

Error	Meaning	Solution
`failed to connect to agent`	Coder agent not running	Check startup script, restart workspace
`runtime "sysbox-runc" not found`	Sysbox not installed on host	Install Sysbox on Coder agent nodes
`permission denied` (Docker)	User not in docker group	Check `group_add` in template.tf
`no space left on device`	Disk full	Clean up Docker: `docker system prune -a`
`OOMKilled`	Out of memory	Increase workspace memory parameter
`port is already allocated`	Port conflict	Stop conflicting containers or change ports
`failed to mount volume`	Volume mount issue	Check volume paths in template.tf
`unable to pull image`	Image not found or auth failed	Check registry, credentials, image tag

Emergency Recovery¶

Workspace completely broken:

# 1. Backup data if possible
coder ssh my-workspace -- tar -czf ~/backup.tar.gz ~/projects

# 2. Copy backup to local machine (requires coder config-ssh)
scp my-workspace.coder:~/backup.tar.gz ./

# 3. Delete and recreate workspace
coder delete my-workspace --yes
coder create --template freeform my-workspace --yes

# 4. Restore data
scp ./backup.tar.gz my-workspace.coder:~/
coder ssh my-workspace -- tar -xzf ~/backup.tar.gz

Docker daemon completely broken:

# Inside workspace
sudo systemctl stop docker
sudo rm -rf /var/lib/docker/*
sudo systemctl start docker

# Or restart workspace
exit
coder restart my-workspace

Host out of resources:

# On Coder host
docker system prune -a --volumes -f
df -h
du -sh /coder-workspaces/* | sort -h
# Delete unused workspaces

/var partition full (containerd not in /data):

containerd runs as a daemon separate from Docker and has its own data root (/var/lib/containerd by default). If the server was set up without configuring containerd’s root, image snapshots accumulate in /var rather than /data. Fix:

sudo systemctl stop docker containerd
sudo mv /var/lib/containerd /data/containerd
sudo sed -i 's|^root = .*|root = "/data/containerd"|' /etc/containerd/config.toml
sudo systemctl start containerd && sudo systemctl start docker
docker ps  # verify healthy

New server setups should configure containerd’s root during initial setup — see server-setup.md.

Getting Help¶

Information to Collect¶

When reporting issues, provide:

Workspace details:

coder show my-workspace

Logs:

coder logs my-workspace > workspace-logs.txt

Template version:

grep image_version freeform/template.tf
cat VERSION

Environment:
Coder version: coder version
Docker version (in workspace): docker --version
DDEV version: ddev version
Host OS and kernel version
Error messages:
Full error output
Browser console errors (for UI issues)

Additional Resources¶

Operations Guide - Template deployment
User Management - User and permission issues
Coder Troubleshooting - Official docs
Sysbox Troubleshooting - Sysbox-specific issues
DDEV Troubleshooting - DDEV-specific issues
GitHub Issues - Report bugs or ask questions