Troubleshooting Kubernetes and GCE deployment manager

I've been using the GCE deployment manager to create a GCE deployment of a complex server application running multiple docker containers.

Here's a basic command cheatsheet:

gcloud deployment-manager deployments create my-first-deployment --config test_config.yaml
gcloud deployment-manager deployments describe my-first-deployment

Once you're at the point where the actual deployment works you probably need to debug other issues. Use the GUI to ssh into your container host VM. If you only see the /pause container, something is wrong. If you do "ps -a" you should get a list of containers that have failed to start properly (it will just keep retrying).

sudo docker ps -a

You can see the configuration Kubernetes passed to the container at creation time with "inspect". This is useful for debugging configuration problems:

sudo docker inspect [container id]

You can see STDOUT for the container launch with:

sudo docker logs [container id]

One trap I fell into is that the Kubernetes use of Cmd is different to docker :( I had a custom entrypoint in my Dockerfile and called it like this with docker:

docker run mycontainer command

But in Kubernetes config speak, cmd gets translated to docker entrypoint, and args gets translated to cmd. Ugh. So assuming your entrypoint is specified in the Dockerfile you want to leave that alone and just set the args:

containers:
  - name: mycontainer
    args: ["command"]
    env:
      - name: EXTERNAL_HOSTNAME
        value: localhost
      - name: ADMIN_PASSWORD
        value: demo

When run by Kubernetes it looks something like this:

   "Config": {
        "Hostname": "mydeployment-host",
        "Domainname": "",
        "User": "",
        "AttachStdin": false,
        "AttachStdout": false,
        "AttachStderr": false,
        "Tty": false,
        "OpenStdin": false,
        "StdinOnce": false,
        "Env": [
            "EXTERNAL_HOSTNAME=localhost",
            "ADMIN_PASSWORD=demo",
        ],
        "Cmd": [
            "command"
        ],
        "Image": "mydocker:latest",
        "Entrypoint": [
            "/mycustom-entrypoint.sh"
        ],
    }