Spinnaker & Kubernetes on macOS

Recently I was playing with the open-source release of Spinnaker, the continuous delivery tool that we use inside of Netflix. Spinnaker is designed for deploying apps on cloud backends (e.g., AWS, GCE), but I wanted to run everything entirely on my MacBook Pro.

Here’s how I set it up.


This uses Docker for running Spinnaker, and Kubernetes for the backend. The simplest way to get Docker and Kubernetes running on your local machine is:

  1. Install Docker for Mac
  2. Install corectl.app
  3. Install Kubernetes Solo cluster for macOS

Once all of these apps have been installed and running, you should have a Docker icon, Corectl icon, and Kube-Solo icon in your menu bar.

Verify that Kubernetes is running by clicking on the Kube-Solo icon and choosing “Kubernetes Dashboard”

Increase Docker memory

I had to increase the amount of memory in the Docker configuration to get Spinnaker working properly. I set mine to 8GB.

  1. Click on the Docker app in the menu bar
  2. Click the “Preferences” menu option
  3. Choose the amount of memory (8GB worked for me).
  4. Click “Apply & Restart”

Docker hub account

You will also need an account on Docker Hub, since we will configure Spinnaker to use Docker Hub as our image registry, and Spinnaker needs to know what your Docker Hub credentials are to download container images.

Grab the Spinnaker source

We will run Spinnaker from containers that have been previously uploaded to the Quay container registry, but we need the local code because it contains the Docker compose config files that we will use to run Spinnaker.

$ git clone https://github.com/spinnaker/spinnaker.git
$ cd spinnaker

Copy your kubeconfig file to config directory

Spinnaker requires your kubeconfig file which tells it how to connect to your Kubernetes deployment.

$ cp ~/kube-solo/kube/kubeconfig config/kubeconfig

The experimental/docker-compose/docker-compose.override.yml file is configured by default to mount the config directory into the clouddriver container as /opt/spinnaker/config.

Edit config/clouddriver.yml

In the Spinnaker repository that you just checked out, the config/clouddriver.yml file contains configuration for the backend cloud providers that Spinnaker uses. We need to configure it for Kubernetes and for the Docker registry.

Here’s how I configured mine, by modifying the kubernetes and dockerRegistry fields.

  enabled: true
      - name: kubesolo
        kubeconfigFile: /opt/spinnaker/config/kubeconfig
            - accountName: docker-hub

    enabled: true
        - name: docker-hub
          address: https://index.docker.io
          username: your-docker-hub-username-goes-here
          password: your-docker-hub-password-goes-here
               - library/nginx

The example above will only allow you to run containers from the nginx repository. In a real deployment, you’d add more repositories, but nginx containers are just fine for local testing.

Start up Spinnaker using Docker Compose

Switch to the directory that contains the Docker Compose config files and start up all of the Spinnaker containers:

$ cd experimental/docker-compose
$ DOCKER_IP=localhost docker-compose up -d

It will take a few minutes for Spinnaker to come up.

Open Spinnaker in your browser

Docker Compose is configured to expose the Spinnaker web UI at http://localhost:9000.

Next, you can check out the Kubernetes Source To Prod code lab on the Spinnaker website for how to launch an app.

Posted in Uncategorized | Tagged , , | 1 Comment

Go: Pointer to methods

Go: Pointer to methods

Sometimes I need a pointer to a method that is not bound to an explicit object.

Method pointers are just function pointers where the first argument is the type.

Imagine we have a Pet type with a Speak method:

type Pet interface {
    Speak() string

Here’s how we would declare, assign, and invoke the pointer

var fp func(Pet) string
fp = Pet.Speak

pet := GetPetFromSomewhere()
fmt.Sprintf("The pet says: %s\n", fp(pet))
Posted in Uncategorized | Tagged | Leave a comment

Getting timestamp from v1 uuid

Getting timestamp from v1 uuid

Translated from J.B. Langston’s Converting TimeUUID Strings to Dates Java version to Python:

import arrow
import fileinput
import uuid

def main():
    for line in fileinput.input():
        line = line.rstrip()
        value = uuid.UUID(line)
        epoch_millis = (value.time - 0x01b21dd213814000) / 10000
        epoch = epoch_millis / 1000
        ts = arrow.get(epoch)

if __name__ == "__main__":
Posted in Uncategorized | Leave a comment

Starting a new Minecraft mod

Starting a new Minecraft mod

My kids are really into Minecraft, and are interested in mods. Here’s a quickstart to getting an OS X dev environment set up with IntelliJ IDEA.


You should have the following already installed on your machine:

  • Minecraft (with purchased license)
  • JDK
  • IntelliJ IDEA

Get Forge Mod Development Kit (MDK)

  1. Go to the Minecraft Forge downloads page.
  2. In the “Download Recommended” section, click on the “Mdk” icon. You’ll be redirected to an ad-based download page.
  3. In a few seconds, you’ll see a red “SKIP” button in the top-right, click on
    it to begin the download.

As of this writing, the recommended file is forge–1.8–

Set up a folder for development

I’m calling mine ~/dev/minecraft-potion-mod, since the initial mod I’m writing involves new potions. Here we extract it and create a new git repository:

mkdir ~/dev/minecraft-potion-mod
unzip ~/Downloads/forge-1.8- -d ~/dev/minecraft-potion-mod
cd ~/dev/minecraft-potion-mod
git init
git add .
git commit -m 'initial commit'
./gradlew setupDecompWorkspace


Generate the project files

./gradlew idea
  1. Open IntelliJ IDEA
  2. Click “Open”
  3. Open the generated .ipr file. In my case, it’s

IDEA will likely show you pop-ups:

  • Unregistered VCS root detected.
  • Unlinked Gradle project?

These are safe to ignore. Alternately, you can register your root git and link the gradle projects so IDEA stops nagging you.

Run Minecraft with the mod

  1. In the menu bar, to the left of the (likely grayed out) “Run” button that looks
    like a “play” triangle, click the drop-down, and choose “Minecraft Client”.
  2. Click the (now active) “Run” button.

This should launch Minecraft. If you click the “Mods” button, it should say
“Example Mod” as one of the mods.

You’re now ready for Minecraft mod development.

Posted in Uncategorized | Tagged | Leave a comment

Testing the Vert.x kinesis module

I needed a simple way to test queueing a Kinesis message with the Vert.x Kinesis module. This Jython script will do it:

import vertx
from core.event_bus import EventBus
from org.vertx.java.core.json import JsonObject
from org.python.core.util import StringUtil

module = "com.zanox.vertx.mods~mod-kinesis~1.4.13"

def handler(_, x):
    print "Deployed kinesis module: %s" % x
    data = JsonObject()
    data.putBinary("payload", StringUtil.toBytes("this is a test"))

def send_message(msg, address="kinesis.verticle"):
    print "sending a message"
    EventBus.send(address, msg, reply_handler) 

def reply_handler(message):
    print "I received a reply %s" % message.body

vertx.deploy_module(module, vertx.config(), 1, handler)

Assuming the script is named test.py, run it by doing:

vertx run test.py -conf conf.json

See the module’s README for the conf.json format, should look something like this:

    "address": "kinesis.verticle",
    "streamName": "test-stream",
    "partitionKey": "partitionKey",
    "region": "us-east-1"

You’ll also need to create a stream first. You can use the AWS CLI for this:

aws kinesis create-stream --stream-name test-stream --shard-count 1
Posted in Uncategorized | Leave a comment

Using mitmproxy to log Docker API calls

The other day I needed to log the API calls made from my Docker client to debug a difference in behavior between the docker command-line tool and the docker-py Python client. Here’s how I did it.

Disable TLS on boot2docker

I’m using boot2docker on OS X. I disabled TLS to simplify things. To disable TLS:

boot2docker ssh
sudo -i
echo DOCKER_TLS="no" > /var/lib/boot2docker/profile
boot2docker start

You’ll also need to update environment variables:

export DOCKER_HOST=tcp://

The IP address of your boot2docker instance may be different, you can do boot2docker ip to find out what the IP is onyour system.

Run mitmproxy as a reverse proxy

Run mitmproxy as a reverse proxy:

mitmproxy -R

Tell your docker client to connect to mitmproxy

By default, mitmproxy listens on port 8000, so point your Docker client there:


Note that we’re hitting localhost ( and not the IP address of the boot2docker instance.

Save request/response bodies

When you interact using the docker client, the requests and responses should be captured by mitmproxy. Hit “tab” to toggle between request and response, and hit “B” to save the request or response body that you’re currently viewing.

Posted in Uncategorized | Tagged , | Leave a comment

Plotting a counting process in an IPython notebook

There are lots of interesting timeseries data in software systems: events that occur over time such as numbers of users signed up, or errors of a certain type.

I like visualizing these as a counting process, which is a cumulative count of the events over time.

Once upon a time, I would’ve done this in R, but I want to learn pandas so I can do more with IPython notebooks. However, I don’t want to give up R’s excellent ggplot2 library. Fortunately, there’s a Python port.

Here’s an IPython notebook that shows an example of a cumulative plot. The example data I used is the commit history of the OpenStack Neutron project. I suspect there’s a simpler way to do the data manipulation with pandas, but I’m just a beginner with the library.

The plot appears below:

OpenStack Neutron commits

Posted in Uncategorized | Tagged , , , | Leave a comment