Generating Dash cheat sheets

I’m a big fan of the Dash documentation tool for macOS. In particular, I often create my own Dash cheat sheets to help me remember commands I frequently forget.

I make enough of these that I created a simple Go app named cheat to generate a skeleton cheat sheet. I also created an UltiSnips snippet to make it easier to generate categories and entries in cheat sheets.

snippet entry "cheatsheet entry"
entry do
    name '$1'
    notes <<-'END'
    \`\`\`
    $2
    \`\`\`
    END
end
endsnippet

snippet category "cheatsheet category"
category do
    id '$1'
end
endsnippet

On Neovim, this would go in ~/.config/nvim/UltiSnips/ruby.snippets

Paste as plain text with Quicksilver

I used to use Plain Clip for converting clipboard contents to plain text before pasting, but now I do it in one step with Quicksilver.

I created the following custom trigger:

Item: Clipbord Contents
Action: Paste as Plain Text
Shortcut: alt-v

Now if I do alt-v instead of cmd-v for pasting, it pastes as plain text.

Lock screen or sleep from Quicksilver

I recently got a new laptop, and I’ve been transferring over all of my settings. I often use Quicksilver to trigger a screen lock or put my laptop to sleep. Here’s how I do it.

I use a shell script to put my machine to sleep:

sleep.sh

#!/bin/bash
pmset sleepnow

In order to be able to trigger the shell script from Quicksilver, you’ll need the “Terminal Plugin” installed.

I use an AppleScript script to lock my screen:

lockscreen.scpt

tell application "System Events"
    set ss to screen saver "Flurry"
    start ss
end tell

 

I use the QuickSilver “Add to Catalog” action to add sleep.sh and lockscreen.scpt to the catalog. That action isn’t enabled by default, to enable it, you need to open QuickSilver preferences, click “General”, choose “Actions”, search for “Add to Catalog”, and enable it.

By default, the “Open” action has a higher precedence than the “Run” action for shell scripts. You can change the precedence order of actions by dragging, I dragged the “Run […]” action (Run a Shell Script [with optional arguments]) above the “Open” action.

 

Spinnaker & Kubernetes on macOS

Recently I was playing with the open-source release of Spinnaker, the continuous delivery tool that we use inside of Netflix. Spinnaker is designed for deploying apps on cloud backends (e.g., AWS, GCE), but I wanted to run everything entirely on my MacBook Pro.

Here’s how I set it up.

Prereqs

This uses Docker for running Spinnaker, and Kubernetes for the backend. The simplest way to get Docker and Kubernetes running on your local machine is:

1. Install Docker for Mac
2. Install corectl.app
3. Install Kubernetes Solo cluster for macOS

Once all of these apps have been installed and running, you should have a Docker icon, Corectl icon, and Kube-Solo icon in your menu bar.

Verify that Kubernetes is running by clicking on the Kube-Solo icon and choosing “Kubernetes Dashboard”

Increase Docker memory

I had to increase the amount of memory in the Docker configuration to get Spinnaker working properly. I set mine to 8GB.

1. Click on the Docker app in the menu bar
2. Click the “Preferences” menu option
3. Choose the amount of memory (8GB worked for me).
4. Click “Apply & Restart”

Docker hub account

You will also need an account on Docker Hub, since we will configure Spinnaker to use Docker Hub as our image registry, and Spinnaker needs to know what your Docker Hub credentials are to download container images.

Grab the Spinnaker source

We will run Spinnaker from containers that have been previously uploaded to the Quay container registry, but we need the local code because it contains the Docker compose config files that we will use to run Spinnaker.

$ git clone https://github.com/spinnaker/spinnaker.git
$ cd spinnaker

Copy your kubeconfig file to config directory

Spinnaker requires your kubeconfig file which tells it how to connect to your Kubernetes deployment.

$ cp ~/kube-solo/kube/kubeconfig config/kubeconfig

The experimental/docker-compose/docker-compose.override.yml file is configured by default to mount the config directory into the clouddriver container as /opt/spinnaker/config.

Edit config/clouddriver.yml

In the Spinnaker repository that you just checked out, the config/clouddriver.yml file contains configuration for the backend cloud providers that Spinnaker uses. We need to configure it for Kubernetes and for the Docker registry.

Here’s how I configured mine, by modifying the kubernetes and dockerRegistry fields.

kubernetes:
  enabled: true
  accounts:
      - name: kubesolo
        kubeconfigFile: /opt/spinnaker/config/kubeconfig
        dockerRegistries:
            - accountName: docker-hub

dockerRegistry:
    enabled: true
    accounts:
        - name: docker-hub
          address: https://index.docker.io
          username: your-docker-hub-username-goes-here
          password: your-docker-hub-password-goes-here
          repositories:
               - library/nginx

The example above will only allow you to run containers from the nginx repository. In a real deployment, you’d add more repositories, but nginx containers are just fine for local testing.

Start up Spinnaker using Docker Compose

Switch to the directory that contains the Docker Compose config files and start up all of the Spinnaker containers:

$ cd experimental/docker-compose
$ DOCKER_IP=localhost docker-compose up -d

It will take a few minutes for Spinnaker to come up.

Open Spinnaker in your browser

Docker Compose is configured to expose the Spinnaker web UI at http://localhost:9000.

Next, you can check out the Kubernetes Source To Prod code lab on the Spinnaker website for how to launch an app.

Go: Pointer to methods

Sometimes I need a pointer to a method that is not bound to an explicit object.

Method pointers are just function pointers where the first argument is the type.

Imagine we have a Pet type with a Speak method:

type Pet interface {
    Speak() string
}

Here’s how we would declare, assign, and invoke the pointer

var fp func(Pet) string
fp = Pet.Speak

pet := GetPetFromSomewhere()
fmt.Sprintf("The pet says: %s\n", fp(pet))

Getting timestamp from v1 uuid

Translated from J.B. Langston’s Converting TimeUUID Strings to Dates Java version to Python:

import arrow
import fileinput
import uuid

def main():
    for line in fileinput.input():
        line = line.rstrip()
        value = uuid.UUID(line)
        epoch_millis = (value.time - 0x01b21dd213814000) / 10000
        epoch = epoch_millis / 1000
        ts = arrow.get(epoch)
        print(ts)

if __name__ == "__main__":
    main()

Starting a new Minecraft mod

My kids are really into Minecraft, and are interested in mods. Here’s a quickstart to getting an OS X dev environment set up with IntelliJ IDEA.

Prereqs

You should have the following already installed on your machine:

• Minecraft (with purchased license)
• JDK
• IntelliJ IDEA

Get Forge Mod Development Kit (MDK)

1. Go to the Minecraft Forge downloads page.
2. In the “Download Recommended” section, click on the “Mdk” icon. You’ll be redirected to an ad-based download page.
3. In a few seconds, you’ll see a red “SKIP” button in the top-right, click on
   it to begin the download.

As of this writing, the recommended file is forge–1.8–11.14.4.1563-mdk.zip

Set up a folder for development

I’m calling mine ~/dev/minecraft-potion-mod, since the initial mod I’m writing involves new potions. Here we extract it and create a new git repository:

mkdir ~/dev/minecraft-potion-mod
unzip ~/Downloads/forge-1.8-11.14.4.1563-mdk.zip -d ~/dev/minecraft-potion-mod
cd ~/dev/minecraft-potion-mod
git init
git add .
git commit -m 'initial commit'
./gradlew setupDecompWorkspace

IntelliJ

Generate the project files

./gradlew idea

1. Open IntelliJ IDEA
2. Click “Open”
3. Open the generated .ipr file. In my case, it’s
   /Users/lorin/dev/minecraft-potion-mod/minecraft-potion-mod.ipr

IDEA will likely show you pop-ups:

• Unregistered VCS root detected.
• Unlinked Gradle project?

These are safe to ignore. Alternately, you can register your root git and link the gradle projects so IDEA stops nagging you.

Run Minecraft with the mod

1. In the menu bar, to the left of the (likely grayed out) “Run” button that looks
   like a “play” triangle, click the drop-down, and choose “Minecraft Client”.
2. Click the (now active) “Run” button.

This should launch Minecraft. If you click the “Mods” button, it should say
“Example Mod” as one of the mods.

You’re now ready for Minecraft mod development.

Testing the Vert.x kinesis module

I needed a simple way to test queueing a Kinesis message with the Vert.x Kinesis module. This Jython script will do it:

import vertx
from core.event_bus import EventBus
from org.vertx.java.core.json import JsonObject
from org.python.core.util import StringUtil

module = "com.zanox.vertx.mods~mod-kinesis~1.4.13"

def handler(_, x):
    print "Deployed kinesis module: %s" % x
    data = JsonObject()
    data.putBinary("payload", StringUtil.toBytes("this is a test"))
    send_message(data)


def send_message(msg, address="kinesis.verticle"):
    print "sending a message"
    EventBus.send(address, msg, reply_handler) 

def reply_handler(message):
    print "I received a reply %s" % message.body


vertx.deploy_module(module, vertx.config(), 1, handler)

Assuming the script is named test.py, run it by doing:

vertx run test.py -conf conf.json

See the module’s README for the conf.json format, should look something like this:

{
    "address": "kinesis.verticle",
    "streamName": "test-stream",
    "partitionKey": "partitionKey",
    "region": "us-east-1"
}

You’ll also need to create a stream first. You can use the AWS CLI for this:

aws kinesis create-stream --stream-name test-stream --shard-count 1

Using mitmproxy to log Docker API calls

The other day I needed to log the API calls made from my Docker client to debug a difference in behavior between the docker command-line tool and the docker-py Python client. Here’s how I did it.

Disable TLS on boot2docker

I’m using boot2docker on OS X. I disabled TLS to simplify things. To disable TLS:

boot2docker ssh
sudo -i
echo DOCKER_TLS="no" > /var/lib/boot2docker/profile
exit
exit
boot2docker start

You’ll also need to update environment variables:

unset DOCKER_TLS_VERIFY
unset DOCKER_CERT_PATH
export DOCKER_HOST=tcp://192.168.59.103:2375


The IP address of your boot2docker instance may be different, you can do boot2docker ip to find out what the IP is onyour system.

Run mitmproxy as a reverse proxy

Run mitmproxy as a reverse proxy:

mitmproxy -R http://192.168.59.103:2375


Tell your docker client to connect to mitmproxy

By default, mitmproxy listens on port 8000, so point your Docker client there:

export DOCKER_HOST=http://127.0.0.1:8080


Note that we’re hitting localhost (127.0.0.1) and not the IP address of the boot2docker instance.

Save request/response bodies

When you interact using the docker client, the requests and responses should be captured by mitmproxy. Hit “tab” to toggle between request and response, and hit “B” to save the request or response body that you’re currently viewing.

Plotting a counting process in an IPython notebook

There are lots of interesting timeseries data in software systems: events that occur over time such as numbers of users signed up, or errors of a certain type.

I like visualizing these as a counting process, which is a cumulative count of the events over time.

Once upon a time, I would’ve done this in R, but I want to learn pandas so I can do more with IPython notebooks. However, I don’t want to give up R’s excellent ggplot2 library. Fortunately, there’s a Python port.

Here’s an IPython notebook that shows an example of a cumulative plot. The example data I used is the commit history of the OpenStack Neutron project. I suspect there’s a simpler way to do the data manipulation with pandas, but I’m just a beginner with the library.

The plot appears below: