Getting timestamp from v1 uuid

Getting timestamp from v1 uuid

Translated from J.B. Langston’s Converting TimeUUID Strings to Dates Java version to Python:

import arrow
import fileinput
import uuid

def main():
    for line in fileinput.input():
        line = line.rstrip()
        value = uuid.UUID(line)
        epoch_millis = (value.time - 0x01b21dd213814000) / 10000
        epoch = epoch_millis / 1000
        ts = arrow.get(epoch)

if __name__ == "__main__":
Posted in Uncategorized | Leave a comment

Starting a new Minecraft mod

Starting a new Minecraft mod

My kids are really into Minecraft, and are interested in mods. Here’s a quickstart to getting an OS X dev environment set up with IntelliJ IDEA.


You should have the following already installed on your machine:

  • Minecraft (with purchased license)
  • JDK
  • IntelliJ IDEA

Get Forge Mod Development Kit (MDK)

  1. Go to the Minecraft Forge downloads page.
  2. In the “Download Recommended” section, click on the “Mdk” icon. You’ll be redirected to an ad-based download page.
  3. In a few seconds, you’ll see a red “SKIP” button in the top-right, click on
    it to begin the download.

As of this writing, the recommended file is forge–1.8–

Set up a folder for development

I’m calling mine ~/dev/minecraft-potion-mod, since the initial mod I’m writing involves new potions. Here we extract it and create a new git repository:

mkdir ~/dev/minecraft-potion-mod
unzip ~/Downloads/ -d ~/dev/minecraft-potion-mod
cd ~/dev/minecraft-potion-mod
git init
git add .
git commit -m 'initial commit'
./gradlew setupDecompWorkspace


Generate the project files

./gradlew idea
  1. Open IntelliJ IDEA
  2. Click “Open”
  3. Open the generated .ipr file. In my case, it’s

IDEA will likely show you pop-ups:

  • Unregistered VCS root detected.
  • Unlinked Gradle project?

These are safe to ignore. Alternately, you can register your root git and link the gradle projects so IDEA stops nagging you.

Run Minecraft with the mod

  1. In the menu bar, to the left of the (likely grayed out) “Run” button that looks
    like a “play” triangle, click the drop-down, and choose “Minecraft Client”.
  2. Click the (now active) “Run” button.

This should launch Minecraft. If you click the “Mods” button, it should say
“Example Mod” as one of the mods.

You’re now ready for Minecraft mod development.

Posted in Uncategorized | Tagged | Leave a comment

Testing the Vert.x kinesis module

I needed a simple way to test queueing a Kinesis message with the Vert.x Kinesis module. This Jython script will do it:

import vertx
from core.event_bus import EventBus
from import JsonObject
from org.python.core.util import StringUtil

module = "com.zanox.vertx.mods~mod-kinesis~1.4.13"

def handler(_, x):
    print "Deployed kinesis module: %s" % x
    data = JsonObject()
    data.putBinary("payload", StringUtil.toBytes("this is a test"))

def send_message(msg, address="kinesis.verticle"):
    print "sending a message"
    EventBus.send(address, msg, reply_handler) 

def reply_handler(message):
    print "I received a reply %s" % message.body

vertx.deploy_module(module, vertx.config(), 1, handler)

Assuming the script is named, run it by doing:

vertx run -conf conf.json

See the module’s README for the conf.json format, should look something like this:

    "address": "kinesis.verticle",
    "streamName": "test-stream",
    "partitionKey": "partitionKey",
    "region": "us-east-1"

You’ll also need to create a stream first. You can use the AWS CLI for this:

aws kinesis create-stream --stream-name test-stream --shard-count 1
Posted in Uncategorized | Leave a comment

Using mitmproxy to log Docker API calls

The other day I needed to log the API calls made from my Docker client to debug a difference in behavior between the docker command-line tool and the docker-py Python client. Here’s how I did it.

Disable TLS on boot2docker

I’m using boot2docker on OS X. I disabled TLS to simplify things. To disable TLS:

boot2docker ssh
sudo -i
echo DOCKER_TLS="no" > /var/lib/boot2docker/profile
boot2docker start

You’ll also need to update environment variables:

export DOCKER_HOST=tcp://

The IP address of your boot2docker instance may be different, you can do boot2docker ip to find out what the IP is onyour system.

Run mitmproxy as a reverse proxy

Run mitmproxy as a reverse proxy:

mitmproxy -R

Tell your docker client to connect to mitmproxy

By default, mitmproxy listens on port 8000, so point your Docker client there:


Note that we’re hitting localhost ( and not the IP address of the boot2docker instance.

Save request/response bodies

When you interact using the docker client, the requests and responses should be captured by mitmproxy. Hit “tab” to toggle between request and response, and hit “B” to save the request or response body that you’re currently viewing.

Posted in Uncategorized | Tagged , | Leave a comment

Plotting a counting process in an IPython notebook

There are lots of interesting timeseries data in software systems: events that occur over time such as numbers of users signed up, or errors of a certain type.

I like visualizing these as a counting process, which is a cumulative count of the events over time.

Once upon a time, I would’ve done this in R, but I want to learn pandas so I can do more with IPython notebooks. However, I don’t want to give up R’s excellent ggplot2 library. Fortunately, there’s a Python port.

Here’s an IPython notebook that shows an example of a cumulative plot. The example data I used is the commit history of the OpenStack Neutron project. I suspect there’s a simpler way to do the data manipulation with pandas, but I’m just a beginner with the library.

The plot appears below:

OpenStack Neutron commits

Posted in Uncategorized | Tagged , , , | Leave a comment

Start stopped instances that all have the same name

It’s worth learning the AWS CLI and jq in order to do one-off batch operations to EC2 instances. I needed to start a group of stopped instances that all had the same name. Here’s the one-liner:

aws ec2 describe-instances --filters \
Name=tag:Name,Values="name goes here" | \
jq ".Reservations[].Instances[].InstanceId" -r | \
xargs aws ec2 start-instances --instance-ids
Posted in sysadmin | Tagged | Leave a comment

mitmproxy and ec2-api-tools

Here’s how you can mitmproxy on OS X to see which URLs the ec2-api-tools are querying against.
1. Install mitmproxy:

    sudo pip install mitmproxy
  1. Start it up:

    mitmproxy -p 8080
  2. Configure the Java keystore to trust the mitmproxy CA certificate:

    sudo keytool -importcert -alias mitmproxy -storepass "changeit" \
    -keystore /System/Library/Java/Support/CoreDeploy.bundle/Contents/Home/lib/security/cacerts \
    -trustcacerts -file ~/.mitmproxy/mitmproxy-ca-cert.pem

(Type yes when asked to trust the certificate)

  1. Configure the EC2 tools to use the mitm proxy:

    export EC2_JVM_ARGS="-DproxySet=true -DproxyHost= -DproxyPort=8080 -Dhttps.proxySet=true -Dhttps.proxyHost= -Dhttps.proxyPort=8080"
  2. Run an ec2-api command, e.g.:


Don’t forget to delete the mitmproxy CA cert when you’re done:

    sudo keytool -delete -alias mitmproxy -storepass "changeit" \
    -keystore /System/Library/Java/Support/CoreDeploy.bundle/Contents/Home/lib/security/cacerts
Posted in Uncategorized | Tagged | Leave a comment

mod_auth_openid on OSX

I wanted to play with mod_auth_openid on my Macbook Pro. OS X ships with Apache installed, so all I needed to do was build the module and edit the Apache configuration.

I wasn’t able to build mod_auth_openid from the git repository because of issues with autotools on OSX, but I was able to build from the latest release tarball (in this case, mod_auth_openid-0.7.tar.gz).

mod_auth_openid needs libopkele (a C++ OpenID library), which can be installed via Homebrew:

brew install libopkele

My initial attempt to build mod_auth_openid failed with:

/usr/share/apr-1/build-1/libtool: line 4575: /Applications/ No such file or directory

For some reason, there’s a reference in an Apache config file to a non-existent path for the C compiler. I tried to edit the file in question, but that failed to resolve the issue. In the end, I just added a symlink:

cd /Applications/
sudo ln -s XcodeDefault.xctoolchain OSX10.8.xctoolchain

Then it was just configure, make, sudo make install.

I then created a /Library/WebServer/Documents/protected directory with an index.html file inside and configured Apache to only allow access via OpenID by adding the following to /private/etc/apache2/httpd.conf:

<Directory "/Library/WebServer/Documents/protected">
    AuthType OpenID
    require valid-user

And I restarted Apache via launchctl:

sudo launchctl unload -w  /System/Library/LaunchDaemons/org.apache.httpd.plist
sudo launchctl load -w  /System/Library/LaunchDaemons/org.apache.httpd.plist

Voila! An OpenID consumer on my laptop.

Posted in Uncategorized | Tagged | Leave a comment

Networking Heisenbugs

While debugging an issue with OpenStack and floating ips, I ran into a strange issue where running tcpdump on the bridge interface on the network controller would cause packets to be forwarded successfully to a compute node, but if I stopped running tcpdump than the packets wouldn’t get forwarded.

Somebody on serverfault provided the solution: tcpdump puts the interface into promiscuous mode. And, indeed, if I set the interface into promiscuous mode, the packets got forwarded. This is a classic Heisenbug.

Posted in Uncategorized | Tagged | Leave a comment

XPath and Chrome dev tools

Here’s a simple way to get the XPath of an element on an HTML page in page.

  1. Right-click on the element on the web page, choose “Inspect Element” from the context-menu
  2. Right-click on the highlighted HTML line that appears in the Chrome Developer Tools view at the bottom of the browser window, choose “Copy XPath” from the context menu.

Very handy for use with something like Splinter’s find_by_xpath method.

Posted in Uncategorized | Tagged , | Leave a comment