Start stopped instances that all have the same name

It’s worth learning the AWS CLI and jq in order to do one-off batch operations to EC2 instances. I needed to start a group of stopped instances that all had the same name. Here’s the one-liner:

aws ec2 describe-instances --filters \
Name=tag:Name,Values="name goes here" | \
jq ".Reservations[].Instances[].InstanceId" -r | \
xargs aws ec2 start-instances --instance-ids
Posted in sysadmin | Tagged | Leave a comment

mitmproxy and ec2-api-tools

Here’s how you can mitmproxy on OS X to see which URLs the ec2-api-tools are querying against.
1. Install mitmproxy:

    sudo pip install mitmproxy
  1. Start it up:

    mitmproxy -p 8080
    
  2. Configure the Java keystore to trust the mitmproxy CA certificate:

    sudo keytool -importcert -alias mitmproxy -storepass "changeit" \
    -keystore /System/Library/Java/Support/CoreDeploy.bundle/Contents/Home/lib/security/cacerts \
    -trustcacerts -file ~/.mitmproxy/mitmproxy-ca-cert.pem
    

(Type yes when asked to trust the certificate)

  1. Configure the EC2 tools to use the mitm proxy:

    export EC2_JVM_ARGS="-DproxySet=true -DproxyHost=127.0.0.1 -DproxyPort=8080 -Dhttps.proxySet=true -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=8080"
    
  2. Run an ec2-api command, e.g.:

    ec2-describe-instances
    

Don’t forget to delete the mitmproxy CA cert when you’re done:

    sudo keytool -delete -alias mitmproxy -storepass "changeit" \
    -keystore /System/Library/Java/Support/CoreDeploy.bundle/Contents/Home/lib/security/cacerts
Posted in Uncategorized | Tagged | Leave a comment

mod_auth_openid on OSX

I wanted to play with mod_auth_openid on my Macbook Pro. OS X ships with Apache installed, so all I needed to do was build the module and edit the Apache configuration.

I wasn’t able to build mod_auth_openid from the git repository because of issues with autotools on OSX, but I was able to build from the latest release tarball (in this case, mod_auth_openid-0.7.tar.gz).

mod_auth_openid needs libopkele (a C++ OpenID library), which can be installed via Homebrew:

brew install libopkele

My initial attempt to build mod_auth_openid failed with:

/usr/share/apr-1/build-1/libtool: line 4575: /Applications/Xcode.app/Contents/Developer/Toolchains/OSX10.8.xctoolchain/usr/bin/cc: No such file or directory

For some reason, there’s a reference in an Apache config file to a non-existent path for the C compiler. I tried to edit the file in question, but that failed to resolve the issue. In the end, I just added a symlink:

cd /Applications/Xcode.app/Contents/Developer/Toolchains
sudo ln -s XcodeDefault.xctoolchain OSX10.8.xctoolchain

Then it was just configure, make, sudo make install.

I then created a /Library/WebServer/Documents/protected directory with an index.html file inside and configured Apache to only allow access via OpenID by adding the following to /private/etc/apache2/httpd.conf:

<Directory "/Library/WebServer/Documents/protected">
    AuthType OpenID
    require valid-user
</Directory>

And I restarted Apache via launchctl:

sudo launchctl unload -w  /System/Library/LaunchDaemons/org.apache.httpd.plist
sudo launchctl load -w  /System/Library/LaunchDaemons/org.apache.httpd.plist

Voila! An OpenID consumer on my laptop.

Posted in Uncategorized | Tagged | Leave a comment

Networking Heisenbugs

While debugging an issue with OpenStack and floating ips, I ran into a strange issue where running tcpdump on the bridge interface on the network controller would cause packets to be forwarded successfully to a compute node, but if I stopped running tcpdump than the packets wouldn’t get forwarded.

Somebody on serverfault provided the solution: tcpdump puts the interface into promiscuous mode. And, indeed, if I set the interface into promiscuous mode, the packets got forwarded. This is a classic Heisenbug.

Posted in Uncategorized | Tagged | Leave a comment

XPath and Chrome dev tools

Here’s a simple way to get the XPath of an element on an HTML page in page.

  1. Right-click on the element on the web page, choose “Inspect Element” from the context-menu
  2. Right-click on the highlighted HTML line that appears in the Chrome Developer Tools view at the bottom of the browser window, choose “Copy XPath” from the context menu.

Very handy for use with something like Splinter’s find_by_xpath method.

Posted in Uncategorized | Tagged , | Leave a comment

Fun with Windows TCP/IP debugging

Today I learned… In Windows, if a process listens on a port, spawns a child, then dies, then no other process can listen on that port until all of the children have been terminated. So, if you were running, say, PowerShellServer, and a process inside of an SSH session hangs, then you can’t restart it until you hunt down the process.

Thank you Server Fault for the answer, TCPView for telling me that a zombie was listening on the port, and Process Explorer for identifying the orphaned processes.

Posted in Uncategorized | Tagged | Leave a comment

Test, test

Test, test

Testing out the new Byword functionality of posting to WordPress.

Posted in Uncategorized | Leave a comment