PythonWise

Adding Key Navigation to Your Web Site

2012-02-05T20:14:00.001+02:00

I don't like using the mouse and thankful for every web site that adds keyboard navigation (like Google Reader). Be nice to your users and add some yourself.

jQuery makes it super easy to add keyboard navigation to your site, below is a simple example adding keyboard navigation. "k/j" for up/down, "o" for opening an item and "?" for toggling help. JavaScript code starts at line 90.

fastavro with Cython

2012-01-11T21:55:00.000+02:00

Added an optional step of compiling fastavro with Cython. Just doing that, with no Cython specific code reduced the time of processing 10K records from 2.9sec to 1.7sec. Not bad for that little work.

Also added a __main__.py so you can use fastavro to process Avro files:

python -m fastavro weather.avro # Dump records in JSON format
python -m fastavro --schema weather.avro # Dump schema

fastavro

2012-01-06T00:47:00.003+02:00

Just released fastavro to PyPI. It has way less features than the official avro package, but according to my tests it's about 5 times faster.

How To Use A Chat To Amplify Your Team

2011-12-27T22:31:00.001+02:00

A chat room is a great tool for any team. I've worked both in distributed and"local" team and in both cases a central chat room was one of the mosteffective tools we used. The chat room is the central location for updates,conversations, shared knowledge and more. It's also highly effective in many open source projects.

Starting is super easy, you can either set an internal server in your company with one of many open IRC/Jabber servers or try an external service (like Campfire, or HipChat).

The success of the chat room depends on the signal/noise ratio in the mainroom. Try to have every new conversation start in the main room, and only if itbecomes to noisy move it to another room. 1/1 conversations are hiding information fromthe team and should be used only for personal communication.

Don't forget add services to the chat room. It's easy to add build notification,source commits, reminders and other smart bots. We use (Jenkins Jabberplugin) to get build notification. However be careful not to generatetoo much noise and tweak all these bots.

Logging and searching is another critical feature. Some services (likeCampfire) provide it out of the box. If your chat server does not provide logging, it's pretty easy to add it with one of the many logging bots out there,and hooking search on top of the logs is pretty easy as well. (That's what wedid with selenium.saucelabs.comwhich is a combination of Supybot and Omega).

Another issue is presence, people need to make it clear when they are "in" or "out" of the room. In some cases when the server logs it's as easy as login/logout. In other cases just notify the team your are away (we use the IRC /me lunching a lot). Stating that it's acceptable to be "out" of the room allows people to close the chat when they need to be in the zone.

So go ahead, set a chat for you team. Soon you'll wonder how you managed before.

Decimals Class Decorator

2011-12-15T04:35:00.001+02:00

A friend at work had a problem where he wanted to make sure some attributes are always Decimal.

We considered using the excellent traits library, but it's an overkill and we didn't want to introduce another dependency. For me this looked like a job for properties but we wanted some way to generate the properties automatically. Lucky for us we now have class decorators (in the old days I'd probably use a metaclass).

branches

2011-12-02T04:54:00.001+02:00

At work, our workflow involves JIRA and feature branches.We name the feature branches in the name of the issue they are solving.

However when doing hg branches it's hard to know what branch you want.
Below is a small script that annotates the output of hg branches with the issue description from JIRA (and also added the branch parent).

VirtualBox and USB

2011-11-11T17:16:00.001+02:00

A little something I found out, writing it here in hope someone else will find it useful.

I'm on a Ubunbu system (11.10), and needed to connect to a USB device from an XP VM (wanted to sync and upgrade iPod touch, there is no iTunes for Linux).I couldn't connect the XP VM to any USB port (the USB menu was empty). The solution is to add yourself to the vboxusers group.

Open Users and Group, click on Manage Groups and double click on vboxusers and make sure your name is checked. After that logout and login, you should see the USB devices in the USB menu now.

Summarize tuples

2011-11-01T21:03:00.000+02:00

Raymond asked "Group the list of tuples on a given field and sum or count selected other fields.", here's my solution using sqlite3.(original answer here).

Using dbus To Control Pidgin

2011-10-28T05:02:00.000+02:00

I using Pidgin as my IM client, however after work I'd like to disconnect from our jabber server. Being the command like geek I am, I wrote the following. (Earlier version was written in bash using purple-remote.)You can learn more about Pidgin dbus interface here.

Finding Module Version

2011-10-15T07:26:00.000+02:00

A friend at work asked me how can he find which version of Python module is currently installed. The quick answer is to use pip:

pip freeze | grep <module>

Old Berlios Projects

2011-09-30T20:11:00.002+03:00

Seems like Berlios is closing down. I've placed a backup of my old projects here. Let me know if you like to take ownership on one/several of them.

Using Generators in nose Tests

2011-09-30T00:19:00.000+03:00

Sometimes you have many tests that do exactly the same thing but with different data. nose provides an easy way to do that with tests that return generators.

Feynman On The Importance Of Playing

2011-09-24T17:57:00.000+03:00

I'm reading "Surely You're Joking, Mr. Feynman" (very good read).
What he says about burnout, playing and doing the things you love is priceless:

But when it came time to do some research. I couldn't get to work. I was a little tired; I was not interested; I couldn't do research! ...

And then I thought to myself, "You know, what they think of you is so fantastic, it's impossible to live up to it. You have no responsibility to live up to it!"...

Then I had another thought; Physics disgusts me a little bit now, but I used to enjoy doing physics.Why did I enjoy it? I used to play with it. I used to do whatever I felt like doing - it didn't have to do with whether it was important for the development of nuclear physics...

So I get this new attitude ... I'm going to play with physics, whenever I want to, without worrying about any importance whatsoever.

Within a week I was in the cafeteria and some guy, fooling around, throws aplate in the air. ...

I had nothing to do, so I start to figure out the motion of the rotating plate...

And before I knew it (it was a very short time) I was "playing" - working, really - with the same old problem that I loved so much, that I had stopped working on when I went to Los Alamos; my thesis-type problems; all those old-fashioned wonderful things.

It was effortless. It was easy to play with these things. It was like uncorkinga bottle: Everything flowed out effortlessly. ...

There was no importance to what I was doing, but ultimately there was. The diagrams and the whole business that I got the Nobel Prize for came from that piddling around with the wobbling plate.

Go out and play, I'm sure you'll make wonderful things.

EDIT: Thanks for HN for proof reading this. Also found a more complete excerpt here.

istail and ipeek

2011-09-08T03:31:00.000+03:00

Prompted by a question in comp.lang.python:

"Top X" UI

2011-08-29T00:40:00.001+03:00

Doing some experiments with data at work, one of the common tasks was "show top X items". However X was changing all the time. Finally I came out with a web based UI where you can set the top yourself. Hopefully someone else will find it useful.

Key points:

Using jQuery, jQuery UI and flot
Site is static, no backend needed

Data is generated to a .js file

You can see a demo on word frequency of "Alice In Wonderland" here.

As usual, the code is in bitbucket.

Crucible Command Line Client

2011-08-18T02:43:00.001+03:00

At work, we use Crucible (which I mostly like). However we do reviews post commit (using feature branches). And creating a review from a patch is too much clicks to my taste. Hence crucible command line client. You can install it with easy_install crucible and then run it.

crucible has many options, but you can save many defaults in ~/.cruciblerc (see example at top of the crucible script).

logio

2011-07-13T01:28:00.000+03:00

The logging module is nice, one think I find myself missing here and there is the ability to treat logs as file objects. This help them "play nice" with many modules.

Here is a quick hack to do that, at the bottom there's an example on how to create CSV log files (and then we can use TimedRotatingFileHandler to rotate the logs).

PDB - The Movie

2011-07-07T06:14:00.000+03:00

I did it again, this time showing how to work with PDB (the Python debugger).

Could it be the proximity to Hollywood that causes me to do these movies?

ingress

2011-06-25T04:12:00.000+03:00

One of the many cool features in Twisted is "manhole". It lets you connect to any running server and get a shell (over ssh or telnet) in the server environment. This is very helpful when debugging.

It's very easy to implement this using SocketServer which is in the standard library, this way even if you're not running under Twised, you can still have this feature. I've created ingress just for this (easy_install ingress), using it is super easy:

import ingress
ingress.install()

The code itself is not that complicated. (Could have been simpler if there was a way to override writing to stdout in code.InteractiveConsole).

3 IDLE Tips - The Movie

2011-06-09T03:33:00.001+03:00

I've managed to make a screencast, giving tips on how to use IDLE.
(Yeah, I know - it sucks, but that's how I learn ... ;)

avrocat

2011-05-29T21:10:00.002+03:00

After playing a bit with simpleavro, I've decided it'll be better done in the Unix tradition of "doing one thing well" and using pipes. Hence avrocat was born, a "cat" like utility for avro files.

usage: avrocat [-h] [-n COUNT] [-s SKIP] [-f {json,csv}] [--header]
               [--filter FILTER] [--schema]
               filename

`cat` for Avro files

positional arguments:
  filename              avro file (- for stdin)

optional arguments:
  -h, --help            show this help message and exit
  -n COUNT, --count COUNT
                        number of records to print
  -s SKIP, --skip SKIP  number of records to skip
  -f {json,csv}, --format {json,csv}
                        record format
  --header              print CSV header
  --filter FILTER       filter records (e.g. r['age']>1)
  --schema              print schema

EDIT: Seems like avrocat is finding itself into the "avro" package.

simpleavro

2011-05-28T03:55:00.000+03:00

simpleavro 0.1.0 is out!

Delicous Monkey

2011-05-18T17:03:00.001+03:00

Sadly, the official Delicious Firefox addon does not support Firefox 4.
I couldn't find any other addon online the gives me keyboard shortcut for saving the current page in Delicious.
Lucky for me, there is GreaseMonkey. Below is a short script to have CTRL-M save the current page in Delicious.

Convert PDF to JPEG

2011-05-07T19:47:00.000+03:00

The wife needed a quick and dirty solution to convert PDF files to JPEG. Below is a quick UI (using Tkinter) about ImageMagic's "convert" utility.

A simpler named decorator

2011-04-05T21:44:00.000+03:00

Everybody loves decorators. However there's is a difference if a decorator is called with or without () (as in @logged vs @logged("foo")). In the first case, the logged will be called with the function it decorates. In the second case, logged is called with a string and should return a function that accepts the function to be decorated. Below is a simple solution to support both cases in one decorator.

A Whirlwind of Python

2011-03-26T06:15:00.001+02:00

I gave a fast paced talk about Python at work, you can download it here. (Unzip and open index.html)

Inside the zip file are also the files used to create the talk. I've used s5 as framework and generated slides from Python source files using Pygments.

Convert oggs to mp3 - the fast way

2011-03-03T02:23:00.001+02:00

I've created a quick script to convert all .ogg files in a given directory to .mp3 (using oggdec and lame). However it was running too slow to my taste, a good excuse to play with multiprocessing.Pool. On a short list of 4 .ogg files the processing time went from 32 seconds to 12 seconds.

Adding XML support to ctags

2011-03-02T22:19:00.000+02:00

I use ctags a lot with Vim. It let me jump to definitions quickly.

At work we have a lot of spring XML configuration files, and it was pretty easy to add support for XML to ctags. Just add the following to ~/.ctags.

--langdef=XML
--langmap=XML:.xml
--regex-XML=/id="([a-zA-Z0-9_]+)"/\1/d,definition/

AppEngine Work Environment

2011-02-18T18:52:00.000+02:00

I'm doing a little AppEngine project (boy, my wife is one tough customer :). On the way I've developed some scripts to help me with my work flow (which is mostly coding with Vim and trying stuff on the REPL).

I have python2.5 (which is the version AppEngine uses) at /opt/python2.5 and I've "pip installed" pyflakes and ipython. The AppEngine SDK is at /opt/google_appengine.

gae.py

repl.sh

run-local.sh

check.sh

push.sh

pypath.sh

JSON decorator for CherryPy

2011-01-31T05:22:00.000+02:00

#!/usr/bin/env python

from functools import wraps
import json
from cherrypy import response, expose

def jsonify(func):
    '''JSON decorator for CherryPy'''
    @wraps(func)
    def wrapper(*args, **kw):
        value = func(*args, **kw)
        response.headers["Content-Type"] = "application/json"
        return json.dumps(value)

    return wrapper

def example():
    from cherrypy import quickstart
    from datetime import datetime
    class Time:
        @expose
        @jsonify
        def index(self):
            now = datetime.now()
            return {
                "date" : now.strftime("%Y-%m-%d"),
                "time" : now.strftime("%H:%M"),
                "day" : now.strftime("%A"),
            }

    quickstart(Time())

if __name__ == "__main__":
    example()

$curl -i localhost:8080
HTTP/1.1 200 OK
Date: Mon, 31 Jan 2011 03:18:34 GMT
Content-Length: 56
Content-Type: application/json
Server: CherryPy/3.1.2

{"date": "2011-01-30", "day": "Sunday", "time": "19:18"}

Having hg output svn diffs

2011-01-20T00:31:00.000+02:00

In my current workplace we use Subversion as the main VCS. I use Mercurial on top of it to get easy feature branches. My problem was that we use svn patches in our review process, and mercurial (hg diff -r default) gave me different format patches. The solution (as suggested by durin42 on #mercurial) was to use hgsubversion.

After installing hgsubversion using pip (or easy_install), you need to add it to your ~/.hgrc:

    [extensions]
    hgsubversion =

Then you can create patches using "hg diff -svn -r default", I use the following genpatch script for that:

#!/bin/bash
# Generate svn style diff for current hg feature branch

branch=$(hg branch)
if [ -z $branch ]; then
    echo "error: not a mercurial repo" 1>&2
    exit 1
fi

if [ "$branch" == "default" ]; then
    echo "error: in default branch" 1>&2
    exit 1
fi

hg diff --svn -r default > ${branch}.patch

gcalc - Command line interface to Google calculator

2011-01-16T19:52:00.004+02:00

#!/usr/bin/env python
'''Command line interface to Google calculator
    gcalc 100f c -> 37.7777778 degrees Celsius
'''

# Idea taken from http://bit.ly/dVL4H3

import json
from urllib import urlopen
import re

def main(argv=None):
    import sys
    from optparse import OptionParser

    argv = argv or sys.argv

    parser = OptionParser("%prog FROM TO\n\t%prog 100f c")
    opts, args = parser.parse_args(argv[1:])
    if len(args) != 2:
        parser.error("wrong number of arguments") # Will exit

    url = "http://www.google.com/ig/calculator?q=%s=?%s" % tuple(args)
    try:
        # We decode to UTF-8 since Google sometimes return funny stuff
        result = urlopen(url).read().decode("utf-8", "ignore")
        # Convert to valid JSON: {foo: "1"} -> {"foo" : "1"}
        result = re.sub("([a-z]+):", '"\\1" :', result)
        result = json.loads(result)
    except (ValueError, IOError), e:
        raise SystemExit("error: %s" % e)

    if result["error"]:
        raise SystemExit("error: %s" % result["error"])

    print result["rhs"]

if __name__ == "__main__":
    main()

svncommit

2010-12-31T18:15:00.002+02:00

#!/bin/bash
# Template for "svn commit"
# Add "export SVN_EDITOR=/path/to/this/file" to your .bashrc

# Fail one first error
set -e

filename=$1
editor=${EDITOR-vim}

mv $filename /tmp

# The template
cat << EOF > $filename
Bug # 
Reviewed-by:

EOF

# Add file list to template
cat /tmp/$filename >> $filename
mtime=$(stat -c %y $filename)
$editor $filename

# Restore old file so svn will see it didn't change
if [ "$(stat -c %y $filename)" == "$mtime" ]; then
    mv -f /tmp/$filename $filename
fi

Delays

2010-11-03T15:15:00.000+02:00

Sorry for long pause.

I just stared a new job which is not in Python. My interested was also shifted to Clojure.

I'll try to post here more from time to time, but the frequency be lower for the seen future.

In the meanwhile, keep on hacking!

Getting last N items of iterator

2010-09-07T08:40:00.000+03:00

islice allow you to get the first N items of an iterator, but since it doesn't accept negative indexes - you can't use it to get the last N. The solution is very simple - use deque.

Configuring Ubuntu For Asus Eee

2010-08-29T01:53:00.001+03:00

Lately I lost my main laptop (belonged to the workplace), my older one allows me to use her Eee for now. The Eee came with Windows 7 starter edition, which I grew tired of fast. I've decided to try Ubuntu on it. Trying not to cause too much damage, I used Wubi.

Installing the main distro went without a problem, and after a reboot I had default Gnome desktop up. What I'm describing next is my attempt to get as much screen real estate as possible. Note that I'm spending most of my time in Firefox and in the shell.

General Setup

Delete the bottom panel (right click and "Delete This Panel")
Auto hide the top panel (right click, "Properties" and "Autohide")
Install "Gnome Do"

Firefox Setup

Right click on the toolbar, "Customize..." and check "Use small icons"
Click on the "View" menu and leave only the navigation toolbar
Uncheck the "Status bar" in the "View" menu
Install the "Hide Menubar" Firefox addon (clicking on "ALT" will show the menu)
I use GMail and Google Reader, so "Better GMail" and "Better GReader" addons helped
Install "Adblock Plus" to get more content inside web pages
This is just a personal preference, but I think installing "Chromifox Extreme" theme freed up some space as well

Terminal Setup

In the "Edit" menu click on "Profile preferences". And on the "General" tab, uncheck "Show menubar by default on new terminals" (use SHIFT-F10 to get a context menu to enable it)

Two More Things

If your kids play webkinz, install Google Chrome. It works much better there (there's always a scroll bar they can use).

As you probably figured out, I am looking for a job right now. If you know someone who is hiring, please point them to my resume.

translate

2010-08-21T01:39:00.000+03:00

curl and couchdb - a love story

2010-07-07T22:17:00.000+03:00

CouchDB API is JSON over HTTP. Which makes curl my default tool to play with the database.

Here's a small demo:

import_any

2010-06-11T03:34:00.001+03:00

Import any file as a Python module. This is a nice big security risk, so beware ...

from types import ModuleType

def import_any(path, name=None):
    module = ModuleType(name or "dummy")
    execfile(path, globals(), module.__dict__)

    return module

Stress Is Good For You

2010-06-09T16:49:00.002+03:00

Tagging last good build in git

2010-06-03T16:12:00.001+03:00

git (and mercurial) has a nice feature that you can "move" tags (using -f). At Sauce Labs we use that to have a track of our last good build and the last deployed revision.

In our continuous integration system, we mark the last good build in the last step which is executed only if the test suite passed.

tag-ok-build.sh

#!/bin/bash

# Error on 1'st failure
set -e

tag=last-green-build
revision=$(git rev-parse HEAD)

echo "Tagging $commit as $tag"
git tag -f $tag $commit
git pull origin master
git push --tags origin master

To deploy, we use the following script who also tags the last deployed version.

deploy.sh

#!/bin/bash
# Deploy, meaning sync from last successful build

tag=last-green-build

# Fetch get the latest changes from remote but does not update working
# directory, just the index
git fetch --tags

# Merge to the last successful bulid
git merge ${tag}

# Tag currently deployed revision
git tag -f deployed ${tag}
git push --tags

couchnuke

2010-05-15T03:28:00.001+03:00

A simple script to "nuke" a couchdb server, use with care!

#!/bin/bash

# Delete all databases on a couchdb server

# Exit on 1'st error
set -e

if [ $# -gt 1 ]; then
    echo "usage: $(basename $0) [DB_URL]"
    exit 1
fi

URL=${1-http://localhost:5984}

echo -n "Delete *ALL* databases on $URL? [y/n] "
read ans
if [ "$ans" != "y" ]; then
    exit
fi

for db in $(curl -s $URL/_all_dbs | tr '",[]' ' \n  ');
do
    echo "Deleting $db"
    curl -X DELETE $URL/$db
done

Run a temporary CouchDB

2010-04-12T23:14:00.002+03:00

One nice solution to isolate tests, it to point the code to a temporary database. Below is a script to launch a new instance of CouchDB on a given port.

#!/bin/bash
# A script to run a temporary couchdb instance

if [ $# -ne 1 ]; then
    echo "usage: $(basename $0) PORT"
    exit 1
fi

port=$1

base=$(mktemp -d)
dbdir=$base/data
config=$base/config.ini
mkdir -p $dbdir

cat <<EOF > $config
[httpd]
port = $port

[couchdb]
database_dir = $dbdir
view_index_dir = $dbdir

[log]
file = ${base}/couch.log
EOF

trap "rm -fr $base" SIGINT SIGTERM

echo "couchdb directory is $base"
couchdb -a $config

Sourcing a shell script

2010-04-09T02:10:00.002+03:00

Sometime you want to emulate the action of "source" in bash, settings some environment variables.
Here's one way to do it:

from subprocess import Popen, PIPE
from os import environ

def source(script, update=1):
    pipe = Popen(". %s; env" % script, stdout=PIPE, shell=True)
    data = pipe.communicate()[0]

    env = dict((line.split("=", 1) for line in data.splitlines()))
    if update:
        environ.update(env)

    return env

Showing Solr Search Results Usign jQuery

2010-03-25T20:06:00.001+02:00

Solr is a wonderful product. However it does not have a integrated search page (there's one in the admin section, but it's nothing you want to expose).

Luckily, Solr supports JSON results and even JSOP, which means we can do all the code for searching in our web page.

Note that I'm using the Solr demo that comes with the distribution, your search results might have different attributes depending on the schema.

<html>
    <head>
        <title>Solr Search</title>
    </head>
    <body>
        <h3>Solr Search</h3>

        Query: <input id="query" /> 
        <button id="search">Search</button>
        <hr />
        <div id="results">
        </div>
    </body>
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script>
    <script>
        function on_data(data) {
            $('#results').empty();

            var docs = data.response.docs;
            $.each(docs, function(i, item) {
                $('#results').prepend($('<div>' + item.name + '</div>'));
            });

            var total = 'Found ' + docs.length + ' results';
            $('#results').prepend('<div>' + total + '</div>');
        }

        function on_search() {
            var query = $('#query').val();
            if (query.length == 0) {
                return;
            }

            var url='http://localhost:8983/solr/select/?wt=json&json.wrf=?&' +
                    'q=' + query;
            $.getJSON(url, on_data);
        }

        function on_ready() {
            $('#search').click(on_search);
            /* Hook enter to search */
            $('body').keypress(function(e) {
                if (e.keyCode == '13') {
                    on_search();
                }
            });
        }

        $(document).ready(on_ready);
    </script>
</html>

To run the Solr demo do extract the distribution, then

cd example 
java -jar start.jar > /dev/null &
cd exampledocs 
./post.sh *.xml

Summarizer

2010-03-12T07:02:00.000+02:00

Just a quick note that I've uploaded the code to "Text Summarizer" here.

Parse HTTP Response

2010-02-11T06:39:00.005+02:00

Steve commented (justly) that there is no Python library function to parse HTTP transportation. Here is one way to do it:

#!/usr/bin/env python

# Parse HTTP response

from StringIO import StringIO
from httplib import HTTPResponse

class FakeSocket(StringIO):
    def makefile(self, *args, **kw):
        return self

def httpparse(fp):
    socket = FakeSocket(fp.read())
    response = HTTPResponse(socket)
    response.begin()

    return response

if __name__ == "__main__":
    from os import popen

    pipe = popen("curl -si http://google.com")
    response = httpparse(pipe)

    print response.getheaders()

Publish Buildbot builds to Twitter

2010-02-04T06:56:00.006+02:00

Buildbot exposes a simple XMLRPC protocol. Using this we can write a small daemon that pools the build status and publish it somewhere (or do other interesting stuff with it). In our case we'll publish the results to twitter using its publishing API.

#!/usr/bin/env python
# Watch builds on buildbot and publish to twitter

from time import time, sleep
from xmlrpclib import ServerProxy
from urllib import urlopen

user, password = "tebeka", "SECRET_PASSWORD"
bbot_url = "http://buildbot.example.com/xmlrpc"
tweet_url = "http://%s:%s@twitter.com/statuses/update.xml" % (user, password)

def main():
    proxy = ServerProxy(bbot_url)
    last_time = time()

    while 1:
        now = time()
        builds = proxy.getAllBuildsInInterval(last_time, now)
        for build in builds:
            builder, build, status = build[0], build[1], build[5]
            status = "OK" if status == "success" else "BROKEN"
            message = "[%s] build %s is %s" % (builder, build, status)
            urlopen(tweet_url, "status=%s" % message)

        last_time = now if builds else last_time
        sleep(10)

if __name__ == "__main__":
    main()

github-url

2010-01-15T01:34:00.001+02:00

#!/bin/bash
# Show github URL for a given file

if [ $# -ne 1 ]; then
    echo "usage: $(basename $0) FILENAME"
    exit 1
fi

if [ ! -e $1 ]; then
    echo "error: can't find $1"
    exit 1
fi

if [ -d $1 ]; then
    cd $1
    dest=.
else
    cd $(dirname $1)
    dest=$(basename $1)
fi

url=$(git config remote.origin.url | cut -d: -f2)
if [ $? -ne 0 ]; then
    echo "error: can't get remote url"
    exit 1
fi
url=${url/.git/}

path=$(git ls-files --full-name $dest | head -1)
if [ $? -ne 0 ]; then
    echo "error: git does not know about $1"
    exit 1
fi

branch=$(git branch --no-color 2>/dev/null | grep '^* ' | sed 's/* //')
prefix="https://github.com/$url"
if [ "$dest" == "." ]; then
    path=$(dirname $path)
    url="${prefix}/tree/${branch}/$path"
else
    url="${prefix}/blob/${branch}/$path"
fi

echo $url

Making TeamSpeak 3 work on Ubuntu

2009-12-30T21:06:00.010+02:00

TeamSpeak 3, you can download the Linux client from their site. The only problem is that currently (beta8) the client don't play nice with pulseaudio.

EDIT: Seems like beta-20 is working well with Ubuntu 10.4. Use the ts3client_runscript.sh (make sure you change directory to the TeamSpeak directory first).

After digging in the Ubuntu forums (can't find the exact post link), I came with the following solution:

Extract the client (cd /opt && sh ~/Downloads/TeamSpeak3-Client-linux_amd64-3.0.0-beta8.run)
Create the following script

#!/bin/bash

# Make teamspeak work on Ubuntu (basically disable pulseaudio before launching)

conf=$HOME/.pulse/client.conf
backup="${conf}.bak"

if [ -f "$conf" ]; then
    mv "$conf" "$backup"
fi
echo "autospawn = no" > "$conf"
pulseaudio --kill

cleanup() {
rm "$conf"
if [ -f "$backup" ]; then
        mv "$backup" "$conf"
fi
    pulseaudio -D
}

cd /opt/TeamSpeak3-Client-linux_amd64/
./ts3client_linux_amd64&
pid=$!
trap "kill -9 $pid; cleanup" SIGINT SIGTERM
wait $pid
cleanup

Good luck! (The other option is to use mumble.)

Multi VCS status

2009-12-29T05:47:00.003+02:00

Since I use mercurial, git, subversion and sometimes bzr. I use the following script (called st) to find the status of the current directory.

#!/bin/bash

# Repository status

root=$PWD
while [ "$root" != "/" ];
do
    if [ -d "$root/.hg" ]; then
        hg status 
        break
    fi
    if [ -d "$root/.git" ]; then
        git status --untracked-files=no
        break
    fi
    if [ -d "$root/.svn" ]; then
        svn status
        break
    fi
    if [ -d "$root/.bzr" ]; then
        bzr status
        break
    fi
    root=$(dirname "$root")
done

if [ "$root" == "/" ]; then
    echo "error: can't find a repository here"
    exit 1
fi

Add Subversion files to git

2009-12-11T02:08:00.002+02:00

I you need to start a git repository from a Subversion one, and you don't care about history. Then the following should be helpful.

#!/usr/bin/env python

# Add subversion repsotory to a git one
# Run (at top of svn repository)
#   git init
#   git-add-svn
#   git ci -a -m "Initial import from svn $(svn info | grep Revision)"

__author__ = "Miki Tebeka <miki@mikitebeka.com>"

from os import walk
from os.path import join
from fnmatch import fnmatch
from itertools import ifilter
from subprocess import call
from sys import stdout
import re

is_vcs = re.compile("\.(git|svn)", re.I).search

def make_pred(exclude):
    def is_ok(name):
        return not any((fnmatch(name, ext) for ext in exclude))
    return is_ok

def all_files():
    for root, dirs, files in walk("."):
        if is_vcs(root):
            continue
        for name in files:
            yield join(root, name)

def git_add_svn(exclude):
    pred = make_pred(exclude)
    for filename in ifilter(pred, all_files()):
        print filename
        stdout.flush()
        call(["git", "add", "-f", filename])

def main(argv=None):
    import sys
    from optparse import OptionParser

    argv = argv or sys.argv

    parser = OptionParser("%prog [options]")
    parser.add_option("-e", "--exclude", dest="exclude", action="append",
            help="extension to exclude")

    opts, args = parser.parse_args(argv[1:])
    if args:
        parser.error("wrong number of arguments") # Will exit

    git_add_svn(opts.exclude or [])


if __name__ == "__main__":
    main()

Making GMail The Default Email Application On Ubuntu

2009-12-04T02:43:00.004+02:00

Open System->Preferences->Preferred Applications
Change "Mail Reader" to "Custom"
Write bash -c 'to="%s"; gnome-open "https://mail.google.com/mail?view=cm&tf=0&to=${to#mailto:}"' in the "Command:" box

Find Cell Number Carrier

2009-11-07T19:57:00.001+02:00

#!/bin/bash

# Find the carrier for a cell number using whitepages

# Miki Tebeka <miki@mikitebeka.com>

if [ $# -ne 1 ]; then
    echo "usage: $(basename $0) NUMBER"
    exit 1
fi

# Fake Firefox
agent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) "
agent="$agent Gecko/20061204 Firefox/2.0.0.1"

url="http://www.whitepages.com/carrier_lookup?"
url="${url}carrier=other&number_0=${1}&name_1=&number_1=&name_2=&number_2="
url="${url}&name_3=&number_3=&response=1"

curl -A "$agent" -L -s "$url" | \
    grep -A1 carrier_result | tail -1 | \
    sed -e 's/.*(\(.*\))/\1/' -e 's/^ \+//'

EnSalvage

2009-10-27T07:20:00.002+02:00

I wrote a little utility to help a fried whose Entourage database got corrupted.

If anybody cares, here it is, you can even download a .app for Mac :)

binary

2009-10-23T21:01:00.000+02:00

#!/bin/bash
# Show binary representation of a number

if [ $# -ne 1 ]; then
    echo "usage: `basename $0` NUMBER" 1>&2
    exit 1
fi

echo "obase=2; $1" | bc -l

strftime

2009-10-09T16:50:00.002+02:00

#!/usr/bin/env python
'''Convert seconds since epoch to human readable time format.
   Useful in cases where logs dump time information in epoch.

    $ echo 1255099899 | strftime
    10/09/2009 07:51
    $
'''

from time import strftime, localtime
import re

def main(argv=None):
    import sys
    from optparse import OptionParser

    argv = argv or sys.argv

    default_format = "%m/%d/%Y %H:%M"
    parser = OptionParser("%prog [options] [FILENAME]")
    parser.add_option("--format", dest="format",
        help="time format [%s]" % default_format, default=default_format)
    parser.add_option("--doc", help="show strftime documentation",
        dest="doc", action="store_true", default=0)
    parser.add_option("--quote", help="quote result (e.g. '10/2/2009')",
        dest="quote", default="")

    opts, args = parser.parse_args(argv[1:])
    if len(args) not in (0, 1):
        parser.error("wrong number of arguments") # Will exit

    if opts.doc:
        import webbrowser
        url = "http://docs.python.org/library/time.html#time.strftime"
        webbrowser.open(url)
        raise SystemExit

    infile = args[0] if args else "-"
    if infile == "-":
        info = sys.stdin
    else:
        try:
            info = open(infile)
        except IOError, e:
            raise SystemExit("error: can't open %s - %s" % (infile, e))

    def callback(match):
        t = localtime(float(match.group()))
        human = strftime(opts.format, t)
        if opts.quote:
            human = human.replace(opts.quote, r"\\%s" % opts.quote)
            human = opts.quote + human + opts.quote

        return human

    # We assume 1XXXXXXXXX (2001 +) dates, so we won't convert *everything*
    print re.sub("1\d{9}(\.\d+)?", callback, info.read())


if __name__ == "__main__":
    main()

Uncomment Your Code

2009-09-28T16:29:00.005+02:00

process

2009-09-16T08:28:00.001+03:00

A combination of pgrep and pkill.

#!/usr/bin/env python
'''Find processes by regexp'''

# Miki Tebeka <miki.tebeka@gmail.com>

import re
from os import popen, getpid, kill
import signal
from sys import platform

def ask_user():
    answer = ""
    while not answer:
        answer = raw_input("kill? [y/n]").strip()
        if not answer:
            continue
        if answer.lower() in ("y", "yes"):
            return 1
        elif answer.lower() in ("n", "no"):
            return 0
        else:
            print "Please answer yes or no"

def find_signum(signal_name):
    if signal_name[:3] == "SIG":
        signum = getattr(signal, signal_name, None)
    else:
        try:
            signum = int(signal_name)
        except ValueError:
            signum = None

    return signum

def main(argv=None):
    if argv is None:
        import sys
        argv = sys.argv

    from optparse import OptionParser

    parser = OptionParser("usage: %prog [options] [REGEX]")
    parser.add_option("-k", "--kill", help="kill processes",
            dest="kill", action="store_true", default=0)
    parser.add_option("-i", "--interactive", 
        help="interactive killing (ask before each one)",
        dest="interactive", action="store_true", default=0)
    parser.add_option("-a", "--annotate", help="annotate matching lines",
        dest="annotate", action="store_true", default=0)
    parser.add_option("-s", help="signal to send", default="SIGTERM",
            action="store", dest="signal")
    parser.add_option("-9", help="send SIGKILL", dest="signal", 
            action="store_const", const="SIGKILL")
    parser.add_option("-g", "--gvim", help="kill also gvim sessions",
            action="store_true", default=0, dest="gvim")

    opts, args = parser.parse_args(argv[1:])
    if len(args) not in (0, 1):
        parser.error("wrong number of arguments") # Will exit

    regexp = args[0] if args else "."

    try:
        is_interesting = re.compile(regexp).search
    except re.error:
        raise SystemExit("error: bad regular expression - %s" % args[0])

    signal = find_signum(opts.signal)
    if signal is None:
        raise SystemExit("error: bad signal - %s" % opts.signal)

    found = 0
    first = 1
    if platform == "darwin":
        cmd = "ps ax"
        index = 0
    else:
        cmd = "ps -eff"
        index = 1

    for line in popen(cmd):
        line = line.rstrip()
        # Print header line
        if first:
            first = 0
            print line
            continue

        if (not opts.annotate) and (not is_interesting(line)):
            continue

        fields = line.split()
        pid = int(fields[index])
        if pid == getpid():
            continue

        # Color lines
        if is_interesting(line):
            found = 1
            if opts.annotate:
                print "\033[32m%s\033[0m" % line
            else:
                print line
        elif opts.annotate:
            print line

        # Kill only intersting lines
        if not is_interesting(line):
            continue
        do_kill = 0
        if opts.kill and opts.interactive:
            do_kill = ask_user()
        elif opts.kill and not opts.gvim:
            do_kill = "vim" not in line.lower()
        elif opts.kill:
            do_kill = 1

        if do_kill:
            try:
                kill(pid, signal)
                print "%d killed" % pid
            except OSError, e:
                print "error: %s" % e
        elif opts.kill:
            print "not killing %s" % pid

    if not found:
        error = "error: can't find any process matching `%s'" % args[0]
        raise SystemExit(error)

if __name__ == "__main__":
    main()

pyindent

2009-09-05T02:29:00.001+03:00

Indent C files according to PEP 7.
(uses indent)

#!/bin/sh

# Indent C files according to http://www.python.org/dev/peps/pep-0007/
# Miki Tebeka <miki.tebeka@gmail.com>

indent -cli4 -npcs -l78 -bad -bap -br -ce -nbc -di1 -psl -i4 -brs -lp -cdw -ut -ts4 $@

timeat

2009-08-15T23:04:00.003+03:00

Find local time at various locations, this is a small mashup between Google map and EarthTools.

#!/usr/bin/env python
# Small mash between Google maps API and earthtools API to get the current time
# at a given location

from urllib import urlopen, urlencode
import json
from xml.etree.cElementTree import iterparse

google_key = (
   "ABQIAAAAet8IFrmrJ_pomL8rTFXEyxSo7y1pJQguvgOA"
   "XVoQyu7YX5GThxSGUNH1P9UT48nxNUi2Ffu0FQpkWw"
)

def timeat(location):
   query = urlencode([("q", location), ("key", google_key)])
   url = "http://maps.google.com/maps/geo?%s" % query
   try:
       # Sadly, sometimes google sends request that can't be encoded in utf-8
       location = json.load(urlopen(url))
   except UnicodeDecodeError:
       return

   if location["Status"]["code"] != 200:
       return

   for placemark in location["Placemark"]:
       lng, lat, h = placemark["Point"]["coordinates"]
       place = placemark["address"]
       url = "http://www.earthtools.org/timezone/%s/%s" % (lat, lng)
       for event, element in iterparse(urlopen(url)):
           if element.tag == "localtime":
               yield place, element.text
               break

def main(argv=None):
   import sys
   from optparse import OptionParser

   argv = argv or sys.argv

   parser = OptionParser("%prog LOCATION ...")
   opts, args = parser.parse_args(argv[1:])
   if len(args) < 1:
       parser.error("wrong number of arguments") # Will exit

   location = " ".join(args)
   try:
       matches = list(timeat(location))
       if not matches:
           raise SystemExit("error: can't find time for `%s`" % location)

       for addr, time in matches:
           print "%s: %s" % (addr, time)
   except IOError, e:
       raise SystemExit("error: %s" % e)

if __name__ == "__main__":
   main()

Dynamicall Generating Static Content

2009-08-15T00:54:00.004+03:00

Sometimes you want to generate a resource (say a chart) once and then keep it. This way you don't need to keep track of all the objects you've created and can easily clean (since if someone want the resource again it will be generated again).

One way is to do it with lighttpd 404 handlers (view complete code).

lighttpd.conf

# lighttpd configuration file
server.modules  = ( "mod_cgi" )

# This is generated by by web-up
include "lighttpd.inc"

server.document-root = var.root
server.pid-file = var.root + "/lighttpd.pid"

# files to check for if .../ is requested
index-file.names            = ( "index.cgi" )

## set the event-handler (read the performance section in the manual)
# server.event-handler = "freebsd-kqueue" # needed on OS X

# mimetype mapping
mimetype.assign             = (
 ".pdf"          =>      "application/pdf",
 ".sig"          =>      "application/pgp-signature",
 ".spl"          =>      "application/futuresplash",
 ".class"        =>      "application/octet-stream",
 ".ps"           =>      "application/postscript",
 ".torrent"      =>      "application/x-bittorrent",
 ".dvi"          =>      "application/x-dvi",
 ".gz"           =>      "application/x-gzip",
 ".pac"          =>      "application/x-ns-proxy-autoconfig",
 ".swf"          =>      "application/x-shockwave-flash",
 ".tar.gz"       =>      "application/x-tgz",
 ".tgz"          =>      "application/x-tgz",
 ".tar"          =>      "application/x-tar",
 ".zip"          =>      "application/zip",
 ".mp3"          =>      "audio/mpeg",
 ".m3u"          =>      "audio/x-mpegurl",
 ".wma"          =>      "audio/x-ms-wma",
 ".wax"          =>      "audio/x-ms-wax",
 ".ogg"          =>      "application/ogg",
 ".wav"          =>      "audio/x-wav",
 ".gif"          =>      "image/gif",
 ".jpg"          =>      "image/jpeg",
 ".jpeg"         =>      "image/jpeg",
 ".png"          =>      "image/png",
 ".xbm"          =>      "image/x-xbitmap",
 ".xpm"          =>      "image/x-xpixmap",
 ".xwd"          =>      "image/x-xwindowdump",
 ".css"          =>      "text/css",
 ".html"         =>      "text/html",
 ".htm"          =>      "text/html",
 ".js"           =>      "text/javascript",
 ".asc"          =>      "text/plain",
 ".c"            =>      "text/plain",
 ".cpp"          =>      "text/plain",
 ".log"          =>      "text/plain",
 ".conf"         =>      "text/plain",
 ".text"         =>      "text/plain",
 ".txt"          =>      "text/plain",
 ".dtd"          =>      "text/xml",
 ".xml"          =>      "text/xml",
 ".mpeg"         =>      "video/mpeg",
 ".mpg"          =>      "video/mpeg",
 ".mov"          =>      "video/quicktime",
 ".qt"           =>      "video/quicktime",
 ".avi"          =>      "video/x-msvideo",
 ".asf"          =>      "video/x-ms-asf",
 ".asx"          =>      "video/x-ms-asf",
 ".wmv"          =>      "video/x-ms-wmv",
 ".bz2"          =>      "application/x-bzip",
 ".tbz"          =>      "application/x-bzip-compressed-tar",
 ".tar.bz2"      =>      "application/x-bzip-compressed-tar"
)


$HTTP["url"] =~ "\.pdf$" {
 server.range-requests = "disable"
}

# which extensions should not be handle via static-file transfer
static-file.exclude-extensions = ( ".cgi" )

# This is where the magic happens :)
$HTTP["url"] =~ "/charts/.*\.png$" {
   server.error-handler-404 = "/chart.cgi"
}

chart.cgi

#!/usr/bin/env python
'''This CGI script is called by lighttpd 404 error handler when it tries to find
/charts/XXX.png.

This way the image is generated once and then we let lighttpd serve it as static
content.
'''

# The below is needed to tell pylab to work without a screen
import matplotlib; matplotlib.use("Agg")
from pylab import hist, randn, title, savefig

from os import environ
from os.path import basename
from urlparse import urlparse

def generate_chart(name, outfile):
   # Generate a random chart
   hist(randn(1000), 100)
   title("Chart for \"%s\"" % name)
   savefig(outfile)

def main():
   import cgitb; cgitb.enable()

   o = urlparse(environ.get("REQUEST_URI", ""))
   if not o.path:
       print "Content-type: text/plain\n"
       print "ERROR: Can't find path"
       raise SystemExit

   outfile = "charts/%s" % basename(o.path)
   name = basename(o.path).replace(".png", "")
   if not name:
       print "Content-type: text/plain\n"
       print "ERROR: 'name' not specified"
       raise SystemExit

   generate_chart(name, outfile)
   image = open(outfile, "rb").read()

   print "Content-type: image/png\n"
   print image

if __name__ == "__main__":
   main()

Enabling Flash on Google Chrom (OSX)

2009-08-04T03:42:00.004+03:00

The current release of Google Chrom For Mac has support for Flash, but you need to enable it with the --enable-plugins command line switch.

Here's how to make it persistent (well, until the next update):

[17:31] ~ $cd /Applications/Google\ Chrome.app/Contents/MacOS/
[17:32] MacOS $sudo mv Google\ Chrome _chrome
[17:32] MacOS $sudo gvim Google\ Chrome
[17:33] MacOS $sudo chmod +x Google\ Chrome

And the content of the new Google Chrome is:

#!/bin/bash


"/Applications/Google Chrome.app/Contents/MacOS/_chrome" --enable-plugins $*

Happy surfing ...

VIm command to open a Python module

2009-07-29T01:50:00.003+03:00

pyhelp

2009-07-28T02:00:00.001+03:00

#!/usr/bin/env python

# Show help for Python

__author__ = "Miki Tebeka <miki.tebeka@gmail.com>"
__license__ = "BSD"

def main(argv=None):
    import sys
    from optparse import OptionParser
    import webbrowser

    argv = argv or sys.argv

    parser = OptionParser("%prog [MODULE[.FUNCITON]]")
    opts, args = parser.parse_args(argv[1:])

    if len(args) not in (0, 1):
        parser.error("wrong number of arguments") # Will exit

    root = "http://docs.python.org"

    if not args:
        url = "%s/%s" % (root, "modindex.html")
    elif "." in args[0]:
        module, function = args[0].split(".")
        url = "%s/library/%s.html#%s.%s" % (root, module, module, function)
    else:
        url = "%s/library/%s.html" % (root, args[0])

    webbrowser.open(url)

if __name__ == "__main__":
    main()

Two gnupg Wrappers

2009-07-21T03:48:00.002+03:00

encrypt

#!/bin/bash

# Encrypt input using gnupg and shred it
# THIS DELETES THE ORIGINAL FILE

err=0
for file in $@
do
    if [ $file != ${file/.gpg/} ]; then
        echo "$file already encrypted"
        continue
    fi
    if [ -d $file ]; then
        fname=$file.tar.bz2
        tar -cjf $fname $file
        find $file -type f -exec shred -u {} \;
        rm -r $file
    elif [ -f $file ]; then
        fname=$file
    else
        echo "ERROR: Unknown file type $file"
        err=1
        continue
    fi

    gpg -r miki -e $fname
    if [ $? -ne 0 ]; then
        echo "ERROR: can't encrypt"
        err=1
        continue
    fi

    shred -u $fname
    if [ $? -ne 0 ]; then
        echo "ERROR: can't shred"
        err=1
        continue
    fi
done

exit $err

decrypt

#!/bin/bash

# Decrypt a file encrypted with gpg


# Check command line
if [ $# -lt 1 ]; then
 echo "usage: `basename $0` FILE[s]"
 exit 1
fi

err=0
for file in $@
do
 if [ ! -f $file ]; then
  echo "$file: Not found"
  err=1
  continue
 fi

 # Output to current directory
 bfile=`basename $file`
    ext=`echo $file | sed -e 's/.*\.\([!.]*\)/\1/'`
    if [ "$ext" != "gpg" ] && [ "$ext" != "pgp" ]; then
  echo "Cowardly refusing to decrypt a file without gpg/pgp extension"
        err=1
        continue
 fi

 outfile=${bfile%.$ext}

    echo $file '->' $outfile
 gpg -d -o $outfile $file
 if [ $? -ne 0 ]; then
        echo "error decrypting $file"
        err=1
 fi
done

exit $err

What's My IP?

2009-07-10T06:05:00.000+03:00

#!/usr/bin/env python
'''Find local machine IP (cross platform)'''

from subprocess import Popen, PIPE
from sys import platform
import re

COMMANDS = {
    "darwin" : "/sbin/ifconfig",
    "linux" : "/sbin/ifconfig",
    "linux2" : "/sbin/ifconfig",
    "win32" : "ipconfig",
}

def my_ip():
    command = COMMANDS.get(platform, "")
    assert command, "don't know how to get IP for current platform"

    pipe = Popen([command], stdout=PIPE)
    pipe.wait()
    output = pipe.stdout.read()

    for ip in re.findall("(\d+\.\d+\.\d+\.\d+)", output):
        if ip.startswith("127.0.0"):
            continue
        return ip

if __name__ == "__main__":
    print my_ip()

Little Genetics

2009-07-03T01:59:00.001+03:00

#!/usr/bin/env python
'''Playing with genetic algorithms, see 
http://en.wikipedia.org/wiki/Genetic_programming.

The main idea that the "chromosome" represents variables in our algorithm and we
have a fitness function to check how good is it. For each generation we keep the
best and then mutate and crossover some of them.

Since the best chromosomes move from generation to generation, we cache the
fitness function results.

I'm pretty sure I got the basis for this from somewhere on the net, just don't
remeber where :)
'''

from itertools import starmap
from random import random, randint, choice
from sys import stdout

MUTATE_PROBABILITY = 0.1

def mutate_gene(n, range):
    if random() > MUTATE_PROBABILITY:
        return n

    while 1:
        # Make sure we mutated something
        new = randint(range[0], range[1])
        if new != n:
            return new

def mutate(chromosome, ranges):
    def mutate(gene, range):
        return mutate_gene(gene, range)

    while 1:
        new = tuple(starmap(mutate, zip(chromosome, ranges)))
        if new != chromosome:
            return new

def crossover(chromosome1, chromosome2):
    return tuple(map(choice, zip(chromosome1, chromosome2)))

def make_chromosome(ranges):
    return tuple(starmap(randint, ranges))

def breed(population, size, ranges):
    new = population[:]
    while len(new) < size:
        new.append(crossover(choice(population), choice(population)))
        new.append(mutate(choice(population), ranges))

    return new[:size]

def evaluate(fitness, chromosome, data, cache):
    if chromosome not in cache:
        cache[chromosome] = fitness(chromosome, data)

    return cache[chromosome]

def update_score_cache(population, fitness, data, cache):
    for chromosome in population:
        if chromosome in cache:
            continue
        cache[chromosome] = fitness(chromosome, data)

def find_solution(fitness, data, ranges, popsize, nruns, verbose=0):
    score_cache = {}
    population = [make_chromosome(ranges) for i in range(popsize)]
    for generation in xrange(nruns):
        update_score_cache(population, fitness, data, score_cache)
        population.sort(key=score_cache.get, reverse=1)
        if verbose:
            best = population[0]
            err = score_cache[best]
            print "%s: a=%s, b=%s, err=%s" % (generation, best[0], best[1], err)

        base = population[:popsize/4]
        population = breed(base, popsize, ranges)

    population.sort(key=score_cache.get, reverse=1)
    return population[0], score_cache[population[0]]

def test(show_graph=1):
    '''Try to find a linear equation a*x + b that is closest to log(x)'''
    from math import log
    xs = range(100)
    data = map(lambda i: log(i+1) * 100, xs)
    def fitness(chromosome, data):
        '''Calculate average error'''
        a, b = chromosome
        def f(x):
            return a * x + b
        values = map(f, xs)
        diffs = map(lambda i: abs(values[i] - data[i]), xs)

        # We want minimal error so return 1/error
        return 1 / (sum(diffs) / len(diffs))

    # Show a nice plot
    (a, b), err = find_solution(fitness, data, ((0, 100), (0, 100)), 10, 100, 1)
    print "best: a=%s, b=%s (error=%s)" % (a, b, err)

    data2 = map(lambda x: a * x + b, range(100))
    if not show_graph:
        return

    import pylab
    l1, l2 = pylab.plot(xs, data, xs, data2)
    pylab.legend((l1, l2), ("log(x+1)", "%s * x + %s" % (a, b)))
    pylab.show()

if __name__ == "__main__":
    test()

A Twitter Trends / Google News meshup

2009-06-12T17:21:00.002+03:00

This tried to automatically explain why things are trending on Twitter.

The Python Server (trends.py)

#!/usr/bin/env python
'''A twitter trends/google news mesh

loader_thread get news trends every 1min and set _TRENDS_HTML
The web server serves _TRENDS_HTML with is refresed via JavaScript every 30sec
'''

from urllib import urlopen, urlencode
import feedparser
from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
from os.path import dirname, join, splitext
import json
from threading import Thread
from time import sleep

def ascii_clean(text):
    return text.encode("ascii", "ignore") # Pure ACII

def trend_news(trend):
    query = {
        "q" : ascii_clean(trend),
        "output" : "rss"
    }
    url = "http://news.google.com/news?" + urlencode(query)
    return feedparser.parse(url).entries

def current_trends():
    url = "http://search.twitter.com/trends.json"
    return json.load(urlopen(url))["trends"]

def news_html(news):
    html = '<li><a href="%(link)s">%(title)s</a></li>'
    chunks = map(lambda e: html % e, news)
    return "\n".join(["<ul>"] + chunks + ["</ul>"])

def trend_html(trend):
   thtml = '<a href="%(url)s">%(name)s' % trend
   nhtml = news_html(trend_news(trend["name"]))
   return '<tr><td>%s</td><td>%s</td><tr>' % (thtml, nhtml)

def table_html(trends):
    return ("<table>" + 
            "<tr><th>Trend</th><th>Related News</th></tr>" +
            "".join(map(trend_html, trends)) +
            "</table>"
           )

_TRENDS_HTML = ""
def loader_thread():
    global _TRENDS_HTML

    while 1:
        _TRENDS_HTML = ascii_clean(table_html(current_trends()))
        sleep(60)

def run_loader_thread():
    t = Thread(target=loader_thread)
    t.daemon = 1
    t.start()

def trends_html():
    while not _TRENDS_HTML:
        sleep(0.1)

    return _TRENDS_HTML

def index_html():
    return open(join(dirname(__file__), "index.html")).read()

class RequestHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path == "/":
            self.wfile.write(index_html())
        elif self.path.startswith("/trends"):
            self.wfile.write(trends_html())
        elif splitext(self.path)[1] in (".js", ".css"):
            self.wfile.write(open(".%s" % self.path).read())
        else:
            self.send_error(404, "Not Found")

if __name__ == "__main__":
    run_loader_thread()
    server = HTTPServer(("", 8888), RequestHandler)
    server.serve_forever()

The Auto-Refreshing Web Client (index.html)

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
    <head>
        <title>Trends</title>
        <link rel="stylesheet" href="trends.css" />
        <style>
            body {
                font-family: Serif, Helvetica;
            }

            table {
                width: 100%;
                border: 2px solid gray;
            }

            td {
                vertical-align: top;
                border-bottom: 1px solid black;
            }

            h1 {
                text-align: center;
                font-variant: small-caps;
            }

            a {
                text-decoration: none;
                color: black;
            }

            a:hover {
                background: silver;
            }

            ul {
                margin: 0px;
            }
        </style>
    </head>
    <body>
        <h1>Trends</h1>
        <div id="trends">
            Loading ...
        </div>
    </body>
    <script src="jquery.js"></script>
    <script>
        function load() {
            $('#trends').load('/trends');
        }
        $(document).ready(function() {
            load();
            setInterval(load, 30 * 1000);
        });
    </script>
</html>

Show "digits only" version of phone number

2009-06-10T01:11:00.001+03:00

#!/bin/bash
# Convert phone numbers to numeric (1-800-T-MOBILE -> 1-800-8-662453)

if [ $# -ne 1 ]; then
    echo "usage: $(basename $0) PHONE-NUMBER"
    exit 1
fi

echo $1 | \
    tr '[:lower:]' '[:upper:]' | \
    tr ABCDEFGHIJKLMNOPQRSTUVWXYZ 22233344455566677778889999

JSON Pretiffier

2009-06-06T08:08:00.003+03:00

#!/usr/bin/env python
'''JSON prettifier'''

def shift(array):
   try:
       return array.pop(0)
   except IndexError:
       return None

def fopen(name, mode, default):
   if name in (None, "-"):
       return default
   return open(name, mode)

def main(argv=None):
   import sys
   from optparse import OptionParser

   argv = argv or sys.argv

   indent = 4
   parser = OptionParser("%prog [INFILE [OUTFILE]]")
   parser.add_option("-i", "--indent", help="indent size (%s)" % indent,
                     dest="indent", default=indent, type="int")
   opts, args = parser.parse_args(argv[1:])
   if len(args) not in (0, 1, 2):
       parser.error("wrong number of arguments") # Will exit

   try:
       info = fopen(shift(args), "r", sys.stdin)
       outfo = fopen(shift(args), "w", sys.stdout)
   except IOError, e:
       raise SystemExit("error: %s" % e)

   import json
   json.dump(json.load(info), outfo, indent=opts.indent)
      
if __name__ == "__main__":
   main()

Update (Aug 14, 2009): Just found out about python -mjson.tool, oh well ...

strftime for JavaScript

2009-06-04T19:15:00.001+03:00

/* strftime for JavaScript 

   Field description (taken from http://tinyurl.com/65s2qw)

    %a  Locale’s abbreviated weekday name.   
    %A  Locale’s full weekday name.   
    %b  Locale’s abbreviated month name.   
    %B  Locale’s full month name.   
    %c  Locale’s appropriate date and time representation.   
    %d  Day of the month as a decimal number [01,31].   
    %H  Hour (24-hour clock) as a decimal number [00,23].   
    %I  Hour (12-hour clock) as a decimal number [01,12].   
    %j  Day of the year as a decimal number [001,366].   
    %m  Month as a decimal number [01,12].   
    %M  Minute as a decimal number [00,59].   
    %p  Locale’s equivalent of either AM or PM.
    %S  Second as a decimal number [00,61].
    %U  Week number of the year (Sunday as the first day of the week) as a
        decimal number [00,53]. All days in a new year preceding the first
        Sunday are considered to be in week 0.
    %w  Weekday as a decimal number [0(Sunday),6].   
    %W  Week number of the year (Monday as the first day of the week) as a
        decimal number [00,53]. All days in a new year preceding the first
        Monday are considered to be in week 0.
    %x  Locale’s appropriate date representation.   
    %X  Locale’s appropriate time representation.   
    %y  Year without century as a decimal number [00,99].   
    %Y  Year with century as a decimal number.   
    %Z  Time zone name (no characters if no time zone exists).   
    %%  A literal '%' character.
*/

var days = [ 
    'Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday',
    'Saturday' 
];

var months = [
    'January', 'February', 'March', 'April', 'May', 'June', 'July',
    'August', 'September', 'October', 'November', 'December'
];

function shortname(name) {
    return name.substr(0, 3);
}

function zeropad(n, size) {
    n = '' + n; /* Make sure it's a string */
    size = size || 2;
    while (n.length < size) {
        n = '0' + n;
    }
    return n;
}

function twelve(n) {
    return (n <= 12) ? n : 24 - n;
}

function strftime(format, date) {
    date = date || new Date();
    var fields = {
        a: shortname(days[date.getDay()]),
        A: days[date.getDay()],
        b: shortname(months[date.getMonth()]),
        B: months[date.getMonth()],
        c: date.toString(),
        d: zeropad(date.getDate()),
        H: zeropad(date.getHours()),
        I: zeropad(twelve(date.getHours())),
        /* FIXME: j: */
        m: zeropad(date.getMonth() + 1),
        M: zeropad(date.getMinutes()),
        p: (date.getHours() >= 12) ? 'PM' : 'AM',
        S: zeropad(date.getSeconds()),
        w: zeropad(date.getDay() + 1),
        /* FIXME: W: */
        x: date.toLocaleDateString(),
        X: date.toLocaleTimeString(),
        y: ('' + date.getFullYear()).substr(2, 4),
        Y: '' + date.getFullYear(),
        /* FIXME: Z: */
        '%' : '%'
    };

    var result = '', i = 0;
    while (i < format.length) {
        if (format[i] === '%') {
            result = result + fields[format[i + 1]];
            ++i;
        }
        else {
            result = result + format[i];
        }
        ++i;
    }
    return result;
}

Quick search using xapian "omega"

2009-06-04T07:42:00.001+03:00

A little hack to search current directory using Xapian's omega.

#!/bin/bash

# Minimal search using Xpians Omega (http://xapian.org/docs/omega/overview.html)
# Use "omsearch -i" to index current directoy, then use "omsearch QUERY" to
# search


index() {
    mkdir .omega
    cat >> .omega/omega.conf << EOF
    database_dir $PWD/.omega
    template_dir /var/lib/xapian-omega/templates
    log_dir /tmp
EOF
    omindex --db .omega/default .
}

search() {
    omega=/usr/lib/cgi-bin/omega/omega
    config=$PWD/.omega/omega.conf
    results=.omega/results.html
    OMEGA_CONFIG_FILE=$config $omega "P=$1" "HITSPERPAGE=1000" > $results
    egrep -o '<B><A HREF=".*"' $results | cut -d\" -f2 | \
        xargs -i printf ".%s\n" {}

}

USAGE="usage: $(basename $0) [-i | QUERY]"

index=no
while getopts "ih" opt
do
    case $opt in
        h | H ) echo $USAGE; exit;;
        i ) index=yes;;
        * ) echo "error: unknown option" >&2; exit 1;;
    esac
done

shift $(($OPTIND - 1))

if [ "$index" == "yes" ]; then
    if [ $# -gt 0 ]; then
        echo "error: -i don't take any parameters"
        exit 1
    fi
    index
    exit
fi

if [ $# -ne 1 ]; then
    echo $USAGE
    exit 1
fi

if [ ! -d .omega ]; then
    echo "error: can't find .omega, run with -i to index first"
    exit 1
fi
search $1

Avoiding "peek" with "itertools.chain"

2009-05-11T18:39:00.003+03:00

Sometimes you need to filter out some header information. However the only clear anchor you have is the start of real data - you need a "peek" function.

It's easy to avoid writing the "peek" logic, just use itertools.chain.

#!/usr/bin/env python
'''Avoiding the need for "peek" with itertools.chain'''

from itertools import chain

def is_first_data(line):
    return line.startswith("Name:")

def skip_header(data):
    data = iter(data)
    for line in data:
        if is_first_data(line):
            return chain([line], data)

    # FIXME: We might want to raise something here
    return []


if __name__ == "__main__":
    data = [
        "this is the header",
        "it might change every time",
        "and you'll never find a good regexp for it",
        "The first line of data is easy to know",
        "Name: Duffy",
        "Type: Duck",
        "Anger: 10",
        "",
        "Name: Bugs",
        "Type: Bunny",
        "Anger: 0",
    ]

    data = skip_header(data)
    for line in data:
        print line

Subversion IRC Bot

2009-05-01T23:55:00.002+03:00

#!/usr/bin/env python
'''Push changes in subversion repository to IRC channel'''

from twisted.words.protocols import irc
from twisted.internet.protocol import ReconnectingClientFactory
import re
from subprocess import Popen, PIPE
from xml.etree.cElementTree import parse as xmlparse
from cStringIO import StringIO

class IRCClient(irc.IRCClient):
    nickname = "svnbot"
    realname = "Subversion Bot"
    channel = "#dev"

    instance = None # Running instance

    def signedOn(self):
        IRCClient.instance = self
        self.join(self.channel)

    def svn(self, revision, author, comment):
        comment = (comment[:57] + "...") if  len(comment) > 60 else comment
        message = "SVN revision %s by %s: %s" % (revision, author, comment)
        self.say(self.channel, message)


class SVNPoller:
    def __init__(self, root, user, password):
        self.pre = ["svn", "--xml", "--username", user, "--password", password]
        self.root = root
        self.last_revision = self.get_last_revision()

    def check(self):
        if not IRCClient.instance:
            return

        try:
            last_revision = self.get_last_revision()
            if (not last_revision) or (last_revision == self.last_revision):
                return

            for rev in range(self.last_revision + 1, last_revision + 1):
                author, comment = self.revision_info(rev)
                IRCClient.instance.svn(rev, author, comment)
            self.last_revision = last_revision
        except Exception, e:
            print "ERROR: %s" % e

    def svn(self, *cmd):
        pipe = Popen(self.pre +  list(cmd) + [self.root], stdout=PIPE)
        try:
            data = pipe.communicate()[0]
        except IOError:
            data = ""
        return xmlparse(StringIO(data))

    def get_last_revision(self):
        tree = self.svn("info")
        revision = tree.find("//commit").get("revision")
        return int(revision)

    def revision_info(self, revision):
        tree = self.svn("log", "-r", str(revision))
        author = tree.find("//author").text
        comment = tree.find("//msg").text

        return author, comment

if __name__ == "__main__":
    from twisted.internet import reactor
    from twisted.internet.task import LoopingCall

    factory = ReconnectingClientFactory()
    factory.protocol = IRCClient
    reactor.connectTCP("irc.mycompany.com", 6667, factory)

    poller = SVNPoller("http://svn.mycompany.com", "bugs", "carrot")
    task = LoopingCall(poller.check)
    task.start(1)

    reactor.run()

Solving Euler Question 24

2009-04-22T03:29:00.002+03:00

I'm learning Clojure by solving project Euler. Here is a python version of one of the solutions.

#!/usr/bin/env python
''' Solving http://projecteuler.net/index.php?section=problems&id=24

Note that Python's 2.6 itertools.permutations return the permutation in order so
we can just write:
    from itertools import islice, permutations
    print islice(permutations(range(10)), 999999, None).next()

And it'll work much faster :)
'''

from itertools import islice, ifilter

def is_last_permutation(n):
    return n == sorted(n, reverse=1)

def next_head(n):
    '''Find next number to be 'head'. 

    It is smallest number if the tail that is bigger than the head.
    In the case of (2 4 3 1) it will pick 3 to get the next permutation
    of (3 1 2 4)
    '''
    return sorted(filter(lambda i: i > n[0], n[1:]))[0]

def remove(element, items):
    return filter(lambda i: i != element, items)

def next_permutation(n):
    if is_last_permutation(n):
        return None

    sub = next_permutation(n[1:])
    if sub:
        return [n[0]] + sub

    head = next_head(n)
    return [head] + sorted([n[0]] + remove(head, n[1:]))

def nth(it, n):
    '''Return the n'th element of an iterator'''
    return islice(it, n, None).next()

def iterate(func, n):
    '''iterate(func, n) -> n, func(n), func(func(n)) ...'''
    while 1:
        yield n
        n = func(n)

def permutations(n):
    return ifilter(None, iterate(next_permutation, n))

if __name__ == "__main__":
    n = range(10)
    m = 1000000
    print "Calculateing the %d permutation of %s" % (m, n)
    print nth(permutations(n), m-1)

pmap

2009-04-03T05:33:00.002+03:00

#!/usr/bin/env python
'''Parallel map (Unix only)'''

__author__ = "Miki Tebeka <miki.tebeka@gmail.com>"

# The big advantage of this implementation is that "fork" is very fast on
# copying data, so if you pass big arrays as arguments and return small values
# this is a win

# FIXME 
# * For too many items, we get "Too many open files"
# * Handle child exceptions

from os import fork, pipe, fdopen, waitpid, P_WAIT
from marshal import dump, load
from itertools import takewhile, count

def spawn(func, data):
    read_fo, write_fo = map(fdopen, pipe(), ("rb", "wb"))
    pid = fork()
    if pid: # Parent
        return pid, read_fo

    # Child
    dump(func(data), write_fo)
    write_fo.close()
    raise SystemExit

def wait(child):
    pid, fo = child
    waitpid(pid, P_WAIT)
    value = load(fo)
    fo.close()
    return value

def pmap(func, items):
    '''
    >>> pmap(lambda x: x * 2, range(5))
    [0, 2, 4, 6, 8]
    '''
    children = map(lambda item: spawn(func, item), items)
    return map(wait, children)

if __name__ == "__main__":
    def fib(n):
        a, b = 1, 1
        while n > 1:
            a, b = b, a + b
            n -= 1
        return b

    items = range(10)
    print "pmap(fib, %s)" % str(items)
    print pmap(fib, items)

subclasses

2009-03-17T01:25:00.003+02:00

New style classes have a __subclasses__ method which I find useful the problem of "find me all instances to run" - without any metaclass black voodoo :)

A trivial example:

#!/usr/bin/env python

class Animal(object):
    pass

class Dog(Animal):
    def talk(self):
        return "whoof whoof"

class Cat(Animal):
    def talk(self):
        return "miao"

class Pig(Animal):
    def talk(self):
        return "oink oink"

def all_animals():
    return Animal.__subclasses__()

if __name__ == "__main__":
    for animal in all_animals():
        print "%s says: %s" % (animal.__name__, animal().talk())

This way, if we add a new animal all_animals will work without any change from us.

glue

2009-03-11T07:33:00.003+02:00

Sometime you need to produce a quick prototype for a site that is an aggregation of several other sites.

One quick way is to glue the parts you want from each site into the page.
We'll use a more relaxed version of trampoline to overcome cross site scripting.

redirect.cgi

#!/usr/bin/env python
'''Get content of "foreign" web pages, enable cross-site AJAX'''

from urllib2 import urlopen, URLError
from cgi import FieldStorage

if __name__ == "__main__":
   import cgitb; cgitb.enable()

   form = FieldStorage()
   url = form.getvalue("url", "")
   if not url:
       raise SystemExit("no url")

   try:
       o = urlopen(url)
   except (URLError, ValueError), e:
       raise SystemExit("error: %s" % e)

   print "Content-type: %(content-type)s\n" % o.headers
   print o.read()

And then in our page we'll glue the parts. I'm using the same part from Weather Underground but with different area. (Ignoring the fact they have a good API :)

index.html

<html>
   <head>
       <title></title>
       <style>
           h1.title {
               text-align: center;
           }
           .header {
               background: blue;
               color: white;
               font-size: 1.3em;
               font-weight: bold;
               font-variant: small-caps;
           }
           div.left {
               float: left;
               width: 50%;
               height: 400px;
               overflow: auto;
           }
           div.right {
               float: right;
               width: 50%;
               height: 400px;
               overflow: auto;
           }
       </style>
   </head>
   <body>
       <h1 class="title">Weather Meshup</h1>

       <div>
           <div class="left">
               <div class="header">Beverly Hills</div>
               <div id="bh">Loading ...</div>
           </div>

           <div class="right">
               <div class="header">New York</div>
               <div id="ny">Loading ...</div>
           </div>
       </div>
       <div>
           <div class="left">
               <div class="header">Chicago</div>
               <div id="ch">Loading ...</div>
           </div>

           <div class="right">
               <div class="header">Washington</div>
               <div id="wa">Loading ...</div>
           </div>
       </div>
   </body>
   <script src="jquery.js"></script>
   <script>
       function weather_url(zipcode) {
           return 'http://www.wund.com/cgi-bin/findweather/getForecast' +
                  '?query=' + zipcode;
       }

       function load_foreign(id, url, selector) {
           $('#' + id).load('redirect.cgi ' + selector, {url: url});
       }

       function load_weather(id, zipcode) {
           var url = weather_url(zipcode);
           load_foreign(id, url, 'div#conditions');
       }


       function on_ready()
       {
           load_weather('bh', '90210');
           load_weather('ny', '10001');
           load_weather('ch', '60290');
           load_weather('wa', '20001');
       }

       $(document).ready(on_ready);
   </script>
</html>

@reboot

2009-02-26T18:45:00.003+02:00

cron has a nice special rule called @reboot which will run every time the machine is rebooted (doh!).

I use it for two main things:

Notify me when one of my servers was rebooted (sometimes IT don't tell me in advance)
Run my services. This is easier than writing an /etc/init.d scripts and I store my servers crontab in the source control, so deploying a new machine is easier.

An example:

# Notify that machine was rebooted
@reboot /path/to/mail -s "Machine $HOSTNANE rebooted" me@somewhere.com < /dev/null
# Run my server
@reboot (cd /path/to/awesome/server/directory && ./run_server)

Where run_server is usually something like:

#!/bin/bash

nohup ./server.py&

calc

2009-02-17T07:41:00.005+02:00

A very simple command line/GUI calculator. The main trick here is to export all of math to the global namespace and then just eval the expression.

twiver

2009-02-12T07:28:00.002+02:00

A Twitter search client the updates every 10min.

#!/usr/bin/env python

from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
from urlparse import urlparse
from urllib2 import urlopen
from operator import itemgetter
from rfc822 import parsedate_tz
from time import strftime
import re
from functools import partial

try:
  import json
except ImportError:
  import simplejson as json

HTML = '''
<html>
    <head>
        <title>Twiver - Refreshing Twitter Search</title>
        <style>
            h2 {
                font-variant: small-caps;
            }
            table#results {
                border: 1px solid black;
                width: 100%;
            }
            table#results tr:hover {
                background: silver;
            }
            span#updated {
                font-family: Monospace;
            }
        </style>
    </head>
    <body>
        <h2>Twiver - Refreshing Twitter Search</h2>
        Query: <input id="query" size="60" /> <button id="run">Go</button>
        <table id="results">
        </table>
        <span id="updated"></span>
        <hr />
        By <a href="mailto:miki.tebeka@gmail.com">Miki</a>
    </body>
    <script src="jquery.js"></script>
    <script>
        var running = 0;

        function handle_result(data) {
            if (!running) {
                return;
            }

            var table = $('#results');
            table.empty();
            $.each(data, function (i, text) {
                var tr = $('<tr><td>' + text + '</td></tr>');
                table.append(tr);
            });
            $('#updated').html('Updated: ' + new Date());

            setTimeout(update, 10 * 1000);
        }

        function update() {
            var query = $.trim($('#query').val());
            url = '/search?q=' + query;
            $.getJSON(url, handle_result);
        }

        function run() {
            var button = $('#run');
            if (button.text() == "Go") {
                var query = $.trim($('#query').val());
                /* FIXME: The best way will be to disable the button until there
                  is text
                */
                if (query.length == 0) {
                    alert("Please enter *something*");
                    return;
                }
                button.text("Stop");
                running = 1;
                update();
            }
            else {
                running = 0;
                button.text("Go");
            }
        }

        function on_ready()
        {
            $('#run').click(run);
        }
        $(document).ready(on_ready);
    </script>
</html>
'''

def format_time(time):
  # Wed, 11 Feb 2009 00:10:36 +0000
  time = parsedate_tz(time)
  return strftime("%m/%d/%Y %H:%M", time[:9])

# 'http://mikitebeka.com' -> 
# '<a href="http://mikitebeka.com">http://mikitebeka.com</a>'
inject_links = partial(
  re.compile("(http://[^ ]+)").sub,
  "<a target=\"_new\" href=\"\\1\">\\1</a>")

def format_result(result):
  time = format_time(result["created_at"])
  text = inject_links(result["text"])
  return "[%s] %s" % (time, text)

class RequestHandler(BaseHTTPRequestHandler):
  def do_GET(self):
      if self.path == "/":
          self.wfile.write(HTML)
      elif self.path.startswith("/search"):
          self.updates()
      elif self.path.endswith(".js"):
          self.wfile.write(open(".%s" % self.path).read())
      else:
          self.send_error(404, "Not Found")

  def updates(self):
      o = urlparse(self.path)
      url = "http://search.twitter.com/search.json?" + o.query
      data = urlopen(url).read()
      obj = json.loads(data)
      results = map(format_result, obj["results"])
      self.wfile.write(json.dumps(results))

if __name__ == "__main__":
  server = HTTPServer(("", 8888), RequestHandler)
  server.serve_forever()