Saturday, May 29, 2010

iPad - Africa - PyCon

I bought an iPad, then I read this sobering commentary. Hmmm.

In other news, I'm heading to the first PyCon AU in a few weeks to talk about giant robots and meet new people.

Monday, May 24, 2010

A GUI Skin for Unity3D

Easier to build than I thought.

Tuesday, May 11, 2010

Watching the house burn down...

Apple's new TOS actually effects peoples livelihoods. This developer considers a move to Android. I suspect many Unity3D shops will do the same.

This will have the net effect that all the best games will now run on Android, and not on the iPhone or iPad. Silly Apple.

Tuesday, May 04, 2010

Are you going to 2010? is happening. Are you going?

I know we have plenty of Python afficandos here in Perth, so what are you waiting for? Sign Up!

Monday, May 03, 2010

Consistent Hashing in Python, Redux.

import hashlib
import bisect

class ConsistentHash(object):
def __init__(self, D=5):
self.keyspace = []
self.D = D

def partition(self, arg):
md5 = hashlib.md5(str(arg)).hexdigest()
return int(md5[:4], 16)

def add(self, hash):
for i in xrange(self.D):
k = self.partition("%s.%s"%(hash,i))
bisect.insort(self.keyspace, (k, hash))

def remove(self, hash):
self.keyspace = [i for i in self.keyspace if i[1] != hash]

def __getitem__(self, i):
return self.keyspace[i%len(self.keyspace)][1]

def hash(self, key):
p = self.partition(key)
i = bisect.bisect_left(self.keyspace, (p,None))
return self[i-1]

Consistent Hashing in Python

Update: This is not really a consistent hash. I'll fix the issues then repost. (Fixed here.)

Consistent Hashing is useful. I'm using it to build a Redis client with support for redundancy, and failover / node insertion.

import hashlib

class ConsistentHash(object):
def __init__(self, partition_size=2):
self.partition_size = partition_size
self.hashes = []

def partition(self, arg):
md5 = hashlib.md5(str(arg)).hexdigest()
partition = int(md5[:self.partition_size], 16)
return partition

def add(self, hash):
lambda a,b: cmp(self.partition(a), self.partition(b)))

def __getitem__(self, i):
return self.hashes[i%len(self.hashes)]

def hash(self, key):
i = self.partition(key) % len(self.hashes)
return self[i], self[i+1], self[i+2]

Testing with the below code produces this beautiful graph. Perfect. :-) If you want to add more weight to a particular hash (server address) just add it more than once.
if __name__ == "__main__":
from pygooglechart import PieChart2D
c = ConsistentHash()
keys = "ABCDEFG"
print len(keys)
for k in keys:
d = {}
for i in xrange(10000):
h = c.hash(i)[0]
count = d.get(h, 0)
d[h] = count + 1
chart = PieChart2D(128, 128)
chart.add_data([d[i] for i in keys])
print chart.get_url()

Popular Posts