Tuesday, May 22, 2007

A memcached helper class.

This little class wraps the memcache functionality:
  • It allows objects to expire after a default or custom age.
  • It allows lazy evaluation of default values (think dict.setdefault, with real lazy behavior) by hiding the default value behind a lambda.

import memcache
import time


class Cache(object):
def __init__(self, cache_addresses, default_timeout=30*60):
self.default_timeout = default_timeout
self.cache = memcache.Client(cache_addresses)

def get(self, key, default=lambda:None, timeout=None):
obj_timestamp = self.cache.get(key)
if obj_timestamp is None:
obj = default()
self.set(key, obj)
else:
obj,timestamp = obj_timestamp
if timeout is None: timeout = self.default_timeout
if time.time() - timestamp > timeout:
self.cache.delete(key)
obj = default()
self.set(key, obj)
return obj

def set(self, key, obj):
self.cache.set(key, (obj, time.time()))

>>> tc = Cache(['127.0.0.1:1979'])
>>> results = tc.get('some_big_list', lambda:build_some_big_list())

An advantage of using this class, is that if the memcached process dies, your process will continue to work as normal, as long as a default is provided to the .get method call.

2 comments:

wavy said...

Hey there

You don't need to wrap your function in a lambda when calling Cache.get - just pass the function itself!

>>> results = tc.get('some_big_list', build_some_big_list)

lambda's are good for default callable arguments (like above), but there's so much more to them than that!

Simon Wittber said...

You don't need to wrap your function in a lambda when calling Cache.get - just pass the function itself!

Ah yes, that is true, tho in my particular use case, I'm using slow list comprehensions, which I should have used in the example instead of a contrived function call.

Popular Posts