Monday, July 23, 2007

Generators hold references too!

For some reason, our TG app was leaking transactions, causing predictable crashes. Every page view, Postgres was holding onto transaction, which never committed.

Argh.

After much brute force debugging, I discovered the problem. A controller was calling a method in the model, and passing the return value through to the template, where incidentally, the value was no longer being used. "Easy fix", I thought, "just remove the method call."

But, I couldn't leave it at that, I had to find out what this method was doing differently which was causing transaction leakage... As it turns out, the method was returning a generator, via a generator expression. The generator itself was holding references to the transaction, preventing it from closing. The transaction would only close once the generator had been iterated through completely, finally calling the clean up code.

Woohoo. It's a good feeling when you kill a nasty bug like this. I'll be more careful with generator expressions from now on!

2 comments:

Andrew said...

Sounds like the real problem is relying on garbage collection to do your finalisation, instead of explicitly closing the transaction at the end of the web request.

Simon Wittber said...

Yeah, thats the issue in a sentence. In theory TG is supposed to look after this. It's think it is probably another symptom of SQLAlchemy having second-class support in TG, though I haven't investigated any further yet.

Popular Posts