Saturday, April 11, 2009

yield from - redundant syntax?

Greg Ewing is putting together a proposal to adding a "yield from" keyword to Python.

The Spam Server code he provides as a supporting example can already be achieved using the Fibra scheduler, without the new syntax. Notice that Fibra allows exceptions to propagate from an inner task to the outer task.

Fibra also allows an inner task to return values to its parent task, by yielding a "Return(value)" instance. This is what the line_recv task uses to continually yield lines from a socket to the parent task, while maintaining its own internal state.

Perhaps the "yield from" syntax is redundant. Fibra doesn't need it, and Fibra fulfilles the main use-case behind the PEP.

The code below is a direct translation of the Spam Server example, using Fibra.

import socket
import fibra

port = 4200

def handler(transport):
while True:
line = yield transport.recv_line()
n = parse_request(line)
yield transport.send_line("100 SPAM FOLLOWS")
for i in xrange(n):
yield transport.send_line("spam glorious spam")
except BadRequest:
yield transport.send_line("400 WE ONLY SERVE SPAM")
except socket.error:

class BadRequest(Exception):

def parse_request(line):
tokens = line.split()
if len(tokens) != 2 or tokens[0] != "SPAM":
raise BadRequest
n = int(tokens[1])
except ValueError:
raise BadRequest
if n < 1:
raise BadRequest
return n

schedule = fibra.schedule()
schedule.install('localhost', port), handler))

Update: modified handler to use a new transport class.


Ian Bicking said...

Just because someone is able to do okay without language support isn't a reason the language support is wrong to implement. If no one tried to get around the lack of language support, then it would be something not worth doing. That it makes intuitive sense but in terms of implementation is rather subtle is not a sign of a flaw in the proposal, but that the language can serve as a blessing for the truly right technique.

Simon Wittber said...

OK, I see your point. "Blessing the truly right technique" is worth doing; It just might go a short way towards unifying some of the cooperative threading frameworks.

Doug Napoleone said...

The potential for optimizations on generator chains alone makes me like this PEP in the works.

I am still not crazy about the 'yield from' syntax (this is really how its implemented). I can not think of any other two word keywords in the language. If 'pass' were not already a keyword I would say that would be better. Given what it s doing, 'yield from' seems to be the best option I have heard yet.

(from a chronic lurker on the dev lists)

Unknown said...

yield from can be optimized a lot easier than a yield loop. Using yield from, you can call only the subgenerator. Using a yield loop, you have to call the generator and the subgenerator every iteration. If you stack generators a little higher than that, you often end up with 3 calls per iteration or more (e.g. in Genshi) which is slow as fucking molasses in current Python. Try it out in a current version of Trac.

Popular Posts