Friday, September 13, 2013

Django: building a HttpResponse object with a generator results in empty 0-byte responses

Interesting gotcha with Django http responses I hit today. Say you want to stream back content in a http response because gathering it from its source is expensive, and/or the content is huge so you can't hold it all in memory at once. Since Django's HttpResponse method accepts a generator, you might think just yielding content in the generator would work. It might, but the problem is that if something else in your code accesses the HttpResponse before it gets put on the wire (like middleware for example), then you're going to end up delivering an empty response. See example below where the second read comes up empty.
In [42]: from django import http

In [43]: def Generator():
   ...:     yield "aaa"
   ...:     yield "bbb"
   ...:     

In [44]: r=http.HttpResponse(content=Generator())
In [45]: print r
Content-Type: text/html; charset=utf-8

aaabbb

In [46]: print r
Content-Type: text/html; charset=utf-8


Whereas if you send all the content at once, the problem doesn't exist.
In [47]: r=http.HttpResponse(content="aaa")
In [48]: print r
Content-Type: text/html; charset=utf-8

aaa

In [49]: print r
Content-Type: text/html; charset=utf-8

aaa
If marshalling all the content up-front isn't practical, you can try the StreamingHttpResponse object as of Django 1.5.