If it won't be simple, it simply won't be. [Hire me, source code] by Miki Tebeka, CEO, 353Solutions

Wednesday, January 11, 2012

fastavro with Cython

Added an optional step of compiling fastavro with Cython. Just doing that, with no Cython specific code reduced the time of processing 10K records from 2.9sec to 1.7sec. Not bad for that little work.

Also added a __main__.py so you can use fastavro to process Avro files:

  • python -m fastavro weather.avro # Dump records in JSON format
  • python -m fastavro --schema weather.avro # Dump schema

Friday, January 06, 2012

fastavro

Just released fastavro to PyPI. It has way less features than the official avro package, but according to my tests it's about 5 times faster.

Blog Archive