If it won't be simple, it simply won't be. [source code] by Miki Tebeka, CEO, 353Solutions

Wednesday, January 11, 2012

fastavro with Cython

Added an optional step of compiling fastavro with Cython. Just doing that, with no Cython specific code reduced the time of processing 10K records from 2.9sec to 1.7sec. Not bad for that little work.

Also added a __main__.py so you can use fastavro to process Avro files:

  • python -m fastavro weather.avro # Dump records in JSON format
  • python -m fastavro --schema weather.avro # Dump schema

Post a Comment

Blog Archive