If it won't be simple, it simply won't be. [Hire me, source code] by Miki Tebeka, CEO, 353Solutions

Tuesday, November 13, 2018

direnv

I use the command line a lot. Some projects require different settings, say Python virtual environment, GOPATH for installing go packages and more.

I'm using direnv to help with settings per project in the terminal. For every project I have a .envrc file which specifies required settings, this file is automatically loaded once I change directory to the project directory or any of it's sub directories.

You'll need the following in your .zshrc

if whence direnv > /dev/null; then
    eval "$(direnv hook zsh)"
fi

Every time you create or change your .envrc, you'll need to run direnv allow to validate it and make sure it's loaded. (If you did some changes and want to check them, run "cd .")

Here are some .envrc examples for various scenarios:

Python + pipenv

source $(pipenv --venv)/bin/activate

Go

GOPATH=$(pwd | sed s"#/src/.*##")
PATH=${GOPATH}/bin:${PATH}

This assumes your project's path that looks like /path/to/project/src/github.com/project

If you're using the new go modules (in 1.11+), you probably don't need this.

Python + virtualenv

source venv/bin/activate

Python + conda

source activate env-name

Replace env-name with the name of your conda environment.

Wednesday, November 07, 2018

Go, protobuf & JSON

Sometimes you'd like more than one way to serve an API. In my case I'm currently working on serving both gRPC and HTTP. I'd like to have one place where objects are defined and have a nice way to serialize both from protobuf (which is the serialization gRPC uses) and JSON .

When producing Go code, protobuf adds JSON struct tags. However since JSON comes from dynamic languages, fields can have any type. In Go we can use map[string]interface{} but in protobuf this is a bit more complicated and we need to use oneof. The struct generated by oneof does not look like regular JSON and will make users of the API write complicated JSON structures.

What's nice about Go, is that we can have any type implement json.Marshaler and json.Unmarshaler. What's extra nice is that in Go, you can add these methods to the generated structs in another file (in Python, we'd have to change the generated source code since methods need to be inside the class definition).

Let's have a look at a simple Job definition


And now we can add some helper methods to aid with JSON serialization (protoc generates code to pb directory)


As a bonus, we added job.Properties that returns a "native" map[string]interface{}

Let's look at a simple example on how we can use it

And its output:
$ go run job.go
[j1]  user:"Saitama" count:1 properties: > properties:
[json]  {"user":"Saitama","count":1,"properties":{"retries":3,"target":"Metal Knight"}}

[j2]  user:"Saitama" count:1 properties: > properties:

Blog Archive