JQ - An easy way to manipulate JSON from the command line

A coworker of mine introduced me to the jq tool today, and I was immediately impressed at how neat it is. It's a tool for parsing JSON directly from the command line, or as stated on the project's page:

"jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text."

My particular problem was that I had a JSON file of user information with various properties and wanted to only print out the ids. The JSON file was in array format, and looked something like:

[
    { "name": "user 1", "user_id": "1", ... },
    { "name": "user 2", "user_id": "2", ... },
    ...
]

With the help of jq, getting the list of user ids was as easy as:

cat user_data.json | jq '.[].user_id'

Which would output the following if applied on the sample data above:

"1"
"2"
...

Another use case I had was to convert a JSON file to CSV. I was able to select the poperties I wanted on the new file and pipe them out using the @csv jq provides, like so:

jq -r '.[] | [ .name, .user_id ]  | @csv' user_data.json >> user_data.csv

The syntax is explained in their docs. They also provide a nice playground for experimenting with the expressions.

Jq can be installed on macOS through homebrew:

brew install jq

That's it. The tool made my day a bit better so might be worth checking out.