Pretty-Printing JSON Response with HTTP Headers

Often while using curl with URLs that return a JSON response, I need to print the HTTP response headers along with the JSON response. Here is an example that shows how this can be done:

$ curl -sSi https://susam.net/code/lab/json/book.json
HTTP/1.1 200 OK
Server: nginx/1.18.0
Date: Sun, 14 Apr 2024 22:20:49 GMT
Content-Type: application/json
Content-Length: 80
Last-Modified: Sun, 14 Apr 2024 22:17:09 GMT
Connection: keep-alive
ETag: "661c55e5-50"
Accept-Ranges: bytes

{"title": "The Music of the Primes", "author": "Marcus du Satoy", "pages": 366}

The above output is obtained using curl 7.77.0 (x86_64-apple-darwin21.0). The -i option is responsible for including the HTTP response headers. The -s and -S options are not too important for the current discussion but I usually happen to use them out of habit. The -s option suppresses the progress meter and error messages but the -S re-enables the display of error messages. This helps me avoid the progress meter in the output without having to lose visibility of any errors that may arise.

So far so good! But can we also have the JSON response pretty-printed with say jq? The above command prints both the HTTP headers and the response to the standard output, so piping the standard output to jq does not work. The jq command fails with an error as soon as it encounters the HTTP headers.

If, however, we manage to send the HTTP header and the response to different streams or files, then we could utilise jq to pretty-print the stream or file that contains the JSON response. Here is an example that shows how to do this:

$ curl -sSD head.txt -o out.json https://susam.net/code/lab/json/book.json && cat head.txt && jq . out.json
HTTP/1.1 200 OK
Server: nginx/1.18.0
Date: Sun, 14 Apr 2024 22:31:35 GMT
Content-Type: application/json
Content-Length: 80
Last-Modified: Sun, 14 Apr 2024 22:17:09 GMT
Connection: keep-alive
ETag: "661c55e5-50"
Accept-Ranges: bytes

{
  "title": "The Music of the Primes",
  "author": "Marcus du Satoy",
  "pages": 366
}

Alternatively, we can achieve this using a single command by printing the the HTTP headers to standard error. This ensures that only the JSON response is printed to standard output, which we can then pretty-print using jq. Here is an example:

$ curl -sSD /dev/stderr https://susam.net/code/lab/json/book.json | jq .
HTTP/1.1 200 OK
Server: nginx/1.18.0
Date: Sun, 14 Apr 2024 22:34:26 GMT
Content-Type: application/json
Content-Length: 80
Last-Modified: Sun, 14 Apr 2024 22:17:09 GMT
Connection: keep-alive
ETag: "661c55e5-50"
Accept-Ranges: bytes

{
  "title": "The Music of the Primes",
  "author": "Marcus du Satoy",
  "pages": 366
}