Add new element to existing JSON array with jq
ArraysJsonBashJqArrays Problem Overview
I want to append an element to an array in a JSON file using the jq``add
command, but it's not working.
report-2017-01-07.json
file:
{
"report": "1.0",
"data": {
"date": "2010-01-07",
"messages": [
{
"date": "2010-01-07T19:58:42.949Z",
"xml": "xml_samplesheet_2017_01_07_run_09.xml",
"status": "OK",
"message": "metadata loaded into iRODS successfully"
},
{
"date": "2010-01-07T20:22:46.949Z",
"xml": "xml_samplesheet_2017_01_07_run_09.xml",
"status": "NOK",
"message": "metadata duplicated into iRODS"
},
{
"date": "2010-01-07T22:11:55.949Z",
"xml": "xml_samplesheet_2017_01_07_run_09.xml",
"status": "NOK",
"message": "metadata was not validated by XSD schema"
}
]
}
}
I am using this command:
$ cat report-2017-01-07.json
| jq -s '.data.messages {"date": "2010-01-07T19:55:99.999Z", "xml": "xml_samplesheet_2017_01_07_run_09.xml", "status": "OKKK", "message": "metadata loaded into iRODS successfullyyyyy"}'
jq: error: syntax error, unexpected '{', expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
.data.messages {"date": "2010-01-07T19:55:99.999Z", "xml": "xml_samplesheet_2017_01_07_run_09.xml", "status": "OKKK", "message": "metadata loaded into iRODS successfullyyyyy"}
jq: 1 compile error
Here's how I want the output to look:
{
"report": "1.0",
"data": {
"date": "2010-01-07",
"messages": [{
"date": "2010-01-07T19:58:42.949Z",
"xml": "xml_samplesheet_2017_01_07_run_09.xml",
"status": "OK",
"message": "metadata loaded into iRODS successfully"
}, {
"date": "2010-01-07T20:22:46.949Z",
"xml": "xml_samplesheet_2017_01_07_run_09.xml",
"status": "NOK",
"message": "metadata duplicated into iRODS"
}, {
"date": "2010-01-07T22:11:55.949Z",
"xml": "xml_samplesheet_2017_01_07_run_09.xml",
"status": "NOK",
"message": "metadata was not validated by XSD schema"
}, {
"date": "2010-01-07T19:55:99.999Z",
"xml": "xml_samplesheet_2017_01_07_run_09.xml",
"status": "OKKKKKKK",
"message": "metadata loaded into iRODS successfullyyyyy"
}]
}
}
Arrays Solutions
Solution 1 - Arrays
The |= .+
part in the filter adds a new element to the existing array. You can use jq
with filter like:
jq '.data.messages[3] |= . + {"date": "2010-01-07T19:55:99.999Z", "xml": "xml_samplesheet_2017_01_07_run_09.xml", "status": "OKKK", "message": "metadata loaded into iRODS successfullyyyyy"}' inputJson
To avoid using the hardcoded length value 3
and dynamically add a new element, use . | length
which returns the length, which can be used as the next array index, i.e.,
jq '.data.messages[.data.messages| length] |= . + {"date": "2010-01-07T19:55:99.999Z", "xml": "xml_samplesheet_2017_01_07_run_09.xml", "status": "OKKK", "message": "metadata loaded into iRODS successfullyyyyy"}' inputJson
(or) as per peak's suggestion in the comments, using the +=
operator alone
jq '.data.messages += [{"date": "2010-01-07T19:55:99.999Z", "xml": "xml_samplesheet_2017_01_07_run_09.xml", "status": "OKKK", "message": "metadata loaded into iRODS successfullyyyyy"}]'
which produces the output you need:
{
"report": "1.0",
"data": {
"date": "2010-01-07",
"messages": [
{
"date": "2010-01-07T19:58:42.949Z",
"xml": "xml_samplesheet_2017_01_07_run_09.xml",
"status": "OK",
"message": "metadata loaded into iRODS successfully"
},
{
"date": "2010-01-07T20:22:46.949Z",
"xml": "xml_samplesheet_2017_01_07_run_09.xml",
"status": "NOK",
"message": "metadata duplicated into iRODS"
},
{
"date": "2010-01-07T22:11:55.949Z",
"xml": "xml_samplesheet_2017_01_07_run_09.xml",
"status": "NOK",
"message": "metadata was not validated by XSD schema"
},
{
"date": "2010-01-07T19:55:99.999Z",
"xml": "xml_samplesheet_2017_01_07_run_09.xml",
"status": "OKKK",
"message": "metadata loaded into iRODS successfullyyyyy"
}
]
}
}
Use jq-play to dry-run your jq-filter
and optimize any way you want.
Solution 2 - Arrays
Rather than using |=
, consider using +=
:
.data.messages += [{"date": "2010-01-07T19:55:99.999Z", "xml": "xml_samplesheet_2017_01_07_run_09.xml", "status": "OKKK", "message": "metadata loaded into iRODS successfullyyyyy"}]
Prepend
On the other hand, if (as @NicHuang asked) you want to add the JSON object to the beginning of the array, you could use the pattern:
.data.messages |= [ _ ] + .
Solution 3 - Arrays
Summary: ". +" is your saviour
Details:
For adding an entry to a list: You can append [list1] + [list2] (and not [list] + data)
$ echo '[ "data1" ]' | jq '. + [ "data2" ]'
[ "data1", "data2"]
$ echo '[ {"key1": "value1"} ]' | jq '. + [{"key2": "value2"}]'
[ { "key1": "value1" }, { "key2": "value2" }]
For adding a key/value to a dictionary:
$ echo '{"key1": "value1"}' | jq '. + {"key2": "value2"}'
{
"key1": "value1",
"key2": "value2"
}
References: