elasticsearch - Elastic Search Bulk Import from JSON without ID -
is there way import data json file elasticsearch without having provide id each document?
i have data in json file. contains around 1000 documents no id has been specified document. here's how data looks like:
{"business_id": "aasd231as", "full_address": "202 mcclure 15034", "hours":{}} {"business_id": "123123444", "full_address": "1322 lure 34", "hours": {}} {"business_id": "sd231as", "full_address": "2 mccl 5034", "hours": {}}
it not have {"index":{"_id":"5"}}
before document. trying import data elasticsearch using following command:
curl -xpost localhost:9200/newindex/newtype/_bulk?pretty --data-binary @path/file.json
but throws following error:
"type" : "illegal_argument_exception", "reason" : "malformed action/metadata line [1], expected start_object or end_object found [value_string]"
this because of absence of id in line before each document.
is there way import data without providing {"index":{"_id":"5"}}
before each document. highly appreciated!!
how using logstash suited task. use following config file , you're done:
save following config in logstash.conf
:
input { file { path => "/path/to/file.json" start_position => "beginning" sincedb_path => "/dev/null" codec => "json" } } filter { mutate { remove_field => [ "@version", "@timestamp", "path", "host" ] } } output { elasticsearch { hosts => ["localhost:9200"] index => "newindex" document_type => "newtype" workers => 1 } }
then start logstash with
bin/logstash -f logstash.conf
Comments
Post a Comment