Extract elasticsearch query output in csv with java

Question

I'm using java for querying Elasticsearch; I want to export the data of my query in csv file. I'm breaking my head over this.. please can anyone help me with this code? I would be grateful.

This is my query code in java

       try {
        RangeQueryBuilder rangeQ = QueryBuilders
                .rangeQuery("@timestamp")
                .gte("1663632000000")
                .lte("1663804799000")
                .format("epoch_millis");

        TermsAggregationBuilder termsAggregation = AggregationBuilders
                .terms("term_by_client_id")
                .field("labels.client_id")
                .size(100000)
                .minDocCount(1);

        termsAggregation
                .subAggregation(
                        AggregationBuilders
                                .sum("sum_by")
                                .field("labels.row_count")
                );
        termsAggregation
                .subAggregation(
                        AggregationBuilders
                                .terms("term_By_job")
                                .field("labels.job_id")
                );


        SearchRequest searchRequest = new SearchRequest();
        searchRequest.indices("*itm*");


        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(rangeQ);
        searchSourceBuilder.aggregation(termsAggregation);
       // searchSourceBuilder.size(100000);
        searchRequest.source(searchSourceBuilder);

        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        System.out.println(searchResponse);

        Aggregations aggregations = searchResponse.getAggregations();
        Map<String, Aggregation> aggregationMap = aggregations.asMap();
        for (Map.Entry<String, Aggregation> each : aggregationMap.entrySet()){
            System.out.println((each.getValue()));
        }

    } catch (IOException e) {
        throw new RuntimeException(e);
    }
}

And output of query is like this

"buckets":[{"key":"1741433","doc_count":1},{"key":"1741435","doc_count":1},{"key":"1741436","doc_count":1},{"key":"1741440","doc_count":1},{"key":"1741441","doc_count":1},{"key":"1741442","doc_count":1},{"key":"1741443","doc_count":1},{"key":"1741444","doc_count":1},{"key":"1741450","doc_count":1},{"key":"1741451","doc_count":1}]},"sum#sum_by":{"value":1.0951264E7}},{"key":"86206","doc_count":383,"sterms#term_By_job":{"doc_count_error_upper_bound":6,"sum_other_doc_count":361,"buckets":[{"key":"1211310","doc_count":3},{"key":"1211316","doc_count":3},{"key":"1210943","doc_count":2},{"key":"1210945","doc_count":2},{"key":"1210946","doc_count":2},{"key":"1210947","doc_count":2},{"key":"1210948","doc_count":2},{"key":"1210949","doc_count":2},{"key":"1210987","doc_count":2},{"key":"1211010","doc_count":2}]}

Pls help me here with this code that would be appreciated. Thanks.

What data do you want to save in the csv file? Values only like this? 1741433,1\n1741435,1\n... — Raymond Choi
– Raymond Choi, Commented Sep 25, 2022 at 16:41

Fares · Accepted Answer · 2022-09-25 15:17:43Z

0

It's not exactly what you're asking for, but I think it might help you, so I suggest it.

You can use this lightweight tool like this

# Backup the results of a query to a file
elasticdump \
  --input=http://production.es.com:9200/my_index \
  --output=query.json \
  --searchBody="{\"query\":{\"term\":{\"username\": \"admin\"}}}"

Then convert the json to csv using a Java lib, or any online tool if it suits you.

answered Sep 25, 2022 at 15:17

Fares

6501 gold badge4 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Shub Over a year ago

i dont want use any online tool to covert, i want direct implimention export code in my program

Collectives™ on Stack Overflow

Extract elasticsearch query output in csv with java

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related