Delete all documents from index using ElasticsearchOperations

Question

I'm trying to delete all the documents from particular index of ES using the following code:

@Autowired
protected ElasticsearchOperations elasticsearchOperations;

@BeforeEach
void beforeEach() {
    Query query = new NativeSearchQueryBuilder().withQuery(matchAllQuery()).build();
    elasticsearchOperations.delete(query, elasticsearchOperations.getIndexCoordinatesFor(ReviewRequestDocument.class));
}

which fails with

java.lang.IllegalArgumentException: id must not be null

I'm puzzled, because similar approach works for counting documents in index:

private long countReviewRequestDocuments() {
    Query query = new NativeSearchQueryBuilder().withQuery(matchAllQuery()).build();
    return elasticsearchOperations.count(query, ReviewRequestDocument.class);
}

So my question is what is the correct way to remove all documents from index?

What version of ElasticSearch? And what ES client are you using exactly as mentioned in your build.gradle or pom.xml file? — Ermiya Eskandary
– Ermiya Eskandary, Commented Feb 15, 2022 at 8:56
Google DeleteByQueryRequestBuilder - that should find you something. You're using the search query while you need the delete by query API called. — Ermiya Eskandary
– Ermiya Eskandary, Commented Feb 15, 2022 at 9:17
are there documents in the index or might it be empty? Can you provide a minimal reproducable example and open an issue in Spring Data Elasticsearch (github.com/spring-projects/spring-data-elasticsearch/issues) — Abacus
– Abacus, Commented Feb 15, 2022 at 9:19

Sergey Tsypanov · Accepted Answer · 2022-02-15 09:55:36Z

1

I've found the solution:

@BeforeEach
void beforeEach() {
  IndexCoordinates coordinates = elasticsearchOperations.getIndexCoordinatesFor(ReviewRequestDocument.class);

  Query query = new NativeSearchQueryBuilder().withQuery(matchAllQuery()).build();
  String[] ids = elasticsearchOperations.search(query, ReviewRequestDocument.class, coordinates)
          .stream()
          .map(SearchHit::getId)
          .toArray(String[]::new);

  Query idsQuery = new NativeSearchQueryBuilder().withQuery(idsQuery().addIds(ids)).build();
  elasticsearchOperations.delete(idsQuery, ReviewRequestDocument.class, coordinates);

  assertThat(countReviewRequestDocuments()).isZero();
}

it looks like ES does not allow bulk removal of all documents in index, so the solution is to fetch ids and then pass them into delete query.

answered Feb 15, 2022 at 9:55

Sergey Tsypanov

4,6124 gold badges21 silver badges51 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Abacus Over a year ago

This is highly ineffective. What if your index contains 100.000 entries? You'd retrieve them all, then reduce them to an array of ids and send this back. I can run your delete query without any problems.

Sergey Tsypanov Over a year ago

Yeah, this is far from optimal. How would you make it faster?

Abacus Over a year ago

as I wrote, the delete(Query) with the matchall works for me

Sergey Tsypanov Over a year ago

@P.J.Meisch could you please put the whole code as the answer to my question?

Abacus Over a year ago

I just use the code from your first code snippet, nothing else.

warkolm · Accepted Answer · 2022-02-15 23:11:26Z

0

deleting all the documents from an index is super inefficient in Elasticsearch and is not the way to go about this

you are far better off deleting the index and then recreating it from code

answered Feb 15, 2022 at 23:11

warkolm

2,0741 gold badge6 silver badges13 bronze badges

Collectives™ on Stack Overflow

Delete all documents from index using ElasticsearchOperations

2 Answers 2

5 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related