Flink Sink parallelism = 1?

Question

I'm trying to understand how parallelism in Flink works. This doc https://ci.apache.org/projects/flink/flink-docs-release-1.9/concepts/programming-model.html seems to suggest that the sink has parallelism equal to 1. In my case, I'm writing to HBase in my sink - does this mean that there is only one task (thread?) which will be writing to HBase? Does it not get the global parallelism set for the application?

David Anderson · Accepted Answer · 2019-10-04 05:58:15Z

5

The sink gets the global parallelism. To reduce it on one operator, like in that example, requires explicitly changing the parallelism on that operator.

answered Oct 4, 2019 at 5:58

David Anderson

44.3k4 gold badges41 silver badges73 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Jiayi Liao · Accepted Answer · 2019-10-04 05:34:58Z

0

Does this mean there is only one task(thread) writing to HBase when parallelism equals to one?

Yes. Generally, the parallelism is the number of an operator's tasks that are running at the same time. By default it equals to the global parallelism you set.

answered Oct 4, 2019 at 5:34

Jiayi Liao

1,0095 silver badges16 bronze badges

3 Comments

frodo Over a year ago

I guess my question was - does the sink always have parallelism 1? or does it get the global parallelism?

Jiayi Liao Over a year ago

@frodo, it doesn't has any limitation on parallelism and you can set it to any value as long as being less than max parallelism and your resources limit.

Cecilia Over a year ago

What is global parallelism?@frodo

Collectives™ on Stack Overflow

Flink Sink parallelism = 1?

2 Answers 2

Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related