Slow consumer using selectors on ActiveMQ Classic

Question

We have an ActiveMQ Classic broker on the default configuration (no specific destination policy configured, and with default KahaDB storage).

Many producers and consumers use the same queue for specific operations. They use a selector to filter what each consumer should read. For example:

String selector = "COMMAND = 'RUN_CALC' AND CATEGORY='CALCULATION'"; 
QueueReceiver queueReceiver = session.createReceiver(destination, selector);

and producers push messages without any timetolive by setting properties:

MessageProducer producer = session.createProducer(destination);
StringBuilder command = new StringBuilder("RUN_CALC;CATEGORY;CALCULATION;.....");
BytesMessage message = session.createBytesMessage();
message.setStringProperty("COMMAND", "RUN_CALC");
message.setStringProperty("CATEGORY", "CALCULATION");

byte[] byteData = command.toString().getBytes();
message.writeBytes(byteData);
producer.send(message);

Of course this is just a simplification of the code. The system has been running fine since many years with thousands of messages daily consumed with filter and selector by around 40 consumers, but lately we had a scenario where 500 pending messages were present in the queue (non consumed) and all consumers became very slow to receive the messages, like very very slow! I always thought that setStringProperty allowed the broker to properly index messages but it does not seem to work. I understand best practices are to not have non consumed messages, but it's weird that this low amount of pending messages in the queue made selectors that slow! I tested many different configuration in the ActiveMQ, enabled JMX to monitor, but I didn't find a way to fix this (and our server memory and CPUs are very good so it's not resource issue).

What version of ActiveMQ Classic are you using?

Justin Bertram
– Justin Bertram

2024-12-23 20:07:28 +00:00
Commented Dec 23, 2024 at 20:07 — Justin Bertram
– Justin Bertram, Commented Dec 23, 2024 at 20:07

Matt Pavlovich · Accepted Answer · 2024-12-28 17:59:31Z

0

This sounds like a common scenario when using selectors on a queue. The number of pending messages that do not have a matching selector outruns the available window for the broker's queue to find matches. Now the queue appears 'slow' or 'stuck'.

Best practice-- don't do a 'queue-in-a-queue' pattern with queues, as it leads to this and other types of issues. The patterns that fall into this include using selectors, and message groups as they cause runtime headaches.

If you have messages that need filtering, split them out to separate queues.

If you need the same consumer to select multiple, use a composite queue to consume from one or more queues.

// This will create one consumer listening on two queues
Destination compositeQueue = session.createQueue("US,UK");
MessageConsumer compositeConsumer = session.createConsumer(compositeQueue));

answered Dec 28, 2024 at 17:59

Matt Pavlovich

4,3361 gold badge11 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Slow consumer using selectors on ActiveMQ Classic

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related