Messages with exceptions are not being acknowledged.

[ryanwi]

Repro:

• Consumer method raises an exception
• Error routes through handle_error and notifies configured handlers
• Message remains in the Unacked state
• When service is stopped, message returns to the ready queue

Expected:
Consumer exception would be handled and notified, then the message would be acknowledged.

Findings:
Appears to be an issue with the break here:

https://github.com/teamsnap/pipe-ruby/blob/534ca366cfdf1d4dd6a60b2fe39949a0f233b9ad/lib/pipe/reducer.rb#L18

which prevents emque-consuming from reaching the RabbitMQ ack here:

https://github.com/teamsnap/emque-consuming/blob/3b7112eb43bd7c1e1dc9dddc71a3c70409629df0/lib/emque/consuming/adapters/rabbit_mq/worker.rb#L70

[ryanwi]

@kyleries this may be a clue for the lack of shutdown notification, could be a similar issue with a break in a pipe array preventing the callback to the shutdown handler. Just a hunch at this point now, but similar symptoms in that code is not being reached.

It also explains why we had to purge messages during the Dozer outage last Sunday.

@semmons99 this is probably why we had trouble firing up Dozer initially, since it will just re-process messages that cause errors.

@dmvt what do you think about the break in the reducer rescue? Can we allow the pipe to handle the error and then continue processing.

[ryanwi]

Ok, it's not the break that's stopping it, it's the handle error.

https://github.com/teamsnap/pipe-ruby/blob/534ca366cfdf1d4dd6a60b2fe39949a0f233b9ad/lib/pipe/reducer.rb#L35

So, I will change the raise on error config and give that a shot.