the lemmy changes are causing excessive resource use on my 'bin instance. so yeah, not using lemmy, but being directly affected by the lemmy snafu.
my failed messaging queue is filling, which has its own retry logic… that queue buildup also takes disk space… extra processing, extra disk space… this leads to ‘worker’ slowdown and then system failures and timeouts.
Lastly RabbitMQ allows message prioritisation. So you can drop the priority of things the older/more retries they contain.
Most of this is either RabbitMQ policy or Queue rules based on Headers in the AMQP message. Depending on how KBin is generating messages you might be able to do this as a system admin
Kbin is not lemmy
the lemmy changes are causing excessive resource use on my 'bin instance. so yeah, not using lemmy, but being directly affected by the lemmy snafu.
my failed messaging queue is filling, which has its own retry logic… that queue buildup also takes disk space… extra processing, extra disk space… this leads to ‘worker’ slowdown and then system failures and timeouts.
When I looked at Kbin the “caddy” was wrapped around RabbitMQ. You can get RabbitMQ to solve a lot of those issues.
Firstly with Rabbit you can set a Time To Live header in messages.
By default RabbitMQ queues have no limit in size, you can set a limit.
Lastly RabbitMQ allows message prioritisation. So you can drop the priority of things the older/more retries they contain.
Most of this is either RabbitMQ policy or Queue rules based on Headers in the AMQP message. Depending on how KBin is generating messages you might be able to do this as a system admin