Tuesday, September 11, 2018

[google-cloud-sql-discuss] Publisher confirms and non existing queues

Hello, I posted something about this a while ago. I though I got it but it turns out I am having problems understanding.

Here is my scenario. I have sensors that send data to the server. I have a webapp that receives the data and sends it to RabbitMQ in order to be consumed by workers.
I need to make sure of 2 things:
  1. Messages have definetly enqueued (including fsync'd to the disk) before I respond with 200 OK to the client
  2. Catch errors so I can signal the clients.
As far as point number 1, I have implemented a confirm channel, made the queue durable and send messages with the persistent flag set to true.

This is what I do:

try {
  for (let i = 0; i < messages.length; i++) {
    ch.sendToQueue(queueName, Buffer.from(messages[i]), {persistent: true});
  }
  a
wait ch.waitForConfirms();
} catch(err){
  // respond to client with error (500, for instance)
  res.sendStatus(500);
}
// here I am sure messages have been enqueued
res.sendStatus(200);

This works fine up to a certain point. For instance, for I had created the queue with a maxLength and overflow: 'reject-publish' options, and I have reached the maximum queue size, the waitForConfirms will throw an error and I am able to respond to the client with an error. This is a desirable behaviour because I want to answer with an error if a message is not enqueued.

Now, instead, I have noticed I don't get anything if the queue does not exists. 
To make a test, if I start the application and manually delete the queue from RabbitMQ while my application is running, sendToQueue and waitForConfirms will not throw any errors, but will continue as if everything was normal.
In my case, this would result into a problem because I would respond to the client with a 200 OK, when in reality this should be an error of some sort for me.

How do I deal with this?
Thanks

--
You received this message because you are subscribed to the Google Groups "Google Cloud SQL discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-sql-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-sql-discuss/09589345-5585-4250-9fb5-91ac8ca4c85f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

No comments:

Post a Comment