[Meego-distribution-tools] ruote-amqp-pyclient crashes when declaring exchange

Fri Oct 28 13:27:09 UTC 2011

Hello,

The first part (allowing usage with rabbitamqp > 2.2) just got merged to
master branch.

On 10/28/2011 10:51 AM, ext Ed Bartosh wrote:
> Any news on this?
> 
>> -----Original Message-----
>> From: meego-distribution-tools-bounces at lists.meego.com [mailto:meego-
>> distribution-tools-bounces at lists.meego.com] On Behalf Of Ed Bartosh
>> Sent: Tuesday, October 18, 2011 11:33 AM
>> To: 'David Greaves'
>> Cc: meego-distribution-tools at lists.meego.com
>> Subject: Re: [Meego-distribution-tools] ruote-amqp-pyclient crashes
>> when declaring exchange
>>
>>> -----Original Message-----
>>> From: David Greaves [mailto:david at dgreaves.com]
>>> Sent: Monday, October 17, 2011 11:56 PM
>>> To: Bartosh, Eduard
>>> Cc: islam.amer at nokia.com; meego-distribution-tools at lists.meego.com
>>> Subject: Re: [Meego-distribution-tools] ruote-amqp-pyclient crashes
>>> when declaring exchange
>>>
>>> OK, I've merged your first patch locally and we will push that
>> through
>>> testing
>>> soon. That should resolve your rabbit 2.6 issues I hope.
>> Thanks.
>>
>>> (Although I
>>> suspect it
>>> won't work if you run a participant against a new rabbit server
>> without
>>> running
>>> BOSS first - a really minor issue.)
>> Why do you think so? Can you elaborate a bit?
>>
>>> Looking at the second patch then.
>>>
>>> The objective for ruote-amqp-pyclient (being renamed and packaged as
>>> python-ruote-amqp)
>> Where can I see its git repo? Does this mean that ruote-amqp-pyclient
>> is going to be dropped or renamed on gitorious?
>>
>>> is to be a fairly robust implementation of an amqp
>>> client for
>>> ruote; so yes it is related to both ruote and amqp - but the minimal
>>> specialist
>>> overlap rather than a generalisation of both.
>>>
>>> Being able to specify amqp_exchange is a sensible option for AMQP
>> 0.9.x
>>> and
>>> should eventually IMHO default to the 'ruote' exchange - but use of
>>> that value
>>> must also correspond to a move at the ruby side; so defaulting to ""
>> is
>>> a good
>>> interim step. So that part is fine.
>>>
>> OK. If we don't agree on the next part I'll redo the patch. At least
>> part of it can be accepted.
>>
>>> Since you are making changes in the Participant class my concern
>> about
>>> the key
>>> is that there are no circumstances that I can see where having key !=
>>> queue will
>>> work with ruote - so providing this flexibility will only allow
>> people
>>> to shoot
>>> themselves in the foot.
>>>
>>> The reason ruote uses queue=routing_key is that AMQP is being used
>>> purely to
>>> deliver direct messages to a single queue shared by all participants.
>> A
>>> ruote
>>> message *must not* be consumed by more than one participant.
>>>
>> That's perfectly understood. However, it doesn't contradict with
>> possibility to have routing_key!=queue.
>>
>>> Of course I could be missing something : can you provide an example
>>> where 2
>>> participants would bind the same queue with different keys?
>>>
>> Correct me if I'm wrong, but it looks like you're confusing two
>> different things here. Having routing_key != queue doesn't mean that
>> there have to be two participants binding to the same queue with
>> different keys. It only means that it's possible to have queue !=
>> routing key even for the single participant. As soon as queue and
>> routing key are different matters they can potentially be different.
>>
>>> Oh, as a simple RPC over AMQP:
>>> https://meego.gitorious.org/meego-infrastructure-tools/air
>>>
>>> I thought we may need this to do things like query the ruote server
>> for
>>> things
>>> like a list of registered participants etc etc.
>>>
>> I can see the same approach in this code: hardcoded exchange and
>> queue=routing key. The same artificial limitation. Why here? This code
>> has nothing to do with ruote limitations at all, why do you use its
>> approach?
>>
>>> David
>>>
>>> On 17/10/11 13:54, Bartosh, Eduard wrote:
>>>> Hi David,
>>>>
>>>> First of all there are two merge requests there. First is a simple
>>> fix, which was reproduced and tested by Islam. Your question is not
>>> about this one. Does it mean that you're going to merge it into
>> master?
>>>>
>>>> Now about your question regarding second fix. My idea was that
>> ruote-
>>> amqp-pyclient related to both ruote and amqp. I don't know much about
>>> ruote, but from amqp point of view having possibility to use non-
>>> default exchange, queue and routing key is rather good than bad.
>>> Speaking about ruote I have an impression that they're using
>>> queue=routing_key just because they decided to use default exchange
>> for
>>> some reason. For default binding to default exchange you have to have
>>> queue=routing_key as far as I remember. I'm not sure it makes sense
>> to
>>> stick to this for both ruote and ruote-amqp-pyclient. It looks like
>>> artificial limitation from my point of view.
>>>>
>>>> Regards,
>>>> Ed
>>>>
>>>>> -----Original Message-----
>>>>> From: David Greaves [mailto:david at dgreaves.com]
>>>>> Sent: Monday, October 17, 2011 2:35 PM
>>>>> To: Bartosh, Eduard
>>>>> Cc: islam.amer at nokia.com; meego-distribution-tools at lists.meego.com
>>>>> Subject: Re: [Meego-distribution-tools] ruote-amqp-pyclient
>> crashes
>>>>> when declaring exchange
>>>>>
>>>>> So I personally can't think of a situation when ruote should have
>>> key
>>>>> != queue
>>>>>
>>>>> Could you clarify this?
>>>>>
>>>>> David
>>>>>
>>>>> On 11/10/11 11:43, Bartosh, Eduard wrote:
>>>>>> Hi guys,
>>>>>>
>>>>>> Here is new merge request I was talking you about today:
>>>>> https://meego.gitorious.org/meego-infrastructure-tools/ruote-amqp-
>>>>> pyclient/merge_requests/3
>>>>>> It adds 2 new parameters to Participant.__init__:
>> amqp_routing_key
>>>>> and amqp_exchange.
>>>>>> This allows to use not only default exchanges and routing key
>>>>> different from queue name.
>>>>>>
>>>>>> Regards,
>>>>>> Ed
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: meego-distribution-tools-bounces at lists.meego.com
>>>>> [mailto:meego-
>>>>>>> distribution-tools-bounces at lists.meego.com] On Behalf Of
>> Bartosh,
>>>>>>> Eduard
>>>>>>> Sent: Tuesday, October 11, 2011 12:17 PM
>>>>>>> To: islam.amer at nokia.com
>>>>>>> Cc: meego-distribution-tools at lists.meego.com
>>>>>>> Subject: Re: [Meego-distribution-tools] ruote-amqp-pyclient
>>> crashes
>>>>>>> when declaring exchange
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Make sense to me.
>>>>>>>
>>>>>>> You can see my merge request here:
>>>>> https://meego.gitorious.org/meego-
>>>>>>> infrastructure-tools/ruote-amqp-pyclient/merge_requests/2
>>>>>>>
>>>>>>> Next one is coming :)
>>>>>>>
>>>>>>> Regards,
>>>>>>> Ed
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: islam.amer at nokia.com [mailto:islam.amer at nokia.com]
>>>>>>>> Sent: Tuesday, October 11, 2011 11:59 AM
>>>>>>>> To: Bartosh, Eduard
>>>>>>>> Cc: meego-distribution-tools at lists.meego.com
>>>>>>>> Subject: Re: [Meego-distribution-tools] ruote-amqp-pyclient
>>> crashes
>>>>>>>> when declaring exchange
>>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> That sounds nice. However, smaller patches that gradually
>>> introduce
>>>>>>>> changes will make it easy to discuss and spot backward
>>> incompatible
>>>>>>>> ones.
>>>>>>>>
>>>>>>>> Just my €0.02 !
>>>>>>>>
>>>>>>>> Thanks :)
>>>>>>>>
>>>>>>>> On 10/11/2011 11:35 AM, ext Bartosh, Eduard wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Sorry for misunderstanding. I thought that you couldn't
>>> reproduce
>>>>>>> the
>>>>>>>> crash as it was happened to me.
>>>>>>>>>
>>>>>>>>> Please, don't apply this patch now. I have some ideas how to
>>> make
>>>>>>> it
>>>>>>>> a bit more useful. I'd like to pass couple of new parameters
>>>>>>>> (routing_key and exchange to be exact) and skip exchange
>>>>> declaration
>>>>>>>> when exchange is empty and queue binding if queue and routing
>> key
>>>>> are
>>>>>>>> the same.
>>>>>>>>>
>>>>>>>>> I'll let you know when it's ready. Most probably I'll make
>> merge
>>>>>>>> request and send its url to this list.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Ed
>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: islam.amer at nokia.com [mailto:islam.amer at nokia.com]
>>>>>>>>>> Sent: Tuesday, October 11, 2011 10:50 AM
>>>>>>>>>> To: Bartosh, Eduard
>>>>>>>>>> Cc: meego-distribution-tools at lists.meego.com
>>>>>>>>>> Subject: Re: [Meego-distribution-tools] ruote-amqp-pyclient
>>>>>>> crashes
>>>>>>>>>> when declaring exchange
>>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> I think my email was a bit unclear.
>>>>>>>>>>
>>>>>>>>>> After upgrading from 1.8 to 2.6 , I recreated the vhost ,
>>>>> username
>>>>>>>>>> and permissions for boss (can be done with apt-get install
>>>>>>>>>> --reinstall boss). Then the participants were showing the
>>>>> familiar
>>>>>>>>>> error like you reported.
>>>>>>>>>>
>>>>>>>>>> After applying your patch everything worked again. I did some
>>>>>>> quick
>>>>>>>>>> tests and messages were flowing both ways fine.
>>>>>>>>>>
>>>>>>>>>> Then I downgraded and tested again.
>>>>>>>>>>
>>>>>>>>>> I purged rabbitmq and made sure the ondisk data was gone and
>>> then
>>>>>>>>>> installed 1.8 and tested _with the patch applied_. It still
>>>>>>> worked.
>>>>>>>>>>
>>>>>>>>>> I purged and installed 2.6 and tested again with the patch
>> and
>>> it
>>>>>>>>>> works.
>>>>>>>>>>
>>>>>>>>>> So what I am saying is after some more testing I'd like to
>>> apply
>>>>>>> the
>>>>>>>>>> patch as it allows working with both old and new version.
>>>>>>>>>>
>>>>>>>>>> On 10/11/2011 10:33 AM, ext Bartosh, Eduard wrote:
>>>>>>>>>>> Hi Islam,
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> It happened to me as well. I couldn't reproduce this crash.
>>>>> First
>>>>>>>>>> time I saw this traceback month ago or so. I downgraded
>>>>>>>>>> rabbitmq-server to 1.8 and it started to work. Then we
>> upgraded
>>>>>>>>>> server to 2.6, because we liked new admin web UI coming with
>>> it.
>>>>>>> It
>>>>>>>>>> also worked just fine for some time. And yesterday, when
>>> college
>>>>>>> of
>>>>>>>>>> mine tried to setup the whole thing on different host he got
>>> the
>>>>>>>> same crash.
>>>>>>>>>>>
>>>>>>>>>>> Try to remove vhost, stop and then start rabbitmq server.
>> That
>>>>>>>> might
>>>>>>>>>> help with reproducing this error. I'm not sure it will
>> though.
>>>>>>>>>>>
>>>>>>>>>>> Just to prove my pov:
>>>>>>>>>>> 1. Here is the source of rabbitmq server 2.1.1 where you can
>>>>>>> search
>>>>>>>>>> for check_not_default_exchange and see that it's used in
>>> several
>>>>>>>>>> places like exchange declaration, exchange deletion and queue
>>>>>>>> binding:
>>>>>>>>>>> https://github.com/rabbitmq/rabbitmq-
>>>>>>>>>> server/blob/rabbitmq_v2_1_1/src/r
>>>>>>>>>>> abbit_channel.erl
>>>>>>>>>>>
>>>>>>>>>>> 2. And here is the quote from rabbitmq admin UI help:
>>>>>>>>>>> The default exchange is implicitly bound to every queue,
>> with
>>> a
>>>>>>>>>> routing key equal to the queue name. It it not possible to
>>>>>>>> explicitly
>>>>>>>>>> bind to, or unbind from the default exchange. It also cannot
>> be
>>>>>>>>>> deleted.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Ed
>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: meego-distribution-tools-bounces at lists.meego.com
>>>>>>>>>> [mailto:meego-
>>>>>>>>>>>> distribution-tools-bounces at lists.meego.com] On Behalf Of
>>>>>>>>>>>> islam.amer at nokia.com
>>>>>>>>>>>> Sent: Tuesday, October 11, 2011 9:39 AM
>>>>>>>>>>>> To: meego-distribution-tools at lists.meego.com
>>>>>>>>>>>> Subject: Re: [Meego-distribution-tools] ruote-amqp-pyclient
>>>>>>>> crashes
>>>>>>>>>>>> when declaring exchange
>>>>>>>>>>>>
>>>>>>>>>>>> No this error appears on the participant (consumer or
>> client)
>>>>>>>> side.
>>>>>>>>>> I
>>>>>>>>>>>> tested several upgrade / downgrade cycles from 1.8 to 2.6
>> and
>>>>>>> back
>>>>>>>>>>>> and the on disk data was fine.
>>>>>>>>>>>>
>>>>>>>>>>>> On 10/11/2011 09:17 AM, ext Jian-feng Ding wrote:
>>>>>>>>>>>>> Maybe the persistent data files are incompatible for
>>> different
>>>>>>>>>>>>> versions of rabbitmq. If current persistent data can be
>>>>>>> dropped,
>>>>>>>>>> how
>>>>>>>>>>>> about to clean them up and restart the services.
>>>>>>>>>>>>>
>>>>>>>>>>>>> - jf.ding
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Oct 10, 2011 at 06:57:16PM +0800, Bartosh, Eduard
>>>>>>> wrote:
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Recently I've upgraded rabbitmq-server to 2.6.1 and as a
>>>>>>> result
>>>>>>>> I
>>>>>>>>>>>> see this traceback when running my participant:
>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>     File "/usr/bin/boss-participant-gerrit", line 190, in
>>>>>>> <module>
>>>>>>>>>>>>>>       sys.exit(main())
>>>>>>>>>>>>>>     File "/usr/bin/boss-participant-gerrit", line 185, in
>>> main
>>>>>>>>>>>>>>       run_participant(config)
>>>>>>>>>>>>>>     File "/usr/bin/boss-participant-gerrit", line 140, in
>>>>>>>>>>>> run_participant
>>>>>>>>>>>>>>       config.get('obs_webui'))
>>>>>>>>>>>>>>     File "/usr/bin/boss-participant-gerrit", line 35, in
>>>>>>> __init__
>>>>>>>>>>>>>>       amqp_routing_key=amqp_settings['amqp_routing_key'])
>>>>>>>>>>>>>>     File "/usr/lib/python2.7/site-
>>>>>>>>>> packages/RuoteAMQP/participant.py",
>>>>>>>>>>>> line 53, in __init__
>>>>>>>>>>>>>>       auto_delete=False)
>>>>>>>>>>>>>>     File "/usr/lib/python2.7/site-
>>>>>>>>>>>> packages/amqplib/client_0_8/channel.py", line 843, in
>>>>>>>>>>>> exchange_declare
>>>>>>>>>>>>>>       (40, 11),    # Channel.exchange_declare_ok
>>>>>>>>>>>>>>     File "/usr/lib/python2.7/site-
>>>>>>>>>>>> packages/amqplib/client_0_8/abstract_channel.py", line 105,
>>> in
>>>>>>>> wait
>>>>>>>>>>>>>>       return amqp_method(self, args)
>>>>>>>>>>>>>>     File "/usr/lib/python2.7/site-
>>>>>>>>>>>> packages/amqplib/client_0_8/channel.py", line 273, in
>> _close
>>>>>>>>>>>>>>       (class_id, method_id))
>>>>>>>>>>>>>> amqplib.client_0_8.exceptions.AMQPChannelException: (403,
>>>>>>>>>>>>>> u'ACCESS_REFUSED - operation not permitted on the default
>>>>>>>>>>>>>> exchange', (40, 10), 'Channel.exchange_declare')
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It worked just fine with rabbitmq 1.8.0.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ruote-amqp-pyclient version is 1.12
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Any idea how to fix this?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Ed
>>>>>>>>>>>>>> ---------------------------------------------------------
>> --
>>> --
>>>>> -
>>>>>>> --
>>>>>>>> -
>>>>>>>>>>>>>> -
>>>>>>>>>> -
>>>>>>>>>>>>>> -
>>>>>>>>>>>> -
>>>>>>>>>>>>>> Intel Finland Oy
>>>>>>>>>>>>>> Registered Address: PL 281, 00181 Helsinki Business
>>> Identity
>>>>>>>> Code:
>>>>>>>>>>>>>> 0357606 - 4 Domiciled in Helsinki
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This e-mail and any attachments may contain confidential
>>>>>>>> material
>>>>>>>>>>>> for
>>>>>>>>>>>>>> the sole use of the intended recipient(s). Any review or
>>>>>>>>>>>> distribution
>>>>>>>>>>>>>> by others is strictly prohibited. If you are not the
>>> intended
>>>>>>>>>>>>>> recipient, please contact the sender and delete all
>> copies.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> MeeGo-distribution-tools mailing list
>>>>>>>>>>>>>> MeeGo-distribution-tools at lists.meego.com
>>>>>>>>>>>>>> http://lists.meego.com/listinfo/meego-distribution-tools
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> MeeGo-distribution-tools mailing list
>>>>>>>>>>>>> MeeGo-distribution-tools at lists.meego.com
>>>>>>>>>>>>> http://lists.meego.com/listinfo/meego-distribution-tools
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Islam Amer
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------------------------------------
>> --
>>> --
>>>>> -
>>>>>>> --
>>>>>>>> -
>>>>>>>>>>> -
>>>>>>>>>>> Intel Finland Oy
>>>>>>>>>>> Registered Address: PL 281, 00181 Helsinki Business Identity
>>>>>>> Code:
>>>>>>>>>>> 0357606 - 4 Domiciled in Helsinki
>>>>>>>>>>>
>>>>>>>>>>> This e-mail and any attachments may contain confidential
>>>>> material
>>>>>>>>>>> for the sole use of the intended recipient(s). Any review or
>>>>>>>>>>> distribution by others is strictly prohibited. If you are
>> not
>>>>> the
>>>>>>>>>>> intended recipient, please contact the sender and delete all
>>>>>>>> copies.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Thanks,
>>>>>>>>>> Islam Amer
>>>>>>>>>
>>>>>>>>> --------------------------------------------------------------
>> --
>>> --
>>>>> -
>>>>>>> --
>>>>>>>>> Intel Finland Oy
>>>>>>>>> Registered Address: PL 281, 00181 Helsinki Business Identity
>>> Code:
>>>>>>>>> 0357606 - 4 Domiciled in Helsinki
>>>>>>>>>
>>>>>>>>> This e-mail and any attachments may contain confidential
>>> material
>>>>>>> for
>>>>>>>>> the sole use of the intended recipient(s). Any review or
>>>>>>> distribution
>>>>>>>>> by others is strictly prohibited. If you are not the intended
>>>>>>>>> recipient, please contact the sender and delete all copies.
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Thanks,
>>>>>>>> Islam Amer
>>>>>>>
>>>>>>> ----------------------------------------------------------------
>> --
>>> --
>>>>> -
>>>>>>> Intel Finland Oy
>>>>>>> Registered Address: PL 281, 00181 Helsinki
>>>>>>> Business Identity Code: 0357606 - 4
>>>>>>> Domiciled in Helsinki
>>>>>>>
>>>>>>> This e-mail and any attachments may contain confidential
>> material
>>>>> for
>>>>>>> the sole use of the intended recipient(s). Any review or
>>>>> distribution
>>>>>>> by others is strictly prohibited. If you are not the intended
>>>>>>> recipient, please contact the sender and delete all copies.
>>>>>>> _______________________________________________
>>>>>>> MeeGo-distribution-tools mailing list
>>>>>>> MeeGo-distribution-tools at lists.meego.com
>>>>>>> http://lists.meego.com/listinfo/meego-distribution-tools
>>>>>> -----------------------------------------------------------------
>> --
>>> --
>>>>>> Intel Finland Oy
>>>>>> Registered Address: PL 281, 00181 Helsinki
>>>>>> Business Identity Code: 0357606 - 4
>>>>>> Domiciled in Helsinki
>>>>>>
>>>>>> This e-mail and any attachments may contain confidential material
>>> for
>>>>>> the sole use of the intended recipient(s). Any review or
>>> distribution
>>>>>> by others is strictly prohibited. If you are not the intended
>>>>>> recipient, please contact the sender and delete all copies.
>>>>>> _______________________________________________
>>>>>> MeeGo-distribution-tools mailing list
>>>>>> MeeGo-distribution-tools at lists.meego.com
>>>>>> http://lists.meego.com/listinfo/meego-distribution-tools
>>>>>
>>>>>
>>>>> --
>>>>> "Don't worry, you'll be fine; I saw it work in a cartoon once..."
>>>> -------------------------------------------------------------------
>> --
>>>> Intel Finland Oy
>>>> Registered Address: PL 281, 00181 Helsinki
>>>> Business Identity Code: 0357606 - 4
>>>> Domiciled in Helsinki
>>>>
>>>> This e-mail and any attachments may contain confidential material
>> for
>>>> the sole use of the intended recipient(s). Any review or
>> distribution
>>>> by others is strictly prohibited. If you are not the intended
>>>> recipient, please contact the sender and delete all copies.
>>>
>>>
>>> --
>>> "Don't worry, you'll be fine; I saw it work in a cartoon once..."
>>
>> _______________________________________________
>> MeeGo-distribution-tools mailing list
>> MeeGo-distribution-tools at lists.meego.com
>> http://lists.meego.com/listinfo/meego-distribution-tools
> 

-- 
Thanks,
Islam Amer

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: OpenPGP digital signature
URL: <http://lists.meego.com/pipermail/meego-distribution-tools/attachments/20111028/b06c1f21/attachment.pgp>