Advertisements
RSS

Multiple OSB domains connect in single Coherence OSB-cluster

09 Aug

When starting one of our Oracle Service Bus development domains (11g R1 PS4 11.1.1.5) it’s managed servers logging showed the following:


####<Aug 8, 2012 2:34:50 PM CEST> <Info> <Coherence> <rbxdev24.local> <rbx_osb_dev_server_2> <Logger@9213666 3.6.0.4> <<anonymous>> <> <0000J^6Ed2wBd5G_ux8DyX1G8_n3000001> <1344429290942> <BEA-000000> <Oracle Coherence 3.6.0.4 (member=n/a): Started cluster Name=OSB-cluster

Group{Address=228.8.8.8, Port=9888, TTL=4}

MasterMemberSet
 (
 ThisMember=Member(Id=14, Timestamp=2012-08-08 14:34:50.57, Address=10.0.10.93:7890, MachineId=27485, Location=site:local,machine: ,process:29347, Role=OSB-node)
 OldestMember=Member(Id=7, Timestamp=2012-07-08 12:42:49.99, Address=10.0.10.170:7890, MachineId=27562, Location=site:local,machine:rbxtst52,process:23992, Role=OSB-node)
 ActualMemberSet=MemberSet
 (
 Size=12, BitSetCount=2
 Member(Id=1, Timestamp=2012-08-06 14:55:54.263, Address=10.0.10.180:7890, MachineId=27572, Location=site:local,machine:rbxtst62,process:1746, Role=OSB-node)
 Member(Id=2, Timestamp=2012-08-06 14:56:03.677, Address=10.0.10.179:7890, MachineId=27571, Location=site:local,machine:rbxtst61,process:11379, Role=OSB-node)
 Member(Id=3, Timestamp=2012-07-23 07:53:26.121, Address=10.0.10.167:7890, MachineId=27559, Location=site:local,machine:rbxdev52,process:21259, Role=OSB-node)
 Member(Id=6, Timestamp=2012-07-23 07:53:58.54, Address=10.0.10.166:7890, MachineId=27558, Location=site:local,machine:rbxdev51,process:24888, Role=OSB-node)
 Member(Id=7, Timestamp=2012-07-08 12:42:49.99, Address=10.0.10.170:7890, MachineId=27562, Location=site:local,machine:rbxtst52,process:23992, Role=OSB-node)
 Member(Id=8, Timestamp=2012-07-08 12:42:52.28, Address=10.0.10.169:7890, MachineId=27561, Location=site:local,machine:rbxtst51,process:28782, Role=OSB-node)
 Member(Id=9, Timestamp=2012-07-18 08:33:29.546, Address=10.0.10.91:7890, MachineId=27483, Location=site:local,machine:rbxtst24,process:23872, Role=OSB-node)
 Member(Id=10, Timestamp=2012-07-18 08:33:30.24, Address=10.0.10.90:7890, MachineId=27482, Location=site:local,machine:rbxtst23,process:6073, Role=OSB-node)
 Member(Id=11, Timestamp=2012-07-30 15:07:59.069, Address=10.0.10.177:7890, MachineId=27569, Location=site:local,machine:rbxdev62,process:32088, Role=OSB-node)
 Member(Id=12, Timestamp=2012-07-30 15:08:00.223, Address=10.0.10.176:7890, MachineId=27568, Location=site:local,machine:rbxdev61,process:30222, Role=OSB-node)
 Member(Id=13, Timestamp=2012-08-08 14:34:50.401, Address=10.0.10.92:7890, MachineId=27484, Location=site:local,machine:rbxdev23,process:9584, Role=OSB-node)
 Member(Id=14, Timestamp=2012-08-08 14:34:50.57, Address=10.0.10.93:7890, MachineId=27485, Location=site:local,machine:rbxdev24,process:29347, Role=OSB-node)
 )
 RecycleMillis=1200000
 RecycleSet=MemberSet(Size=0, BitSetCount=0)
 )
TcpRing{Connections=[13]}
IpMonitor{AddressListSize=11}
>

As you can see the memberset of the Coherence cluster contains 12 machines, while this OSB domain only excists of 2 managed servers (rbxdev23 and rbxdev24). The other machines are part of different development and test Oracle Service Bus domains (yes we have multiple OSB domains). Inspecting the other loggings show identical behaviour on all OSB managed servers mentioned.

Looking at the group address:

Group{Address=228.8.8.8, Port=9888, TTL=4}

It looks like Multicast is used and looking at the individual machines ip-addresses you can guess that all these machines are part of the same network subnet.

The coherence OSB-Cluster configuration files are located here: /%domainhome%/config/osb/coherence
./osb-coherence-cache-config.xml
./osb-coherence-override.xml

(not to be confused with your own created Coherence Cluster configuration stored here: /%domain_home%/config/coherence

The file osb-coherence-override.xml contains the configuration for the Coherence cluster. Multiple solutions were possible:

  • keep using multicast but configure each domain with an unique address
  • use unicast and it’s well-known-address option

We decided to solve this by using the Unicast option. We had multicast in the past for our Weblogic 8 and 9 clusters (before Unicast became available and the default setting) and it’s a pain because:

  • somebody needs to Administer these ranges
  • we had unknown network hosts trying to connect to our multicast addresses and filling our logging

So editing the osb-coherence-override.xml

</pre>
<cluster-config>
 <unicast-listener>
 <well-known-addresses>
 <socket-address id="1">
 <address system-property="OSB.coherence.wka1">rbxdev23.local</address>
 <port system-property="OSB.coherence.wka1.port">7890</port>
 </socket-address>
 <socket-address id="2">
 <address system-property="OSB.coherence.wka2">rbxdev24.local</address>
 <port system-property="OSB.coherence.wka2.port">7890</port>
 </socket-address>
 </well-known-addresses>
 <address system-property="OSB.coherence.localhost">127.0.0.1</address>
 <port system-property="OSB.coherence.localport">7890</port>
 </unicast-listener>
 <multicast-listener>
 <address system-property="OSB.coherence.clusteraddress">228.8.8.8</address>
 <port system-property="OSB.coherence.clusterport">9888</port>
 </multicast-listener>
 </cluster-config>
</coherence>

After restarting our OSB domain the following information is stored in the logging:


####<Aug 8, 2012 4:00:54 PM CEST> <Info> <Coherence> <rbxdev23.local> <rbx_osb_dev_server_1> <Logger@9226098 3.6.0.4> <<anonymous>> <> <0000J^6YNsRBd5G_ux9DiX1G8b40000001> <1344434454571> <BEA-000000> <Oracle Coherence 3.6.0.4 (member=n/a): Started cluster Name=OSB-cluster

WellKnownAddressList
(
 Size=2,
 WKA{Address=10.0.10.93, Port=7890}
 WKA{Address=10.0.10.92, Port=7890}
)

MasterMemberSet
(
 ThisMember=Member(Id=1, Timestamp=2012-08-08 16:00:51.238, Address=10.0.10.92:7890, MachineId=27484, Location=site:local,machine:rbxdev23,process:10439, Role=OSB-node)
 OldestMember=Member(Id=1, Timestamp=2012-08-08 16:00:51.238, Address=10.0.10.92:7890, MachineId=27484, Location=site:local,machine:rbxdev23,process:10439, Role=OSB-node)
 ActualMemberSet=MemberSet
 (
 Size=1, BitSetCount=2 Member
 (Id=1, Timestamp=2012-08-08 16:00:51.238, Address=10.0.10.92:7890, MachineId=27484, Location=site:local,machine:rbxdev23,process:10439, Role=OSB-node)
 )
 RecycleMillis=1200000
 RecycleSet=MemberSet(Size=0, BitSetCount=0)
)
TcpRing{Connections=[]}
IpMonitor{AddressListSize=0}
>

Logging shows only 2 managed servers left in the Coherence cluster.

Conclusion (and opinion be aware):

I don’t really understand why Oracle decided to use Multicast as default for their OSB-Cluster coherence configuration. Personally I hate Multicast due to earlier experience, so that is more of a gut feeling. Maybe for Weblogic servers running Java Applications using Coherence it’s the best default. I don’t know. But for the OSB-Cluster coherence which is used for Business Service caching I don’t see the point. When you create a domain (or extend it with the OSB template) all the information is there at the moment it’s config.xml and other files are created/extended. Why not go the extra mile and make sure that the OSB coherence cluster uses unicast (with well-known-address) as default instead of the multicast mechanism ?
References:

http://coherence.oracle.com/display/COH35UG/cluster-config
http://coherence.oracle.com/display/COH35UG/well-known-addresses
http://coherence.oracle.com/display/COH35UG/multicast-listener

Advertisements
 
2 Comments

Posted by on 09-08-2012 in Coherence, Oracle, OSB

 

Tags: , ,

2 responses to “Multiple OSB domains connect in single Coherence OSB-cluster

  1. Malcolm

    16-02-2015 at 21:54

    I tried editing the osb-coherence-override.xml in ../config/osb/coherence/ and when I restarted OSB it got overwritten by the “default” so it didn’t work. Am I supposed to copy the default to some other location and edit it and if so what location should it be stored to?

     
    • James Humphrey

      18-11-2016 at 17:28

      We are having the same problem as Malcom. We edit osb-coherence-override.xml but upon restart the changes made to that file are gone.

       

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: