Tag Archives: Weblogic

Changing the Weblogic user password

Since Weblogic stores it’s primary admin user account “weblogic” encrypted on disc, simply changing this users password in the console is not all there is.

Basic step: changing the password:

  • Go to: Security Realm -> myrealm > Users and Groups >weblogic -> Passwords
  • Change the password of the user, as you would normally do for every user

Stopping all managed servers

During step 3 we will change the NodeManager password. After this step the Admin console can no longer communicatie with the NodeManager so stopping/starting instances will not succeed.

Changing the Nodemanager password in the Weblogic Console:

  • Use the Weblogic /console
  • Go to Domain -> General -> Security -> Advanced
  • Change the value for the Nodemanager Password

Attention!!!: The next steps will be necessary on each physical server of the domain:

Changing the Nodemanager password:

If you start/stop the Managed Servers through the NodeManager you will need to edit the so called nm_properties file

  • Navigate your Linux/Windows file system and go to: %domain_home%/config/nodemanager
  • Open nm_properties
  • The file will hold encrypted values, replace all content with:


  • Restart the NodeManager
  • Check nm_properties file for encrypted values

#Node manager user information

Changing the Weblogic security file

If you decided to not use the NodeManager for start/stop Managed Servers but use the Weblogic start/stop scripts you will probably have your good reasons and you will need to edit the so called file

  • Navigate your Linux/Windows file system and go to: %domain_home%/servers/myserver01/security
  • Open the file
  • Change the values to


Launching the ManagedServer through the Startscript will test the result and encrypt the content.
Example:  %domain%\bin\startManagedWeblogic rbx_dev_wls01 http://server01:7001


Posted by on 08-01-2012 in Weblogic


Tags: ,

Configuring SSL for Oracle Weblogic (and OFMW)

We have a very basic domain: 3 physical servers, 1 containing the admin server, 2 containing 2 managed servers each, running in a cluster. Nice picture to follow. :)

name of the servers: server01 (Admin) / server02 (MS1+MS2) / server03 (MS3+MS4)

Step 0 – The Weblogic Demo Certificates (the reason why)

If you ever checked your Weblogic production logs and found the Security Alert below, then your Weblogic domain has a huge gaping security hole. Even if you don’t want SSL communication for your webservices or applications. The internal (administrative) processes in your Weblogic domain still relies on the default demotrust and with this everyone can access your domain.


<Alert> <Security> <SERVER02> <rbx_dev_soa_ms01> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <BEA-090152> <Demo trusted CA certificate is being used in production mode: [
  Version: V3
  Subject: CN=CACERT, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US
  Signature Algorithm: MD5withRSA, OID = 1.2.840.113549.1.1.4

  Key:  Sun RSA public key, 512 bits
  modulus: 95501928778692415440753....
  public exponent: 65537
  Validity: [From: Thu Mar 21 21:12:27 CET 2002,
               To: Tue Mar 22 21:12:27 CET 2022]
  Issuer: CN=CACERT, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US
  SerialNumber: [    33f10648 fcde0deb 4199921f d64537f4]

Certificate Extensions: 1
[1]: ObjectId: Criticality=true
KeyUsage [

  Algorithm: [MD5withRSA]
0000: 9D 26 4C 29 C8 91 C3 A7   06 C3 24 6F AE B4 F8 82  .&L)......$o....
0010: 80 4D AA CB 7C 79 46 84   81 C4 66 95 F4 1E D8 C4  .M...yF...f.....
0020: E9 B7 D9 7C E2 23 33 A4   B7 21 E0 AA 54 2B 4A FF  .....#3..!..T+J.
0030: CB 21 20 88 81 21 DB AC   90 54 D8 7D 79 63 23 3C  .! ..!...T..yc#<

] The system is vulnerable to security attacks, since it trusts certificates signed by the demo trusted CA.>

Step 1 – generate the identity java keystore

We first generate a java keystores containing a private key for each physical server, since SSL hostname verifiation is based on machine names. We use “welcome1” as password here, but make sure to choose a complex password here, use it during these steps and then lock it safely away.

keytool.exe -genkey -keysize 2048 -keyalg RSA -alias server01 -keystore server01_identity.jks

Enter keystore password: welcome1
 Re-enter new password: welcome1
 What is your first and last name?
   [Unknown]:  server01
 What is the name of your organizational unit?
 What is the name of your organization?
   [Unknown]:  Rubix
 What is the name of your City or Locality?
 What is the name of your State or Province?
 What is the two-letter country code for this unit?
   [Unknown]:  NL
 Is CN=server01, OU=, O=Rubix, L=, ST=, C=NL correct?
   [no]:  yes

Step 2 – generate signing request

keytool.exe -certreq -alias server01 -file server01.csr -keystore server01_identity.jks</div>
Repeat step 1 + 2 for each physical server, replacing the -alias parameter, -keystore parameter and CN name with the correct server0* value.


Step 3 – the Certificate Authority (CA)

Now comes the tricky part that you probably need some outside forces. You will need the correct Certificate Authority to sign your certificate requests (the .CSR files you generated). You can create your own CA and self-sign them, but I would not recommend that unless you know what you are doing and understand the consequences. You could use an external Internet CA provider, but this becomes very costly and time consuming when you need such an external provider to sign every SSL enabled server in your landscape. The best situation for us would be if the current organization already has an internal CA provider, especially when the rootCA is trusted by the servers and machines in your landscape.
Let’s assume however that you receive the signed response, including the public keys of the rootCA and a intermediateCA. CA instances often use a intermediate CA to authorize certificates, so if they have “problems” they can withdraw this intermediate CA and don’t need to withdraw the whole RootCA making it useless.
So the situation should be that you know hold 3 PEM/DER files.


Step 4 – importing the CA response

Import the certificates in your keystore, starting with the rootCA, then the intermediateCA, then the specific server alias. Example for server01:
keytool -importcert -trustcacerts -alias rootca -file rootCA.cer -keystore server01_identity.jks

keytool -importcert -alias intermediateca -file intermediateCA.cer -keystore server01_identity.jks

keytool -importcert -alias server01 -file server01.cer -keystore server01_identity.jks
Repeat these steps for server02 and server03. So we know have 3 identity keystores for each server containing the signed private key and it’s certificate chain.


Step 5 – generate the trust java keystore

Now let’s generate a truststore for each server. Altough you can use the identical keystore for both identity and trust, it’s always a good practice to separate them. The identity keystone contains the private key and normally never changes during it’s lifetime, so you should secure it with a complex password. The trust store contains the certificates that you trust and as your middleware landscape grows, this file might grow and more administrators will need access to it.
Example generating the truststore for server01, repeat for 02 & 03.
keytool -importcert -trustcacerts -alias rootca -file rootCA.cer -keystore server01_trust.jks
keytool -importcert -alias intermediateca -file intermediateCA.cer -keystore server01_trust.jks

Step 6 – copy the java keystore files to each server

 Make sure that each server had the files stored on disk. We use the windows server example “c:\oracle\security” in this blogpost. So server01 should hold:
– c:\oracle\security\server01_identity.jks
– c:\oracle\security\server01_trust.jks


Step 7 – configure the Node Manager for SSL

The nodemanger by default uses the demoIdentity and demoTrust keystores.
So on each server hosting a managed server (server02 and server03) we will need to change this.
Open the and add the following lines:
### SSL config
# CustomIdentityKeyStoreType=JKS
# CustomTrustKeyStoreType=JKS
Restart the NodeManager to activate the changes. The passwords in the properties file will automatically become encrypted.
Possible error:
For server rbx_dev_soa_ms03, the Node Manager associated with machine server03 is not reachable. All of the servers selected are currently in a state which is incompatible with this operation or are not associated with a running Node Manager or you are not authorized to perform the action requested. No action will be performed.
Possible solution:
Your AdminServer can not communicate with a node manager. Make sure it’s running, check Environment -> Machines in the Weblogic console if the machine has state reachable, check if the address configured there, and the address in the and the CN-name used in SSL have the same value, check if Weblogic and Nodemanager use the same SSL configuration (trust+identity).


Step 8 – configure Weblogic for SSL

Log in the Weblogic console, in our example: http://server01:7001/console
Step 8.1 – configure Weblogic for SSL, enable SSL
Log in the Weblogic console (in our case http://server01:7001/console) and make sure that each server has the SSL port enabled. You can find the setting “SSL Listen Port Enabled” for each server under “Configuration -> General”. Enable it and make sure you use a correct and free port.

Step 8.2 – configure Weblogic for SSL, changing keystores

For each server (Admin + MS) we have to change the KeyStore configuration.
So for each server go to: Configuration -> Keystores
  • Keystores -> “Change” = “Custom Identity and Custom Trust”
  • Custom Identity Keystore = c:/Oracle/security/serverXX_identity.jks
  • Custom Identity Keystore Type = JKS
  • Custom Identity Keystore Passphrase = welcome1
  • Custom Trust keystore = c:/Oracle/security/serverXX_trust.jks
  • Custom Trust Keystore Type = JKS
  • Custom Trust Keystore Passphrase = welcome1

Step 8.3 – configure Weblogic for SSL, changing identity:

For each server (Admin + MS) we have to change the SSL configuration.
So for each server go to: Configuration -> SSL
  • Private Key Alias = serverXX
  • Private Key Passphrase = welcome1

Step 9 – Weblogic start scripts:

By default Weblogic uses the DemoTrust.jks truststore. Since we changed this in the console configuration you would hope this would be enough. However the / SetDomainEnv.cmd script still contains a reference. So for each server:
  • Go to %domain_home%/bin/
  • Open the SetDomainEnv script
  • Search the argument:
  • Make sure the value is not referring to the demo trust.jks but “c:/oracle/security/serverXX_trust.jks
When you are finished restart your whole Weblogic environment (Admin + MS).

Step 10 – Configure Enterprise Manager:

This took me some research and the Oracle Forums to get it working. Navigate to the enterprise manager console (/em)

Step 10.1 – Security Provider Configuration:

I’m actually not really sure if this step is still necessary in 11g, but hey .. why not be sure:
  • Click Weblogic -> right-click domain
  • Select Security -> Security Provider Configuration -> KeyStore Configure
  • Keystore Path = c:/oracle/security/server01_identity.jks

Step 10.2 – SOA Infra properties

  • Click SOA -> right-click soa-infra
  • Select SOA Administration -> Common Properties
  • Click More SOA Infra Advanced Properties
  • Keystore Location = c:/oracle/security/serverXX_identity.jks

Step 10.3 – Credential Mapping:
The next step is to configure the credential mapping keys

  • Click Weblogic -> right-click domain name
  • Select Security -> Credentials
  • Create a Map named “SOA”
  • Click Create a key
  • Key = KeyPassword
  • User Name = KeyPassword
  • Password = welcome1
  • Click Create Key:
  • Key = KeyStorePassword
  • User Name = KeyStorePassword
  • Password = welcome1
Possible error:

<Warning> <> <J2EE JMX-46041> <The resource for bundle "" with key "oracle.soa.BPELConfigBean.auditFlushByteThreshold" cannot be found.>
INFO: SSLSocketFactoryManagerImpl.getKeystoreLocation SOA Keystore location: c:/oracle/security/server04_identity.jks
INFO: SSLSocketFactoryManagerImpl.getKeystorePassword Obtained null or empty keystore password
INFO: SSLSocketFactoryManagerImpl.getKeyPassword Obtained null or empty key password
INFO: SSLSocketFactoryManagerImpl.getSSLSocketFactory Could not obtain keystore location or password

Possible solution:
You did not configure the SOA key mapping correctly as mentioned above in 9.3



Enabling SSL takes a lot of steps, and we didn’t even mentioned it for enabling SSL client identity in OSB (Service Key Provider) and communication with an LDAP over SSL. But there’s more to come I guess.
If you have any remarks please let me know in the comments. Hope it helps. :)



Posted by on 16-12-2011 in Security, SOA Suite, Weblogic


Tags: ,

Why is there no Weblogic WLST Node Manager nmShutdown command ?

The great thing about the Weblogic Nodemanager is that you can control your Weblogic Admin and Managed servers through Weblogic Scripting Tool (WLST) without the AdminServer being online. Simply said, with a few basic WLST commands you can perform important administrative tasks to your Weblogic domain and don’t need a running AdminServer(!).

For instance, starting a server:

nmConnect('weblogic', 'welcome1', 'myserver3', '5555', 'rbx_dev_domain', 'd:/oracle/projects/domains/rbx_dev_domain')

The problem arises when you want to gracefully shutdown a server through the Node Manager. As shown here there is no nmStop/nmShutdown or nmSuspend command, only a very blunt knife called nmKill. I tried nmKill to experiment with it, and concluded that it does exactly what it claims. However applications (and administrators) might not like this forcefully termination.

So how can we stop a Managed Server through WLST then ? The answer is simple, we need to connect with WLST to the running AdminServer and execute our commands in online mode:

# connect to the Admin running on server01, port 7001
wls:/offline> connect("weblogic","welkome1","t3://server01:7001")

# shutdown MS01, ignoreSessions=false, timeout=300sec, force=false, block=false
wls:/rbx_dev_domain/serverConfig> shutdown('rbx_dev_soa_ms01','Server','false',300000,'false','false')

wls:/rbx_dev_domain/serverConfig> disconnect()

The online (meaning you need to connect to a running Admin) WLST life cycle commands are all there: shutdown, start, suspend, resume and even migrate. Creating a powerful toolset to WLST script against your Weblogic domain.


I personally think this is a limitation of the Weblogic Node Manager. The power of the NodeManager should be exactly what it name claims to be, managing a certain ‘node’. And I think that both starting and stopping should be a very basic requirement.

In this case I think it fails, since you can not perform this administrative tasks to your domain without connecting to a running AdminServer. Which is a shame if you don’t want/need your AdminServers to be running constantly. Yes you can script around it, using the nmStart() command to start the AdminServer if not running, then nmDisconnect() and connect() to the now running AdminServer but it’s a lot of effort if you could simply succeed with a nmShutdown.



Posted by on 13-12-2011 in Weblogic, WLST


Tags: , ,

Fixing BEA-090504 Certificate Chain warnings after enabling Weblogic Administration port

After we enabled SSL on all our Weblogic machines in our environment (multiple hosts) we enabled the Administration port for our domain:

The domain-wide administration port enables you to start a WebLogic Server instance in STANDBY state. It also allows you to separate administration traffic from application traffic in your domain. Because all servers in the domain must enable or disable the administration port at once, you configure the default administration port settings at the domain level. If you enable the administration port:
The administration port accepts only connections that specify administrator credentials. Connections that specify administrator credentials can use only the administration port. Because the administration port uses SSL, enabling the administration port requires that SSL must be configured for all servers in the domain. (source: Weblogic console)

Everything started up fine, however in the weblogic logfiles of every managed server we did see the following message:

<Warning> <Server> <server2> <rbx_dev_wls_01> <[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1322835992044> <BEA-090504> <Certificate chain received from server1.rubix.local - failed hostname verification check. Certificate contained server1 but check expected server1.rubix.local>

<Notice> <WebLogicServer> <server2> <rbx_dev_wls_01> <main> <<WLS Kernel>> <> <> <1322835992137> <BEA-000365> <Server state changed to RUNNING>

<Notice> <WebLogicServer> <server2> <rbx_dev_wls_01> <main> <<WLS Kernel>> <> <> <1322835992153> <BEA-000360> <Server started in RUNNING mode>

<Warning> <Log Management> <server2> <rbx_dev_wls_01> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1322835996646> <BEA-170011> <The LogBroadcaster on this server failed to broadcast log messages to the admin server. The Admin server may not be running. Message broadcasts to the admin server will be disabled.>

<Warning> <Server> <server2> <rbx_dev_wls_01> <[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1322835992044> <BEA-090504> <Certificate chain received from server1.rubix.local - failed hostname verification check. Certificate contained server1 but check expected server1.rubix.local>

The CN value of all our SSL certificates are based upon the basis hostname of the server. In this case the admin runs on server1 and the managed server on server2. The Managed Server performs a SSL hostname verification and this fails due to the fact that server1 is trying to communicate with server2 over channel server1.rubix.local but server receives the identity based upon CN=server1.

We needed to override Weblogic to use the server1 communication channel instead of the server1.rubix.local channel which it default is trying to use. This could be easily fixed by going to the Weblogic console and check each Managed Server and Admin server for it’s Listen Address.

I’ve seen more problems with Weblogic environments (also on Windows) where the Listen Address is not configured. For me reason enough to use a best-practice to always configure the Listen Address with a fixed value.


Posted by on 02-12-2011 in Weblogic


Tags: ,

Creating Weblogic JMS components with WLST

The previous post explained the different Weblogic JMS components. As promised I would share some WLST scripts which I use to create the different JMS components as an example.

The example set of scripts exist of 3 files:

  • a generic file which hold the connection settings
  • JMSCreateInfra.jy which configures the JMS persistent stores, servers, modules and CF
  • JMSCreateQueue.jy which creates a queue and configures redirect
The very basic
# Weblogic connection parameters                                 #

And the WLST script to configure the Persistent Store, JMS Server, JMS Module and CF on your environment. The script is based upon a Weblogic domain containing a Weblogic cluster with 2 Managed Servers. If you want to use these scripts make sure to change the:

– path for persistent store
– cluster name (rbx_dev_wls_cluster)
– managed server name (rbx_dev_wls_0*)
– persistent store name (FileStore_wls0*)
– jms server name (JmsServer_wls0*)
– jms module name (JmsModuleRubix)
– CF name (ConnFactoryGenericRubix*)
– queue names (SR01.getMyData.*)
– sub deployment name (RBX.myDivision)

Execute the script:  java weblogic.WLST scriptname.jy

from import FileInputStream
import sys

# Generic definitions

def loadProps(configPropFile):
propInputStream = FileInputStream(configPropFile)
configProps = Properties()
return configProps

# Load properties and connect to Weblogic server

importConfigFile = sys.argv[1]
exportConfigProp = loadProps(importConfigFile)
adminUrl = exportConfigProp.get("adminUrl")
importUser = exportConfigProp.get("importUser")
importPassword = exportConfigProp.get("importPassword")

# Start the script

connect(importUser, importPassword, adminUrl)

# Build JMS Filestores
# Filestores store the messages

set('Targets', jarray.array([ObjectName('com.bea:Name=rbx_dev_wls_01,Type=Server')], ObjectName))
set('Targets', jarray.array([ObjectName('com.bea:Name=rbx_dev_wls_02,Type=Server')], ObjectName))

# Build JMS Server(s)
# For every MS in cluster define a JMSserver and target a single MS server

set('Targets', jarray.array([ObjectName('com.bea:Name=rbx_dev_wls_01,Type=Server')], ObjectName))

## Threshold (values are default)
# cmo.setBytesThresholdHigh(-1)
# cmo.setBytesThresholdLow(-1)
# cmo.setMessagesThresholdHigh(-1)
# cmo.setMessagesThresholdLow(-1)
## Quotas (values are default, except MaxMsgSize set to 10MB)
# cmo.setBytesMaximum(-1)
# cmo.setMessagesMaximum(-1)
# cmo.setBlockingSendPolicy('FIFO')

set('Targets', jarray.array([ObjectName('com.bea:Name=rbx_dev_wls_02,Type=Server')], ObjectName))
## Threshold (values are default)
# cmo.setBytesThresholdHigh(-1)
# cmo.setBytesThresholdLow(-1)
# cmo.setMessagesThresholdHigh(-1)
# cmo.setMessagesThresholdLow(-1)
## Quotas (values are default, except MsgMaxSize is 10MB)
# cmo.setBytesMaximum(-1)
# cmo.setMessagesMaximum(-1)
# cmo.setBlockingSendPolicy('FIFO')

# Build JMS Module
# target preferrable cluster, single-server DEV domain use server

set('Targets',jarray.array([ObjectName('com.bea:Name=rbx_dev_wls_cluster,Type=Cluster')], ObjectName))

# Build JMS Connection Factory


# Build JMS Connection Factory XA


# Finalize


# End

Create 3 queues, a request, response and a error queue.
Failure is configured to use redirect to the error queue.
A read service (hence the SR naming) mechanism I like to combine for the Oracle Service Bus

Execute the script:  java weblogic.WLST scriptname.jy

# Build JMS Subdeployment

set('Targets',jarray.array([ObjectName('com.bea:Name=rbx_dev_wls_cluster,Type=Cluster')], ObjectName))

# Build Queues: SR01.getMyData

# Error Queue

# Request queue

# Response Queue

Posted by on 01-12-2011 in Weblogic, WLST


Tags: , ,

Understanding the Oracle Weblogic JMS infrastructure

I decided to create some blogposts about the wonderful world of Weblogic JMS. This due to the fact I ran into the same questions at different clients.

I will start of with this blogpost about the different Weblogic JMS components, their relation with each other and what their main configuration, logging and monitoring settings are.  The idea of this post is to provide a quick start in understanding the components of the JMS infrastructure within Oracle Weblogic.

Weblogic JMS components overview:

The following overview gives a simple idea of the relations between the JMS components in Oracle Weblogic:

Where you would like to use a Weblogic Cluster as target for JMS Modules instead of specific targeting a Managed Server. Clusters can contain 1-n Managed Servers, so even if you have a small Weblogic domain you can create an environment which already uses clustering and is ready for future domain expansion.

Persistent Stores:

  • 2 types: filestore (disc) and JDBCstore (database)
  • Target: 1 Managed Server
  • Some situation require dedicated persistent stores due to security/audit regulations of the information        stored. Keep this in mind in your JMS infra design.
  • Important configuration:
    • location (folder on disc) or connection (database)
    • maximum file size of persistent filestore (default = 1.342.177.280 bytes)
    • send policy: First-In-First-Out as default, but preemptive allows more speed for huge small messages
    • write-policy: Important for shared storage solutions
  • Monitoring:
    • Statistics (create/read counts, buffers)
    • Active filestore connections (from JMS Server)

JMS Servers:

  • Target: 1 Managed Server
  • Connected to a Persistent Store
  • Important configuration:
    • Threshholds bytes/messages: becomes “ARMED” and instructs producers to slow dow, default disabled !!!!
    • Maximum bytes/messages: maximum for incoming messages, Oracle recommends total amount of system memory available after accounting rest of application load. Default disabled !!!!
    • Maximum individual Message Size: Default is 2.147.483.647 !!!! which is huge and in most of my client cases totally unrealistic. Prevent future issues and tune this down.
  • Monitoring en Logging:
    • Configuratie where JMS logging is located, not WHAT to log
    • Monitoring of messages ( total, pending, current) and size

JMS Module:

  • Target: Preferred to Weblogic Cluster(s) for scaling, but also possible to single or multiple managed server(s)
  • Owner of 0-n SubDeployments
  • Contains artifacts such as queues, topics and connections factories (CF)
  • Important configuration:
    • Security, users and policies (specific hours, etc)
    • Location for JMS Connection Factory used by client connecting

JMS Modules – SubDeployments

  • Configured in the JMS Module
  • Target: Weblogic Cluster
  • Allows grouping of queues and topics

JMS Module – Queue/Topic

  • Most commonly used JMS artifacts
  • Configured in the JMS Module
  • Configured in the SubDeployment (preferred)
  • Important configuration (there is much much more):
    • Java Naming and Directory Interface (JNDI) which provides a lookup facility
    • Individual threshold and quotas
    • Individual override for time-to-live, priority and delivery mode (persistency)
    • Failure options: amount of redelivery, redirect to error queue/topic or discard
    • Monitoring en Logging:
      • Defines WHAT to log, full body/header or specific header fields as CorrelationID, MessageID.
      • Amount of consumers/subscribers

My personal Weblogic JMS best practices:

  • Certain Oracle products, like the Oracle SOA suite, come with their own JMS artifacts like modules, queues, topics and connection factories (CF). Just leave them and do NOT create your own queues/topics/etc in these modules and “abuse” their Persistant Stores. They are not yours ;)

  • Use queues for point-to-point and topics for publish/subscribe. If you design a publish/subscribe point-to-point connection try to use a topic for future 1-n delivery. If not possible, start with a queue and think about your JNDI naming so you can always change the configuration from queue to topic in the future.
  • If you need queues/topics that are JTA user-transaction aware you need a XA Connection Factory. Make sure your timeout (default = 3600 seconds) is enough for any long running transactions.
  • When you don’t configure the Persistent FileStore location, it will create a %filestore_name% folder in your weblogic %domain% folder. If no complex shared storage solutions are used, I prefer to  create a /filestores/ folder in the domain root and target all filestores there (example: ../filestores/myFilestoreA/) to prevent a mess in your %domain% folder. Make sure that the folder exists, to prevent errors like:  Caused by: [Store:280044] The file store directory “….\rbx_dev_domain\filestores” does not exist
  • Use Distributed Queues and Distributed Topics to make use of the scalability Weblogic clustering technology provides
  • Think about your naming convention for both internal and external (JNDI)
  • Tune your Weblogic JMS instance by defining quotas, thresholds and sizes. See this blogpost from Darrel for more information.

As I wrote before, I am not all-knowing, so if you have different insights or can add useful information to this post please do.


1 Comment

Posted by on 30-11-2011 in Weblogic


Tags: ,

Using the ADRCI utility with Oracle Weblogic

ADRCI (Automatic Diagnostic Repository Command Interface) is a command line utility that originated as a Oracle database software utility. Recent versions of Oracle Weblogic also include this utility. For the experienced DBA this utility will probably hold no secrets, but for the friendly neighborhood middleware admins some quick start instructions might save you some time.

What does it do:
If your Weblogic Managed Server detects a problem (and incident) it will generate a diagnostics dump and store it in the folder: %DOMAIN_HOME%/servers/myManagedServer1/adr/diag

The command-tool (adrci) is located in your %WL_HOME%\server\adr folder.
Before starting the tool, make sure this folder is added to your PATH (I explain later why).
Windows => Set PATH=C:\Oracle\Middleware\wlserver_10.3\server\adr;%PATH%
Linux => export PATH=$PATH:/opt/oracle/middleware/wlserver_10.3/server/adr/

First we need to set the adr base folder, and then possibly check it:

adrci> set base c:/Oracle/Middleware/user_projects/domains/myDomain/servers/osb_server1/adr
adrci> show home
ADR Homes:

Now lets look at the problems registered:

adrci> show problem
-------------------- ------------------------------------------------------------
2           BEA-802 [Kernel]                          14              2011-11-22 18:00:00.000000 +01:00
3           BEA-101017 [HTTP][]    13              2011-11-21 17:00:00.000000 +01:00

A problem may occur multiple times, each occurance is registered as an incident. The problem view only shows it’s last incident ID and occurance.

To get an basic overview of all the incidents:

adrci> show incident -mode BASIC

ADR Home = c:\oracle\middleware\user_projects\domains\myDomain\servers\osb_server1\adr\diag\ofm\myDomain\osb_server1:

INCIDENT_ID     PROBLEM_KEY                             CREATE_TIME
11              BEA-802 [Kernel]                        2011-11-19 15:00:00.000000 +01:00
12              BEA-802 [Kernel]                        2011-11-20 16:00:00.000000 +01:00
13              BEA-101017 [HTTP][]  2011-11-21 17:00:00.000000 +01:00
14              BEA-802 [Kernel]                        2011-11-22 18:00:00.000000 +01:00

To get a detailed overview of a specific incident:

adrci> show incident -mode DETAIL -p "incident_id=13"

ADR Home = c:\oracle\middleware\user_projects\domains\myDomain\serves\osb_server1\adr\diag\ofm\myDomain\osb_server1:

INCIDENT_ID                   13
STATUS                        ready
CREATE_TIME                   2011-11-21 17:00:00.000000 +01:00
PROBLEM_ID                    3
CLOSE_TIME                    <NULL>
FLOOD_CONTROLLED              none
ERROR_FACILITY                BEA
ERROR_NUMBER                  101017
ERROR_ARG1                    <NULL>
ERROR_ARG2                    <NULL>
ERROR_ARG3                    <NULL>
ERROR_ARG4                    <NULL>
ERROR_ARG5                    <NULL>
ERROR_ARG6                    <NULL>
ERROR_ARG7                    <NULL>
ERROR_ARG8                    <NULL>
ECID                          da261c9725811311
IMPACTS                       0
PROBLEM_KEY                   BEA-101017 [HTTP][]
FIRST_INCIDENT                8
FIRSTINC_TIME                 2011-11-04 12:00:00.000000 +01:00
LAST_INCIDENT                 13
LASTINC_TIME                  2011-11-21 17:00:00.000000 +01:00
IMPACT1                       0
IMPACT2                       0
IMPACT3                       0
IMPACT4                       0
KEY_NAME                      ECID
KEY_VALUE                     da261c9725811311:......
OWNER_ID                      1
INCIDENT_FILE                 ...\osb_server1\inciden\incdir_13\
OWNER_ID                      1
INCIDENT_FILE                 ...\osb_server1\inciden\incdir_13\dms_metrics2_i13.dmp
OWNER_ID                      1
INCIDENT_FILE                 ...\osb_server1\inciden\incdir_13\odl_logs4_i13.dmp
OWNER_ID                      1
INCIDENT_FILE                 ...\osb_server1\inciden\incdir_13\readme.txt
OWNER_ID                      1
INCIDENT_FILE                 ...\osb_server1\inciden\incdir_13\odl_logs3_i13.dmp
OWNER_ID                      1
INCIDENT_FILE                 ...\osb_server1\inciden\incdir_13\jvm_threads1_i13.dmp
1 rows fetched

We can create a package for the specific incident 13, or a package for the whole problem 3

Created package 4 based on incident id 13, correlation level typical

Created package 5 based on problem id 3, correlation level typical

We now have 2 logical packages, which means they were created but not yet written to disk.To be able to send the packages to Oracle Support we have to fire off the following command:

adrci> IPS GENERATE PACKAGE 4 IN c:/temp
Generated package 4 in file c:/temp\, mode complete

adrci> IPS GENERATE PACKAGE 5 IN c:/temp
Generated package 5 in file c:/temp\, mode complete

Voila, you have your package ready to send to Oracle Support.

If you FORGOT to set the PATH before starting ADRCI, you may receive this error message:

adrci> IPS GENERATE PACKAGE 1 IN c:/temp
 'perl' is not recognized as an internal or external command,
 operable program or batch file.
 'perl' is not recognized as an internal or external command,
 operable program or batch file.
 'zip' is not recognized as an internal or external command,
 operable program or batch file.
 DIA-49450: Non-zero return code from archiving utility [0] [1]

*Update 28-11-2011*
Found a excellent blogpost from Uwe Hesse which also covers the ADRCI commands for managing trace files. Added Link below.


Leave a comment

Posted by on 25-11-2011 in Weblogic


Tags: ,