Category Archives: Uncategorized

Getting started with Apache Kafka

Apache Kafka is a publish-subscribe messaging solution rethought as a distributed commit log.


The original use case for Kafka was to be able to rebuild a user activity tracking pipeline as a set of real-time publish-subscribe feeds. This means site activity (page views, searches, or other actions users may take) is published to central topics with one topic per activity type. These feeds are available for subscription for a range of use cases including real-time processing, real-time monitoring, and loading into Hadoop or offline data warehousing systems for offline processing and reporting.

Some use cases for Kafka are stream processing, event sourcing, metrics and all other (large sets of) data that go from publisher to 1-n subscriber(s). A single Kafka broker can handle hundreds of megabytes of reads and writes per second from thousands of clients making it a very efficient (and also easy to scale) high volume messaging solution.

So actually Kafka is a good alternative for any more traditional (JMS / MQ) message broker. Message brokers are used for a variety of reasons (to decouple processing from data producers, to buffer unprocessed messages, etc). In comparison to most messaging systems Kafka has better throughput, built-in partitioning, replication, and fault-tolerance which makes it a good solution for large scale message processing applications. And this all, is free.

Getting Started

The Kafka website has an excellent quickstart tutorial here. Download the latest version here and work through the tutorial to send and receive your first messages from console.

Playing around with Java

First we create a test topic.

bin/ –create –zookeeper localhost:2181 –replication-factor 1 –partitions 1 –topic testIteration
Created topic “testIteration”.

The earlier versions of Kafka came with default serializer but that created lot of confusion. With 0.8.2, you would need to pick a serializer yourself from StringSerializer or ByteArraySerializer that comes with API or build your own. Since both our key and value in the example will be a string, we use the StringSerializer.

Use the following Apache Kafka library as a Maven dependency (pom.xml).


The following lines of code produces / publishes 10 messages on the Kafka Topic.

public void ProduceIteration()
int amountMessages = 10; // 10 is enough for the demo

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("serializer.class", "kafka.serializer.StringEncoder");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

org.apache.kafka.clients.producer.Producer<String, String> producer = new KafkaProducer<String, String>(props);

for(int i = 1; i <= amountMessages; i++)
ProducerRecord<String, String> data = new ProducerRecord<String, String>("testIteration", Integer.toString(i), Integer.<em>toString</em>(i));
System.out.println ("Publish message " + Integer.toString(i) + " - " + data);


The messages can be received from the topic:

jvzoggel$ bin/ –zookeeper localhost:2181 –topic testIteration –property print.key=true –property print.timestamp=true

CreateTime:1474354960268        1       1
CreateTime:1474354960284        2       2
CreateTime:1474354960284        3       3
CreateTime:1474354960285        4       4
CreateTime:1474354960285        5       5
CreateTime:1474354960285        6       6
CreateTime:1474354960285        7       7
CreateTime:1474354960285        8       8
CreateTime:1474354960285        9       9
CreateTime:1474354960285        10      10


1 Comment

Posted by on 20-09-2016 in Uncategorized


Tags: , , ,

What about CMMN (Case Management Model and Notation) ?

When we started our Case Management adventure the Case Management Model and Notation (CMMN) standard was still relatively new and unknown. (May 2014 the official 1.0 release was released). Both our design tool (Enterprise Architect) and implementation software (Oracle  Adaptive Case Management) did not support the CMMN notation so we created our own “way-of-modelling”.

In our teams that means that the process analysts/designers use powerful tools like SparxSystem Enterprise Architect (EA) with its integrated BPM(N) and SOA support. However manually adding text documents describing the relation the specific process or task has in the whole case. You would think that the CMMN notation could easily be integrated in one of the leading architecture and design tools out there. There are some niche products out there that see a market for CMMN modelling like Trisotech, however SparxSystem seems to have no plans at all.

What about the CMMN adoption by software vendors

It seems that only Camunda and IBM adopted CMMN in their business process management offering. Camunda even supports the beta CMMN 1.1 definition. The other (major) BPM vendors seem to hold back. If a small company like Camunda can do it, you would expect the other large vendors to be able to adopt the standard fairly easy as well.

I assume that the move to cloud, trends like big data and mobile (including new challenges like better API management) has been the primary focus for most software vendors. A big distraction away from extending the core functionality of their current BPM/CM offering. A shame, especially since Gartner states that currently case management software is a 8 billion dollar untapped market. Adoption of standards and especially gaining market share should sound interesting.

So these questions pop up and make me wonder:

  • What is the future of CMMN ?
  • Will companies like Oracle, Appian, TIBCO and PEGA eventually adopt CMMN in their on-premise BPM offering?
  • And with the Gartner prediction in mind and the unstoppable move to cloud, can we expect any Case Management cloud based solution in the near future ? And if so, will it support CMMN ?

I guess only time will tell.


Leave a comment

Posted by on 08-09-2016 in Uncategorized


Tags: ,

Case management for the knowledge worker era

Companies have been using workflow and content management for many, many years. In IT we got used to the term Business Process Management (BPM) whenever we spoke of automated process execution. Not so strange since the term BPM was adopted by all the major software vendors and has received a lot of attention as “the” way to achieve process automation and integration.

There are however many different definitions of BPM, where some are technical and others are business oriented. But if we look at the Gartner definition:

Business process management (BPM) is a discipline that uses various methods to discover, model, analyze, measure, improve, and optimize business processes.  A business process coordinates the behavior of people, systems, information, and things to produce business outcomes in support of a business strategy. Processes can be structured and repeatable or unstructured and variable. Though not required, technologies are often used with BPM. BPM is key to align IT/OT investments to business strategy

 An interesting (and correct) remark by Gartner that technology is not required for BPM, however the general idea (in IT) has been that BPM equals process automation. The business process is summarized to a complete definition until the level that it can be executed as a program. The efficiency gains of automating the business process much exceed the up-front effort to fully identify it and make it executable. An effort which can initially be very large where the cost is only repaid over many instances of the process. So this approach will only work on highly predictable which is highly repeatable. It’s in some way identical to a factory production line where a huge amount of products justifies the initial up-front factory costs.


The Tesla factory production line

Using this approach to discover, implement, and execute complex and dynamic processes will be less economically beneficial. We assume that the less complex a process is, the more likely a lower initial investment is required. And the more often the process will be executed, the greater the financial gain and/or business case.

When we talk about low complexity high volume processes, we can also identify this as so called “routine work”. Routine work is well known and can be planned to a high level of detail. The process uses a general pattern and is done in a very similar way every time. Because routine work is so predictable and repeatable it has been an ideal candidate for successful BPM implementations. Simply due to the fact that the return of investment for routine work is more likely and easily gained.

Agility and Adaptability

Back in 1958 James March & Herbert Simon wrote the management science classis Organizations [2] and concluded that an organization is successful not because it does everything in exactly the same way every time, but because it is flexible enough to adapt and respond to the changes around it. Looking at the present, with changing regulations, new laws, new markets and changing business models this statement seems more true than ever. We therefore can conclude that a rigidly specified organization, that has every process fully planned in detail, will find itself unable to respond to changes.

Maybe we might accept that in real life there is never work that is 100% predictable, nor is there work that is 100% unpredictable. So it seems that the approach to automation of more complex and dynamic business processes requires more flexibility.

In 1992 the legendary business management guru Peter Drucker explains that matching a company’s strengths to the changes that have already taken place produces, in effect, a plan of action. Competitive advantage comes not from steely corporate rigor, but from organizational agility and the adaptability of support systems. [4]

Unpredictable work

Scientific management as once defined by Frederick Winslow Taylor has been at the heart of business process automation to date. And we have seen how it works for predictable and repeatable processes since they are an ideal candidate for executable BPM. But not all work is predictable and many organizations hold processes which are even unpredictable. When we talk about actions and events in processes being unpredictable, we mean that the sequence of human acts is not known in advance, and the course of the process may vary greatly for every instance. We can easily think of examples like medical care, law enforcement and complex industrial and financial processes. 

Rise of the Knowledge Worker

Whenever we read up on the knowledge worker terminology we always come across one individual, namely management guru Peter F. Drucker. Drucker made the first reference to knowledge work in his 1959 book Landmarks of Tomorrow. He loosely defined a knowledge worker as “someone who knows more about his or her job than anyone else in the organization.” What is important is that Drucker already then understood the uniqueness of the job that a knowledge worker performs.


Peter F. Drucker 

Drucker predicted during the 90’s that that the most valuable asset of a 21st-century institution (business or non-business) will be its knowledge workers and their productivity [5]. He even wends further and claimed that the productivity of knowledge and knowledge workers is likely to become the decisive competitive factor in the world economy [6].

So even back then there was already an understanding by business management leaders that the most valuable assets during the industrial revolution were a company’s mass production equipment. Nowadays we can recognize high volume business processes executed in BPM as a form of mass production. Upfront investment which generates value overtime by repetitively producing the same expected result. So if we recognize common grounds between industrial mass production and BPM production work nowadays, what can we say about automation knowledge work in the present or near future? Keeping in mind that Davenport more recently (2005) expresses the importance of knowledge-worker productivity stating that “Within organizations, knowledge workers tend to be closely aligned with the organization’s growth prospects [7]. So with this in mind, how can we efficiently support these knowledge workers ?

Today’s jobs are more dynamic, more ad hoc and require more skills and interaction with other specialists. Besides that, the amount of information available and necessary for a single worker has increased dramatically. So knowledge work cannot be matched on a traditional rigid business process (where work is performed according to a detailed plan prepared in advance). Because as knowledge work proceeds, the sequence of activities depends on the situational information. This is the nature of knowledge work. As the available information or external influences change the process has to be dynamic. It is not simply a matter of plan and execute but a continuous dynamic set of activities in an adaptive flow of execution. And while rules like regulations, laws and organization standards will certainly constrain the actions of the knowledge worker no plan is ever final until the end is reached.

Case Management

 To digitally support the knowledge workers to dynamically act in their cases, we need some sort of case management solution. A way to dynamically start activities like processes and tasks throughout the case lifecycle. In such a way that the knowledge worker can act on the information available to determine which actions to take. The case management solution should therefore be “information” driven.


Information drives the case

Forrester’s [3] defines case management as a highly structured but collaborative, dynamic, and information-intensive process driven by outside events requiring incremental and progressive responses from the business domain handling the case.

(Unstructured) data like documents in an enterprise content management (ECM) system and processes in business process management (BPM) suites are clearly key requirements for any case management offering. But only processes and content are insufficient. By adding advanced business rules and clear business analytics in the mix, you get the 4 essential components of a case management solution.


The 4 essential components for case management

Advanced business rules allow for guidance to make sure certain actions are made available or withdrawn during the case lifecycle. And dashboards with case analytics allow for real-time insight on all running cases.

These 4 core components combined with good (knowledge) user experience and the ability to integrate with your service oriented landscape are basically the case management product you want. Add some policies management and even some social collaboration and you have a set of existing proven technologies, which together have a very interesting proposition for the modern knowledge worker needs.



  1. Keith D. Swenson; Mastering the Unpredictable: How Adaptive Case Management Will Revolutionize the Way That Knowledge Workers Get Things Done; 2010
  2. James G. March & Herbert A. Simon; Organizations; 1958
  3. Connie Moore, Craig Le Chair & Ralph Vitti (Forrester Research); Dynamic Case Management, an old idea catches new fire; 2009
  4. Peter F. Drucker; Planning for Uncertainty; 1992
  5. Peter F. Drucker; Knowledge Worker Productivity, The Biggest Challenge; 1999
  6. Peter F. Drucker; Rise of the Knowledge Worker; 1998
  7. Thomas H. Davenport; Process Management for Knowledge Work; 2005



1 Comment

Posted by on 16-08-2016 in Uncategorized


Tags: ,

Cleaning up your old excess Docker containers

Every docker run command creates a container on your system. When you exit the container (and it is no longer running) it is still there on disk and can be seen with

docker ps -a
ea520a35da1f jvzoggel/kafka "/bin/bash" 2 minutes ago Exited (0) 2 minutes ago kafka1
46067fe01dc4 jvzoggel/kafka "/bin/bash" 3 minutes ago Exited (0) 2 minutes ago kafka2
0b7bcb382e65 jvzoggel/kafka "/bin/bash" 3 minutes ago Exited (0) 3 minutes ago kafka3

The containers that are not running will not consume any system resources except disk space, but it is usually good to clean up after yourself so ..

Automatically clean-up after yourself

The Docker documentation describes how to automatically clean up the container and remove the file system when the container exits:
   –rm=false: Automatically remove the container when it exits (incompatible with -d)
The above shows that by default containers are not removed, so by adding –rm=true or just the short-hand –rm will do the trick:

docker run -i -t --rm jvzoggel/kafka /bin/bash

When you exit from the container it will be automatically removed from disc.

Manually clean-up your stuff

Another method (all credits: Guillaume J. Charmes) is the command:

docker rm `docker ps --no-trunc -aq`

which will remove all containers in a elegant way


Leave a comment

Posted by on 05-08-2016 in Uncategorized



Limitation in Oracle Adaptive Case Management (ACM) revision ID length

All our Oracle BPM projects use a revision id during deployment of the SCA component which is something like [4 digits].[svn-revision] which might look like 1602.71234

The Oracle SCA version format convention

In the Oracle documentation it states that the Oracle SCA composite revision must apply this format:
n0[.n1[.n2[.n3[.n4]]]][-milestone-name[milestone-number] | _patch-number]
Where all but “milestone-name” and “comment” are numeric, composed of one or more digits (0-9) from 0 up to a maximum value of 99999999.

This is the same convention you see on the error when deploying a composite with an incorrect revision format.


So we are good due to the fact that our revision naming standard only use n0.n1 and both numbers will not reach the max value of 99999999 anywhere soon.

The problem

However when testing we discovered that when our revision passed 99999 we have a problem. Not due to the normal SOA or BPM components, they can easily handle the longer n1 digits. But due to the fact that our Oracle ACM projects will throw the following error when deploying a composite with a total(!) revision length of 10+


: Case metadata deployment failed. 
Case metadata deployment failed for MyCase. 
Contact system administrator for assistance. 
at oracle.bpm.casemgmt.fabric.CaseManagementServiceEngine.deploy( 
at oracle.bpm.casemgmt.fabric.CaseManagementServiceEngine.deploy( 
at oracle.bpm.casemgmt.fabric.CaseManagementServiceEngine.deploy( 
Caused By: BPM-72806 
Caused By: javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.3.1.v20111018-r10243): org.eclipse.persistence.exceptions.DatabaseException 
Internal Exception: java.sql.SQLException: ORA-12899: value too large for column "SOA_SOAINFRA"."CM_CASE_DEFINITION"."COMPOSITE_VERSION" (actual: 12, maximum: 10)</pre>

So when checking the SOAINFRA database design we discovered that for ACM components the revision is also stored in a column with a definition of VARCHAR2(10)

Name Null Type
----------------- -------- --------------


The solution

So we logged a service request (SR 3-12014738981) and together with Oracle Support we concluded that this is actually a bug in both 11g and 12c.
Bug 22564283 – Limitation in ACM revision ID due to CM_CASE_DEFINITION.COMPOSITE_VERSION

Yesterday we received confirmation from Oracle Support that the bug is fixed and we will receive a patch.
Meanwhile we can already use the workaround to update COMPOSITE_VERSION column in CM_ACTIVITY_DEFINITION and CM_CASE_DEFINITION to VARCHAR2(200).
So actually an easy fix on the SOAINFRA schema we assumed would work if we “hacked” it in our self, but we’re still very glad that it was quickly handled and solved through Oracle Support with an official supported patch.

alter table CM_ACTIVITY_DEFINITION modify (COMPOSITE_VERSION varchar2(200))
alter table CM_CASE_DEFINITION modify (COMPOSITE_VERSION varchar2(200))

So, if you run into the same problem you can refer with Oracle Support to the SR and BUG numbers mentioned above

Leave a comment

Posted by on 17-02-2016 in Uncategorized


Tags: , , , ,

Cloud integration using federation between Microsoft Office 365 Azure Active Directory (AAD) and Amazon Web Service (AWS)

Not an Oracle blog for a change, but when an organization uses both Amazon Web Services (AWS) and Microsoft Office 365 it is possible to allow single sign-on with the internal LDAP Microsoft uses (Azure AD). Since RubiX uses both cloud products since day 1, I decided to look into integration between both products when Microsoft recently allowed SAML federation.

In this blog I will demonstrate how to connect Amazon Web Services (AWS) to the internal Azure Active Directory (AAD) that is used by Microsoft. As a result of this blog your users should be able to login to AWS from the Office 365 menu.


1. Configure Microsoft Office 365 / Azure Active Directory (AAD)

Go to your Administration console and select Azure AD from ADMIN


In the Azure AD console select “Active Direcory”, click on your Office 365 domain name and the AD menu should open. Click on “Applications” from the top menu


By defaut you will see a lot of Microsoft web applications, so we click on “Add+” on the bottom menu. Then select “Add application from the gallery”


The AWS application will be added to your list. Select Configure Single Sign-On next.


We will select the 1st option (MS AAD SSO) to establish federation between AAD & AWS. The Federated Single Sign-On enables the users in your organization to be automatically signed in to a third-party application like AWS by using the AAD user account information. In this scenario, when you have already been logged into Office 365 the federation eliminates the need for you to login again to AWS.


In this case, we don’t need to perform any extra advanced settings. So NEXT


Download the metadata XML and store it for future use and make sure to accept the checkbox


Go to the users tab and assign (bottom button) your users that are allowed to login to AWS


Before we can finalize our SSO from AAD, we first need to setup AWS.

2. Configure Amazon Web Service (AWS)

Login to your AWS account and select the Identity & Access Management


First we will create an Identity Provider for AAD.
Select SAML as Provider Type and choose a logical name (I use “Office365” in my example).
Browse to the exported metadata we downloaded from the AAD console earlier.

Important: check your metadata xml file

  • The exported metadata XML file from Azure might be encoded as UTF-8 with byte order mark (BOM). Make sure to convert it to UTF-8 without BOM otherwise the AWS console will not be able to import it.
  • Make sure to remove the <?xml version=”1.0″?> on line 1, otherwise AWS will not be able to parse the file


As a result we now have a SAML provider configured, so time to set some roles.
Select Roles in the IAM menu, select “Create New Role” and give your role a logical name (I use “RubixUsers” here)
In the Role Type select “Grant Web Single Sign-On (WebSSO) access to SAML providers“.


  • Select the SAML provider we trust, so we use the earlier created “Office365” provider here.
  • Next step we can customize the policy, which we won’t do so next.
  • In the next step we can select the policy you want to attach to your SSO users.
    You can go fine grained with policies, but for now I will use the default PowerUser policy
  • The last screen you will receive a review of the configuration, make sure to note down the Role ARN and Thrusted Entities
    Role ARN = arn:aws:iam::[customerID]:role/[RoleName]
    Trust = arn:aws:iam::[customerID]:saml-provider/[ProviderName]



3. Configure Microsoft Office 365 / Azure Active Directory (AAD) – part 2

Go back to the AAD management console (
Select applications -> Amazon Web Services (AWS) -> Attributes

Add the following 2 attributes:



4. Result

With these configuration steps you are now able to login to AWS from your Office 365 apps tile.


Leave a comment

Posted by on 16-10-2015 in Uncategorized


Tags: , , , , , , ,

Patching your JDeveloper 12.1.3 with SOA Bundle Patch

The SOA Bundle Patch is available for download from Oracle Support (
Just use the search on patch number ‘20900599’ and it will bring you the patch details page with the download (approx. 765MB).
You can deploy the patch on top of your basic or the earlier released SOA/BPM Bundle Patch or SOA/BPM Bundle Patch

Note: For some reason Oracle no longer seems to use 1 bundle patch for both Oracle SOA 12.1.3 & BPM 12.1.3
The patches 20900599 (SOA) and 20645340 (BPM) are also conflicting according to OPatch.
So you can either patch your environment (server runtime or local JDev) for BPM or SOA.

Patch 19707784: SOA/BPM Bundle Patch
Patch 20423408: SOA/BPM Bundle Patch
Patch 20900599: SOA Bundle Patch
Patch 20645340: BPM Bundle Patch

So (again) I patched my own JDeveloper 12.1.3 environment with the OPATCH tool.
Extract the downloaded patch, I’ll use C:\temp\p20900599_121300_Generic\20900599
This folder is called ‘patch_TOP’ in opatch terminology.

Since I use Windows as local OS you will need to First open a command prompt with Administrator rights (right-click, run as Administrator).
If we don’t use the right permissions we will get the error: Unable to lock Central Inventory. OPatch will attempt to re-lock


First we set our ORACLE_HOME to the Jdeveloper 12.1.3 folder:
SET ORACLE_HOME=C:\ORACLE\middleware\12.1.3
And make sure opatch is in our path:
SET PATH=%PATH%;C:\ORACLE\middleware\12.1.3\OPatch
We browse to the patch_top folder:
cd C:\temp\p20900599_121300_Generic\20900599
And run opatch:
opatch apply

Oracle Interim Patch Installer version
Copyright (c) 2014, Oracle Corporation. All rights reserved.

Oracle Home : C:\ORACLE\middleware\12.1.3
Central Inventory : C:\Program Files\Oracle\Inventory
 from : n/a
OPatch version :
OUI version :
Log file location : C:\ORACLE\middleware\12.1.3\cfgtoollogs\opatch\20900599_Aug_14_2015_09_29_34\apply2015-08-14_09-29-13AM_1.log

OPatch detects the Middleware Home as C:\ORACLE\middleware_12.1.3

aug 14, 2015 9:29:35 AM oracle.sysman.oii.oiii.OiiiInstallAreaControl initAreaControl
INFO: Install area Control created with access level 0
Applying interim patch '20900599' to OH 'C:\ORACLE\middleware\12.1.3'
Verifying environment and performing prerequisite checks...
Patch 20900599: Optional component(s) missing : [ oracle.integration.bpm, ] , [ oracle.mft.apache, ] , [ oracle.bpm.mgmt, ] , [ oracle.hwf.standalone, ] , [ oracle.soa.b2b.client, ] , [ oracle.mft, ] , [ oracle.bpm.processspaces, ] , [ oracle.soa.workflow.wc, ] , [ oracle.oep.examples, ]

Patch [ 20900599 ] conflict with patch(es) [ 19707784 ] in the Oracle Home.

To resolve patch conflicts please contact Oracle Support Services.
If you continue, patch(es) [ 19707784 ] will be rolled back and the new Patch [ 20900599 ] will be installed.

Do you want to proceed? [y|n] y
User Responded with: Y
OPatch will roll back the subset patches and apply the given patch.
All checks passed.

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = 'C:\ORACLE\middleware\12.1.3')

Is the local system ready for patching? [y|n] y
User Responded with: Y
Backing up files...
Rolling back interim patch '19707784' from OH 'C:\ORACLE\middleware\12.1.3'

Patching component oracle.bpm.addon,
Patching component oracle.rules,
Patching component oracle.bpm.plugins,
Patching component oracle.soa.mgmt,
Patching component oracle.integration.bam,
Patching component oracle.soa.common.adapters,
Patching component oracle.soacommon.plugins,
Patching component oracle.integration.soainfra,
RollbackSession removing interim patch '19707784' from inventory

OPatch back to application of the patch '20900599' after auto-rollback.

Patching component oracle.oep,
Patching component oracle.bpm.addon,
Patching component oracle.rules,
Patching component oracle.mft.client,
Patching component oracle.rcu.soainfra,
Patching component oracle.oep.plugins,
Patching component oracle.bpm.plugins,
Patching component oracle.soa.mgmt,
Patching component oracle.integration.bam,
Patching component oracle.soa.common.adapters,
Patching component oracle.soacommon.plugins,
Patching component oracle.integration.soainfra,

Verifying the update...
Patch 20900599 successfully applied
Log file location: C:\ORACLE\middleware\12.1.3\cfgtoollogs\opatch\20900599_Aug_14_2015_09_29_34\apply2015-08-14_09-29-13AM_1.log

OPatch succeeded.


Dave Shaffer gave me a good tip. When he patched his JDeveloper it became unstable and did not resolve any of the performance issues. However, he used jdev -clean once to fix it.


Posted by on 14-08-2015 in Uncategorized