Integration flows vs Business Processes

Integration flows and business flows.. what is the difference? which technology should be used for each? This is an area where some get confused.

Integration flows are for integrating different systems by matching protocols / message formats, transforming messages, routing messages to different destinations based on message data, performing simple service compositions, etc. An example would be a system A wants to send a JSON/REST message to system B, which can only receive SOAP/WS messages. Further, system B needs some addition information with the request that has to be fetched from a database. Such integration flow can be implemented in an ESB, so that system A can just invoke an endpoint available in the ESB without worrying about matching message formats or fetching additional data.

Business processes are for a different purpose, that is to capture day-to-day operations of an organization. An example would be the process for purchasing equipment for a particular division of an organization. Multiple steps may be involved in this process, such as an employee filling a procurement form, approval by division head, approval by higher management if the value exceeds certain amount, selection of suppliers, obtaining quotes, buying the requested item, notifying interested parties, etc. Things may not always follow the happy path in such a process.. for example manager may not approve the request. Therefore, many conditional flows have to be modeled. For some exceptional flows, we may need to revert previous operations.. for example if a order is canceled, we have to revert fund allocations. Another important thing to note is the duration of running this process. It involves multiple human interactions (e.g. manager approval), which can take days to complete. Therefore, the whole process may take weeks or months to complete. Such flows have to be implemented as business processes and deployed in Business Process Servers (BPS).

This can be summarized as below, which helps in selecting the right technology for the right purpose:

Business flows	Integration flows
Focus on high level end user flows	Focus on low level system interactions
Long running (days, months, years)	Short running (milli seconds, seconds)
Involves humans and systems	Involves systems
Complex flows (escalations, compensations, loops)	Relatively simple flows
Requires persistence	Stateless flows
Relatively high response time	Relatively low response time
Usually implemented in BPS	usually implemented in ESB

Why IoT matters in BPM?

A major part of Business Process Management (BPM) is to understand how processes were executed and to find out ways to improve current processes. In order to do this we need execution data. But how much data is needed? Answer is, the more data we can collect, the more we can discover.

How do we collect data during process executions? If there is a process step which involves an employee filling a web form, it can easily generate data about who filled the form, start and end times of filling the form, values entered, etc. But what about process steps that do not involve someone entering data into a computer.

Let's take the below shipment process. When a package has to be shipped, an employee fills shipment details in a system. Then the package is transported to a port and shipped. Once the package is received, a user sends an acknowledgment message through the shipment system.

We can easily collect data on first and last steps as those are done through the shipment system. We can perform some process analysis from that data as well. For example, by considering start time of the first task and end time of the last task along with shipping destination for each shipment, we can derive average time for shipping a package to a given destination. However, we can't say much on what happens during the shipment process as we don't have data on loading the package, transporting the package using a truck and shipping the package. This is where IoT helps in business processes...

Let's assume that each package is tagged with a RFID. Further, each forklift, truck and ship has RFID readers, which can track all packages it currently holds. In addition, operators of forklifts and trucks wear RFID tags so that corresponding forklify/truck can track its operator. Then each vehicle sends events about its actions. For example, forklift sends the below event when it picks up a package:

<forklift id, package id, pick up time, operator id, location>

Furthermore, an example event sent by a truck when a package is loaded can be:

<truck id, package id, loaded time, driver id, location>

These events can be captured by the shipment system, so that it has data on each intermediate process step in addition to the two steps with direct user involvement. With this addition data, we can do more detailed process analysis such as which steps have most waiting time (i.e. in which places a package sits idle without being transported), how many packages got delayed due to traffic congestion, who are the most efficient forklift operators, etc. Thus, if a process step is performed using a IoT enabled device, execution data on that step can be captured and used in process analysis.

Process execution analysis is not the only area where IoT can help. Events sent by IoT enabled devices can be used for other important functions such as enhancing process models (e.g. by identifying missing steps), discovering new process models from event streams, automatic task completion by integrating events directly into process runtimes, etc.