Integration flows vs Business Processes

Integration flows and business flows.. what is the difference? which technology should be used for each? This is an area where some get confused.

Integration flows are for integrating different systems by matching protocols / message formats, transforming messages, routing messages to different destinations based on message data, performing simple service compositions, etc. An example would be a system A wants to send a JSON/REST message to system B, which can only receive SOAP/WS messages. Further, system B needs some addition information with the request that has to be fetched from a database. Such integration flow can be implemented in an ESB, so that system A can just invoke an endpoint available in the ESB without worrying about matching message formats or fetching additional data.

Business processes are for a different purpose, that is to capture day-to-day operations of an organization. An example would be the process for purchasing equipment for a particular division of an organization. Multiple steps may be involved in this process, such as an employee filling a procurement form, approval by division head, approval by higher management if the value exceeds certain amount, selection of suppliers, obtaining quotes, buying the requested item, notifying interested parties, etc. Things may not always follow the happy path in such a process.. for example manager may not approve the request. Therefore, many conditional flows have to be modeled. For some exceptional flows, we may need to revert previous operations.. for example if a order is canceled, we have to revert fund allocations. Another important thing to note is the duration of running this process. It involves multiple human interactions (e.g. manager approval), which can take days to complete. Therefore, the whole process may take weeks or months to complete. Such flows have to be implemented as business processes and deployed in Business Process Servers (BPS).

This can be summarized as below, which helps in selecting the right technology for the right purpose:

Business flows	Integration flows
Focus on high level end user flows	Focus on low level system interactions
Long running (days, months, years)	Short running (milli seconds, seconds)
Involves humans and systems	Involves systems
Complex flows (escalations, compensations, loops)	Relatively simple flows
Requires persistence	Stateless flows
Relatively high response time	Relatively low response time
Usually implemented in BPS	usually implemented in ESB

Why IoT matters in BPM?

A major part of Business Process Management (BPM) is to understand how processes were executed and to find out ways to improve current processes. In order to do this we need execution data. But how much data is needed? Answer is, the more data we can collect, the more we can discover.

How do we collect data during process executions? If there is a process step which involves an employee filling a web form, it can easily generate data about who filled the form, start and end times of filling the form, values entered, etc. But what about process steps that do not involve someone entering data into a computer.

Let's take the below shipment process. When a package has to be shipped, an employee fills shipment details in a system. Then the package is transported to a port and shipped. Once the package is received, a user sends an acknowledgment message through the shipment system.

We can easily collect data on first and last steps as those are done through the shipment system. We can perform some process analysis from that data as well. For example, by considering start time of the first task and end time of the last task along with shipping destination for each shipment, we can derive average time for shipping a package to a given destination. However, we can't say much on what happens during the shipment process as we don't have data on loading the package, transporting the package using a truck and shipping the package. This is where IoT helps in business processes...

Let's assume that each package is tagged with a RFID. Further, each forklift, truck and ship has RFID readers, which can track all packages it currently holds. In addition, operators of forklifts and trucks wear RFID tags so that corresponding forklify/truck can track its operator. Then each vehicle sends events about its actions. For example, forklift sends the below event when it picks up a package:

<forklift id, package id, pick up time, operator id, location>

Furthermore, an example event sent by a truck when a package is loaded can be:

<truck id, package id, loaded time, driver id, location>

These events can be captured by the shipment system, so that it has data on each intermediate process step in addition to the two steps with direct user involvement. With this addition data, we can do more detailed process analysis such as which steps have most waiting time (i.e. in which places a package sits idle without being transported), how many packages got delayed due to traffic congestion, who are the most efficient forklift operators, etc. Thus, if a process step is performed using a IoT enabled device, execution data on that step can be captured and used in process analysis.

Process execution analysis is not the only area where IoT can help. Events sent by IoT enabled devices can be used for other important functions such as enhancing process models (e.g. by identifying missing steps), discovering new process models from event streams, automatic task completion by integrating events directly into process runtimes, etc.

From process descriptions to executable business processes

Many organizations have a considerable number of business processes documented in various formats such as plain text and flow charts. Some of these documented processes may have to be automated. A core part in automating a business process is to create an executable process from a documented process (e.g. plain text process). Let's see how we can do this using BPMN and how such executable processes can be enacted in WSO2 BPS.

We will take a procurement process for buying equipments for employees as an example. Below is the plain text description of the process:

An employee fills a procurements form and gets the approval from his manager. Then the employee hands over the form to the procurements department. Procurements department gets quotations from approved suppliers. Once quotations are obtained, it submits an appropriate form to the finance department for funds. If the funds are not approved, new quotations are taken and sent for approval. Once funds are approved, procurements department buys the item and hands it over to the employee. 

Our target is to create a BPMN model from the above process description. Tasks involved in the process, the flow of the process and people responsible for carrying out different tasks are described in the above text. We can identify the below tasks and roles responsible for those tasks:

• Fill procurements form - Employee
• Approve procurement - Manager
• Get quotations - Procurements department staff
• Approve funds - Finance department staff
• Buy item - Procurements department staff
• Handover item to employee - Procurements department staff

Each of the above tasks become a User Task in our BPMN model. We use the Eclipse based Activiti BPMN designer for modelling the process. To start with, create a new BPMN project from the menu File -> New -> Project. Select "Activiti Project" and give the project name as "XYZOrganization". Then right click on the created project, click New -> Other and select "Activiti Diagram". Give process model name as "procurementsProcess" and click "Finish". Now we have an empty BPMN model.

First drag and drop a start event from the pallet. Then drag and drop seven User Tasks to the canvas and rename them with above task names. Next step is to model the control flow, that is to model the connections among tasks. We can identify these connections easily from the process description. For example, "Fill procurements form" precedes "Approve procurement". To model such flows, simple connect the two tasks with a sequence flow. The only part in the control flow that needs bit more attention is the outgoing flow from the "Approve funds" task. There are two possible outcomes of this task: Funds can be approved or rejected. According to the process description if the funds are not approved, new quotations have to be taken and sent for approval. If funds are approved process can proceed to the "Buy item" task. Therefore, we have introduce a decision point here, which is modeled using an exclusive gateway in BPMN. Drag and drop an exclusive gateway to the canvas and connect it to the "Approve funds" task using a sequence flow. This exclusive gateway models a decision point. Accordingly it has to have two outgoing flows to model different paths taken by the decision. One outgoing flow simply connects to the "Buy item" task. The other outgoing flow should be connected to the "Get quotations" task. However the flow from "Approve procurement" task also connect to the "Get quotations" tasks. Therefore, this is a converging point in the process as two flows join to form a single flow. Only one converging flow can be taken at a given time, which means they are exclusive converging flows. Such converging points can also be modeled using exclusive gateways in BPMN. Just drag and drop an exclusive gateway and connect the flows as shown in the below diagram. Finally, we mark the completion point of the process by adding an end event and connecting the "Handover item to employee" task to the end event. Now we have captured the control flow of the process as shown in the below figure. Although the figure contains pools and lanes, we can ignore them as the current version supports them only for display purpose.

The next step is to enrich this model with details required for process execution. First, we have to specify the role that is responsible for performing each task. In the BPMN designer, click on the "Fill procurement form" task to view the properties of this task. Click on the "Main config" tab in properties and enter "employee" in the "Candidate groups" field as shown in the below figure:

Similarly, assign appropriate tasks to "manager", "procurementsStaff" and "financeStaff" roles. Now we are done with the control flow and resource assignment steps of our process model. Next step is the data modelling, which is to specify which data elements are required for each task and how those data elements are used by other tasks. Let's assume that the employee has to specify what equipment is required and the purpose of the equipment in the procurements form. Therefore, "Fill procurement form" task has three output data elements called, for example "emplyeeId", "equipment" and "purpose". Then the manager needs to know the equipment applied for and its purpose for approving the request. Further, he will specify whether the application is approved or not. Thus, "Approve procurement" task has two input data elements: "equipment" and "purpose". The output data element of this task is "approvalStatus". Similarly, we have to identify input and output data elements of each task and specify them in our BPMN model.

Click on the "Fill procurement form" task and select "Form" tab in the properties window. Click "New" button to bring up the new form property window. Specify the property ID as "equipment", name as "Equipment" and type as "string" as shown in the below figure. Property ID is used to refer to the property within the process model. Property name is the human readable
name of the property, which is displayed in forms presented to end users.

 

Similarly, add the "purpose" and "employeeId" form properties to the "Fill procurement form" task. Once properties are added, form tab of the properties window will look like below:

Next is to fill up data elements for the "Approve procurement" task. Here we have to give "equipment" and "purpose" as inputs and get "approvalStatus" as the output. Go to the form properties view of the "Approve procurement" task and add two properties with IDs "equipment" and "purpose". As we are giving the same IDs that we gave in the "Fill procurement form" task, we are referring to the properties used in that task. As we do not want users to modify those two properties we can set the "Writable" property to false. Then add another property with ID: "approvalStatus", name: "Approve the request" and type: "string". In the same way, we can fill up the data elements for other tasks.

One task that needs particular attention is the "Approve funds" task as the output of this task is used in deciding the subsequent process flow (i.e. in the exclusive gateway). Therefore, let's add a property with ID: "approveFunds", name: "Approve funds" and type: "string". Now if the "approveFunds" property is set to "no", user should be redirected to "Get quotations" task. If it is set to "yes", process will continue to "Buy item" task. Select the sequence flow connecting two exclusive gateways and go to the "Main config" tab in the properties window. Enter ${approveFunds=='no'} in the "Condition" field. Then select the sequence flow connecting the exclusive gateway and "Buy item" task and in the "General" tab of the properties window, enter "approvedFlow" in the "Id" field. Then select the exclusive gateway next to "Approve funds" task. In the "General" tab select "approvedFlow" under the "Default flow" field.

We are done with modelling the process. Now we have to export the process as an archive so that it can be deployed in WSO2 BPS. Right click on the project and select "Export" from the menu. In the export dialog, select "Archive File" under the "General" section and click "Next". Select "procurementsProcess.bpmn" file from the "XYZOrganization" project. Fill in the export destination and make sure to name the archive with extension "bar" (e.g. xyzprocurements.bar). Filled export dialog is shown below:

Now we have our procurements process ready to be executed. Let's try it out.. Start WSO2 BPS and go to the management console (https://localhost:9443/carbon/). Under "Processes" -> "Add" click "BPMN". Browse the exported BPMN archive (i.e. xyzprocurements.bar file) and click "Upload". Wait for few seconds until the process is deployed and click Processes -> List -> BPMN to view the uploaded BPMN process.

Business processes have many stakeholders and those stakeholders involve with processes in different ways. Business analysts work with process editors to build models as we did with the Activiti BPMN designer. IT administrators deploy and manage processes as we just did with the BPS management console. Now it comes to real process participants working with process to perform their day-to-day business tasks. In our case, it is organizational staff working on a procurements procedure. WSO2 BPS offers a separate web application called BPMN Explorer for such process participants to work with the process. Let's view the BPMN Explorer by pointing the browser to https://localhost:9443/bpmn-explorer. Login to BPMN Explorer using username: admin and password: admin. Click the "Processes" tab to display the process model that we just deployed as below:

Click "Start" to start this process. However, we won't be able to do anything with the process other than starting it as none of the tasks in our procurements process are assigned to the admin role. Therefore, go to the BPS management console and create roles "employee", "manager", "procurementsStaff" and "financeStaff" and assign some users to those roles. Then first login to the BPMN Explorer from a user in the "employee" role. We can see that the "Fill procurement form" task is listed under the "Claimable tasks" tab. Click on the task name to brings up task details as shown below:

One thing to note here is the list of data items to be filled to complete the task. These are the properties that we specified when modelling the "Fill procurement form" task. However, we cannot fill anything until we claim the task. Click "Claim Task" to claim the task, so that the task will be assigned to the current user. Now the task will be listed under "My tasks" tab. Click on the task name to view task details as below:

Here we can fill the form with appropriate data and complete the task. Then the task lists of the current user becomes empty as no more tasks are assigned to users in "employee" role. Then login again using a user in the "manager" role and go to the "Claimable tasks" tab. We can see that the "Approve procurement" task is listed there. Claim it and open the task from "My tasks" tab. As shown in the below figure, we can notice that the equipment and purpose fields are already filled with the values we have entered before.

Fill in the approval field and complete the task. Then log in using "procurementsStaff" and "financeStaff" users to complete subsequent tasks accordingly.

We just saw the execution of our procurements process in WSO2 BPS. Basically, what BPS does is to follow the process flow of the BPMN model and assign tasks to users in relevant roles. Once a task is completed, BPS follows the process flow to the next task and assign it to appropriate users. From the users point of view, what they get is a list of tasks for them to work with. They can claim any task from the list, fill necessary details and complete it. Once a task is completed, new tasks will be added to the same user or to some other users depending on the process model. This is how an automated business process coordinates users to perform tasks required for a business goal.

Towards business friendly process modelling with BPMN

WSO2 Business Process Server (BPS) will support BPMN starting from the upcoming release. You can try out the beta release from here.

So what's in it for WSO2 BPS users? How would an organization benefit from BPMN based business processes? Let's start by looking at the purpose of business processes and business process modelling.

Business process is a collection of interrelated tasks used to achieve a business goal. Let's take an example of buying a new laptop for an employee. First, the employee has to fill a procurement form and get the approval from his manager. Then the employee hands over the form to the procurements department. Procurements department gets quotations from approved suppliers. Once quotations are obtained, it submits appropriate form to the finance department for funds. If the funds are not approved, new quotations are taken and sent for approval. Once funds are approved, procurements department buys the laptop and hands it over to the employee. Here we see a (simplified) business process for procurements. As mentioned above, this process has tasks such as "Fill procurements form", "Get manager approval", "Get quotations", etc. The process also has interrelations among tasks such as "Fill procurements form" precedes "Get manager approval", "Get quotations" precedes "Get funds", etc.

A typical organization can have a large number of business processes for achieving various business goals such as handling sales orders, processing employee leave applications, recruiting new employees, launching new products, etc. To complicate this further, each of these processes may have multiple subprocesses. For example, the process of recruiting new employees may contain subprocesses for scheduling interviews, preparing employment offers and provisioning IT services for new employees.

So what is the purpose of modelling these processes? Although we like it or not, there are many processes in any organization. Further, these processes exists to fulfill business goals at various levels. Therefore, if we want to know how we are going to achieve our business goals we have to know on our processes. This is why we need business process modelling. By modelling business processes using a suitable notation, we can easily understand details of a process such as what are involved tasks, how certain tasks depend on other tasks and who are responsible for performing different tasks. In addition, if we use a standard notation for modelling, those process models can be used to communicate with process stakeholders. For example, once business analysts prepare a process model, they can hand over the model to the division head for approval. Then the same model can be used by process participants to understand tasks to be performed at different stages of the process. Wait.. what if the same model can be executed in a Business Process Server (BPS)? This is where BPMN comes in..

BPMN provides a rich notation for modelling business processes while trying to be simple enough to be understood by business users. At the same time, BPMN is an executable language, which means it can be executed in process servers in order to automate business processes. As an illustration, below is a BPMN model of the laptop procurement process we discussed above:

A business analyst working with process stakeholders can drag and drop BPMN elements and connect them as necessary to draw up such BPMN model. This model can be used understand and analyze the process. For example, it is clear that the current model has no action to take if the management approval cannot be obtained or if no quotations could be obtained for the given laptop. Once we visualize the process using BPMN, such points become more visible and allows us to introduce to possible improvements. If the staff of the organization has some minimal BPMN knowledge, this model can be used to communicate the process. For example, an employee working in the procurements departments understands that quotations have to be obtained before submitting for funds approval. Such benefits of clearly understanding and communicating business processes become more prominent when processes contain large number of tasks with complex interrelations and hierarchies of subprocesses.

Then comes the best part... If we add little more details to this process, it can be executed directly on a process server such as WSO2 BPS. That is, we can get the procurement process automated. Once automated, an employee who needs to get a new laptop just have to trigger the procurement process. Then BPS will assign relevant tasks to appropriate users and coordinate the flow of tasks according to the BPMN model. In this way, users just have to work on tasks assigned to them while BPS performs all coordinations among users, other processes and external systems.

As we have discussed, BPMN is a great tool for capturing real business processes and automating them. Therefore, the support for BPMN in WSO2 BPS allows BPS users to move towards business friendly process automation.

BPM and SOA

Business Process Management (BPM) can be thought of as the identification, design, modeling, execution, monitoring and optimization of business processes involving people and systems, within and external to organizational units. Automation of suitable business processes and use of tools for simplifying the different phases of BPM is a key to the success of a BPM approach. There are large number of vendors offering various solutions for different aspects of BPM, including process modeling tools, process execution engines, Business Activity Monitoring (BAM) solutions and complete BPM suites.

What ever the tools and solutions used for BPM, it involves interactions with people and systems. Specially the process execution and monitoring phases have to interact heavily with the entities (i.e. people and systems) that perform actual tasks of the business process. For example, some where in the process, it may be required to query an inventory system to check the availability of an item. Or it may be required to get an approval from the branch manager to continue the process. So how does the process execution engine (in the process execution phase) communicate with these entities, which are outside the process engine itself. Let's ignore the interaction with the people part for the moment and focus on interactions with external systems (e.g. inventory system). Clearly, we need some interface from the system. But what should be the interface? Such external systems could provide some sort of interfaces to expose its functionality and data. But in some cases, it may not be the case and we may have to end up querying and updating files and databases of such systems. Either of these approaches are not good enough. Why? If we use the interfaces provided by different systems, we end up using large variety technologies and message formats in our deployments. Process execution engine may have to use RMI to communicate with the inventory system, file based communication method to communicate with CMS, some other messaging technology to interact with the HR system. This introduces enormous management and technological burden. Process engine has to understand all those communication technologies. It may have to use various plug-ins to support these different technologies. This is only one part of the problem. In addition, we have to deal with all the usual system to system integration problems as we are doing exactly a such integration here.

So, what is the role of SOA to play here? Well, system integration is what SOA is good at. And we have a big system integration problem in our hand. That is to integrate our BPM system (BPMS) with the existing and possibly new business systems. As the simplest solution, we can decide on set of services to be provided by such business systems. And implement those services using a suitable technology (preferably web services). Then it is a matter of invoking these services as required by the BPMS. We can decide on the required security level, message format, etc and make them common to all the services, thus reducing the burden of different communication methods.

We are not done yet. If you think of a typical system with a BPMS, you would find out that BPMS is one of the components that make most of the connections to other components of the system. So, use of an ESB would obviously help us here. We can use the pattern described here to minimize and manage the number of connections with the BPMS.

We can do even more with SOA. As the organizations grow larger and business processes become more complex, usually it is harder to capture all the steps of a process to a single model. Various organizational units may take different approaches for the (sub)processes and the modeler may not be aware of the details of such activities. And some departments could already have developed process models for some parts of the processes. This makes it obvious that it is better to let business units to define the processes that are most suitable to them and combine such sub-processes when it is required to develop a larger process involving many departments. Going inline with SOA, we can identify and expose sub-processes as services. Then invoke these services (thus executing the sub-process) from larger processes as required. This in turn creates a hierarchy of processes, and makes it possible to reuse existing processes.

As mentioned earlier, sometimes the interface provided by existing systems may not be sufficient for the BPMS. Specially, BAM solutions need to collect detailed information about the process execution. In such cases, we may have to query the databases and files of such systems. Again it would be much neater to wrap such data sources with data services and expose them as usual services. So the BPMS could simply invoke these services and the common concerns like authentication, authorization, message formats, etc are left to the SOA infrastructure.



Above diagram demonstrates above uses of SOA, in the context of BPM systems. Note that the use of three BPMS, each deploying one or more processes and exposing those processes as services. For example, BPMS3 may contain set of smaller processes, BPMS2 may contain set of smaller processes as well as set of composite processes integrating sub-processes of BPMS3 and BPMS2. And the BPMS1 may contain organization wide complex processes which extensively use the processes deployed in BPMS2 and BPMS3. As shown in the diagram, (authorized) external systems could also invoke these processes using the services they expose, thus creating cross-organizational business processes.

Enterprise Service Bus (ESB): a critical part of SOA

SOA deployments could end up having large number of services and similar number of servers hosting these services. Such system would have many business process servers, coordinating the business activities using available services. It may also have several web/application servers and other systems to provide user interaction, etc, which communicate with various services hosted in the deployment. Such SOA deployment would look like the below diagram, where there are large number of components and even larger (possibly unmanageable) amount of interconnections.



Figure 1: A deployment with front end web servers, process servers running various business processes and set of servers with business systems (e.g. HR systems, inventory). Note that these business systems may be wrapped as services (e.g. web services).

It's obvious that an improvement is needed for such deployments. But, let's first consider the problems with this type of system. In this deployment, each and every component (service, BPMS, web server, etc) is directly connect with other components. So if we want to change one component (e.g. move a service to a different server), we have to reconfigure all the components connecting to that component. Another major problem is that, it is very difficult to get an idea about how different components are interconnected, once the system is deployed. We have to study the connections between each and every component (by examining their configurations, etc) to get a complete view on the system. Therefore, troubleshooting the deployment and introducing new components would be a tremendously painful process. Added to these complexities, we have very limited control over the messages passed between the components. For example, different components may react differently to failures and components may use different payload formats and other attributes.

So it is clear that we have a problem. One part of the problem is due to the large number of components making up the SOA deployment. We can't do much to prevent the number of components from growing, as it is a natural consequence of expanding the business units or organizations. But, of course, we can do a lot to better organize and manage the growing number of components, which I am hoping to discuss later. For now, let's consider the other part of the problem. That is the exponential increase of connections between components. There is a neat way of solving this problem, which addresses almost all the problems mentioned above and also provides lot more benefits. That is to use an Enterprise Service Bus (ESB) as the backbone of all the communications of the SOA deployment. The simple idea is:

(1) Do not make direct connections between components
(2) Connect all the components to an ESB (or to multiple ESBs)
(3) Configure the ESB to route messages between components

Again, there are no strict rules. Decisions like how many ESBs to use in a deployment, how to configure ESBs, whether a direct link is preferable in a particular situation, etc are mostly depend on the requirements of the deployment. Let's consider how we can construct above architecture using the WSO2 ESB. We can easily build this using the feature called proxy services, which acts as a virtual service hosted in the ESB. Once a proxy service is configured, it provides an endpoint similar to a real service, so that external parties can invoke it. But internally, the ESB can be configured to do whatever we want with the message. In this case, what we can do is to create proxy services for all the components in the SOA deployment and configure those proxy services to direct the messages to the actual services. Then we have to configure all the components to interact only with the proxy services defined in the ESB. So we effectively end up with the architecture mentioned in above 3 steps. SOA deployment after the introduction of the ESB is shown below.



Figure 2: SOA deployment after introducing an ESB. Note that two new components, an administrative server and a central log is added, which are difficult to support without this type of architecture.

So, how do we gain advantages over the previous deployment. In this architecture, all the interconnections are handled through the ESB. Therefore, if we want to relocate a component, it is simply a matter of changing the configuration of its proxy service. We don't have to touch any other components. The next problem is getting a detailed view about all the interconnections in the SOA deployment. In this architecture, all the messages are passed through the ESB. Using the ESB configuration, we can enable logging at all required points with necessary level of details. These details could be client address, target address, message type, etc. Then we can analyze these logs (with the support of visual tools) to get an in-depth knowledge about interactions in the SOA deployment. Next, we can easily impose constraints on message formats passing through the ESB. ESB can be configured to examine all or required set of incoming/outgoing messages and perform desired corrective actions on malformed messages. These actions may include reformatting the messages on-the-fly, dropping the message and sending an error message to the client, blocking the client and informing an administrative service, etc. This type of ESB based deployment also gives us more control over the errors and failures within different components of the SOA deployment. Different components may react differently to errors occurred while processing requests. But in this type of deployment, all those error messages (possibly in different formats) are sent to the ESB. So we can gain overall control of such failure situations and configure the ESB to take necessary actions. These could be one or combination of actions like redirecting the request to another similar component, temporarily block the failed component, sending an error message to the client, logging the error according to standardized format, etc. Whatever the actions are, they can be configured centrally with a clear view of the overall architecture, so that the SOA deployment reacts consistently and gracefully to failures.

Use of ESBs could provide much more advantages to SOA deployments than mentioned here. But if a SOA deployment is experiencing the problems addressed above, it is time to start evaluating how an ESB fits in to the picture.

A simple introduction to SOA

Organizations use software systems to improve their operations, in terms of productivity, accuracy, reactivity, security, cost, etc (of course, some of these points are interrelated). But the problem is that, there is no strictly defined way of how to use software in various business operations. And the worst part is, you may not achieve anything or may even make it more complex and inefficient, if software systems are not used appropriately.

What usually happens is that individual and isolated areas of the business will be automated using software systems. These can either be broad areas like customer handling, HR management, inventory control, etc or small and more specific sections. Organizations may use commercial grade systems like CRM systems, SCM systems and inventory control systems to automate some of these areas and may use outsourced or in-house developed software systems for more specific business tasks. At the end of the day, what it has is a set of independent software systems working in isolation to perform individual business tasks of the organization. This method works fine and may introduce some improvements for the business as well. But if you think carefully, there is a huge potential for further improvement, which can be tapped by going only one step forward. That is to integrate of existing systems and make them work collaboratively.

So, how do we achieve this? There are many systems developed by different vendors and also custom built systems. Possibly, they have different interfaces and different data formats. And how do we establish a flow of control/data among these systems to make them work collaboratively. Hold on, this is where SOA can help you.

Again, how SOA can be used in a particular environment varies according the nature of the problem, scalability concerns and many other factors. Simply, there are no hard and fast rules on how you should use SOA. But we can use a simple example for explaining a application of SOA. Let's consider a online book store. Book store has a web site for browsing and ordering books, an inventory system, a shipping system, payment system, etc. A trivial way to develop the online system for the book store is to build a web based program (e.g. JSP/servlets), which accesses all other systems and perform all the business operations. One problem with this approach is that the web based program has to access each and every system using their specific interfaces, which requires separate client modules for all those systems. And if there is a need to do a simple change to the business operation, we have to change the program and redeploy it. And the worst part is yet to come. Let's say we want to introduce another business process involving these systems. Then we have to write a new program from the beginning. Even though, we go for something similar to three tier architecture and separate the web interface and the business logic, there won't be much difference. This situation is illustrated below.



So how could SOA improve this scenario? As the first step of SOA, we need to identify units of functionality in our systems and make them services. For simplicity we can assume each system (inventory, payment system, etc) as a single service. But in real-life situations, we might identify multiple services in a single system or expose features of multiple systems as a single service. Once services are identified, we have to implement them as standard services. One recommended way is to implement them as web services. There are many products to help you in creating and deploying web services and automating this process in various levels. WSO2 Web Services Application Server can be very useful in this phase of SOA implementation. These web services should wrap the existing systems (hiding their internal complexities) and expose well defined and required functionalities. Once this step is completed, we have a set of services, which have well defined functionalities (using WSDL) and communicates using a well known protocol (e.g. SOAP).

Next step is to make them work together to perform business operations. Business Process Management Systems (BPMS) can be used for this purpose. We can define a business process by specifing how the control flows between different systems (and users) and what data is used for input and output of each system. Once a business process is defined, we can deploy it in a BPMS and the BPMS will expose our business process as a service. So, the only thing that our web interface has to do is to call the service exposed in BPMS with required parameters. Again, there are many BPMS products, which can be used for this purpose. Specially, the WSO2 Business Process Server provides many features to simplify the deployment and management of business processes.

Now, let's examine how this SOA based approach improves the process over the previous methods. First, we have standardized the interfaces of all the systems. Therefore, other systems can communicate with these systems with same protocol (same client modules). And their functionalities are defined in well known format (WSDL), so that any third party systems can understand what they exactly do. Then, we have moved the business logic to a BPMS. Business processes are defined using text based files in BPMS. So it is very easy to change and redeploy business processes according to the changing requirements. Most BPMS have graphical interfaces to simply this process further. Introducing a new business operation is also straight forward in this environment. We just have to write a new business process definition and deploy it in the BPMS. It already knows how to communicate with existing systems (as they are exposed as web services) and all the infrastructure is already in-place for execution. This is shown in below diagram.



This is only a simple example of how SOA can help you in improving the business operations. As I mentioned earlier, how we can (and how we should) apply SOA varies greatly based on the scenario and other conditions.