Performance and scalability benchmark

<< Click to Display Table of Contents >>

Navigation:  Automation Server > Automation Server requirements > Performance and optimization >

Performance and scalability benchmark

Overview

For Bizagi, it is of critical importance that its projects deliver business processes which meet customers expectations.

Among these expectations and while supporting powerful functional aspects as well, Bizagi is committed to comply with non-functional aspects such as robustness and adequate performance, especially for volumes which are representative to the customer requirements.

In order to set up such business processes in a platform that offers adequate performance, it is important that customers follow the recommendations as issued by Bizagi, which includes official online documentation or any additional documents as provided by authorized Bizagi personnel.

 

This document presents a load test report which may be used as a benchmark analysis, in order to guide and support decisions oriented towards choosing an adequate system architecture in terms of hardware requirements and an appropriate configuration while provisioning the infrastructure for Automation Server.

 

Software version

The load tests apply to Automation Server version 10.7, running on a JEE platform.

 

Methodology

The load tests carried out in this document were worked by using an automated application that runs multiple instances of a sample business process.

These load tests were run under 3 different benchmark scenarios: A Small load, Medium load and a High load.

 

The objective is to provide results from these load tests in terms of utilization of: CPU, memory, and IO. 

Results from the load tests are focused on the processing and overall resources as demanded by back-end operations in the Bizagi server (where Automation Server runs).

Note that the processing carried out inside user interfaces is not considered, since user interfaces run on each client device and not on the server.

 

 

Benchmark profile and environment configuration

The following image illustrates the system architecture and setup used for the load tests:

 

Benchmark_Overview2

 

All servers employed for load tests were dedicated to these tests solely.

No other business-oriented applications that would interfere with the availability of resources at the servers were running simultaneously (a controlled environment).

 

note_pin

If you are planning to run other applications where Automation Server or its database is running, then you will need to make sure that your hardware sizing and configuration is appropriate in a way in which such additional applications do not affect the performance of Bizagi processes.

 

Hardware configuration

Refer to the following characteristics describing the hardware configuration in each layer and relevant element of the image shown above.

 

Digital process layer

The digital process layer hosts Automation Server.

For the load tests, Automation Server was set in a clustered configuration using 2 nodes using Weblogic version 12.1 in a CentOS Linux release 7.2.1511 (Core).

 

Each node having the following characteristics:

 

EACH NODE OF THE BIZAGI CLUSTER

HARDWARE

RAM

16 GB

Hard Disk

Two disks of 80 GB - RAID 1, disk speed of 10,000 rpm.

Processor

Intel(R) Xeon(R) CPU E5-2665 0, using a quad-core of 2.40 GHz (64-bit)

 

 

Database layer

The database layer hosts the database used by Bizagi processes.

For the load tests, Oracle 11g database (version 11.2.0.1) in a CentOS Linux release 7.2.1511 (Core).

 

DATABASE SERVER

HARDWARE

RAM

16 GB

Hard Disk

Two disks of 80 GB - RAID 1, using solid state drives

Processor

Intel(R) Xeon(R) CPU E5-2665 0, using a quad-core of 2.40 GHz (64-bit)

 

Network characteristics

The database was setup in the same network segment of the Bizagi cluster, with a speed of 10 Gbps (full duplex).

 

Benchmark business process

As noted in the Methodology, the load tests consider a sample business process which is presented as a baseline.

Therefore, it is important to acknowledge that this sample process has its own characteristics which may not be the same ones of your own project's implementation.

This means that the test results presented in this document use linear extrapolations and should be interpreted as minimum requirements (your results may differ).

 

Results are expected to be complemented with detailed sizing analysis to be conducted for your implementation, i.e  while considering any other factors that directly influence the execution of processes in Bizagi.

Such factors include: any additional significant processing being executed in your processes' business rules and integration points (whether Bizagi executes third party APIs, external web services or others that involve heavy resource-consuming logic), the estimated size and amount of business documents handled as attachments in Bizagi, and any other aspects of the customer premises and infrastructure, especially while integrating with your corporate systems, among others.

 

Characteristics of the sample process

The test results reported in this document consider a Travel request template (as available in Bizagi Process Xchange) as the sample business process.

Since this Travel request template has more than one possible path, the path taken for testing purposes was the one in which a full process cycle is completed while having no business exceptions.

Such path presents the most possible quantity of processed BPMN shapes (e.g activities, Sub-Processes, events, gateways).

 

The following image presents the BPMN shapes defining the Travel request process and the path used for these tests:

 

Travel Request_workitems2

 

A total of 24 BPMN shapes are processed for each instance of the Travel request process.

 

Benchmark scenarios and concepts

Since the objective is to provide results from these load tests in terms of utilization of: CPU, memory, and IO, and overall throughput for the 3 different benchmark scenarios (Small loadMedium load and High load), the following section describes the load used by each scenario and the definition for throughput in Bizagi projects.

 

Throughput definition

Throughput for Bizagi projects is primarily defined as number of BPMN shapes processed per unit of time.

In Bizagi's terminology, this is referred to as the number of processed workitems (per second).

Workitems can usually be seen mainly as activities of the modeled process, though it considers others such as Sub-Process, events and gateways since these involve processing and additional logic behind.  

 

Small load scenario

This scenario consists of an automated application generating 250 new workitems per minute.

 

Medium load scenario

This scenario consists of an automated application generating 500 new workitems per minute.

This load is two times the one presented in the small load scenario (2x).

 

High load scenario

This scenario consists of an automated application generating 1000 new workitems per minute.

This load is four times the one presented in the small load scenario (4x).

 

Results

The load tests results presented in this document are classified in terms of utilization and performance of: CPU, memory, and IO.

The following results are displayed for a 600-seconds time frame sample. 

 

 

CPU utilization

CPU % utilization given for the 3 different scenarios, as recorded throughout elapsed seconds for the given time frame sample.

 

Small load results

The following image illustrates CPU % utilization:

 

CPU_S

 

The following facts summarize CPU % utilization:

oMaximum:  50%

oAverage:  20.6180%

 

 

Medium load results

The following image illustrates CPU % utilization:

 

CPU_M

 

The following facts summarize CPU % utilization:

oMaximum:  61%

oAverage:  38.87013%

 

High load results

The following image illustrates CPU % utilization:

 

CPU_L

 

The following facts summarize CPU % utilization:

oMaximum:  100 %

oAverage:  69.16264 %

 

 

Average CPU % utilization comparison

The following image presents the 3 scenarios:

 

CPU_All

 

 

 

Memory utilization

Memory % utilization given for the 3 different scenarios, as recorded throughout elapsed seconds for the given time frame sample.

 

Small load results

The following image illustrates memory % utilization:

 

Mem_S

 

The following facts summarize memory % utilization:

oMaximum:  43.5311 %

oAverage:  28.7644 %

 

 

Medium load results

The following image illustrates memory % utilization:

 

Mem_M

 

The following facts summarize memory % utilization:

oMaximum:  46.3461 %

oAverage:  30.3962 %

 

 

High load results

The following image illustrates memory % utilization:

 

Mem_L

 

The following facts summarize memory % utilization:

oMaximum:  53.3583 %

oAverage:  33.1713 %

 

 

Average memory % utilization comparison

The following image presents the 3 scenarios:

 

Mem_All

 

 

 

 

 

IO utilization

IO % utilization given for the 3 different scenarios, as recorded throughout elapsed seconds for the given time frame sample.

 

Small load results

The following image illustrates IO % utilization:

 

IO_S

 

The following facts summarize IO % utilization:

oMaximum KB read per second:  56

oAverage KB read per second:   0.0881

 

oMaximum KB written per second:  496

oAverage KB written per second:   42.0994

 

 

Medium load results

The following image illustrates the IO % utilization:

 

IO_M

 

The following facts summarize IO % utilization:

oMaximum KB read per second:  4.6

oAverage KB read per second:   0.0062

 

oMaximum KB written per second:  664

oAverage KB written per second:   65.6982

 

High load results

The following image illustrates the above CPU % utilization:

 

IO_L

 

The following facts summarize IO % utilization:

oMaximum KB read per second:  4.58

oAverage KB read per second:   0.0122

 

oMaximum KB written per second:  1164

oAverage KB written per second:   125.4052

 

 

Average IO % utilization comparison

The following image presents the 3 scenarios:

 

IO_All

 

 

Important

The information presented here is provided for information purposes only, and the contents hereof are subject to change without notice. 

This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability of fitness for a particular purpose.

Bizagi specifically disclaims any liability with respect to this document and no contractual obligations are formed either directly or indirectly by this document.

 

Further documentation and resources

Recall that at anytime, you may scale-out your Bizagi project.

Bizagi provides an Excel spreadsheet for your project to carry out a sizing analysis for your specific requirements in terms of concurrent users and amount of activities to be processed per unit of time.

This way, you can determine if additional nodes should be added to your cluster setup. The Excel spreadsheet is referred to as the Sizing estimator, further details and its download link is specified at Sizing estimation.