Services monitoring

<< Click to Display Table of Contents >>

Navigation:  Bizagi Engine > Bizagi system administration > Maintenance and administration > Monitoring >

Services monitoring

Overview

We recommend a platform administrator to monitor Bizagi services, as offered by both its components: the Work portal and the Scheduler.

Ensure that both services are running and operational at all times, and that these show an adequate performance.

Adequate and pro-active monitoring will allow you detect if there are potential issues or bottlenecks to resolve and where to do so.

 

Monitoring the application/web server

For the Bizagi servers, and either if you are using WebSphere, Weblogic, JBoss or IIS, make sure you constantly monitor the service provided by such application/web servers.

You may rely on their logs. The following table illustrates which logs to follow up (or as instructed by the specific vendors):

 

SERVER

LOG

LOCATION

WebSphere

SystemErr.log, SystemOut.log, StartServer.log, and StopServer.log

<WEBSPHERE_HOME>\profiles\[NodeName]\logs\server1

Weblogic

AdminServer.log, [your_Weblogic_domain].log

<WEBLOGIC_HOME>\user_projects\domains\[DOMAIN]\servers\log


[NodeName].log, [NodeName].out,

for horizontal clustered installations.

<WEBLOGIC_HOME>\user_projects\domains\[DOMAIN]\servers\[NodeName]\log

JBoss

Server.log

<JBOSS_HOME>\standalone\log

IIS

Event viewer logs

C:\inetpub\logs

 

In addition to these logs, monitor the events recorded by Bizagi as well.

For instance, on Windows operating systems, you may review the Windows Event Viewer (eventvwr).

 

Regarding Asynchronous Activities

You should also monitor how Bizagi Asynchronous Activities are processed.

Recall that these may run in the background, and may involve retries (for instance, due to an external interface being off-service).

 

If your project relies strongly on Asynchronous Activities, it is recommended to closely watch over their execution.

You may also rely on the administration option provided at the Work portal, called the Asynchronous Activities Console, which will display those tasks awaiting for processing (due to a previous failed attempt).

 

note_pin

Bizagi provides an additional log which is recommended to monitor as well, which keeps track of those external services which present a delay or are unresponsive.

This means that whenever Bizagi invokes external web services, a new entry will be recorded in a .csv file as long as such external service exceeds the expected time (threshold) to return a response (or, produces a timeout).

This file is saved at the .\Temporary\SOA\Log\ path of your project (one file being stored per web service, as C:\Bizagi\Projects\[your_project]\Temporary\SOA\Log\[your_interface]Log_1.csv) .

And the file will have one line per each of these alerts, presenting further detail including: DateTime, ErrorDescription, idCase, Task_Name, URL, Method_Name, and InterfaceTime(MM:ss:mmm).

 

 

Regarding the IIS

When explicitly monitoring the IIS, it is recommended to monitor the size of the IIS requests' queue in order to detect bottlenecks produced at the Bizagi servers.

Having a growing size of requests being queued up at the IIS (implying an increase in the number of requests in queue -a degrading pattern), suggests that additional nodes will be needed in your clustered configuration.

Delays to process requests, which are not produced by external services (i.e in integration points in Bizagi) or not produced by the database server's capacity to deliver information, suggest that your Bizagi servers should be tuned (scaled-out if the delay has a growing trend).

 

Log files of the Web thread provide detail found by default at the W3SVC folders of C:\inetpub\logs\LogFiles:

 

TimeBA_TimeDB

 

ASPECTS TO MONITOR

INTERPRETATION

Log files of the Web threads

 

Such files present the following parameters having a time measure in milliseconds:

timedb: The time it took for the database to deliver the final information as requested by the Bizagi server.

timeba: The time it took Bizagi's Work portal to process the request.

 

The sum of these above parameters adds up the total time it took to process a request once it started to process in the IIS (does not consider the time a request was waiting in queue, if any).

 

The above suggests that through IIS performance counters you may narrow down if requests are being left in a queue in an undesired/unexpected behavior.

 

Recall that you may monitor the Current queue size counter  (available under the HTTP Service Request queue classification) for the dedicated Bizagi application pool (by default named as Bizagi 64-Bit ASP.NET v4.0) to determine queue behavior in the nodes.

 

 

Perfmon_IIS

 

You may also involve additional IIS-role specific performance counters, such as:

ASP.NET Applications\Requests/Sec: Shows the throughput of the ASP.NET application on the server. It is monitored along with other counters to determine whether the server is handling the application as it’s supposed to.

ASP.NET\Application Restarts: Indicates the number of restarts of the application in the server’s uptime. A high value of this indicator should be monitored. The general counters can help you identify whether it is caused by a bottleneck in the system or by the application itself.

ASP.NET\Request Wait Time: Shows the amount of time (in milliseconds) that the last request was held in the queue. It should be close to 0 ms. If this indicator is usually greater than 1000 ms, the performance of the IIS server is suffering.

ASP.NET\Requests Queued: The queue fills up with requests that wait to be processed. This counter should be monitored to find out when an application is overwhelmed. Then, an analysis of the application and server performance altogether can help the administrator identify the cause for the filled queue.

.NET CLR Memory\# Total Committed Bytes: Shows the amount of virtual memory reserved for the application on the paging file. It should be monitored along with the general counters to identify issues with the performance of the IIS. Problems can be caused by a small amount of memory installed or by an application overusing the memory.

Web Service\Get Requests/sec: Measures the amount of GET requests processed in a second. If the value is not as optimal for a particular IIS server (i.e, when compared to other servers), then load balancing or clustering technologies can be applied to lower the burden of the server in question. For the above, consider if hardware and software characteristics of servers are alike, otherwise you need to include them as variables to your analysis.

Web Service\Post Requests/sec: Measures the amount of POST requests processed in a second. If the value is not as optimal for a particular IIS server (i.e, when compared to other servers), then load balancing or clustering technologies can be applied to lower the burden of the server in question. For the above, consider if hardware and software characteristics of servers are alike, otherwise you need to include them as variables to your analysis.

Web Service\Current Connections: Shows the number of active connections with that server's service. If the value is not as optimal for a particular IIS server (i.e, when compared to other servers), then load balancing or clustering technologies can be applied to lower the burden of the server in question. For the above, consider if hardware and software characteristics of servers are alike, otherwise you need to include them as variables to your analysis.

 

note_pin

As with any IIS application, it is strongly recommended that you tune and watch after the application pool's recycling parameters so that such settings do not affect your working or peak hours significantly.

 

 

Monitoring the Scheduler

Monitoring the Scheduler is fundamental to ensure an adequate operation of Bizagi.

The Scheduler must be permanently in a started mode in order to keep Bizagi system maintenance tasks up to date (e.g, this service should not be unexpectedly interrupted).

 

note_pin

Note that in some scenarios it may be useful to scale-out the Scheduler and use multiple instances running simultaneously (involving additional nodes as necessary).

This is specially useful to distribute the processing involved in the tasks carried out by the Scheduler, which mainly are:

Importing and synchronizing users with an LDAP repository.

Importing and synchronizing information of parameter entities via Bizagi's Data Replication technology.

Executing Asynchronous activities and their retries.

Executing custom jobs.

Sending alarms or other specific notifications.

Others, such as triggering timers or other scheduled/time-based delays.

 

Therefore and according to the sizing of your project, further pro-active monitoring or other characteristics of your implementation, evaluate if using additional instances of the Scheduler service is needed.

Note that system maintenance is also performed by the Scheduler, but in a cluster, these tasks are always performed by the master instance and not distributed/load balanced.

 

Should any error or unexpected behavior happen, it will be recorded at the server's log (for instance, in a Windows operating system, at the Event viewer).

Additional aspects to monitor are:

 

ASPECTS TO MONITOR

INTERPRETATION

Scheduler executing jobs

A growing trend in the creation of jobs to be executed by the Scheduler, when compared to the actual Scheduler's processing capacity, suggest reviewing its configuration.

 

While observing an increase of jobs to be executed by the Scheduler which is being queued up, consider additional instances for a clustered Scheduler configuration (scaling-out such component).

 

Important notes regarding the database

It is very important that you monitor the database engine in order to ensure that it is operational and that it can process the requested information in a timely fashion.

By monitoring the database and its performance, you will also be able to determine if you need to scale-up the servers hosting the database (or scale-out, when using an active-active cluster scheme such as Oracle RAC).

For more information about guidelines on both monitoring and maintenance of the database, refer to Database maintenance.

 

Additionally, make sure that the specific database of your Bizagi project is accessible that its Bizagi system user (domain\admon) is active and enabled.