Modifying values in EWS server low performance

Lihov · ‎2017-07-18

Hello!

My SmartConnector Extension get data from 600 electrical counters. Part of reading data from devices works fine, but updating EwsValueItems in EWS server performs very slow with high CPU usage. The number of measured parametrs is about 2500. Update rate - 30s. So, when I invoke DataAdapter.ModifyValueItemValue for each item in loop for upadting items vlaue, the CPU usage is increases up to 30-40% by Mogoose.Service. The duration measurement in log file present.

In Log.txt ForEach duration means execution time of bottom red rectangle (ms). Join Duration - top rectangle (ms). Therefore, ModifyValueItemValue operates slowly - but what a reason?

What is a best approach for updating Ews Items Values with high Update Rate?

sesa180908_brid · ‎2017-07-19

If you have long running "tight" loops in a Processor you are required to monitor the CancellationToken. Section "5.5.3.6.2 Looping" in the SmartConnector Developers Guide talks about this a bit. This is important if you want to have control over your running processes.

Also when you have tight/long running loops (for, for each, while etc) it is best to allow other threads to have priority. This can easily be done by calling Thread.Sleep for a short time frame (on the order of a few mSec).

The Processor base class will do both of these for you in the "NoBusyWait" call. I will typically put a NoBusyWait(5) call at the bottom of most arbitrarily long loops. My guess will be that this will lessen the CPU spikes you see in your looping.

As for general performance of the EwsServerDataAdapter. It is a known issue that writes to the underlying database (Add and Modify methods) are slow when used in "chatty" use cases like this one. We are currently working on an enhancement (in the 2.3 release) which will provide a way to delay the actual commit to the database. Preliminary testing does show some considerable improvements in performance.

See Answer In Context

sesa180908_brid · ‎2017-07-19

If you have long running "tight" loops in a Processor you are required to monitor the CancellationToken. Section "5.5.3.6.2 Looping" in the SmartConnector Developers Guide talks about this a bit. This is important if you want to have control over your running processes.

Also when you have tight/long running loops (for, for each, while etc) it is best to allow other threads to have priority. This can easily be done by calling Thread.Sleep for a short time frame (on the order of a few mSec).

The Processor base class will do both of these for you in the "NoBusyWait" call. I will typically put a NoBusyWait(5) call at the bottom of most arbitrarily long loops. My guess will be that this will lessen the CPU spikes you see in your looping.

As for general performance of the EwsServerDataAdapter. It is a known issue that writes to the underlying database (Add and Modify methods) are slow when used in "chatty" use cases like this one. We are currently working on an enhancement (in the 2.3 release) which will provide a way to delay the actual commit to the database. Preliminary testing does show some considerable improvements in performance.

Lihov · ‎2017-07-26

Thak you, Mark, for the response. I know about monitoring of CancellationToken - it was added to final version of method.

I achived a reduction of processor load to 5-10% by NoBusyWait calling, but summary duration of method greatly increased.

When is the release of version 2.3 planned?

sesa180908_brid · ‎2017-07-26

We don't have a definitive date for the 2.3 release. It should be sometime in early Q4 though.