Worker is being aborted because the processor is not completing in a timely manner. Total elapsed runtime is 603.2563196

[mike_meirovitz]

Hi,

We have a setup processor that runs for more than 10 minutes. After 10 minutes, the processor is aborted with the following messages in the log file:

2020-12-23 18:24:11.0170,Info,WorkerManager,<no principal>,Worker is being aborted because the processor is not completing in a timely manner.  Total elapsed runtime is 603.2563196,

2020-12-23 18:24:11.0831,Error,Processor,<no principal>,The operation was canceled.

   at System.Threading.CancellationToken.ThrowOperationCanceledException()

   at Mongoose.Process.Processor.CheckCancellationToken()

   at SmartConnector.EBOOPCClient.SetupProcessor.EnsureContainerItem(String altId, String name, String description, EwsContainerTypeEnum type, EwsContainerItem parent)

   at SmartConnector.EBOOPCClient.SetupProcessor.AddOPCServerItems(OpcClientManager opcClientManager)

   at SmartConnector.EBOOPCClient.SetupProcessor.Execute_Subclass()

   at Mongoose.Process.Processor.Execute(CancellationToken token, ProcessorExecutionMode mode)

,["EBO-OPC Client Setup Processor"],

Any idea what can be done?

Thank you.

Mike

[mike_meirovitz · Accepted solution]

Hi,

I understand why the processor was aborted. The "Processor Runtime Limit" value in the SmartConnector service settings was "600" which means 10 minutes.

After increasing the value to "1800", the processor was not aborted and completed successfully.  

However, I have noticed continues degradation in the performance of adding an items/values to the EWS server. The time to add one value at the beginning of the process is ~29 milliseconds. If I add 10,000 items/values, the time that it takes to add the last items/values is ~182 milliseconds. 

Can someone explain why the performance of adding items is getting worse when adding large amount of items/values? 

See the our log file below.

Creating first items/values:

2020-12-28 20:35:39.674 [18] DEBUG, add EWS item '1K.A0001', # of items 14 of 10016
2020-12-28 20:35:39.703 [18] DEBUG, add EWS item '1K.A0002', # of items 15 of 10016     dif = (703-674)=29 mili

Creating last items/values: 

2020-12-28 20:52:23.084 [18] DEBUG, add EWS item '10K.B4998', # of items 10014 of 10016
2020-12-28 20:52:23.266 [18] DEBUG, add EWS item '10K.B4999', # of items 10015 of 10016  dif =  (266-084)=182 mili

Thank you.

Mike

[JeffBowman]

Hi Michael,

The processor is taking longer than 600 seconds to run, and this is the default timeout for processors.

To fix this you have two options:

1. Add the ILongRunningProcess interface to your processor, so the framework doesn’t automatically stop it from running long.
2. Increase the Processor Runtime Limit from 600 to something higher, this is in the Settings in the Smart Connector portal.

Best Regards,

-Jeff

[JeffBowman · Accepted solution]

Hi Michael,

I can't be sure, but I suspect this has something to do with the fact that the database is larger, and it takes longer to execute queries. In this case, the Smart Connector framework needs to look through each object and verify that what is being added, has not already been created, so this could explain why it takes longer..

Best Regards,

-Jeff