Netcontroller is running slowly (or going offline)

[Product_Support]

Issue

Netcontroller is running slowly or occasionally goes offline (bCX NC2 ACX CX)

Product Line

Andover Continuum

Environment

• Continuum
• Netcontroller, Net Controller, Infinet, MSTP, Bacnet, Xdriver, x-driver

Cause

There are various reasons why the operation of one or more Netcontrollers and or attached controllers is slow and/or goes offline occasionally.  This guide is primarily to help collate the right information which may identify the issues.  If the reason cannot be identify then this information can be provided to support for further analysis.

Resolution

General troubleshooting techniques should be used.

1. Define the exact problem.
   • What exactly is slow? eg. Sending to the controller or its connected devices, opening or refreshing graphics, opening or refreshing points, updating Xdriver points, saving programs, creating new points.
   • Is the controller slow from all cyberstations?
   • Is the controller always slow? If not then try to find what makes it go slow, it is time related? Can certain steps be taken to cause the slowness or offline to happen?
   • Get a screen capture of any errors.
2. Gather site information.
   1. Continuum version, Continuum hotfixes and service packs applied, number of Cyberstation. Cyberstation operating system, details of Ethernet network and subnets used, with BBMD information of Bacnet sites (Asbuilt or a spreadsheet with IPs, etc.)
   2. Continuum Analyzer report, site controller map or spreadsheet or text dump
   3. Check the PCs running Cyberstation, do they meet the system specifications?
3. Gather controller information
   1. What is the Scan rate? Is the controller time sync. correct? How many objects are on the controller, how many programs are running? What is the firmware version of the netcontroller and b3 or i2 controllers?
   2. Ethernet connection - Is the controller on the same subnet as the Cyberstations? A wireshark trace may help to establish the quantity of traffic on the Ethernet (See Using WireShark to analyze communications on an Ethernet network)
   3. Comms Ports - XDriver being used?
      What is on the comms ports? Xdriver (how much communications, points etc)
   4. Comms Ports - Bacnet MSTP - How many b3 controllers? How much extended logging? How many alarms being sent? How many 3rd party devices? Look at bacnet settings, how many values are being sent across the network? Check COV settings. Try removing the MSTP cable from the bCX (there could be an issue with an MSTP device causing the bCX to go offline/go slow)
   5. Comms Port - Infinet - How many infinet controllers? How much extended logging? How many alarms being sent? Look at infinet settings, how many values are being sent across the network? Check port reconfigs and errors. Try removing the infinet cable from the NetController (there could be an issue with an infinet device causing the NetController to go offline/go slow)
   6. Comms Port - other. Again try removing the comms cable from the NetController (there could be an issue with a device on this network causing the NetController to go offline/go slow)
   7. Run the Controller error log tool to gather the controller errors See Gather Continuum Controller error logs if requested by Product Support for troubleshooting
   8. If the controller goes offline, run the alarm listview.  Does it line up with the Controller error log?
   9. Check the CPU LED flash rate on all connected devices (i1, i2, b3 etc) as Can't bring bCX online in Continuum but do get a response to pinging its IP address, also the RD and TD LEDs should be checked for anything unusual
4. This guide is primarily to help the on site engineer by helping them to assess the site. If there is still a need to contact support, please review and record as many items above as possible and send them to the case owner. 
5. Further tests can be carried out including an ACCtrace to determine if the restriction could be at the Cyberstation, See How to capture Continuum Cyberstation activity using ACCTrace
   NOTE: Ensure all Cyberstations, Controllers and other laptops used for logging are in time synchronization before recording any logs.
6. If it is believed there is an issue with the Infinet or MSTP network see Infinet and mstp bus troubleshooting
7. If it is believed the problem is with extended logging see Extended Logging gaps, performance and load balancing
8. It is possible that something connected to the controller causes the controller to go offline, try disconnecting all other cables and connections to the controller and then re-testing
9. Where a controller is offline when connected directly to a PC/Laptop, try resetting the IP address, otherwise the controller will probably need replacing.

NOTE
Disconnecting all networks from the NetController is often a good starting point (reconnected IP/Ethernet cable)