Issue
Network scanning software causes a loss of Jace network communication.
Product Line
TAC IA Series
Environment
I/A Series Jace 8000 N4.9 and later
Cause
As network security awareness continues to expand, software-based scanners, such as industry-standard Qualys and Nessus, intended to detect and report on vulnerabilities within internal networks will likely continue to gain popularity. Today, Tridium is seeing these scanners being deployed and run against Niagara-based platforms like the JACE-8000 and Edge 10. In some cases, these scans are causing Niagara platforms to become unresponsive or reboot via an Engine Watchdog Timeout, all of which are not acceptable for the critical applications that Niagara facilitates.
Resolution
While Tridium has no control over how these scanners behave, or how and when they are executed within an organization, Niagara 4.9 introduces a number of changes intended to allow a Niagara-based hardware platform to appropriately respond to the scanning utilities, and also maintain operation. Below is a brief explanation of how Niagara will function under the different known circumstances currently employed by these scanners, and how to interpret the results.
Recognition of non-Niagara Traffic on the platform
In the event a scanner is interrogating a Niagara 4.9+ platform connection, the Niagara Daemon has been modified to recognize non-Niagara traffic over a period of time, shut down the connection if necessary, and wait for a pre-determined amount of time before re-enabling connectivity. Under these conditions, a scanning utility may report that the Niagara instance has encountered denial of service, when in fact, Niagara has simply disabled the communication mechanism by which the scanner was attempting its interrogation. During this time, normal platform communication will also be affected; however, the Niagara platform and station will continue to run.
Prioritization of Internal vs. External Communication on the Niagara station
In the event, a scanner is interrogating a Niagara 4.9+ station (external communication) and Niagara detects that this interrogation may cause an Engine Watchdog Timeout, the station’s web server will be stopped and restarted. Under these conditions, a scanning utility may report that the Niagara instance abruptly stopped communicating, and may have encountered denial of service. During this time period, normal/expected client web connections to the station will also be affected; however, the Niagara platform and station will continue to run.
As mentioned, these scanners are outside the control of Tridium, and likely always evolving to meet the needs of the various threats they are intended to protect against. As a best practice, Tridium recommends not scanning in production if possible, as any findings would be just as legitimate during scheduled downtime. Additionally, it may be prudent to work with the scanning tools to configure the appropriate priority of a scan, as the intensity of which you scan a production multicore, failover redundant web server host, is likely not the best choice for scanning a single-core JACE.
Should you encounter an issue with a network scanning utility and Niagara 4.9 and above, please contact your support organization.
