Security: CVE-2016-2183 - Sweet32 Vulnerability found on APC UPS NMCs & PDUs

[BillP]

A relatively new vulnerability, CVE-2016-2183 has been found across APC UPS with NMCs and APC PDUs. The firmware versions on these NMCs are relatively new - version 6.4.x.

Does anyone knows if there is a fix to this vulnerability for APC NMCs - AP96xx?

Reference:

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-2183

[B-DoubleE_apc]

Hi Anthony,

Do you know if you are currently running v6.4.4 or v6.4.6 when scanning for this vulnerability? 

[BillP]

APC PDUs (AP8xxx) - Firmware versions 6.4.0, 6.4.4 and latest 6.4.6 are flagged as vulnerable to this 'sweet32' CVEs. More firmware versions within the 6.4.x branch may be affected but we need time to tabulate the firmware versions flagged since there is no APC tool that can help on this centrally.

APC PDUs (AP7xxx) - still checking but there may not be newer firmware available for quite a while (months if not more than a year).

APC UPS NMC2s (AP9631xx)- Latest firmware version 6.4.6 (with HTTPS setting already configured to use TLS1.2) is also flagged as vulnerable to this 'sweet32' CVEs. 

[B-DoubleE_apc]

What was the scan that flagged the vulnerabilities? Nessus?

[BillP]

On 4/12/2017 12:56 AM, Matt said:

What was the scan that flagged the vulnerabilities? Nessus?

Hi Matt,

Unfortunately, information around the tool/appliance we use for security scan is a confidential information which cannot be shared on public domain. There is an APC case (#38079530) opened and escalated higher up within APC.

[BillP]

Hi Anthony,

We are providing answers to you through the regional Schneider representative that escalated this. I just looked the case up and saw it was the same case that we on the product teams already provided some information about. If you don't get some of the info and feedback by tomorrow (due to time differences), then I'd follow up.

[BillP]

On 4/12/2017 11:44 PM, Angela said:

Hi Anthony,

We are providing answers to you through the regional Schneider representative that escalated this. I just looked the case up and saw it was the same case that we on the product teams already provided some information about. If you don't get some of the info and feedback by tomorrow (due to time differences), then I'd follow up.

Hi Angela,

I have not receive any response yet at time of this posting. Based on my assessment, the cipers setting is no longer exposed in both GUI and CLI in the current firmware branch (v6.4.x), unlike the previous generation of firmwares (v3.9.x). It is likely the vulnerability lies with the OpenSSL daemon within the firmware and without ability to configure, a new firmware may likely be required to fix this vulnerability.

Hope to hear from Schneider soon. 

[BillP]

Hi Anthony,

We don't use OpenSSL on the NMC2 based products in any firmware, nor do we use it in NMC1 actually. Just NetBotz like NBWL0355 uses OpenSSL and no affected ciphers are enabled on that system. (So, NetBotz units using 4.5.4 (the latest firmware) are not vulnerable.) The KVMs like KVM1116P use OpenSSL as well which was being checked.

Older NMC1 units allowed users to enable/disable ciphers for performance reasons. On NMC2, performance is not an issue so that is why a user doesn't have the ability to manually enable/disable ciphers. But, personally, I think we need to re-think this internally because of questions like yours and potential ciphers users would like to just turn off for their own internal security policies. We did recently just allow the user to set the TLS protocols they want to allow.

I hope to have some confirmations and feedback on the NMC later which we'll relay back to your Schneider contact.

[BillP]

Hi Angela,

I believe the option to allow configuration of TLS cipher in newer firmware version 6.4.4 onwards for NMC2 was partly due to contributions and vulnerability findings which we (my firm/team) submitted months ago if not longer. Till date, there is still vulnerabilities for other APC/Schneider products still not resolved/remediated.

We have confirmation that this 'Sweet32' does not affect any of the NMC1 with firmware version 3.9.x which we still have. The largest hit is NMC2 across a few UPS series such as Smart-UPS Online and Symmetra. There appears to be no hit on NetBotz which came as a surprise to us.

Here is a summarized of what we believe (internal confirmation after checks) are affected/non-affected by 'Sweet32':

• NMC1 (AP961x) - not affected 
• NMC2 (AP963x) - affected even with latest 4.6.6 firmware
• PDU Gen 1 (AP7xxx) - not affected
• PDU Gen 2 (AP8xxx) - affected even with latest 4.6.6 firmware
• NetBotz Gen 1 (x5x) - not affected
• KVM - none so far (sampling size too small to confirm)

[BillP]

Hi Anthony,

Is this list based on what you were told from your local Schneider reps or what you've determined internally?

From what I understood, most scanner tools out there are going to say that the vulnerability exists just because a device supports one or more 64 block ciphers that you need in order to do the other steps involved in the exploit. I also thought it required 32GB of data to be sent or something like that to truly exploit the vulnerability.

We do scans in house with different security scanning tools and then determine what is a false positive and what is not and prioritize it all as to what needs to be fixed and when. There are other steps as part of a release that relate to cyber security and overall adherence to an SDL. Sometimes also, we just can't remove or disable certain ciphers as they are needed for legacy communications or applications.

Relating to your list, I think your comments are based on whether or not the 3DES cipher is present and that is all. Or, are you saying that you or the tools you used successfully exploited this?

Based on my thinking and what I mentioned above, I would think on NMC1/Gen 1 PDUs you would say affected but I think you're saying not affected because maybe you're disabling the ciphers you don't want individually? On the NMC2 and Gen 2 devices too, the ciphers are there but the user cannot modify them. The only option you may be able to do is force TLS 1.2 where I didn't think the cipher was necessarily used or based on researching, would bypass the vulnerability.

With the KVMs, I am still following up internally with the concerned team but I do know the KVMs rely on OpenSSL but I am not sure of the configuration or what version.

In general though and because of what I was thinking because of how scanner tools work, I don't know if all of the products are straight up vulnerable based on the way the NMC works as a system and other mechanisms it supports to prevent the exploit. I don't have the technical details unfortunately and am going based on what some of the firmware engineers have said.

[BillP]

On 19-Apr-17 01:51, Angela said:

Hi Anthony,

Is this list based on what you were told from your local Schneider reps or what you've determined internally?

From what I understood, most scanner tools out there are going to say that the vulnerability exists just because a device supports one or more 64 block ciphers that you need in order to do the other steps involved in the exploit. I also thought it required 32GB of data to be sent or something like that to truly exploit the vulnerability.

We do scans in house with different security scanning tools and then determine what is a false positive and what is not and prioritize it all as to what needs to be fixed and when. There are other steps as part of a release that relate to cyber security and overall adherence to an SDL. Sometimes also, we just can't remove or disable certain ciphers as they are needed for legacy communications or applications.

Relating to your list, I think your comments are based on whether or not the 3DES cipher is present and that is all. Or, are you saying that you or the tools you used successfully exploited this?

Based on my thinking and what I mentioned above, I would think on NMC1/Gen 1 PDUs you would say affected but I think you're saying not affected because maybe you're disabling the ciphers you don't want individually? On the NMC2 and Gen 2 devices too, the ciphers are there but the user cannot modify them. The only option you may be able to do is force TLS 1.2 where I didn't think the cipher was necessarily used or based on researching, would bypass the vulnerability.

With the KVMs, I am still following up internally with the concerned team but I do know the KVMs rely on OpenSSL but I am not sure of the configuration or what version.

In general though and because of what I was thinking because of how scanner tools work, I don't know if all of the products are straight up vulnerable based on the way the NMC works as a system and other mechanisms it supports to prevent the exploit. I don't have the technical details unfortunately and am going based on what some of the firmware engineers have said.

Hi Angela,

The tool/appliance the firm use is among the industry-leading and the 'Security' teams (worldwide) specialise in what they do best. The list I shared is the list of specific devices being flagged with this 'sweet32' vulnerability. In short, no first generation NMC and PDU (both with the last latest firmware version which may be as old as few years ago) have been flagged. Only those second generation - NMC2 and PDU AP8xxx which runs firmware tree of version 6.4.x are affected. I do not have the means to reverse engineer APC firmwares to determine what daemon and its version runs in them. One obvious analysis is indicating anything with firmware tree 4.6.x is affected. Anything with firmware tree 4.5.x (e.g. NetBotz) and 3.9.x (e.g. 1st gen NMC and PDU) are not affected.

All our NMC2 with firmware tree 4.6.x are already configured with TLS1.2 which was to fix another vulnerability we raised to APC months ago and APC added that configurable option.

Time is running short...and the timeline we have been given to mitigate or mark as manufacturer has no solution is coming up soon...Unfortunately, I (or my team) have so far not hear from APC besides being escalated higher up within APC.

[BillP]

Hi Anthony,

The only thing further beyond what I explained that I can offer personally is to say we'll log it as a bug but if you're looking for us to disable 3DES or allow the user to choose individual ciphers, both of those will have to be evaluated for a future release and cannot happen overnight. Doing something like this isn't clear cut as I am sure you understand as we have to make sure other things don't break and do testing. There are other ways to mitigate Sweet32 in general that would recommended to users if your tool is truly indicating we are vulnerable beyond just supporting 3DES. I'd bet you are aware of those already anyway.

The newer AOSs in NMC2 devices with 6.4.X also have a preferred list of ciphers with AES128 being at the top of that list.

For the bug, it'd be ideal if you could share the scan report offline to me so I could show the proof that this is coming up in the security scans. I've asked that someone triple check our own recent security scans to see if it has come up in there.

Lastly, the KVM team reports back that the KVMs are not affected by this (and they use OpenSSL as we discussed) so if you have something saying otherwise, I would need further details since the original question I saw on behalf of your regional team was "Is KVM affected - yes or no"

I will continue to drive this as a priority to check on further as much as I can but I think it needs more investigation on our side if you're unable to provide further proof or evidence of what you're seeing.

[BillP]

I believe I have stated earlier on for the KVM. I still believe KVM is not affected as we do have both 1st gen and 2nd gen ones around this region but none found affected.

• KVM - none so far (sampling size too small to confirm)

I am afraid I will not be able to share any security information/logs/reports even offline without first going through a series of processes to get clearance.

[BillP · Accepted solution]

Hi Anthony,

Yes, saw your comment on KVM so far and just wanted to add what our team reported internally.

Understood on the second part of your comment as well. We would just be looking for the snippet of what the tool reported and/or what criteria it checked without any IP address or other internal information, etc. I understand if that does not change anything for what you'd have to do to provide it.