Config.ini via DHCP limitations?

[BillP]

I'm having trouble getting a config.ini file to fully process when it's supplied over FTP with DHCP options 66 and 67.  It seems like it gets part of the way through the file and then stops processing for some reason and I can't seem to figure out why.  If I manually FTP the file to the device it seems to process just fine.

The first time the config.ini gets transferred I see this in the log:

07/22/2014	06:29:48	System: Warmstart.
07/22/2014	05:28:44	System: Restarting. FTP boot file download.
07/22/2014	05:28:42	System: Configuration file upload complete, with 27 valid values.
07/22/2014	05:28:42	System: Configuration change. Email 'from' address.
07/22/2014	05:28:42	System: Configuration change. Temperature preference.
07/22/2014	05:28:42	System: Configuration change. Secondary NTP server address.
07/22/2014	05:28:42	System: Configuration change. Primary NTP server address.
07/22/2014	05:28:41	System: Configuration change. NTP enable.
07/22/2014	05:28:41	System: Configuration change. Secondary RADIUS server secret.
07/22/2014	05:28:41	System: Configuration change. Secondary RADIUS timeout.
07/22/2014	05:28:40	System: Configuration change. Secondary RADIUS server address.
07/22/2014	05:28:40	System: Configuration change. Primary RADIUS server secret.
07/22/2014	05:28:40	System: Configuration change. Primary RADIUS timeout.
07/22/2014	05:28:40	System: Configuration change. Primary RADIUS server address.
07/22/2014	05:28:39	System: Configuration change. RADIUS access.
07/22/2014	05:28:39	System: Configuration change. System inactivity timeout.
07/22/2014	05:28:39	System: Configuration change. System contact.
07/22/2014	05:28:38	System: Configuration change. Override Manual DNS Settings.
07/22/2014	05:28:38	System: Configuration change. IPv6 enable.
07/22/2014	05:28:38	System: Configuration change. Domain name.

The next time it renews it's DHCP lease and gets the same config.ini I get this:

07/22/2014	06:39:01	System: Warmstart.
07/22/2014	06:38:49	System: Restarting. FTP boot file download.
07/22/2014	06:38:49	System: Configuration change. System contact.
07/22/2014	06:38:49	System: Configuration change. Override Manual DNS Settings.
07/22/2014	06:38:48	System: Configuration change. IPv6 enable.
07/22/2014	06:38:48	System: Configuration change. Domain name.
07/22/2014	06:38:47	System: Configuration change. System administrator password.
07/22/2014	06:38:47	System: Configuration file upload complete, with 0 valid values.
07/22/2014	06:38:47	System: File transfer failed.

The third time I get this:

07/22/2014	07:00:29	System: Warmstart.
07/22/2014	07:00:17	System: Restarting. FTP boot file download.
07/22/2014	07:00:16	System: Configuration file upload complete, with 0 valid values.
07/22/2014	07:00:16	System: File transfer failed.

I'm not sure why it's reporting "File transfer failed" because on the FTP server it looks the same every time it connects.

(000022)7/22/2014 9:04:15 AM - (not logged in) (10.11.12.13)> Connected, sending welcome message...

(000022)7/22/2014 9:04:15 AM - (not logged in) (10.11.12.13)> 220 Test FTP FileZilla Server version 0.9.45 beta

(000022)7/22/2014 9:04:15 AM - (not logged in) (10.11.12.13)> USER apc

(000022)7/22/2014 9:04:15 AM - (not logged in) (10.11.12.13)> 331 Password required for apc

(000022)7/22/2014 9:04:15 AM - (not logged in) (10.11.12.13)> PASS ***

(000022)7/22/2014 9:04:15 AM - apc (10.11.12.13)> 230 Logged on

(000022)7/22/2014 9:04:15 AM - apc (10.11.12.13)> TYPE I

(000022)7/22/2014 9:04:15 AM - apc (10.11.12.13)> 200 Type set to I

(000022)7/22/2014 9:04:16 AM - apc (10.11.12.13)> PORT 10,35,14,84,11,184

(000022)7/22/2014 9:04:16 AM - apc (10.11.12.13)> 200 Port command successful

(000022)7/22/2014 9:04:16 AM - apc (10.11.12.13)> RETR apc-config.ini

(000022)7/22/2014 9:04:16 AM - apc (10.11.12.13)> 150 Opening data channel for file download from server of "/apc-config.ini"

(000022)7/22/2014 9:04:16 AM - apc (10.11.12.13)> 226 Successfully transferred "/apc-config.ini"

(000022)7/22/2014 9:04:17 AM - apc (10.11.12.13)> QUIT

(000022)7/22/2014 9:04:17 AM - apc (10.11.12.13)> 221 Goodbye

(000022)7/22/2014 9:04:17 AM - apc (10.11.12.13)> disconnected.

If I manually FTP the same config.ini file to the device it seems to process it just fine.  I'm wondering if maybe there is some kind of limitation on the size of the file or maybe something with the encoding that is causing it to blow up.  Is there any detailed config.ini processing log available or anything else that might help me figure this out? 

The device is a Smart-UPS 1500, here are the software versions:

Model Number:	AP9630
Hardware Revision:	05
Manufacture Date:	11/19/2013

Application Module

Name:	sumx
Version:	v5.1.3
Date:	May 4 2010
Time:	14:06:35

APC OS (AOS)

Name:	aos
Version:	v5.1.3
Date:	May 4 2010
Time:	13:55:26

APC Boot Monitor

Name:	bootmon
Version:	v1.0.1
Date:	Aug 3 2009
Time:	16:30:40

Incidentally I tried with 6.0.6 and 6.1.1 firmware but that was even worse, the device got stuck in a reboot loop.  It would ftp the file, process it, reboot and then do it all over again until I stopped the FTP server so it couldn't get the file again.  At least after downgrading to the older version it only gets the file when it renews it's DHCP lease.

[BillP]

Have you tried 5.1.7? That is our latest version of the v5.X.X code. Meanwhile, I am going to see if someone with the resources to test this can look at it.

There are no extra log files you are missing and I know of no limitations. The only thing related to this I was aware of was on our older cards and TFTP (versus FTP) not working properly for this functionality.

I would try 5.1.7 to verify it reacts the same way as 5.1.3 and I will see what other information I can find out in the meantime. I will attached 5.1.7 to the post if you need it.

[BillP]

Pretty much the same results.  It processes some of the config.ini but not all of it and it still says the file transfer failed.

Is it possible the file encoding is wrong?

07/22/2014	13:10:12	System: Network Interface restarted.
07/22/2014	13:09:59	System: Restarting. FTP boot file download.
07/22/2014	13:09:59	System: Configuration change. Override Manual DNS Settings.
07/22/2014	13:09:59	System: Configuration change. IPv6 enable.
07/22/2014	13:09:59	System: Configuration change. Domain name.
07/22/2014	13:09:58	System: Configuration file upload complete, with 0 valid values.
07/22/2014	13:09:58	System: File transfer failed.

[BillP]

It seems to get a few of the settings the 2nd time it downloads the file too.  I thought I was seeing this in the logs before but I figured I was misreading it.

07/22/2014	13:20:38	System: Restarting. FTP boot file download.
07/22/2014	13:20:38	System: Configuration file upload complete, with 26 valid values.
07/22/2014	13:20:38	System: Configuration change. SNMP access control 2 community.
07/22/2014	13:20:38	System: Configuration change. SNMP access control 1 community.
07/22/2014	13:20:37	System: Configuration change. Temperature preference.
07/22/2014	13:20:37	System: Configuration change. Secondary NTP server address.
07/22/2014	13:20:37	System: Configuration change. Primary NTP server address.
07/22/2014	13:20:37	System: Configuration change. NTP enable.
07/22/2014	13:20:36	System: Configuration file upload complete, with 0 valid values.
07/22/2014	13:20:36	System: File transfer failed.

[BillP]

Yes, this is strange. I can't answer much more without having someone look into it for me and test it. I expect a response to help provide responses to you hopefully in a few days to see if we can replicate the problem and figure out what it is wrong on both firmwares (v5 and v6). I feel like something is stopping it from applying all of the values but it's only a guess at this point.

[BillP]

It seems to be cutting off the file somehow.  The last file I tested it quit at 1460 characters (I know this because it cut off one of the settings in mid string.)  I tried Filezilla FTP server and the IIS FTP server and neither one worked properly.  I also did a wireshark capture and it shows that the entire file was transferred so I'm pretty sure it's something on the device end. 

As an aside I see from the capture that it is trying a TFTP transfer first then going to FTP, that's probably why it always shows "file transfer failed" in the logs first.

[BillP]

Yup, it does try TFTP first then FTP. That was where I knew we had a bug on the older cards that TFTP did not work at all.

I will keep you updated and thank you for the extra information, it is appreciated.

[BillP]

I appreciate the attention to this issue very much, thank you!

I put a TFTP server up and it consumes the same INI file and processes all of the settings successfully so it's something about the FTP code apparently.

[BillP]

Just an FYI, I expect to know something early next week between the testing team's previous obligations (an all week training) and time to review/look at it. Will report back ASAP!

[BillP]

That's great, thanks!  I may have stumbled across another issue that's in the same neighborhood but not quite the same.  Testing on a Smart-UPS 5000 RM XL with an AP9617 running AOS v3.7.3 and SUMX v3.7.2 I can't seem to get the thing to take ANY config from the DHCP setup and it never tries TFTP.  Did you say that there was some kind of FTP issue with some earlier hardware?

[BillP]

Yeah, there is. Here is some information:

Issue

Network Management Card 1 (AP9617/18/19) not attempting a TFTP transfer of a configuration/boot file while using BOOTP


Product Line

• Network Management Card 1 (NMC1) - AP9617, AP9618, AP9619

Devices with an embedded Network Management Card 1 include (but are not limited to): Metered/Switched Rack PDUs (AP78XX, AP79XX), Rack Automatic Transfer Switches (AP77XX, Environmental Monitoring Units (AP9320, AP9340, Netbotz 200)

Environment

• All serial numbers
• Firmware versions 3.X.X

Cause

In the user's manual for Network Management Card 1 (AP9617, AP9618, AP9618), the following feature set of the NMC is noted:

When the Management Card reboots, the BOOTP server provides it with the TCP/IP settings.
• If you specified a bootup file name, the Management Card attempts to transfer that file from the BOOTP server using TFTP or FTP.

The Management Card assumes all settings specified in the bootup file. In NMC firmware revisions 1.x and 2.x, this feature works as designed. While using an NMC with BOOTP to assign an IP address, the NMC will accept an IP address from a BOOTP server, and five minutes later, attempt a TFTP transfer of a specified bootup file. If no TFTP server is reachable, it will attempt FTP transfer of a specified bootup file.

In NMC firmware revision 3.x, the NMC does not attempt a TFTP transfer, only an FTP transfer. Customers relying on TFTP will not have success in specifying a bootup file.


Resolution

Though TFTP does not work, customers can use FTP to configure their NMC device using a boot file (config.ini). FTP normally requires a client login. Since the NMC device is at its default settings, the customer will need to create an "apc" user on the FTP server. At default settings, the NMC device will attempt an FTP login using apc for both the username and password. The customer will need to create such an account on their FTP server in order for a successful upload of the boot file.

I think this was discovered after those cards were discontinued so that is why it was not addressed. So TFTP does not work but FTP should with the older cards.

[BillP]

I'm beginning to wonder if I'm chasing a pipe dream here.  The second (older) UPS doesn't seem to even try TFTP (as the article described) but it does get the file from the FTP server OK and then doesn't do anything with it.

07/25/2014	13:51:57	System: Warmstart.
07/25/2014	13:51:39	System: File transfer failed.

I've tried 3 different FTP servers now and they all show the same thing - ups logs in, ups downloads config.ini, ups disconnects.  Once again if I manually FTP the file TO the UPS it processes it just fine.

[BillP · Accepted solution]

I apologize. I got wrapped up the last two days and found out last Thursday we are able to replicate your issues on NMC2. We don't actively develop on the older NMC1 platform anymore so any fixes would go into NMC2 v6.X.X platform. We were going to be trying it the latest available development code and if not fixed there, entering a bug request to address this issue with using the bootfile.