Com'X200 CSV export issues

[matt_azzo_com_a]

Sorry to start yet another Com'X thread, but I haven't seen this addressed anywhere else...

Using a Com'X200 with the latest firmware (1.1.20), I've been trying to test the CSV publication feature and import into PME using the ETL tool and have run into two issues preventing this:

1. The CSV files are exported in a .zip archive. This is fine for moving large amounts of data over slow internet connections, but means an additional step is required before processing (such as running a batch file set as a windows task). Would it be possible for the next firmware release to give an option not to zip?
2. I have noticed an error in the file which is preventing the PME7.2 ETL tool from processing the measurements in the file. As per the screenshot below, the file does not include the Topic ID number of -1 in line 5 of the file. If I add it in manually, ETL works fine. This can't be done manually each time however, so a fix in the next firmware would be appreciated.

Does anyone know of a work-around for these issues, or alternatively can give an indication of when these might be fixed in a firmware update?

[sesa11375]

Hi Matt,

don't be sorry to start a thread on COM'X!

as the Offer Quality Leader I'm very happy to have a direct feedback from the field to help us improve our product with what is important for our customers.

these 2 points are already known by the project team and are currently investigated as change requests.

I will inform you of how it will be managed.

Catherine

[matt_azzo_com_a]

Thanks Catherine, look forward to an update!

If anyone has got ETL into PME working with the current firmware I'd be very interested to know how, as we have committed to implementing this in the next few weeks.

[sesa56543_bridg]

Hi Matt,

I'll get in touch with you over email to dig deeper.  I may be able to provide recommendations on how to confgure ETL to import these files.

Regards,
Kyle

[Anonymous user]

Hello,

Has anyone managed to import ComX'200 CSV into PME7.2? successfully?

Regards
Edward

[matt_azzo_com_a]

Hi Edward,

As per the original post, I could get it to work if the TopicID was added manually - however this isn't really a workable solution if you need it to continually import new files as a scheduled task.

[Anonymous user]

Thanks Matt.

At least we have manual solution as temporary solution which I can advise the client if they can accept that for now.

So all they have to do is enter Topic ID number of -1 in line 5 of the file. Much appreciated for sharing that.

Regards
Edward

[matt_azzo_com_a]

Yes, although I haven't tested it with multiple measurements in the one CSV file - I expect you'd just need to add extra comma separated IDs for each.

[sesa275613_brid]

Hi Matt,

I've made a test with my Com'X and get a csv with TPIDs but anyhow maybe I have a solution for this case:

You could use the EODC which is also an ETL Tool but with more configuration options.

For additional Information about the Tool and the download -->

The configuration of the tasks could look like this:

The first Task can be used to unzip the files. You can programm a Python Script for the Extract Task so the TPID is not necessary any more and after that you can load the values directly into the PME with the Load Task.

Currently I have no time to test it decently but maybe you have the chance to do it. If the Load Task won't work you can also create another CSV File with the EODC and use this for the import into PME. There are several ways to avoid the manual input. If you need help with the configuration just let me know.

You can find the Python Script and the Parse Method for the Extract Task attached.

BR,

Elif

[JamesJeanette · Accepted solution]

Hello Edward.

As I understand, this is being tested now for Com'X 200 Version 1.4 firmware release (planned for this month).  This will give you the ability to send the CSV files compressed, or not, and the format of the CSV file is being changed to work with the PME ETL.

/ / - Anything else to add?

James

[matt_azzo_com_a]

Thanks Elif, this is perfect! It will also be useful for importing data files from third-party sources (BMS systems, weather, etc).

I'm unable to access the files with that Box link (unauthorised) - are you able to open it up to partners?

[sesa275613_brid]

Sorry for that, I was not aware of restrictions for that area. I am sure can help you with this issue.

To get started with the Tool I uploaded it so you can download it with the following link:

Energy Operation Data Connector.zip - Box