ComX510 data Export via HTTP or HTTPS

Sushant_Belote · ‎2019-01-12

Hello Experts,

I am trying to export log data from ComX 510 to EMS server automatically.

I have successfully tested with FTP.

May i know how can i use HTTP & HTTPS methods for data export, as client restrict us to use FTP method.

Thanks in Advance.

Randi_Dolan · ‎2019-01-12

Hello,

To export data log file from the Com'X via HTTP or HTTPs, you will have to setup a web server and configure it to receive files from the Com'X. Typically this is done with a script written in cgi or perl.

Here is a document on how to do this on the EGX300. The Com'X works the same way. Although, the fieldname is different.

EGX 300 - Logged Data Export over HTTP
======================================


Rationale
---------
Provide export of EGX300 logged data over HTTP.  This allows EGX's
inside firewalled and/or NAT'ed (Network Address Translated) networks
to export their logged data to external servers without port-forwarding
or other complications.


Background
----------
EGX 300 supports exporting logged data over Email (SMTP) and FTP.
This project adds the capability to "push" logged data over an HTTP
POST message. By doing so, common HTTP servers can aggregate logged
data from one or many EGX's.


Approach
--------
Provide export of EGX300 logged data over HTTP 'POST' method. Any
WWW server that supports POST messages and installing one's own CGI
handlers is sufficient. Our testing used Apache 1.3.34 (www.apache.org)
running on FreeBSD (www.freebsd.org). The same scripts will work
on a variety of WWW servers and operating systems. (Linux, OpenBSD,
Solaris, etc.)

The EGX will post a message specifying the filename of where to
deposit the POST data. By default this is 'datafile1', but this can
be customized in the EGX web interface. This would be useful in the
case where you already have a POST handler that uses a different
field name for this purpose.


Sample Upload handler - Python
------------------------------
The following is the full text of an EGX-compatible Python script 
that accepts HTTP uploads from EGX's. Install it into the directory
on your chosen web server where CGI scripts reside. (this varies by
hosting provider)

#! /usr/bin/env python
#
# based on http://webpython.codepoint.net/cgi_tutorial
#
import cgi, os
# cgitb module gives specific tracebacks when there is a problem,
#   rather than just a generic failure message.
import cgitb
cgitb.enable()
try: # Windows needs stdio set for binary mode.
    import msvcrt
    msvcrt.setmode (0, os.O_BINARY) # stdin  = 0
    msvcrt.setmode (1, os.O_BINARY) # stdout = 1
except ImportError:
    pass
form = cgi.FieldStorage()
# Generator to buffer file chunks
def fbuffer(f, chunk_size=10000):
   while True:
      chunk = f.read(chunk_size)
      if not chunk: break
      yield chunk
# A nested FieldStorage instance holds the file
fileitem = form['datafile1']
# Test if the file was uploaded
if fileitem.filename:
   # strip leading path from file name to avoid directory traversal attacks
   fn = os.path.basename(fileitem.filename)
   # Customize this directory to match your WWW server
   outfil = open('/usr/local/www/data/uploads/' + fn, 'wb', 10000)
   # Read the file in chunks
   for chunk in fbuffer(fileitem.file):
      outfil.write(chunk)
   outfil.close()
   message = 'The file "' + fn + '" was uploaded successfully\n'
else:
   message = 'No file was uploaded'
print """\
Content-Type: text/html\n
<html><body>
<p>%s</p>
</body></html>
""" % (message,)


Sample Upload handler - PERL
----------------------------
The following is the full text of an EGX-compatible PERL script to
accept HTTP file uploads.

#!/usr/bin/perl -wT
use strict;
use CGI;
# set maximum allowable size to 5MB
$CGI::POST_MAX = 1024 * 5000;
my $safe_filename_characters = "a-zA-Z0-9_.-";
my $upload_dir = "/usr/local/www/data/uploads";
my $query = new CGI;
my $filename = $query->param('datafile1');
if ( !$filename )
{
  print $query->header ( );
  print "There was a problem uploading your file; try a smaller file.";
  exit;
}
my ( $name, $path, $extension ) = fileparse ( $filename, '\..*' );
$filename = $name . $extension;
$filename =~ tr/ /_/;
$filename =~ s/[^$safe_filename_characters]//g;
if ( $filename =~ /^([$safe_filename_characters]+)$/ )
{
  $filename = $1;
}
else
{
  die "Filename contains invalid characters";
}
open ( UPLOADFILE, ">", "$upload_dir/$filename" ) or die "$!";
binmode(UPLOADFILE);
my $upload_filehandle = $query->upload("datafile1");
while ( <$upload_filehandle> )
{
  print UPLOADFILE;
}
close UPLOADFILE;
print $query->header ( );
print <<END_HTML;
<html>
<body>
<p>The file $filename was uploaded successfully</p>
</body>
</html>
END_HTML


Microsoft Internet Information Services (IIS)
---------------------------------------------
IIS can most naturally use Active Server Pages (ASP) or ASP.NET
to upload files. Alternately, IIS can be configured to support CGI.
Unfortunately, IIS does not support CGI as installed "out of the box".
Then one may use PERL/Python/etc. CGI scripts as above.


Sample Upload handler - Microsoft Internet Information Services (IIS) w/ ASP.NET
--------------------------------------------------------------------------------

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.IO;

namespace EnergyDataReceiver
{
    public partial class filepost : System.Web.UI.Page
    {
        protected void Page_Load(object sender, EventArgs e)
        {
            String TempFileName;
            HttpFileCollection myFileCollection = Request.Files;

            for (int Loop1 = 0; Loop1 < myFileCollection.Count; Loop1++)
            {
                string fname = myFileCollection[Loop1].FileName;
                // Create a new file name.
                TempFileName = "C:\\TempFiles\\File_" + Loop1.ToString() + "_" + Path.GetFileName(fname);
                // Save the file.
                myFileCollection[Loop1].SaveAs(TempFileName);
            }
        }
    }
}


References
----------
1. RFC 1867 "Form-based File Upload in HTML",
    http://www.ietf.org/rfc/rfc1867.txt
2. File Upload with ASP.NET,
    http://www.codeproject.com/KB/aspnet/fileupload.aspx
3. Uploading a file to IIS using a browser,
    http://support.microsoft.com/default.aspx/kb/189651
4. Google Search of "IIS cgi" yields quite a few pointers and tutorials:
    http://www.google.com/search?q=IIS+cgi

Best regards,

Randi

See Answer In Context

Randi_Dolan · ‎2019-01-12

Hello,

To export data log file from the Com'X via HTTP or HTTPs, you will have to setup a web server and configure it to receive files from the Com'X. Typically this is done with a script written in cgi or perl.

Here is a document on how to do this on the EGX300. The Com'X works the same way. Although, the fieldname is different.

EGX 300 - Logged Data Export over HTTP
======================================


Rationale
---------
Provide export of EGX300 logged data over HTTP.  This allows EGX's
inside firewalled and/or NAT'ed (Network Address Translated) networks
to export their logged data to external servers without port-forwarding
or other complications.


Background
----------
EGX 300 supports exporting logged data over Email (SMTP) and FTP.
This project adds the capability to "push" logged data over an HTTP
POST message. By doing so, common HTTP servers can aggregate logged
data from one or many EGX's.


Approach
--------
Provide export of EGX300 logged data over HTTP 'POST' method. Any
WWW server that supports POST messages and installing one's own CGI
handlers is sufficient. Our testing used Apache 1.3.34 (www.apache.org)
running on FreeBSD (www.freebsd.org). The same scripts will work
on a variety of WWW servers and operating systems. (Linux, OpenBSD,
Solaris, etc.)

The EGX will post a message specifying the filename of where to
deposit the POST data. By default this is 'datafile1', but this can
be customized in the EGX web interface. This would be useful in the
case where you already have a POST handler that uses a different
field name for this purpose.


Sample Upload handler - Python
------------------------------
The following is the full text of an EGX-compatible Python script 
that accepts HTTP uploads from EGX's. Install it into the directory
on your chosen web server where CGI scripts reside. (this varies by
hosting provider)

#! /usr/bin/env python
#
# based on http://webpython.codepoint.net/cgi_tutorial
#
import cgi, os
# cgitb module gives specific tracebacks when there is a problem,
#   rather than just a generic failure message.
import cgitb
cgitb.enable()
try: # Windows needs stdio set for binary mode.
    import msvcrt
    msvcrt.setmode (0, os.O_BINARY) # stdin  = 0
    msvcrt.setmode (1, os.O_BINARY) # stdout = 1
except ImportError:
    pass
form = cgi.FieldStorage()
# Generator to buffer file chunks
def fbuffer(f, chunk_size=10000):
   while True:
      chunk = f.read(chunk_size)
      if not chunk: break
      yield chunk
# A nested FieldStorage instance holds the file
fileitem = form['datafile1']
# Test if the file was uploaded
if fileitem.filename:
   # strip leading path from file name to avoid directory traversal attacks
   fn = os.path.basename(fileitem.filename)
   # Customize this directory to match your WWW server
   outfil = open('/usr/local/www/data/uploads/' + fn, 'wb', 10000)
   # Read the file in chunks
   for chunk in fbuffer(fileitem.file):
      outfil.write(chunk)
   outfil.close()
   message = 'The file "' + fn + '" was uploaded successfully\n'
else:
   message = 'No file was uploaded'
print """\
Content-Type: text/html\n
<html><body>
<p>%s</p>
</body></html>
""" % (message,)


Sample Upload handler - PERL
----------------------------
The following is the full text of an EGX-compatible PERL script to
accept HTTP file uploads.

#!/usr/bin/perl -wT
use strict;
use CGI;
# set maximum allowable size to 5MB
$CGI::POST_MAX = 1024 * 5000;
my $safe_filename_characters = "a-zA-Z0-9_.-";
my $upload_dir = "/usr/local/www/data/uploads";
my $query = new CGI;
my $filename = $query->param('datafile1');
if ( !$filename )
{
  print $query->header ( );
  print "There was a problem uploading your file; try a smaller file.";
  exit;
}
my ( $name, $path, $extension ) = fileparse ( $filename, '\..*' );
$filename = $name . $extension;
$filename =~ tr/ /_/;
$filename =~ s/[^$safe_filename_characters]//g;
if ( $filename =~ /^([$safe_filename_characters]+)$/ )
{
  $filename = $1;
}
else
{
  die "Filename contains invalid characters";
}
open ( UPLOADFILE, ">", "$upload_dir/$filename" ) or die "$!";
binmode(UPLOADFILE);
my $upload_filehandle = $query->upload("datafile1");
while ( <$upload_filehandle> )
{
  print UPLOADFILE;
}
close UPLOADFILE;
print $query->header ( );
print <<END_HTML;
<html>
<body>
<p>The file $filename was uploaded successfully</p>
</body>
</html>
END_HTML


Microsoft Internet Information Services (IIS)
---------------------------------------------
IIS can most naturally use Active Server Pages (ASP) or ASP.NET
to upload files. Alternately, IIS can be configured to support CGI.
Unfortunately, IIS does not support CGI as installed "out of the box".
Then one may use PERL/Python/etc. CGI scripts as above.


Sample Upload handler - Microsoft Internet Information Services (IIS) w/ ASP.NET
--------------------------------------------------------------------------------

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.IO;

namespace EnergyDataReceiver
{
    public partial class filepost : System.Web.UI.Page
    {
        protected void Page_Load(object sender, EventArgs e)
        {
            String TempFileName;
            HttpFileCollection myFileCollection = Request.Files;

            for (int Loop1 = 0; Loop1 < myFileCollection.Count; Loop1++)
            {
                string fname = myFileCollection[Loop1].FileName;
                // Create a new file name.
                TempFileName = "C:\\TempFiles\\File_" + Loop1.ToString() + "_" + Path.GetFileName(fname);
                // Save the file.
                myFileCollection[Loop1].SaveAs(TempFileName);
            }
        }
    }
}


References
----------
1. RFC 1867 "Form-based File Upload in HTML",
    http://www.ietf.org/rfc/rfc1867.txt
2. File Upload with ASP.NET,
    http://www.codeproject.com/KB/aspnet/fileupload.aspx
3. Uploading a file to IIS using a browser,
    http://support.microsoft.com/default.aspx/kb/189651
4. Google Search of "IIS cgi" yields quite a few pointers and tutorials:
    http://www.google.com/search?q=IIS+cgi

Best regards,

Randi