View Tables of ETL Import Database

DCIM_Support · ‎2020-07-05

Dear all,

After finished configuring ETL import database from DCO web Access, I wanted to see table structure of the database created for the ETL, but I could not find any tables listed on my pgAdmin. Please see the picture below:

I have tried to run the ETL job, but it returned the following error message "ETL Configuration with the specified ID can not be found".

Can anyone explain what I missed? Or It is still under development since the API for ETL configurations are still in "Internal" status.

Any responses will be highly appreciated.

Thank you.

DCIM_Support · ‎2020-07-05

Hello Bayu.

Can you confirm which version of DCO you are running?

Did you try to re-create the import database?

From the DCO web client, can you delete and recreate the import database?

Do you see an error when you create the import database?

If the error/condition continues we'd ask you if possible to share log files from the DCO server.

Regards

Greg Sterling

DCIM_Support · ‎2020-07-05

Hello Sterling,

Thank you very much for your response.

I am running DCO 8.2.12.

Yes, I was creating the import database but actually I have been trying so many time, all combinations, Export and Import database, Internal and external. No tables created, only the database name, especially for the Internal Import (already mentioned on my previous question). I can see the database names, as many as my trials, created in the ISXO database. Here are the pictures showing the databases name in the table etl_settings (in ISXO database):

and the databases listed but with no tables created (Tables = 0):

No errors shown when I was creating the import database using "Manage ETL Integrations" wizard in DCO web access, all run smoothly. But, after finished creating the ETL configuration for import database, when I wanted to run the job, it showed me the error ""ETL Configuration with the specified ID can not be found" (already mentioned as well on my question).

After creating the import database ETL Configuration, the last ETL Configuration just created in DCO Web Access just gone when we revisiting the "Manage ETL Integrations" menu.

The logs you request will be sent later soon.

Thank you very much. Looking forward to your reply.

DCIM_Support · ‎2020-07-05

Among the log files available, which one do you want me to share?

Here are tailed or probably related to ETL log chunk of the 3 of them:

etl.log

2019-07-19 14:18:14,939 INFO [71ho1SwFRc2yvHTenPsRpg] [a3ab747d76de] [org.pentaho.di] (default task-9 71ho1SwFRc2yvHTenPsRpg) RepositoriesMeta - Reading repositories XML file: FromInputStream

2019-07-19 14:18:15,181 INFO [71ho1SwFRc2yvHTenPsRpg] [a3ab747d76de] [org.pentaho.di] (default task-9 71ho1SwFRc2yvHTenPsRpg) RepositoriesMeta - Reading repositories XML file: FromInputStream

2019-07-23 06:00:49,003 INFO [wFXzNtZbQkWjkcCXAjGtag] [a3ab747d76de] [org.pentaho.di] (default task-3 wFXzNtZbQkWjkcCXAjGtag) RepositoriesMeta - Reading repositories XML file: FromInputStream

2019-07-23 06:00:49,145 INFO [wFXzNtZbQkWjkcCXAjGtag] [a3ab747d76de] [org.pentaho.di] (default task-3 wFXzNtZbQkWjkcCXAjGtag) RepositoriesMeta - Reading repositories XML file: FromInputStream

http_acces.log

Jul 19 09:53:18 localhost httpd_access: 192.168.77.99 - - [19/Jul/2019:09:53:18 +0200] "GET /web/api/webclient/v8.1/etl-configuration/driver HTTP/1.1" 200 160

Jul 19 09:53:18 localhost httpd_access: 192.168.77.99 - - [19/Jul/2019:09:53:18 +0200] "GET /web/api/webclient/v8.1/etl-configuration/dialect HTTP/1.1" 200 111

Jul 19 09:53:18 localhost httpd_access: 192.168.77.99 - - [19/Jul/2019:09:53:18 +0200] "GET /web/api/webclient/v8.1/etl-configuration/configuration HTTP/1.1" 200 2

server.log

2019-07-19 10:35:31,592 INFO [] [] [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0010: Unbound data source [java:jboss/datasources/PostgresDS]
2019-07-19 10:35:33,048 INFO [] [] [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0208: Stopped subdeployment (runtime-name: com.apc.external.server-8.2.12.OD-c0a100a9283af158f349eae2e864ad6235804c21.jar) in 10453ms
2019-07-19 10:35:33,048 INFO [] [] [org.jboss.as.server.deployment] (MSC service thread 1-2) WFLYSRV0208: Stopped subdeployment (runtime-name: com.apc.etl.server.synchronizers-8.2.12.OD-c0a100a9283af158f349eae2e864ad6235804c21.jar) in 10582ms
2019-07-19 10:35:33,049 INFO [] [] [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) WFLYJCA0019: Stopped Driver service with driver-name = postgres

Thanks.

DCIM_Support · ‎2020-07-05

If possible, can you share the server.log and etl.log files?

Regards

Greg Sterling

DCIM_Support · ‎2020-07-05

Please find the files on the following links:

I do hope you could find the solution by inspection the log.
Thank you.

Regards,

Bayu Raharjo

DCIM_Support · ‎2020-07-05

Hello Bayu

There are a number of errors in the server.log.

Is this an offline demo configuration or a copy of a demo configuration? There are a number of services which appear they may not be running and there are errors suggesting there has been trouble writing to the postgres database.

How much memory is assigned to this DCO server? It would appear its has approximately 8GB of memory. Its disk is not full or near full, correct?

When was the last time this server was restarted?

Assuming this is not an offline demo configuration, if you can allocate more memory to this configuration I would suggest doing so. Bringing it up to at least 10GB, recommended is 16GB in our system requirements docs.

Regards

Greg Sterling

DCIM_Support · ‎2020-07-05

Hello Greg Sterling,

Thank you for your reply.

Is this an offline demo configuration or a copy of a demo configuration? There are a number of services which appear they may not be running and there are errors suggesting there has been trouble writing to the postgres database.

Yes, DCO I use is a demo version. Some services are not running in a demo version? ETL doesn't work in Demo version?

We already reconfigure RAM to 10 and 16 GB and still ETL database is not created.

Thank you.

DCIM_Support · ‎2020-07-05

I tried the offline demo and had a similar experience. Seems like creating/managing ETL databases is not working. I will pass this on to dev for comment.

DCIM_Support · ‎2020-07-05

Ok. Clear now.
Perhaps, you may update on this thread if you got any comments from dev.

Many thanks.

DCIM_Support · ‎2020-07-05

Hello Bayu. I received confirmation the use of ETL is not supported with the offline demo, so it is not expected to work.

Regards

Greg Sterling

DCIM_Support · ‎2020-07-05

This question is closed for comments. You're welcome to start a new topic if you have further comments on this issue.