ETL export not working in 8.2.7 (Merging)

[DCIM_Support]

This question was originally posted on DCIM Support by DataCenterServices on 2019-01-28

---

After upgrading to 8.2.7 my ETL export database is no longer functioning.

The initial ETL export was successful. The error only appears when merging/updating data. The below is the error received:

2019-01-28 10:00:02,769 ERROR [0CGZi4oBRhy0fy8dpYeBAQ] [datacenterservices] [org.pentaho.di] (swoAsset_v2.0.0 - Merge) Merge - Unexpected error
2019-01-28 10:00:02,769 ERROR [0CGZi4oBRhy0fy8dpYeBAQ] [datacenterservices] [org.pentaho.di] (swoAsset_v2.0.0 - Merge) Merge - org.pentaho.di.core.exception.KettleException:
Invalid layout detected in input streams, keys and values to merge have to be of identical structure and be in the same place in the rows

The data type of field #16 is not the same as the first row received: you're mixing rows with different layout. Field [ci_max_power_draw Number(15)] does not have the same data type as field [ci_max_power_draw BigNumber(17, 17)].

at org.pentaho.di.trans.steps.mergerows.MergeRows.processRow(MergeRows.java:92)
at org.pentaho.di.trans.step.RunThread.run(RunThread.java:60)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.pentaho.di.core.exception.KettleRowException:
The data type of field #16 is not the same as the first row received: you're mixing rows with different layout. Field [ci_max_power_draw Number(15)] does not have the same data type as field [ci_max_power_draw BigNumber(17, 17)].

at org.pentaho.di.trans.step.BaseStep.safeModeChecking(BaseStep.java:1861)
at org.pentaho.di.trans.steps.mergerows.MergeRows.checkInputLayoutValid(MergeRows.java:265)
at org.pentaho.di.trans.steps.mergerows.MergeRows.processRow(MergeRows.java:88)
... 2 more

2019-01-28 10:00:02,769 INFO [0CGZi4oBRhy0fy8dpYeBAQ] [datacenterservices] [org.pentaho.di] (swoAsset_v2.0.0 - Merge) Merge - Finished processing (I=0, O=0, R=2, W=0, U=0, E=1)
2019-01-28 10:00:02,769 INFO [0CGZi4oBRhy0fy8dpYeBAQ] [datacenterservices] [org.pentaho.di] (swoAsset_v2.0.0 - Merge) swoAsset_v2.0.0 - swoAsset_v2.0.0
2019-01-28 10:00:02,769 INFO [0CGZi4oBRhy0fy8dpYeBAQ] [datacenterservices] [org.pentaho.di] (swoAsset_v2.0.0 - Merge) swoAsset_v2.0.0 - swoAsset_v2.0.0
2019-01-28 10:00:02,769 ERROR [0CGZi4oBRhy0fy8dpYeBAQ] [datacenterservices] [org.pentaho.di] (QuartzSharedThreadPool-1 M/Mur0csSGqQjTGc1EWLlw) swoAsset_v2.0.0 - Errors detected!
2019-01-28 10:00:02,770 ERROR [0CGZi4oBRhy0fy8dpYeBAQ] [datacenterservices] [org.pentaho.di] (QuartzSharedThreadPool-1 M/Mur0csSGqQjTGc1EWLlw) swoAsset_v2.0.0 - Errors detected!

DCO updated from 8.0.2 to 8.2.7.

Database Driver:  net.sourceforge.jtds.jdbc.Driver

Dialect: org.hibernate.dialect.PostgreSQL94Dialect / SQLServerUnicodeDialect

DB Destination: MS SQL Server 2017

(CID:138810569)

[DCIM_Support]

This answer was originally posted on DCIM Support by Ed Tarento on 2019-01-29

---

I usually see advice to delete the External system, then delete the export DB, then recreate export DB and then the  external system.  That has solved many issue for me in the past.

Cheers

Ed

(CID:138810660)

[DCIM_Support]

This comment was originally posted on DCIM Support by Greg Sterling on 2019-01-29

---

Are you by chance exporting to an external ETL database, perhaps a MS-SQL version by chance?

Regards

Greg Sterling

(CID:138810679)

[DCIM_Support]

This comment was originally posted on DCIM Support by Greg Sterling on 2019-01-29

---

Yes, sorry. I see you mentioned its an external ETL ms-sql database. There may be a transformation update which can be provided to resolve this issue. I have seen this at least one other customer. I'll ask dev if there's a file which can be shared.

A fix for this should also be in the latest DCO release, 8.2.12.

Regards

Greg Sterling

(CID:138810680)

[DCIM_Support · Accepted solution]

This answer was originally posted on DCIM Support by Jef Faridi on 2019-01-29

---

Hi (DataCenterServices),

This seems to be a known issue, where an ETL export to SQL server may fail after the second transformation.

This issue is fixed in the latest version of DCO (currently 8.2.12),

Kind regards

(CID:138810772)

[DCIM_Support]

This comment was originally posted on DCIM Support by DataCenterServices on 2019-01-30

---

Thanks for the input. Upgrading DCO from 8.2.7 to 8.2.12 resolved the ETL issue. 

Best

(CID:138811665)

[DCIM_Support]

This comment was originally posted on DCIM Support by Jef Faridi on 2019-01-30

---

Sounds great! many thanks for the update.

Kind regards

(CID:138811667)

[DCIM_Support]

This question is closed for comments. You're welcome to start a new topic if you have further comments on this issue.