The error log of the package is difficult to read.
I have one suggestion:
1. instead of failing the component whenever there is an error during DataConversion on Lookup etc, redirect that row and then do a multicast.
2. Now you have two identical sets.
3.As you are aware each column has a lineage ID now generate a collection of column names and Lineage ID.
4. Perform an Inner join with the other replica on Lineage ID.
5. Retrieve the Erroneous Column name, Value and Row number.
6. Finally once you are done with all transformations, join the input obtained as mentioned above, on Rownumber and write out records as Erroneous and Non-Erroneous Data.
No comments:
Post a Comment