Tuesday, March 27, 2012

customized error output?

I am using SSIS to load a lot of Excel, csv files. Some of the files will fail for various formating/validation reason. Is it a good way to capture the error and generate a nice error report so the provider can read it easily and correct the data files?

The error log of the package is difficult to read.

I have one suggestion:

1. instead of failing the component whenever there is an error during DataConversion on Lookup etc, redirect that row and then do a multicast.

2. Now you have two identical sets.

3.As you are aware each column has a lineage ID now generate a collection of column names and Lineage ID.

4. Perform an Inner join with the other replica on Lineage ID.

5. Retrieve the Erroneous Column name, Value and Row number.

6. Finally once you are done with all transformations, join the input obtained as mentioned above, on Rownumber and write out records as Erroneous and Non-Erroneous Data.

No comments:

Post a Comment