Monday, February 20, 2012

Preventing the loading of duplicate data to a table - Best Option

Start at the file level; sort it, and scrub it with a 3GL program
before you load it. I would also look into Sunopsis. This is an ELT
tool -- it uses native SQL tools to move data rather than adding yet
another ETL language on top of everything.Thanks, I appreciate the tip; however, the constraints of the project
require that this be done in SQL Server. I am interested to know if this
(Primary Key or Unique Constraint) is the best way to do this within the
restrictions that are in place.
*** Sent via Developersdex http://www.examnotes.net ***|||A PK or UNIQUE constraint is the obvious way to generate the exception
condition. It's probably useful also to report the invalid data so you
may want to load to a staging table, without the constraint, and then
write a query using HAVING COUNT(*)>1 to find the duplicate rows.
David Portas
SQL Server MVP
--

No comments:

Post a Comment