Category Archives: ETL

Column Change Auditing with SSIS

By dslattery | December 22, 2011

I was recently asked by a client to write an SSIS package to update some data in a database.  One of the requirements was to audit all changes with before and after column values, update user and update date. The client requested that CDC (Change Data Capture) NOT be implemented. And not being very proficient [...]

Posted in ETL | 1 Comment

Partitioned Fact Tables

By achandler | December 4, 2008

Once tables grow into the millions of records, they become candidates for partitioning. Table partitioning offers many benefits, particularly in warehouse environments. Since data is split into smaller units of storage, backups can target filegroups with a higher rate of change. Systems with multiple CPUs see improved query performance as partitioned data leads to greater [...]

Also posted in Warehousing | Leave a comment

Introduction to SAP Business Objects Data Integrator ETL tool

By sfernandes | November 6, 2008

SAP Business  Objects Data Integrator XI R2/R3
The purpose of this presentation is to familiarize the team with some of the features of Business Objects Data Integrator (DI). The presentation is also aimed at providing some insight into how technology can affect the design and implementation of  the ETL processes’.Â
While at a high level it [...]

Posted in ETL | 3 Comments

SSIS data management fundamentals

By elizotte | October 22, 2008

Loading data into a warehouse for use in business intelligence often involves loading data from multiple dissimilar sources. Many times the easiest way to consistently manage the data involves doing extracts from the source systems into a simple text format such as comma delimited. Even legacy systems have some form of reporting engines that allow [...]

Posted in ETL | 1 Comment

ETL Process Control Architecture

By mfortuna | October 20, 2008

Data Volumes are growing at incredible rates across many IT departments. Businesses are dealing with new problems every day, and there is much demand on IT to be flexible and to react to many issues facing the business on a daily basis. Many times, processes become obsolete or disfunct because they cannot keep up with [...]

Posted in ETL | Leave a comment

SCD via SQL Stored Procedure

By srichards | September 8, 2008

We are going to revisit the issue of dealing with Slowly Changing Dimensions in a data warehouse. We have seen a demonstration of using the SCD transformation that is available in SQL Server Integration Services (SSIS); however, this is not always the best option for processing an SCD. When the volume of rows you’re dealing [...]

Also posted in Warehousing | 6 Comments

The BCP Option

By dstrickland | August 28, 2008

Moving data from one repository to the next may sound like a trivial task, and in the sense of the normal mechanical approach it very well may be.  What could be simpler than INSERT INTO Here SELECT * FROM There?  Not much, it would seem, until your data source is something other than a SQL [...]

Posted in ETL | Leave a comment

The SCD Transformation in SSIS

By pross | August 14, 2008

In this post, we will demonstrate the Slowly Changing Dimension (SCD) transformation that is available for use in a Data Flow task within SQL Server Integration Services (SSIS).  We’ll walk through an example of how to use it, explaining the terms as we go. In addition, I’ll demonstrate some “extra effort” kind of modifications that can be [...]

Also posted in Warehousing | Leave a comment