Home > SSIS > Data cleansing with SSIS

Data cleansing with SSIS


The data warehouse is the place where enterprise data reside for analytical and reporting purpose. It helps to keep the business active and grow with the current market competition. At this point, we realized that the data in the data warehouse should be accurate and meaningful. If we read data warehousing (DWH) concept these are the main characteristics which make data storage to warehouse. Those are subject oriented, Time variant, Unchangeable and Integrated data from various source.

Integrating data from various sources leads one to data quality issue. Because, Each ODS source has designed for the specific use. So, data coming from different sources need to be cleaned before going to DWH.

In typical data cleaning process we write script to clean data in the staging tables and make it consistent to store into DWH. There are different overheads while writing SQL query. We have to change the query that again gives lot of time consuming effort and human resources.

Microsoft Integration Services helps to design a easy use ETL process. It has lot of task that can help you to connect to various data sources and transformation tasks to purify the data and load it to the right destination with the help of Destination tasks.

All cleansing task are part of Data flow – data transformation tasks such as Data profiler, Fuzzy group, Fuzzy lookup, Lookup task, etc.,

I will be discussing each task in detail in my next post Data Cleansing with SSIS – Data Profiler.

Thanks for reading this far.

Please do write your comments about this post to make it better next time. You support is most valuable.

– Ayyappan Thangaraj

Advertisements
Categories: SSIS Tags:
  1. No comments yet.
  1. February 2, 2015 at 5:38 am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: