In today's corporate world, data is spread across multiple data sources (ex: DB2, Oracle, Sybase etc) and business users wish to generate types of business reports from a single source, no matter how the data is distributed. This scenario calls for data integration from multiple data sources and demands for new pertinent technical challenges:
a) Data sources with different formats and organized in structured, semi-structured, and unstructured data.
b) Huge data volumes and data populated from source systems arriving at different times.
c) Data quality and transforming the data into a format that is meaningful to business analysts.
Microsoft SQL Server Integration Services (SSIS) package enables developers to address the business users need. However testing of these packages introduces new challenges in testing. This scenario calls for the following needs:a) How do we validate complex sql queries?b) How do we ensure data workflow validation?c) How do we validate multiple data validation checkpoints?d) How do we ensure data correctness between the source and target, with millions of records?e) How do we approach continuous testing?
key features of this tool is:
a) Built-in intelligence, which executes relevant test actions based on the user XML tags
b) COM based tasks and component execution , helps to execute tests in a non-UI environment
c) Listeners to handle extraction of huge amount of data
d) Data comparsion API's, to compare billions of records
e) Multiple execution checkpoint support, which helps in enabling checkpoints in packages, databases, logs etc