What happens when the session/mapping is triggered?

Lookup Transformation
ORDER BY Override in Lookup Transformation
August 22, 2017
Start Informatica Services
Start up procedure for Informatica on LINUX/UNIX
September 14, 2017
Show all

What happens when the session/mapping is triggered?

Threads

Threads

When a PowerCenter session is triggered, Integration service start Data Transformation Manager (DTM), which is responsible to start reader thread, transformation thread and writer thread for the mapping.

  • Reader thread is responsible to read data from the sources.
  • Transformation threads process data according to the transformation logic designed in the mapping.
  • Finally, the writer thread connects to the target and loads the data.

So, if there is a situation where the Source Qualifier takes 1 hr to fetch 50 k rows, then it is considered as a source bottle neck. Becuase, the Transformatica threads and the Writer threads are waiting for the source thread to complete the task so that the other two threads can pick up the tasks.

Does this happen in sequence?

Yes, initially.

First reader reads the data, then passes to transformation and then in the end writer writes it to target. Let’s take the same example where we have 50k rows to be read from the source. About 1k records have been read and being processed, then the transformation thread will start ad will perform the logic as per the design. As and when the transformation completes it’s task, it moves the data for the writer threads. Now, this is a parallel process, only when the data fetch happens from the source.

And, during this process the bottlenecks at the transformation threads and the writer threads are also possible. Take a situation where the reader is reading the rows pretty faster, and the logic in the mapping is complex where the data has to move to 40 different transformations then there will be a transformation thread bottle neck.

Similarly, this can also occur for the target where the reader and the transform threads perform their jobs pretty fast and at the target load the writer takes a lot of time due to some other issue, then it will be a target bottleneck.

Sid
Sid
Business Intelligence Consultant and Trainer with 13+ years of extensive work experience on various client engagements. Delivered many large data warehousing projects and trained numerous professionals on business intelligence technologies. Extensively worked on all facets of data warehousing including requirement gathering, gap analysis, database design, data integration, data modeling, enterprise reporting, data analytics, data quality, data visualization, OLAP. Has worked on broad range of business verticals and hold exceptional expertise on various ETL tools like Informatica Powercenter, SSIS, ODI and IDQ, Data Virtualization, DVO, MDM.

Leave a Reply

Your email address will not be published. Required fields are marked *

Share
+1
Tweet
Pin
Share