Lookup Transformation
ORDER BY Override in Lookup Transformation
August 22, 2017
Start Informatica Services
Start up procedure for Informatica on LINUX/UNIX
September 14, 2017
Show all

What happens when the session/mapping is triggered?



When a PowerCenter session is triggered, Integration service start Data Transformation Manager (DTM), which is responsible to start reader thread, transformation thread and writer thread for the mapping.

  • Reader thread is responsible to read data from the sources.
  • Transformation threads process data according to the transformation logic designed in the mapping.
  • Finally, the writer thread connects to the target and loads the data.

So, if there is a situation where the Source Qualifier takes 1 hr to fetch 50 k rows, then it is considered as a source bottle neck. Becuase, the Transformatica threads and the Writer threads are waiting for the source thread to complete the task so that the other two threads can pick up the tasks.

Does this happen in sequence?

Yes, initially.

First reader reads the data, then passes to transformation and then in the end writer writes it to target. Let’s take the same example where we have 50k rows to be read from the source. About 1k records have been read and being processed, then the transformation thread will start ad will perform the logic as per the design. As and when the transformation completes it’s task, it moves the data for the writer threads. Now, this is a parallel process, only when the data fetch happens from the source.

And, during this process the bottlenecks at the transformation threads and the writer threads are also possible. Take a situation where the reader is reading the rows pretty faster, and the logic in the mapping is complex where the data has to move to 40 different transformations then there will be a transformation thread bottle neck.

Similarly, this can also occur for the target where the reader and the transform threads perform their jobs pretty fast and at the target load the writer takes a lot of time due to some other issue, then it will be a target bottleneck.