Perfomance tuning : Tmap Component in Talend

The settings mostly depend on the volume of data you require for processing while applying a join, i.e., 

 If Lookup table have few rows:

  • select "Lookup model" as Load Once . This will load lookup data into memory only once and uses that to compare all other records from parent table. 
  • This is a faster option as memory loading is less

IF lookup table have large amount of data:
Talend cant store the lookup table data in memory. you will get java heap space Execption. To resolve this issue follow the below steps

  • open tmap
  • go to lookup table
  • click on tmap settings
  • select the value for store temp data property to True 
  • click on ok
  • In Tmap properties basic settings  set Temp Data Directory path by browsing folder
  • Go to Advance settings
  • set  max buffer size(nb of row) to some value based on  lookup condition data type

Comments

  1. UnknownJuly 19, 2016 at 4:52 PM

    This is an awesome post.Really very informative and creative contents. These concept is a good way to enhance the knowledge.I like it and help me to development very well.Thank you for this brief explanation and very nice information.Well, got a good knowledge.

    ccna training in chennai Tnagar

    ReplyDelete
  2. gracylaylaApril 18, 2018 at 5:53 PM

    Hello,
    Very nice post here thanks for it. Know More About Talend Self Paced Training

    ReplyDelete
  3. MounikaNovember 27, 2018 at 12:02 PM

    Thanks for your informative article, Your post helped me to understand the future and career prospects & Keep on updating your blog with such awesome article.
    python training institute in marathahalli
    python training institute in btm
    Python training course in Chennai

    ReplyDelete

Post a Comment

Popular posts from this blog

Increase Java Memory For Pentaho Data Integration

Pentaho runs inside a Java virtual machine, and hence is bound by the properties of that VM. These optimisations can apply to just about any Java application, including the Pentaho BI Server and GUI tools. Method to increase memory allocation: Open the file spoon.sh or spoon.bat in a text editor Look for a section that looks like this: # ****************************************************************** # ** Set java runtime options                                     ** # ** Change 256m to higher values in case you run out of memory.  ** # ****************************************************************** OPT="-Xmx256m -cp $CLASSPATH -Djava.library.path=$LIBPATH -DKETTLE_HOME=$KETTLE_HOME -DKETTLE_REPOSITORY=$KETTLE_REPOSITORY -DKETTLE_USER=$KETTLE_USER -DKETTLE_PASSWORD=$KETTLE_PASSWORD -DKETTLE_PLUGIN_PACKAGES=$KETTLE_PLUGIN_PACKAGES" Change the -Xmx parameter to alter the max...
Read more

Null pointer exception in tmap

You might have faced a common error in Talend i.e. the  NullPointerException in tMap "Exception in component tMap_1 java.lang.NullPointerException" Solutions: 1) Make sure your columns are nullable when you define the schema. 2) The NullPointerException indicate there are some null value on lookup table, so you need to deal with the null value on corresponding columns on expression filed of tmap, for example: !Relational.ISNULL(row13.abc)? row13.xyz:row13.sss:null
Read more

Different Match Models in tMap with example

Unique Match:     (Also called Last match) If several matches are found in the Inner Join, i.e. the last row matching the explicit join as well as the filter will be added to the output flow. First Match: If several matches are found in the Inner Join, i.e. the first row matching the explicit join as well as the filter will be added to the output flow. All Match: If several matches are found in the Inner Join, i.e. rows matching the explicit join as well as the filter, all of them will be added to the output flow Example: deptid   sales 1           1000 2           2000 3           3000 2           5000 3           6000 Outputs: 1. Unique match deptid   sales 1           1000 2           5000 3           6000 2. All matches deptid  ...
Read more