Advantages / Uses of ETL

- Data-warehouse population - ETL tool can be used to populate the dimensions & fact tables of a data-warehouse
- Data migration - ETL tool can be used to migrate or transfer the data between variety of sources like databases, CRMs, cloud, files, emails etc.
- Transferring huge data - ETL tools are used to benefit in the transfer rate as they benefit from different technologies like parallel processing, selective processing, etc.
- Data cleansing- ETL tool can be used to cleanse your data i.e. remove any inconsistent or ambiguous data  
- Source for reports & analysis - ETL readies the data for further representation in the data warehouse in the form of reports or analysis.
- Big data support - ETL tool supports many big data options such as Hadoop, etc.

Comments

  1. konatham prasadOctober 16, 2012 at 4:47 PM

    Hi Could you help me the below problem,

    I have created one ETL job in PDI 4.2.1 and the job contains the following objects
    1) table Input
    2) Insert/Update transformation
    3) dummy
    my problem is if i use the insert/update transformation it is taking long process but if i want update the existing record is there any alternative way

    ReplyDelete
    Replies
    1. Jugal DhrangadhariaOctober 19, 2012 at 3:28 PM

      there are many ways you can update ur row likw use the update step, write a sql in a step, etc. chk if tht helps

      Delete

Post a Comment

Popular posts from this blog

Increase Java Memory For Pentaho Data Integration

Pentaho runs inside a Java virtual machine, and hence is bound by the properties of that VM. These optimisations can apply to just about any Java application, including the Pentaho BI Server and GUI tools. Method to increase memory allocation: Open the file spoon.sh or spoon.bat in a text editor Look for a section that looks like this: # ****************************************************************** # ** Set java runtime options                                     ** # ** Change 256m to higher values in case you run out of memory.  ** # ****************************************************************** OPT="-Xmx256m -cp $CLASSPATH -Djava.library.path=$LIBPATH -DKETTLE_HOME=$KETTLE_HOME -DKETTLE_REPOSITORY=$KETTLE_REPOSITORY -DKETTLE_USER=$KETTLE_USER -DKETTLE_PASSWORD=$KETTLE_PASSWORD -DKETTLE_PLUGIN_PACKAGES=$KETTLE_PLUGIN_PACKAGES" Change the -Xmx parameter to alter the max...
Read more

Null pointer exception in tmap

You might have faced a common error in Talend i.e. the  NullPointerException in tMap "Exception in component tMap_1 java.lang.NullPointerException" Solutions: 1) Make sure your columns are nullable when you define the schema. 2) The NullPointerException indicate there are some null value on lookup table, so you need to deal with the null value on corresponding columns on expression filed of tmap, for example: !Relational.ISNULL(row13.abc)? row13.xyz:row13.sss:null
Read more

Different Match Models in tMap with example

Unique Match:     (Also called Last match) If several matches are found in the Inner Join, i.e. the last row matching the explicit join as well as the filter will be added to the output flow. First Match: If several matches are found in the Inner Join, i.e. the first row matching the explicit join as well as the filter will be added to the output flow. All Match: If several matches are found in the Inner Join, i.e. rows matching the explicit join as well as the filter, all of them will be added to the output flow Example: deptid   sales 1           1000 2           2000 3           3000 2           5000 3           6000 Outputs: 1. Unique match deptid   sales 1           1000 2           5000 3           6000 2. All matches deptid  ...
Read more