...
- The step uses a lot of CPU and you have multiple processor cores in your computer. Example: a JavaScript step
- The step has to deal with network latencies and launching multiple copies can reduce the average latency. If you have a low network latency of say 5ms and you need to do a round trip to the database, the maximum performance you get is 200 (x5) rows per second, even if the database is running smoothly. You can try to reduce the round trips with caching, but if not, you can try to run multiple copies. Example: a database lookup or table output
- Please consider the drawback of running multiple steps in parallel, since they consume more CPU threads.
Manage thread priorities
In versions 3.0.2 and higher, this feature that is found in the "Transformation Settings" dialog (Misc tab) improves performance by reducing the locking overhead in certain situations. This feature is enabled by default for new transformations that are created in recent versions, but for older transformations this can be different.
...
Always think of the whole environment that influences your performance. There can be limitating factors within the transformation but also in the outside world or Pentaho Data Integration. You can play with the optimal commit size for instance, but this is depending on many factors and we can only give you some background and encourage you to test with. It is depending at least on the following: your database, your tables, indexes, the JDBC driver, your hardware, speed of the LAN connection to the database, the row size of data and your transformation itself.
You can try with different commit sizes and also change the number of rows in row sets in your transformation settings or even the buffers in your JDBC drivers or database.
In a lot of cases a smaller row set size actually improves performance since it forces rows through all steps of a (parallel executing) transformation.
Step Performance Monitoring
...