Introduction

Originally created in version 2.0, the Kettle plugin system was designed with the KISS principle in mind.  At the core of it, it's a simple Java URL class loader appropriately named "KettleURLClassLoader".  It was used to load Step and Job entry plug-ins. 

In version 3.0, support for Java annotations were added as well as support for partition plugins.

In version 4 of Kettle we wanted to add Repository and Database plugins as well as plan for Transformation and job plugins. We also want to add support for loading annotation plug-ins from single independend jar files stored anywhere we like.  That would make configuration of plugins a lot easier.  You write a set of plugins of various types, put those in a jar file and you put the jar file in the "plugins" folder.  End of story.  Obviously, having 7 different class loaders around that all do almost the same thing was not an option so we decided to rewrite the plug-in system a bit.

This resulted in the creation of the Kettle 4 Plugin Registry.

PluginRegistry

 The plugin registry is the new one-stop-shop you can use to deal with all the aspects of handling any type of plugin.  For this, a number of new interfaces and classes were created:

• PluginTypeInterface: this describes the type of plugin that is being handled.  For example, StepPluginType implements the interface.   It not only has an ID and a name, it also knows how to search and register plugins of this particular type (searchPlugins()).  Finally, it knows the natural order of the categories in a user interface (getNaturalCategoriesOrder()).
• PluginInterface: this interface gives us all the details we need about a certain plugin, the IDs, the name, the description, needed libraries, cateogory and obviously the classes it represents.
• PluginClassType: This is a simple enumeration that describes which type of class we're dealing with.  Usually, the plugins only define one main class type (MainClassType), but it is quite possible to have others like in the case of the Repository plugin.

Migration of code from 3.x

StepLoader

The step loader singleton was used everywhere we needed to load a StepInterface or if we wanted to learn more about a certain step.

As such, this class was used mostly in Spoon, in StepMeta and in the Repository plugins.   Here are the main changes to make:

• StepLoader.findStepPluginWithID(id) --> PluginRegistry.findPluginWithId(StepPluginType.class, id)
• StepPlugin --> PluginInterface
• StepLoader.getStepClass(StepPlugin) --> (StepMetaInterface)PluginRegistry.loadClass(PluginInterface, PluginClassType.MainClassType)
• StepLoader.getStepPluginID(StepMetaInterface) --> PluginRegistry.getPluginId(StepPluginType..class, StepMetaInterface)
• init() --> taken over by PluginRegistry.init() and StepPluginType.searchPlugins()
• getPluginPackages() --> is now available as getPluginPackages(PluginTypeInterface) returning a list of package names.
• getPluginInformation() --> getPluginInformation(PluginTypeInterface) returning a RowBuffer object

The StepPlugin class and the PluginInterface are quite similar, you'll find your way around.  We took the opportunity to correct a few mistakes from in that class though:

• getID(): since this wasn't a single ID, it got renamed to getIds() but it's an array of String as well.
• getDescription(): this was actually the name of the step so it's now called: getName()
• getTooltip(): this was actually the description of the step so it's now called: getDescription()
• getJarfiles(): moved to a list: getLibraries()
• getClassname(): move to a map: getClassMap()
• getIconFilename(): renamed to: getImageFile();

JobEntryLoader

The job entry loader singleton was used everywhere we needed to load a JobEntryInterface or if we wanted to learn more about a certain step.  Unfortunately a certain developer made a mistake in the naming of a few things.  As such, here are the changes in the objects:

• @Job --> @JobEntry: Since we plan to have Job plugins in the future, the appropriate name for a job entry annotation is .. euh.. @JobEntry.
• Since the Annotation changed anyway we also renamed the members of the JobEntry annotation:
  • name() --> id()
  • description() --> name()
  • tooltip() --> description()
• This better reflects the actual use of the fields (to put it mildly)

The JobEntryLoader class was used mostly in Spoon, JobEntryCopy and in the Repository plugins.   Here are the main changes to make:

• JobEntryLoader.findJobEntriesWithDescription(String) --> PluginRegistry.findPluginWithName(StepPluginType..class, description)
• JobPlugin (also with a wrong name!) --> PluginInterface
• JobEntryLoader.getStepClass(StepPlugin) --> (JobEntryInterface)PluginRegistry.loadClass(PluginInterface, PluginClassType.MainClassType)
• JobEntryLoader.getJobEntryPluginID(JobEntryInterface) --> PluginRegistry.getPluginId(JobEntryPluginType.getInstance(), JobEntryInterface)
• init(): taken over by PluginRegistry.init() and JobEntryPluginType.searchPlugins()
• getPluginPackages(): is now available as getPluginPackages(PluginTypeInterface) returning a list of package names.
• getPluginInformation(): getPluginInformation(PluginTypeInterface) returning a RowBuffer object

The JobPlugin class and the PluginInterface are quite similar, see the StepLoader paragraph above for more information.

DatabaseMeta

Since the creation of DatabaseMeta is now handled by the PluginRegistry as well, we can say a few words about the migration of your code.

Here are a few of the methods that are now missing from DatabaseMeta:

• getDatabaseTypeDescLong() --> You need to look up the PluginInterface for this database in the plugin registry with the pluginId.
  PluginInterface plugin = registry.getPlugin(DatabasePluginType.class, pluginId); 
  With this you can determine the name (previous long description)
• getDatabaseTypeDesc() --> This is in fact the plugin ID of the DatabaseMeta object.

Examples from the field

Job entry example

JobPlugin plugin = JobEntryLoader.getInstance().findJobPluginWithID(typeId);

 becomes:

PluginInterface plugin = PluginRegistry.getInstance().getPlugin(JobEntryPluginType..class, typeId);

Step plugin example

 stepMetaInterface = StepLoader.getInstance().getStepClass(sp);

becomes

stepMetaInterface = (StepMetaInterface)PluginRegistry.getInstance().loadClass(sp);

Pentaho Examples

StepLoader steploader = StepLoader.getInstance();
String fromstepid = steploader.getStepPluginID(tii);

becomes

PluginRegistry registry = PluginRegistry.getInstance();
String fromstepid = registry.getPluginId(tii);

repository = (Repository) RepositoryLoader.createRepository(repositoryMeta);

becomes

repository = (Repository)PluginRegistry.getInstance().loadClass(RepositoryPluginType..class, repositoryMeta.getId(), PluginClassType.MainClassType);
repository.init(repositoryMeta);