OSGI in Kettle

Owned by Former user (Deleted)
Last updated: Jul 23, 2014Version comment
3 min read

Introduction to OSGI in Kettle

The vision of OSGI in Kettle is to eliminate the current PluginRegistry with it's mix of xml-described built-in plugins and annotation/xml-based jar plugins, in favor of an industry standard infrastructure with as little custom code as possible.

While there has been broad acceptance of OSGI as the technology behind this effort, this is not an adoption of OSGI as an integral part of the software. There exists very limited use of OSGI classes and all instances of such are interfaced off such that they may be backed by something else down the road.

By taking the Blueprint Container approach the plugins themselves have no OSGI classes and contain only an Bean descriptor that OSGI consumes. If Kettle itself were to become a series of OSGI bundles all mention of OSGI could be replaced with the Dependency Injection provided by the Blueprint Container (Spring DM). In essence the plugin system would be invisible and acting on the system from the side. This is out of scope of the current effort as it would require extensive refactoring of Kettle to adapt it to Dependency Injection.

Basics of working with PluginTracker

The new plugin layer, PluginTracker, is very similar to the current PluginRegistry in use in Kettle today. This was consciously done in order to smooth migration.

Registering a Plugin Class

Plugin Classes are OSGI services the same as any other. However registering one with the PluginTracker will tell the plugin system to actively monitor all services of this type and notify any PluginLifecycleListeners when a service is added, removed or modified.

Similar to PluginTypes in Kettle today these should be used more as descriptors than functional units. 

PluginTracker.getInstance().registerPluginClass(Class);

You can track add/remove/modify events associated with services of this type by registering a PluginLifecycleListener.

PluginTracker.getInstance().addPluginLifecycleListener(Class, PluginLifecycleListener)

Using the information contained within your plugin type you should be able to find references to the Services and Beans that you need in order to make use of your plugin in Kettle.

Note: this programing model is purposefully close to the existing Kettle system, but you're not limited to what you can do with these PluginTypes.
They can contain direct references to services and beans as they're standard OSGI services.

Looking-up Services

Services are far and away preferred over Beans if at all possible, but there is one terrible weekness. Only one Service is created for the given service props with the highest ranked service matching the props winning. That is if one class asks for a Service and another asks for the same service with identical properties, they'll both have a reference to the same instance. One last thing to note: Services should be stateless as they can come an go. 

PluginTracker.getInstance().getServiceObject(Class serviceClassType, Map serviceProps, [optional] boolean doProxy);

Services are proxied by default and when proxied have a dynamic lifecycle. A bundle backing a service may be stopped, restarted
or replaced by another bundle. The Proxy will automatically resolve a new implementation, but all state held in the original would be lost. Because of this, you may wish to track these changes by adding a ServiceReferenceListener:

PluginTracker.getInstance().addServiceReferenceListener(Object serviceObject, ServiceReferenceListener listener);
Getting bean instances from your plugin.

Beans are either single-instance or prototyped classes from your plugin's bundle. Beans are not proxied and will persist even after their parent Bundle has stopped. This provides built-in factory support to your plugin without having to register numerous XXXFactory as a Services. This is both a strength and a weekness. The best practice is that a bean should implement interfaces from non-transient bundles. Ideally bundles should be split with impls separate which takes care of this issue.

As of now there are two ways to get a bean. In one you must provide a map of properties used to locate the BeanFactory in your Bundle ("pluginID" => "myPlugin", etc). In the other you pass-in an instance of service from the Bundle you wish to get bean instances from. This last one could be considered problematic as another bundle may have registered a service overriding the original, but their bean container wouldn't be guaranteed hold the bean you're looking for.

PluginTracker.getInstance().getBean(Class beanType, Map beanFactoryProps, String beanID);

OR

PluginTracker.getInstance().getBean(Class beanType, Object serviceObject, String beanID);

Known issues / areas needing work

  • Loading a transformation with an OSGI provided step is not working. That loading code needs to look in the PluginTracker.
  • Custom bootstrap code and directory watching needs to be replaced with Apache Karaf (used to be ServiceMix).
  • Unit Tests
  • Solve the built-in plugin issue. Karaf allows you to register Blueprint bean docs standalone that operate with the Host classloader (Fragment Bundles). We need this.
  • So far only steps has seen and modification to run on the new system. All of the other existing plugin areas need the same treatment (DatabaseMeta, Jobs, LifecycleListers, SpoonPlugins, RegistryPlugins, etc.)