Standard MetaStore Element types

Owned by Former user (Deleted)
Last updated: Jun 27, 2016 by Former user (Deleted)Version comment
5 min read

This page is dedicated to highlighting agreed upon global MetaStore definitions within the Pentaho Suite.  The terms and definitions here will map 1 to 1 with how the Pentaho MetaStore API references shared element types across the suite.  

!!! DRAFT !!!

Database connection

Element type name: Database connection
Element type description: This is the official central database connection metadata
Element Attributes:

Attribute

ID

API Version

Type

Required

Description

Description

description

1.0

String

No

The description of the database connection

PDI Database
Plugin ID

plugin_id

1.0

String

No (1)

The PDI (Kettle) database plugin ID to use.
(1) In case no plugin ID is specified, the Driver Class attribute is required.

Driver Class name

driver_class

1.0

String

Yes (2)

The JDBC driver class to use.
(2) In case no driver class name is specified in a PDI environment, the Plugin ID attribute is required.

JDBC URL

jdbc_url

1.0

String

Yes (3)

The JDBC URL used to connect to a database.
(3) Can be derived in a PDI environment in which case it's not required.

Database Access Type

access_type

1.0

String

No

One of Native (JDBC), ODBC, OCI, JNDI.  Defaults to Native in case the attribute is empty.

Host name

hostname

1.0

String

No

(Name change upcoming in 1.1) The server host name or IP address to connect to

Port

port

1.0

String

No

The database connection port

Database name

database_name

1.0

String

No

The name of the database to use

User name

user_name

1.0

String

No

The user name to authenticate with

Password

password

1.0

String

No

The password to authenticate with
NOTE: optionally stored obfuscated

Servername

servername

1.0

String

No

(Name change upcoming in 1.1) Used by certain databases (SQL Server, Informix, ...) to specify which logical server name to use

Data tablespace

data_tablespace

1.0

String

No

The table space (Oracle) to use for storing tables

Index tablespace

index_tablespace

1.0

String

No

The table space (Oracle) to use for storing indices

Extra Attributes

attributes

1.0

Child attributes

No

This attribute can contain extra child elements.  PDI uses (among others) the following commonly used attributes.

  • SQL_CONNECT: The SQL to execute right after connecting to a database (to specify a license and so on)
  • USE_POOLING: Use connection pooling or not (Y/N)
  • STREAM_RESULTS: MySQL specific option to force cursor emulation (Y/N)
  • MSSQL_DOUBLE_DECIMAL_SEPARATOR: forces a double decimal (..) schema/table separator needed for certain environments.
  • QUOTE_ALL_FIELDS: quote all identifier fields
  • FORCE_IDENTIFIERS_TO_LOWERCASE: force all identifier fields to lowercase
  • FORCE_IDENTIFIERS_TO_UPPERCASE: force all identifier fields to uppercase
  • SUPPORTS_BOOLEAN_DATA_TYPE: PDI compatibility flag to prevent conversion of Boolean/Bit type fields to CHAR(1)
  • SUPPORTS_TIMESTAMP_DATA_TYPE: PDI compatibility flag to retain ns precision of timestamp fields
  • PRESERVE_RESERVED_WORD_CASE: Option to make PDI preserve the case of a reserved word.
    Note: Once all Pentaho tools support a certain optional attribute it will be promoted to a top-level attribute.  Until that time these are considered hints.

Splunk connection

Element type name: Splunk connection
Element type description: The necessary information needed to connect to a splunk server
Element Attributes:

Attribute

ID

Type

Required

Description

Host

host

1.0

String

Yes

The IP or Hostname of the Splunk Server to connect to

Port

port

1.0

Integer

No

The port to connect to, defaults to 8000

Username

username

1.0

String

Yes

The username to connect as

Password

password

1.0

String

Yes

The password to authenticate with.
NOTE: optionally stored obfuscated

Big Data Plugin - Hadoop Named Cluster

Element type name: NamedCluster
Element type description: Defines a Hadoop named cluster, these are used throughout the big-data-plugin's job and transformation entries.

Element type source file: https://raw.githubusercontent.com/pentaho/big-data-plugin/master/src/org/pentaho/di/core/namedcluster/model/NamedCluster.java

Element Attributes:

Attribute

ID

API Version

Type

Required

Description

Source File

Cluster
Name

name

1.0

String

Yes

The name of the Hadoop Cluster configuration

o.p.d.c.namedcluster.model.NamedCluster @MetaStoreAttribute

HDFS Hostname

hdfsHost

1.0

String

no

The HDFS hostname to use for the Hadoop Cluster Configuration, can be set as a Kettle Variable

o.p.d.c.namedcluster.model.NamedCluster 
@MetaStoreAttribute

HDFS Port

hdfsPort

1.0

String

no

The HDFS port to use for the Hadoop Cluster Configuration, can be set as a Kettle Variable

o.p.d.c.namedcluster.model.NamedCluster 
@MetaStoreAttribute

HDFS Username

hdfsUsername

1.0

String

no

The HDFS username to use for the Hadoop Cluster Configuration, can be set as a Kettle Variable

o.p.d.c.namedcluster.model.NamedCluster 
@MetaStoreAttribute

HDFS Password

hdfsPassword

1.0

String

no

The HDFS password to use for the Hadoop Cluster Configuration, can be set as a Kettle Variable

o.p.d.c.namedcluster.model.NamedCluster 
@MetaStoreAttribute

JobTracker Hostname

jobTrackerHost

1.0

String

no

The JobTracker hostname to use for the Hadoop Cluster Configuration, can be set as a Kettle Variable

o.p.d.c.namedcluster.model.NamedCluster 
@MetaStoreAttribute

JobTracker Port

jobTrackerPort

1.0

String

no

The JobTracker port to use for the Hadoop Cluster Configuration, can be set as a Kettle Variable

o.p.d.c.namedcluster.model.NamedCluster 
@MetaStoreAttribute

ZooKeeper Hostname

zooKeeperHost

1.0

String

no

The ZooKeeper hostname to use for the Hadoop Cluster Configuration, can be set as a Kettle Variable

o.p.d.c.namedcluster.model.NamedCluster 
@MetaStoreAttribute

ZooKeeper Port

zooKeeperPort

1.0

String

no

The ZooKeeper port to use for the Hadoop Cluster Configuration, can be set as a Kettle Variable

o.p.d.c.namedcluster.model.NamedCluster 
@MetaStoreAttribute

Oozie URL

oozieUrl

1.0

String

no

The Oozie URL to use for the Hadoop Cluster Configuration, can be set as a Kettle Variable

o.p.d.c.namedcluster.model.NamedCluster 
@MetaStoreAttribute

Model Annotation Group

Design Document: https://pentaho.box.com/s/pu3850oq7yztkh2supaz
Element type name: Model Annotation Group
Element type description: Define a group of related model annotations used in Annotate Stream Step
Element Attributes:

Attribute

ID

Type

Required

Description

Source

Name

name

String

Y

Name of the Group

org.pentaho.agilebi.modeler.models.annotations.ModelAnnotationGroup

Description

description

String

N

Description of the Group

org.pentaho.agilebi.modeler.models.annotations.ModelAnnotationGroup

Model Annotations

modelAnnotations

List<ModelAnnotation>

Y

List of Model Annotations

org.pentaho.agilebi.modeler.models.annotations.ModelAnnotationGroup

Shared Dimension Group

Design Document: https://pentaho.box.com/s/pu3850oq7yztkh2supaz
Element type name: Shared Dimension Group
Element type description: Define a group of related model annotations used in Shared Dimension Step
Element Attributes:

Attribute

ID

Type

Required

Description

Source

Name

name

String

Y

Name of the Group

org.pentaho.agilebi.modeler.models.annotations.SharedDimensionGroup

Description

description

String

N

Description of the Group

org.pentaho.agilebi.modeler.models.annotations.SharedDimensionGroup

Data Providers

dataProviders

List<DataProvider>

Y

List of Data Providers of the mapped columns

org.pentaho.agilebi.modeler.models.annotations.SharedDimensionGroup

Model Annotations

modelAnnotations

List<ModelAnnotation>

Y

List of Model Annotations

org.pentaho.agilebi.modeler.models.annotations.SharedDimensionGroup

Model Annotation

Design Document: https://pentaho.box.com/s/pu3850oq7yztkh2supaz
Element type name: Model Annotation
Element type description: Define a set of properties used to augment/modify auto-generated model
Element Attributes:

Attribute

ID

Type

Required

Description

Source

Name

name

String

Y

Name of the annotation

org.pentaho.agilebi.modeler.models.annotations.ModelAnnotation

Field

field

String

Y

Name of the mapped field

org.pentaho.agilebi.modeler.models.annotations.ModelAnnotation

Cube

cube

String

N

Name of the associated cube

org.pentaho.agilebi.modeler.models.annotations.ModelAnnotation

Annotation

annotation

AnnotationType

Y

Annotation type implementation

org.pentaho.agilebi.modeler.models.annotations.ModelAnnotation