Configuring WSO2 product with new single TOML configuration model

dhanushka madushan's photo
dhanushka madushan
·

9 min read

Product configuration is an important part of using any software application. It is bad experience to end-user when a product has too many different configurations to configure. It's become worst when there are multiple files need to be configured and those files are in different formats. In the WSO2 product pack, there are many configuration files with different formats(XML and properties). Single TOML based configuration gives you a way to define all of these configurations in a single TOML file.

TOML file format has been selected as the configuration file format since its highly readable. TOML file format is easy to read due to its obvious semantics. It looks the same as the property file. But it can be used to define more complex data structures.

How it works internally?

Internally product read configurations same as previous. When the server starting up it checks whether deployment.toml file exists inside the CARBON_HOME/conf/ directory. If it does exist, it parses deployement.toml configurations. Parse configuration then goes through a set of phases and these configurations might be changed in these phases. Finally, these configurations generate actual configuration files according to the set of template files that is defined in CARBON_HOME/repository/resources/conf/templates/ folder. There newly generated file will be replaced with the original file.

When the server starting up it reads configuration the same as the previous way. The only difference is that when the server starting up it replaces existing configurations with the configurations provided by the deployment.toml file.

Each time the server getting started, it checks whether deployment.toml file changed or not. If it changed, deployment.toml configurations will be applied upon existing configuration. You can avoid applying configuration by either deleting deployement.toml file or using -DavoidConfigUpdate option in server startup.

TOML configuration format

TOML configuration format has been selected due to high readability. TOML format does support to define from a simple string to complex arrays. TOML format looks the same as the property file format. It does have sections. Each section has a section header and section content. Section headers defined by using a single bracket("[]"). For example, you can define a server hostname as follows.

[server]
hostname = "localhost"

The left-hand side is the key and the right-hand side is the value as a string. TOML format supports the following value formats.

  • String
    hostname = "localhost"
  • Integer
    max_worker_pool_size=500
  • Float
    flt2 = 3.1415
  • Boolean
    listener_enable = true
  • Date time
    odt1 = 1979-05-27T07:32:00Z

  • Arrays

    target_hosts = ["example.com", ".*.sample.com"]

  • Table ```toml

[server] hostname = "localhost" http_content_negotiation = true

[transport.passthru.http.listener] port=8280


- Array of Tables
```toml
[[custom_message_formatters.blocking]]
content_type = "application/json/badgerfish"
class = "org.apache.axis2.json.JSONBadgerfishMessageFormatter"

[[custom_message_formatters.blocking]]
content_type = "text/xml"
class = "org.apache.axis2.transport.http.SOAPMessageFormatter"

If you open up deployment.toml file from fresh WSO2 EI pack, It looks like follows.

[server]
hostname = "localhost"

[user_store]
type = "read_only_ldap"

[keystore.tls]
file_name = "wso2carbon.jks"
password = "wso2carbon"
alias = "wso2carbon"
key_password = "wso2carbon"

[truststore]
file_name = "client-truststore.jks"
password = "wso2carbon"
alias = "symmetric.key.value"
algorithm = "AES"

These configurations are must configured properties(For production usage. You can use default configs for local dev environment). For example, you should generate new keystore and set keystore details under [keystore.tls] and truststore sections.

Using escape characters with a string

TOML file format gives two way's to define strings. You can use a single quote and double quotes to define a string. If you used double quotes, it allows using escape characters inside the string. For example, if you need to define a newline character is inside a string, then you can use \n in inside the double-quotes. Here follow the whole list of escape characters supported by the TOML file.

  • \b - backspace (U+0008)
  • \t - tab (U+0009)
  • \n - linefeed (U+000A)
  • \f - form feed (U+000C)
  • \r - carriage return (U+000D)
  • \" - quote (U+0022)
  • \\ - backslash (U+005C)
  • \uXXXX - unicode (U+XXXX)
  • \UXXXXXXXX - unicode (U+XXXXXXXX)

The single quote is the preferred format that does not allow to use escape characters in it. Therefore you can use it to define simple plain text without worrying about escape characters.

In case you need to use a backslash in double-quotes, always remember to use a double backslash.

Do not define the same section headers twice

Following configuration is invalid since [transport.rabbitmq] defined in two places. sender_enable parameter also should defined along with listener_enable property in [transport.rabbitmq] section.

[server]
hostname = "localhost"
http_content_negotiation = true

[transport.rabbitmq]
listener_enable = true

[[transport.rabbitmq.listener]]
name = "AMQPConnectionFactory"
parameter.hostname = "localhost"
parameter.port = 5672

[transport.rabbitmq]
sender_enable = true

Careful about section ordering

When you define a section somewhere on toml file If you need to add another subsection for that section, be careful to define it below the main section. Here below invalid use case of section ordering.

[mediation.inbound]
core_threads = 20
max_threads = 100

[mediation]
synapse.core_threads = 20
synapse.max_threads = 100

This configuration is wrong since [mediation] section defined below the [mediation.inbound] subsection. The corrected configuration should be as follows.

[mediation]
synapse.core_threads = 20
synapse.max_threads = 100

[mediation.inbound]
core_threads = 20
max_threads = 100

Using custom parameters

In some scenarios, you may need to define properties other than the defined list of parameters. In this case, you can use custom parameters to include custom values. Please refer "Defining synapse properties" section for custom parameter example.

If custom parameters contain dot-separated keys, then you need to surround it with single quotes.

Defining synapse properties

WSO2 MI does provide support to define the following parameters with deployment TOML file to configure synapse related properties.

[mediation]
synapse.core_threads = 20
synapse.max_threads = 100
synapse.threads_queue_length = 10
synapse.global_timeout_interval = "120000"
synapse.enable_xpath_dom_failover=true
synapse.temp_data_chunk_size=3072
synapse.command_debugger_port=9005
synapse.event_debugger_port=9006
synapse.script_mediator_pool_size=15
synapse.enable_xml_nil=false
synapse.disable_auto_primitive_regex = "^-?(0|[1-9][0-9]*)(\\.[0-9]+)?([eE][+-]?[0-9]+)?$"
synapse.disable_custom_replace_regex = "@@@"
synapse.enable_namespace_declaration = false
synapse.build_valid_nc_name = false
synapse.enable_auto_primitive = false
synapse.json_out_auto_array = false
synapse.preserve_namespace_on_xml_to_json=false
flow.statistics.enable=false
flow.statistics.capture_all=false
statistics.enable_clean=true
statistics.clean_interval = 1000
stat.tracer.collect_payloads=false
stat.tracer.collect_mediation_properties=false
inbound.core_threads = 20
inbound.max_threads = 100
internal_http_api_enable = true
internal_http_api_port = 9191
internal_https_api_port = 9154

If you need to define anything other than the above config, you can configure those by using custom parameters as follows.

[synapse_properties]
'parameter.key' = "parameter value"

always remember to use a single quotation surrounded by the key, if the key has separated with dots.

Adding system parameters

There are two ways to pass system parameters into the WSO2 server. Either you can specify the parameter in server startup, or by using TOML file. For example, if you need to pass org.wso2.CipherTransformation = true as system parameter, you can either start server with -Dorg.wso2.CipherTransformation=true or define parameter inside TOML file as follows.

[system.parameter]
'org.wso2.CipherTransformation' = RSA/ECB/OAEPwithSHA1andMGF1Padding

You can add new parameters under [system.parameter] section as you wish.

Configuring message builders and formatters

The latest TOML configuration model does enable the following message builders and formatters by default. Unless you need to change the default classes for builder as formatters it will work by default with following default class names.

[message_formatters]
form_urlencoded =  "org.apache.synapse.commons.formatters.XFormURLEncodedFormatter"
multipart_form_data =  "org.apache.axis2.transport.http.MultipartFormDataFormatter"
application_xml = "org.apache.axis2.transport.http.ApplicationXMLFormatter"
text_xml = "org.apache.axis2.transport.http.SOAPMessageFormatter"
soap_xml = "org.apache.axis2.transport.http.SOAPMessageFormatter"
text_plain = "org.apache.axis2.format.PlainTextFormatter"
application_json =  "org.wso2.micro.integrator.core.json.JsonStreamFormatter"
json_badgerfish = "org.apache.axis2.json.JSONBadgerfishMessageFormatter"
text_javascript = "org.apache.axis2.json.JSONMessageFormatter"
octet_stream = "org.wso2.carbon.relay.ExpandingMessageFormatter"
application_binary =  "org.apache.axis2.format.BinaryFormatter"

[message_builders]
application_xml = "org.apache.axis2.builder.ApplicationXMLBuilder"
form_urlencoded = "org.apache.synapse.commons.builders.XFormURLEncodedBuilder"
multipart_form_data = "org.apache.axis2.builder.MultipartFormDataBuilder"
text_plain = "org.apache.axis2.format.PlainTextBuilder"
application_json = "org.wso2.micro.integrator.core.json.JsonStreamBuilder"
json_badgerfish = "org.apache.axis2.json.JSONBadgerfishOMBuilder"
text_javascript = "org.apache.axis2.json.JSONBuilder"
octet_stream =  "org.wso2.carbon.relay.BinaryRelayBuilder"
application_binary = "org.apache.axis2.format.BinaryBuilder"

If you need to define a message builder or formatter other than above list, then you can add it as follows.

[[custom_message_formatters]]
content_type = "application/json/badgerfish"
class = "org.apache.axis2.json.JSONBadgerfishMessageFormatter"

[[custom_message_builders]]
content_type = "application/json/badgerfish"
class = "org.apache.axis2.json.JSONBadgerfishOMBuilder"

Configuring transport

All transport configurations define under the [transport] section. All the transport follows the following key naming convention.

[transport.<tranport-type>]
<listener-or-sender>.<property>=value

For example, http sample transport configurations will be as follows. Here the transport type is http sender.

[transport.http]
sender.enable = true
sender.warn_on_http_500 = "*"
sender.proxy_host = "$ref{server.hostname}"
sender.proxy_port = 3128

Here another example of VFS listener.

[transport.vfs]
listener.enable = true
listener.keystore.file_name = "$ref{keystore.tls.file_name}" 
listener.keystore.type = "$ref{keystore.tls.type}"
listener.keystore.password = "$ref{keystore.tls.password}"
listener.keystore.key_password = "$ref{keystore.tls.key_password}"
listener.keystore.alias = "$ref{keystore.tls.alias}"

Transport listeners and senders can be individually enabled and disable by using enable property. These configuration conventions apply for following transports.

  • HTTP (Both http/https transport configure under [transport.http] section)
  • VFS
  • FIX
  • WS
  • WSS
  • TCP
  • UDP
  • MSMQ
  • MQTT

Custom parameters can be passed into these transport with following convention.

[transport.<tranport-type>]
parameter.'<custom-parameter>'=value

Always remember to surround custom parameters with single quote to avoid TOML parsing issues(If the custom parameter has dots inside).

Since JMS and RabbitMQ transport can have multiple transport definitions, it follows the following convention to define JMS configs.

[[transport.<transport-type>.<listener-or-sender>]]
parameter.<property>=value

Example JMS configuration with JMS queue and topic as follows.

[[transport.jms.listener]]
name = "myTopicConnectionFactory"
parameter.initial_naming_factory = "com.sun.jndi.fscontext.RefFSContextFactory"
parameter.provider_url = "file:/C:/JNDI-Directory"
parameter.connection_factory_name = "MQ_JMS_MANAGER"
parameter.connection_factory_type = "topic"
parameter.destination = "ivtT"

[[transport.jms.listener]]
name = "default"
parameter.initial_naming_factory = "com.sun.jndi.fscontext.RefFSContextFactory"
parameter.provider_url = "file:/C:/JNDI-Directory"
parameter.connection_factory_name = "MQ_JMS_MANAGER"
parameter.connection_factory_type = "queue"
parameter.destination = "bogusq"

None of these array type transports are defined as default. Therefore, If you need to only enable transport with no parameters, you can use transport enable disable feature according to the following convention.

[transport.<transport-type>]
<listener-or-sender>_enable = true

For example, if you need to only enable JMS sender, add this configuration into the deployment.toml file.

[transport.jms]
sender_enable = true

You can configure RabbitMQ transport in the same way. Example RabbitMQ sender as follows.

[[transport.rabbitmq.sender]]

name = "AMQPConnectionFactory"
parameter.hostname = "localhost"
parameter.port = 5672
parameter.username = "guest"
parameter.password = "guest"

Custom parameters for these types of transport arrays would be follows following convention.

[[transport.rabbitmq.sender]]
parameter.'<custom-parameter>' = value

Configuring blocking transport

Blocking transport definition for TOML file follows same rules as non-blocking transport. It follow following convention to define single transport.

[transport.blocking.<tranport-type>]
<listener-or-sender>.<property>=value

JMS can be configured accroding to the following convention.

[[transport.blocking.<transport-type>.<listener-or-sender>]]
parameter.<property>=value

Configuring secure vault

If you are already familiar with EI 6xx series, configuring secure vault is not that easy task to do. In the latest single TOML config model, configuring secure vault is super simple. You can do secure vault configuration inside the TOML file itself rather than changing multiple different files.

All secrets define inside the [secrets] section. Inside this section, you can define secrets as key-value pairs. Follow an example of how to define a secret inside the TOML file.

[secrets]
carbon_password = "[wso2carbon]"
admin_password = "[admin]"

Here, there are two passwords defined with the relevant value. Always remember to insert plain values surround brackets. To encrypt plain text values, you need to run cipher tools with ./ciphertool.sh -Dconfigure command. Once you run this command, it will generate encrypted values as follows in deployment.toml file.

[secrets]
carbon_password  = "aCjz6PyiZ0k0G9uxzhKNQaHW7QvfR+9O4Na9SedX0KLimtZBPsmYjtL7dbrZWCJm0O77AZyLOPFcR+g+ZDey4iO0dkF9DIQlm7BCL5i9fUcSLQk/4pc9rMrlOnoJ+dkp5XjwtXqs528D/qdfo1ywXkJXS3vt2Mb5+ZirAvkhnI2uMzwbEDjIFJxUQdrE4cPUnnq/9XDBLFb4CwznYdxC6OMF9IgCn/V9ezNjTu8icA50fRNx7jipCMy/RVkq9CUwRJEmKI9DgDurX3LDs3X3jpgeX17OmisFzbx26GzllL0MBf+0AVc7C60YpXEezGSOfdaa2Oti1VvGwrSkLRYapw=="
admin_password  = "YLGU7Kim9nLL9HPNjorixpuhHbc5rQ8+o8vH3gA/xdJunCNY4cD0F8O5a4owP+IZrGldGRzzMz9nXTOljTDDYJMvwj0g2M1hNAr0q9vUuLR2x1hnI+8wRv96rVKr9MJQQyWn76Ru1JTeoHQngcnPTGsfVh3EmCfHQCia8crhV3W55ZWpf4kCjsmNZoKNSnzeeBypFFErT3LBxMja+Su5XTHAkS3jp0IYxAZSYoDqUs2rYEV9El0uRKQdCUB6AyDUE8OTLglnH2I0a3uE/b9HiKK48Yujdit10xxe+qaCDdI4TPObT3U4xOkBAWPUBtXcBj3Bvm7HnMeqzi8EAORTnA=="

You can use these encrypted entities when you need to define a secret value. For example, If you need to set super admin username password, it would be as follows.

[super_admin]
username = "admin"
password = "$secret{admin_password}"
integration