Kubernetes Configuration Guide

Event store Microservice helps to collect events raised from the individual Microservices and forwards it to the respective Microservice event using the streaming platform and this allows the bank to create deployment scripts (charts) based on their needs.

The following table defines the property of Start Scripts in the guide.

PROPERTY DEFAULT VALUE POSSIBLE VALUES DESCRIPTION
Database Properties      
database_Key

MSSQL & MYSQL

sql

ORACLE

orcl

orcl, sql

Specifies the name of the database server.

db_Enable_Secret N Y, N

A Secret is an object that contains a small amount of sensitive data such as a password, a token, or a key and this allows to fetch DB username and password through Kubernetes (k8s) secrets for MySQL DB if db_Enable_Secret is set to Y.

db_Host eventstore-db-service   Specifies the hostname of the sql server.
database_Name

MSSQL & MYSQL

ms_eventstore

ORACLE

orcl

  Specifies the name of the database used in sql server.
db_Username db_Password

MSSQL

sa Rootroot@12345

MYSQL

root password

ORACLE

system Oracle_1

  Indicates the username and password that matches the server login credentials to interact with the database.
driver_Name

MSSQL com.microsoft.sqlserver. jdbc.SQLServerDriver

MYSQL com.mysql.jdbc.Driver

ORACLE oracle.jdbc.OracleDriver

 

The driver provides Java database connectivity from any Java application, application server, or Java - enabled applet.

MSSQL

SQL Server Driver class includes methods for connecting to a SQL Server database and obtaining information about the JDBC driver.

MYSQL

The MySQL JDBC Driver enables users to connect with live MySQL data, directly from any applications that support JDBC connectivity.

ORACLE

Oracle Driver enables users to dynamically load the driver class file into memory, which automatically registers it.

dialect MSSQL org.hibernate.dialect. SQLServer2012Dialect MYSQL org.hibernate.dialect. MySQL5InnoDBDialect ORACLE org.hibernate.dialect. Oracle8iDialect   For connecting any hibernate application with the database, it is required to provide the configuration of SQL dialect.
db_Connection_ Url

MSSQL jdbc:sqlserver://eventstor e-db-service:1433; databaseName=ms_eve ntstore

MYSQL jdbc:mysql://eventstore- db-service-np:3306 /ms_eventstore

ORACLE jdbc:oracle:thin:@host. docker.internal:1521:orcl

 

The general form of the connection URL is as follows:

MSSQL: jdbc:sqlserver://<host_or_ip>:1433;databaseName=<db_name>

MYSQL: jdbc:mysql://<host_or_ip>:3306/<db_name>

Oracle: jdbc:oracle:thin:@<host_or_ip>:1521:<db_name>

host_or_ip: This is the address (optional) of the server to connect to. This can be a DNS or IP address, or it can be localhost or 127.0.0.1 for the local computer. If not specified in the connection URL, the server name must be specified in the properties collection.

port number: This is the port (optional) to connect to serverName. The default is 1433 for mssql , 3306 for mysql, and 1521 for oracle. For the default option, do not specify the port, nor its preceding : in the URL.

db_name: This is the name of the database to be used in sql server.

dbinit_Connection_Url

MSSQL jdbc:sqlserver://eventstor e-db-service.eventstore. svc.cluster.local:1433 MYSQL jdbc:mysql://eventstore- db-service-np. eventstore.svc.cluster. local:3306

ORACLE: jdbc:oracle:thin:@host. docker.internal:1521:orcl

  DB initialization utility URL is used to connect the database. This is used to create tables inside sql database.
max_Pool_Size min_Pool_Size

MSSQL & MYSQL

5

1

ORACLE

150

10

 

Indicates the maximum number of connections maintained in the pool.

Indicates the minimum number of connections maintained in the pool.

Kafka Properties      
kafka_Bootstrap _Servers my-cluster-kafka- bootstrap.kafka:9092  

This contains a list of host or port pairs for establishing the initial connection to the Kafka cluster. A host and port pair uses : as the separator.

  • localhost:9092

  • localhost:9092,another.host:9092

schema_Registry _Url http://schema-registry- svc.kafka.svc.cluster. local   Schema Registry is an application that resides outside of the Kafka cluster and handles the distribution of schemas to the producer and consumer by storing a copy of the schema in its local cache. Schema registry URL is used to connect schema registry in Kafka.
kafka_Aliases N Y, N

hostAliases are used to overwrite the resolution of the hostname and IP at the pod level when adding entries to the /etc/hosts file of the pod.

To enable hostAliases, set the below variable to Y.

kafkaip

kafka0ip

kafka1ip

kafka2ip

    Set the variables of Kafka IP for hostAliases.

kafka_Host_Name

kafka0_Host_Name

kafka1_Host_Name

kafka2_Host_Name

    Set the variables of Kafka hostname for hostAliases.
kafka_Ssl_Enable N Y, N To enable ssl connectivity in Kafka.
kafka_Ssl_Boots trap_Servers kafka:29092   This contains a list of host or port pairs for establishing the initial connection to the Kafka cluster. A host and port pair uses : as the separator. To enable kafka_Ssl_Bootstrap_Servers, set kafka_Ssl_Enable variable to Y.
Image Properties      
tag <releaseversion>   Specifies the release version of the image.

api_Image

ingester_Image

eventdelivery_Image

scheduler_Image

dbinit_Image

temenos/ms-eventstore- service

temenos/ms-eventstore- inboxoutbox

temenos/ms-eventstore- inboxoutbox

temenos/ms-eventstore- catchup-processor

temenos/ms-eventstore- initscripts

  Specifies the name of Images for API, ingester, event delivery, scheduler, dbinit that are pushed to external repositories. Consider our external repository as acr.azurecr.io and tag as 21.0.0.
env_Name     Specifies the environment name which is added as a respective pod prefix.

es_Image_Pull_ Secret

dbinit_Image_Pull_Secret

   

Docker registry secret contains the Oracle Cloud Infrastructure credentials to use when pulling the image. The user needs to specify the image to pull from Container Registry, including the repository location and the Docker registry secret to use, in the application manifest file.

kubectl create secret docker-registry <secret-name> --docker-server=<region-key>.ocir.io --docker- username='<tenancy-namespace>/<oci-username>' --docker-password='<oci-auth-token>' --docker- email='<email-address>' .

es_Image_Pull_Secret and dbinit_Image_Pull_Secret specifies the <secret-name>, that is used in the manifest file to refer to the already created secret.

Scheduler Properties      
inbox_Cleanup 60   Specifies the minutes required to hold the records inside ms_inbox_events table before automatic deletion. Default is 60 minutes.
schedule 5   Specifies the frequency to trigger that the scheduler job is set in this property.
Logger Properties      
logger_Root_Level INFO DEBUG, INFO, ERROR Specifies the configuration parameter for logging.
Rolling Update      
rolling_Update false true, false Rolling updates allow deployment updates with zero downtime by incrementally updating pods instances with new ones. The possible values could be true or false.
api_MaxSurge ingester_MaxSurge 1   Indicates the maximum number of pods that can be created over the desired number of pods.

api_MaxUnavailable

ingester_MaxUnavailable

0   Indicates the maximum number of pods that are unavailable during the update process.
config_Location svc/eventstore- configmap.yaml   The location specified for the configmap.yaml.
Others      
hostpath /c/testing/   The host path variable specifies the path where the file is loaded inside the k8 pod. The local path can be mounted to the specific local path in K8 container, and this path can be mapped inside the camel to load the key store files.
key_Filepath /var/lib/processed /temenosserver.keystore. jks   Specifies the path of keystore.jks file which contains the application server certificate, including its private key. The Keystore file is protected with a password. Change the password using the key tool.
trust_Filepath /var/lib/processed /temenosclient.jks   Specifies the path of .jks file which contains the trusted certificates of the application server, including public keys for other entities. For a trusted certificate, the server has confirmed that the public key in the certificate belongs to the certificate owner. Trusted certificates generally include those of certification authorities (CAs).
keystore_Pass 123456   Indicates the password used to access the Keystore file when SSL is enabled.
keypass 123456   Keypass is the password of a key in the Keystore.
trust_Store_Pass 123456   Indicates the password used to gain access to the trust store.

The following table defines the property of values.yaml in the guide.

PROPERTY DEFAULT VALUE POSSIBLE VALUES DESCRIPTION
Config Map      
location /svc/eventstore- configmap.yaml   Specifies the location for the configmap.yaml.
Rolling Update      
enabled false true, false Rolling updates allow deployment updates to take place with zero downtime by incrementally updating pods instances with new ones.
api & ingester      
maxsurge 1 Indicates the maximum number of pods that can be created over the desired number of pods.
maxunavailable 0 Indicates the maximum number of pods that are unavailable during the update process.
Replica Count      

eventstoreapi

eventstoreingester

1   Replica Count is used to specify the number of pod instances running in a cluster at any given time to prevent users from losing access to their application when a pod fails or is inaccessible.
Kafka Topic     Kafka topic names are the categories used to organize messages. Each topic has a name that is unique across the entire Kafka cluster.

inboxtopic

sourcestream_consumergroupid

errorstream_producerid

ms-eventstore-inbox- topic

ms-eventstore- ingester-consumer

ms-eventstore- ingester-error- producer

 

The eventstore inbox topic is used to log messages of the events produced.

The eventstore ingester consumer topic is used to log messages of the events that are consumed.

The eventstore ingester error producer logs messages when there is any issue or error when the events are produced.

Image      
tag <releaseversion>   Specifies the release version of the image.
pullPolicy IfNotPresent  

When creating the POD, one can specify the imagePullPolicyspecification, which guides the Kubelet service on how to pull the specified image during an update.

  • If the image pull policy is set to Always, Kubernetes always pulls the image from the Repository.

  • With IfNotPresent, Kubernetes only pulls the image when it does not already exist in the node.

  • Wth the image pull policy set to Never, Kubernetes never pulls the image.

Repository      

eventstoreapi

eventstoreingester

eventstorecatchupprocessor

dbinit_Image

temenos/ms- eventstore- service

temenos/ms- eventstore- inboxoutbox

temenos/ms- eventstore- inboxoutbox

temenos/ms- eventstore- catchup- processor

temenos/ms- eventstore- initscripts

Specifies the name of Images for API, ingester, event delivery, scheduler, and dbinit that are pushed to external repositories, Consider external repository as acr.azurecr.io and tag as 21.0.0.
imagePullSecrets    

Docker registry secret contains the Oracle Cloud Infrastructure credentials which the user can use to pull the image. The user needs to specify the image to pull from Container Registry, including the repository location and the docker registry secret to use, in the application manifest file.

kubectl create secret docker-registry <secret-name> --docker-server=<region-key>.o cir.io --docker-username='<tenancy-namespace>/<oci-username>' --docker- password='<oci-auth-token>' --docker-email='<email-address>'.

imagePullSecrets specifies the <secret-name>, that can be used in the manifest file to refer to the already created secret.

Deployment      

apiname

schedulername

ingestername

deliveryname

eventstore-api

eventstore- catchup

eventstore- command- ingester

eventstore- event-delivery

 

Kubernetes deployment informs Kubernetes how to create or modify pod instances that holds a containerized application.

Deployments can scale the number of replica pods, enable the roll out of updated code in a controlled manner, or roll back to an earlier deployment version if necessary.

Specifies the name of the image to be loaded for API, scheduler, ingester, and event delivery during deployment.

Service Name      

apiname

ingestername

eventstore-api- svc

eventstore- command- ingester-svc

 

This service is responsible for enabling network access to a set of pods.

Specifies the Kubernetes service name for API and ingester.

env      
name     Specifies the environment name which is added as a respective pod prefix.
hostpath /c/testing/   The hostpath variable specifies the path where the file can be loaded inside the k8 pod. The local path can be mounted to specific local path in K8 container, and this path can be mapped inside the camel to load the key store files.
keyfilepath     Specifies the path of keystore.jks file which contains the application server certificate, including its private key. The keystore file is protected with a password. Change the password using the key tool.
trustfilepath     Specifies the path of .jks file which contains the trusted certificates of Application Sever, including public keys for other entities. For a trusted certificate, the server confirms that the public key in the certificate belongs to the certificate owner. Trusted certificates generally include those of certification authorities.
keystorepass     The password used to access the Keystore file when SSL is enabled.
keypass     Keypass is the password of a key in the Keystore.
truststorepass     Specifies the password used to gain access to the trust store.
kafkasslenable     Set the following variables to enable SSL connectivity in Kafka
kafkasslbootstrapservers     This contains a list of host or port pairs for establishing the initial connection to the Kafka cluster. A host and port pair uses : as the separator. To enable kafka_Ssl_Bootstrap_Servers, set kafka_Ssl_Enable variable to Y.
Database      
database_key sql orcl, sql Specifies the name of the database server.
MSSQL_CRED MYSQL_CRED N  

A Secret is an object that contains a small amount of sensitive data such as a password, a token, or a key which allows to fetch DB username and password through k8s secrets if secret is set to Y.

host eventstore-db- service   Specifies the hostname of the sql server.
database_name

MSSQL & MYSQL ms_eventstore ORACLE

orcl

  Specifies the name of the database used in sql server.

db_username

db_password

MSSQL

sa Rootroot@12345

MYSQL

root password ORACLE

system Oracle_1

  Indicates the username and password that matches the server login credentials to interact with the database.
driver_name

MSSQL

com.microsoft. sqlserver.jdbc.SQLS erverDriver

MYSQL com.mysql.jdbc. Driver

ORACLE oracle.jdbc.OracleDr iver

 

The driver provides Java database connectivity from any Java application, application server, or Java - enabled applet.

MSSQL

  • SQLServerDriver class includes methods for connecting to a SQL Server database and obtaining information about the JDBC driver.

MYSQL

  • The MySQL JDBC Driver enables users to connect with live MySQL data, directly from any applications that support JDBC connectivity.

ORACLE

  • Oracle Driver enables users to dynamically load the driver class file into memory, which automatically registers it.

dialect

MSSQL org.hibernate.dialect. SQLServer2012Diale ct

MYSQL org.hibernate.dialect. MySQL5InnoDBDiale ct

ORACLE org.hibernate.dialect. Oracle8iDialect

  For connecting any hibernate application with the database, it is required to provide the configuration of SQL dialect.
db_connection_url

MSSQL jdbc:sqlserver://event store-db-service: 1433; databaseName=ms_ eventstore

MYSQL jdbc:mysql://eventsto re-db-service-np: 3306/ms_eventstore ORACLE jdbc:oracle:thin: @host.docker. internal:1521:orcl

 

The general form of the connection URL is as follows:

MSSQL: jdbc:sqlserver://<host_or_ip>:1433;databaseName=<db_name>

MYSQL: jdbc:mysql://<host_or_ip>:3306/<db_name>

ORACLE: jdbc:oracle:thin:@<host_or_ip>:1521:<db_name>

host_or_ip: This is the address (optional) of the server to connect to. This address can be a DNS or IP address, or it can be localhost or 127.0.0.1 for the local computer. If not specified in the connection URL, the server name must be specified in the properties collection.

port number: This is the port (optional) to connect to on serverName. The default is 1433 for mssql , 3306 for mysql , and 1521 for oracle. If the user is using the default, do not specify the port, nor the preceding : in the URL.

db_name: This is the name of the database to be used in sql server.

dbinit_Connection_Url MSSQL jdbc:sqlserver://event store-db-service. eventstore.svc. cluster.local:1433 MYSQL jdbc:mysql://eventsto re-db-service-np. eventstore.svc. cluster.local:3306 ORACLE: jdbc:oracle:thin: @host.docker. internal:1521:orcl   DB initialization utility URL is used to connect the database. This is used to create tables inside sql database.

max_pool_size

min_pool_size

MSSQL & MYSQL

5

1

ORACLE

150

10

 

Indicates the maximum number of connections maintained in the pool.

Indicates the minimum number of connections maintained in the pool.

temn_msf_db_pass_encryption_key

temn_msf_db_pass_encryption_algorithm

temenos

PBEWithMD5A ndTripleDES

 

Indicates the key or password and algorithm to encrypt the plain text.

These two values are mandatory to encrypt the plain text.

Kafka      
kafkabootstrapservers my-cluster-kafka- bootstrap.kafka:9092  

This contains a list of host or port pairs to establish the initial connection of kafka cluster. A host and port pair uses : as the separator.

  • localhost:9092

  • localhost:9092, another.host:9092.

temnmsfstreamvendor kafka   Specifies the vendor used to process messages to a topic.
temnqueueimpl kafka   Specifies the queue used to identify the listener container (processing messages).
schema_registry_url http://schema- registry-svc.kafka. svc.cluster.local   Schema Registry is an application that resides outside of the Kafka cluster and handles the distribution of schemas to the producer and consumer by storing a copy of the schema in its local cache. Schema registry URL is used to connect schema registry in Kafka.
kafka_Aliases N Y, N

hostAliases are used to overwrite the resolution of the host name and IP at the pod level when adding entries to the /etc/hosts file.

To enable hostAliases, set the below variables to Y.

kafkaip

kafka0ip

kafka1ip

kafka2ip

    Set the variables of Kafka IP for hostAliases.

kafka_Host_Name

kafka0_Host_Name

kafka1_Host_Name

kafka2_Host_Name

    Set the variables of Kafka hostname for hostAliases.
Security      
temn_msf_security_authz_enabled false true, false Enable or disable the XACML policy authorization. XACML policy authorization is used to configure security policies and access rights to information for APIs.
Scheduler Properties      
temn_msf_scheduler_inboxcleanup_schedule 60   Specifies the minutes required to hold the records inside ms_inbox_events table before automatic deletion.
temn_msf_scheduler_sqlInboxCatchup_cron_trigger_time 0 0/1 * 1/1 * ?   Indicates the frequency to trigger the scheduler job is set in this property and cron trigger time for sql inbox catchup.
temn_msf_scheduler_sqlOutboxCatchup_cron_trigger_time 0 0/1 * 1/1 * ?   Indicates the frequency to trigger the scheduler job is set in this property and cron trigger time for sql outbox catchup.
schedule 5   Indicates the frequency to trigger the scheduler job is set in this property.
Thread Pool      
tmn_outbox_thread_pool_count 10   Indicates the number of thread counts required to process the outbox events.
Logger      
root_level INFO DEBUG, INFO, ERROR Specifies the configuration parameter for logging. INFO logger level is maintained by default.
SSL      

temn_msf_stream_security_kafka_ssl_keystore_location:

temn_msf_stream_security_kafka_ssl_keystore_password:

temn_msf_stream_security_kafka_ssl_key_password:

temn_msf_stream_security_kafka_ssl_truststore_location:

temn_msf_stream_security_kafka_ssl_truststore_password:

    Keystore stores private key and identity certificates that a specific program should present to both parties (server or client) for verification. Trust store is used to store certificates from Certified Authorities (CA) that verify the certificate presented by the server in SSL connection which is commented by default. If required, kindly enable them.
Ports     Specifies the port, name, protocol, target port for the eventstore ingester, and api service.
name http   This is a formally defined set of rules for communication between a client (the network resource requesting data or services) and a server (the resource that receives and responds to the request).
port 8080   A port number is a way to identify a specific process to which internet or other network message is to be forwarded to a server.
protocol TCP   The default protocol for services is TCP (other supported protocols can also be used).
target port 8080   This service can map any incoming port to a target port. By default, and for convenience, the target port is set to the same value as the port field.
Service      

eventstoreapi:

type: LoadBalancer

port: 7004

eventstoreingester:

type: LoadBalancer

port: 7010

   

This service in Kubernetes is a REST object, similar to a pod. Like all REST objects, the service definition can be posted to the API server to create a new instance.

Specifies the port, type for the eventstore API, and ingester service.

When the Service type is set to load balancer, Kubernetes provides functionality equivalent to type equals ClusterIP to pods within the cluster and extends it by programming the load balancer (external to Kubernetes) with entries for the nodes hosting the relevant Kubernetes pods.

Logging      

# socketHost: "192.168.1.37"

# socketPort: "4560" exportType: #"socket" protocol: #TCP

   

Logging can be collecting, storing, and analyzing distributed logs. Here log stash is used for creating an ingestion pipeline for collecting logs from multiple devices and stored in elastic search for visualizing and analyzing through Grafana.

To enable logging sockethost, SockerPort, and exportType, the protocol should be uncommented, and the required port and host address should be given.

Metrics      

#publisherPort: 9091

#publisherHost: 192.168.228.193 metricsDisabled: "true"

   

Metrics give us insights into the historical and current state of a system. Data instrumented from microservices are pushed to the PushGateway service. Prometheus collects data from PushGateway at a regular interval. Grafana is used to visualize data in UI.

To enable monitoring in micro services publisherPort, publisherHost can be configured with a push gateway port and host address and metricsDisabled can be changed to false (set to true by default).

Metrics is disabled by default.

Resources      

api:

cpulimit: "100m" memorylimit: 1.5G java: heaplimit: ' -XX: +UnlockExperimentalVMOptions - XX: +UseCGroupMemoryLimitForHeap -XX:MaxRAMPercentage=75.0 '

ingester:

cpulimit: "100m" memorylimit: 1.5G java: heaplimit: ' -XX: +UnlockExperimentalVMOptions - XX: +UseCGroupMemoryLimitForHeap -XX:MaxRAMPercentage=75.0 '

   

Resource management for pods or containers - Kubernetes uses to control resources such as CPU and memory, can declare limits for CPU and for all pod containers.

  • cpulimit --The CPU limit defines a hard ceiling on how much CPU time the container can use. It should not be less than 1m (CPU limit can be 1m or 0.001).

  • memorylimit – To specify a memory limit for every container in a pod, Kubernetes can infer the pod-level memory limit by adding up the limits for its containers.

  • heapLimit

    • To change the value of experimental flags, unlock them first. The user can do this by setting -XX:+UnlockExperimentalVMOptions explicitly on the command line before any experimental flags.

    • The flag -XX:+UseCGroupMemoryLimitForHeap lets the JVM detect what the max heap size in a container should be.

    • MaxRAMPercentage parameter allows setting the maximum heap size for a JVM running with a large amount of memory (greater than 200 MB).

Delivery

cpulimit: "100m" memorylimit: 1.5G java: heaplimit: ' -XX: +UnlockExperimentalVMOptions - XX: +UseCGroupMemoryLimitForHeap -XX:MaxRAMPercentage=75.0 '

 

Scheduler

cpulimit: "100m" memorylimit: 1.5G java: heaplimit: ' -XX: +UnlockExperimentalVMOptions - XX: +UseCGroupMemoryLimitForHeap -XX:MaxRAMPercentage=75.0 '

 

Field Name
|
Application/Table name

Temenos Headquarters SA
2 Rue de l'Ecole-de-Chimie
CH - 1205 Geneva
Switzerland

Copyright © 2020- Temenos Headquarters SA

Published on :
Tuesday, August 29, 2023 1:50:10 PM IST

Last Updated R23 AMR