Kubernetes Configuration Guide
Event store Microservice helps to collect events raised from the individual Microservices and forwards it to the respective Microservice event using the streaming platform and this allows the bank to create deployment scripts (charts) based on their needs.
The following table defines the property of Start Scripts in the guide.
| PROPERTY | DEFAULT VALUE | POSSIBLE VALUES | DESCRIPTION |
|---|---|---|---|
| Database Properties | |||
| database_Key |
MSSQL & MYSQL sql ORACLE orcl |
orcl, sql |
Specifies the name of the database server. |
| db_Enable_Secret | N | Y, N |
A Secret is an object that contains a small amount of sensitive data such as a password, a token, or a key and this allows to fetch DB username and password through Kubernetes (k8s) secrets for MySQL DB if db_Enable_Secret is set to Y. |
| db_Host | eventstore-db-service | Specifies the hostname of the sql server. | |
| database_Name |
MSSQL & MYSQL ms_eventstore ORACLE orcl |
Specifies the name of the database used in sql server. | |
| db_Username db_Password |
MSSQL sa Rootroot@12345 MYSQL root password ORACLE system Oracle_1 |
Indicates the username and password that matches the server login credentials to interact with the database. | |
| driver_Name |
MSSQL com.microsoft.sqlserver. jdbc.SQLServerDriver MYSQL com.mysql.jdbc.Driver ORACLE oracle.jdbc.OracleDriver |
The driver provides Java database connectivity from any Java application, application server, or Java - enabled applet. MSSQL SQL Server Driver class includes methods for connecting to a SQL Server database and obtaining information about the JDBC driver. MYSQL The MySQL JDBC Driver enables users to connect with live MySQL data, directly from any applications that support JDBC connectivity. ORACLE Oracle Driver enables users to dynamically load the driver class file into memory, which automatically registers it. |
|
| dialect | MSSQL org.hibernate.dialect. SQLServer2012Dialect MYSQL org.hibernate.dialect. MySQL5InnoDBDialect ORACLE org.hibernate.dialect. Oracle8iDialect | For connecting any hibernate application with the database, it is required to provide the configuration of SQL dialect. | |
| db_Connection_ Url |
MSSQL jdbc:sqlserver://eventstor e-db-service:1433; databaseName=ms_eve ntstore MYSQL jdbc:mysql://eventstore- db-service-np:3306 /ms_eventstore ORACLE jdbc:oracle:thin:@host. docker.internal:1521:orcl |
The general form of the connection URL is as follows: MSSQL: jdbc:sqlserver://<host_or_ip>:1433;databaseName=<db_name> MYSQL: jdbc:mysql://<host_or_ip>:3306/<db_name> Oracle: jdbc:oracle:thin:@<host_or_ip>:1521:<db_name> host_or_ip: This is the address (optional) of the server to connect to. This can be a DNS or IP address, or it can be localhost or 127.0.0.1 for the local computer. If not specified in the connection URL, the server name must be specified in the properties collection. port number: This is the port (optional) to connect to serverName. The default is 1433 for mssql , 3306 for mysql, and 1521 for oracle. For the default option, do not specify the port, nor its preceding : in the URL. db_name: This is the name of the database to be used in sql server. |
|
| dbinit_Connection_Url |
MSSQL jdbc:sqlserver://eventstor e-db-service.eventstore. svc.cluster.local:1433 MYSQL jdbc:mysql://eventstore- db-service-np. eventstore.svc.cluster. local:3306 ORACLE: jdbc:oracle:thin:@host. docker.internal:1521:orcl |
DB initialization utility URL is used to connect the database. This is used to create tables inside sql database. | |
| max_Pool_Size min_Pool_Size |
MSSQL & MYSQL 5 1 ORACLE 150 10 |
Indicates the maximum number of connections maintained in the pool. Indicates the minimum number of connections maintained in the pool. |
|
| Kafka Properties | |||
| kafka_Bootstrap _Servers | my-cluster-kafka- bootstrap.kafka:9092 |
This contains a list of host or port pairs for establishing the initial connection to the Kafka cluster. A host and port pair uses : as the separator.
|
|
| schema_Registry _Url | http://schema-registry- svc.kafka.svc.cluster. local | Schema Registry is an application that resides outside of the Kafka cluster and handles the distribution of schemas to the producer and consumer by storing a copy of the schema in its local cache. Schema registry URL is used to connect schema registry in Kafka. | |
| kafka_Aliases | N | Y, N |
hostAliases are used to overwrite the resolution of the hostname and IP at the pod level when adding entries to the /etc/hosts file of the pod. To enable hostAliases, set the below variable to Y. |
|
kafkaip kafka0ip kafka1ip kafka2ip |
Set the variables of Kafka IP for hostAliases. | ||
|
kafka_Host_Name kafka0_Host_Name kafka1_Host_Name kafka2_Host_Name |
Set the variables of Kafka hostname for hostAliases. | ||
| kafka_Ssl_Enable | N | Y, N | To enable ssl connectivity in Kafka. |
| kafka_Ssl_Boots trap_Servers | kafka:29092 | This contains a list of host or port pairs for establishing the initial connection to the Kafka cluster. A host and port pair uses : as the separator. To enable kafka_Ssl_Bootstrap_Servers, set kafka_Ssl_Enable variable to Y. | |
| Image Properties | |||
| tag | <releaseversion> | Specifies the release version of the image. | |
|
api_Image ingester_Image eventdelivery_Image scheduler_Image dbinit_Image |
temenos/ms-eventstore- service temenos/ms-eventstore- inboxoutbox temenos/ms-eventstore- inboxoutbox temenos/ms-eventstore- catchup-processor temenos/ms-eventstore- initscripts |
Specifies the name of Images for API, ingester, event delivery, scheduler, dbinit that are pushed to external repositories. Consider our external repository as acr.azurecr.io and tag as 21.0.0. | |
| env_Name | Specifies the environment name which is added as a respective pod prefix. | ||
|
es_Image_Pull_ Secret dbinit_Image_Pull_Secret |
Docker registry secret contains the Oracle Cloud Infrastructure credentials to use when pulling the image. The user needs to specify the image to pull from Container Registry, including the repository location and the Docker registry secret to use, in the application manifest file. kubectl create secret docker-registry <secret-name> --docker-server=<region-key>.ocir.io --docker- username='<tenancy-namespace>/<oci-username>' --docker-password='<oci-auth-token>' --docker- email='<email-address>' . es_Image_Pull_Secret and dbinit_Image_Pull_Secret specifies the <secret-name>, that is used in the manifest file to refer to the already created secret. |
||
| Scheduler Properties | |||
| inbox_Cleanup | 60 |
Specifies the minutes required to hold the records inside ms_inbox_events table before automatic deletion. Default
is 60 minutes.
|
|
| schedule | 5 | Specifies the frequency to trigger that the scheduler job is set in this property. | |
| Logger Properties | |||
| logger_Root_Level | INFO | DEBUG, INFO, ERROR | Specifies the configuration parameter for logging. |
| Rolling Update | |||
| rolling_Update | false | true, false | Rolling updates allow deployment updates with zero downtime by incrementally updating pods instances with new ones. The possible values could be true or false. |
| api_MaxSurge ingester_MaxSurge | 1 | Indicates the maximum number of pods that can be created over the desired number of pods. | |
|
api_MaxUnavailable ingester_MaxUnavailable |
0 | Indicates the maximum number of pods that are unavailable during the update process. | |
| config_Location | svc/eventstore- configmap.yaml | The location specified for the configmap.yaml. | |
| Others | |||
| hostpath | /c/testing/ | The host path variable specifies the path where the file is loaded inside the k8 pod. The local path can be mounted to the specific local path in K8 container, and this path can be mapped inside the camel to load the key store files. | |
| key_Filepath | /var/lib/processed /temenosserver.keystore. jks | Specifies the path of keystore.jks file which contains the application server certificate, including its private key. The Keystore file is protected with a password. Change the password using the key tool. | |
| trust_Filepath | /var/lib/processed /temenosclient.jks | Specifies the path of .jks file which contains the trusted certificates of the application server, including public keys for other entities. For a trusted certificate, the server has confirmed that the public key in the certificate belongs to the certificate owner. Trusted certificates generally include those of certification authorities (CAs). | |
| keystore_Pass | 123456 | Indicates the password used to access the Keystore file when SSL is enabled. | |
| keypass | 123456 | Keypass is the password of a key in the Keystore. | |
| trust_Store_Pass | 123456 | Indicates the password used to gain access to the trust store. |
The following table defines the property of values.yaml in the guide.
| PROPERTY | DEFAULT VALUE | POSSIBLE VALUES | DESCRIPTION |
|---|---|---|---|
| Config Map | |||
| location | /svc/eventstore- configmap.yaml | Specifies the location for the configmap.yaml. | |
| Rolling Update | |||
| enabled | false | true, false | Rolling updates allow deployment updates to take place with zero downtime by incrementally updating pods instances with new ones. |
| api & ingester | |||
| maxsurge | 1 | Indicates the maximum number of pods that can be created over the desired number of pods. | |
| maxunavailable | 0 | Indicates the maximum number of pods that are unavailable during the update process. | |
| Replica Count | |||
|
eventstoreapi eventstoreingester |
1 | Replica Count is used to specify the number of pod instances running in a cluster at any given time to prevent users from losing access to their application when a pod fails or is inaccessible. | |
| Kafka Topic | Kafka topic names are the categories used to organize messages. Each topic has a name that is unique across the entire Kafka cluster. | ||
|
inboxtopic
sourcestream_consumergroupid errorstream_producerid |
ms-eventstore-inbox- topic
ms-eventstore- ingester-consumer ms-eventstore- ingester-error- producer |
The eventstore inbox topic is used to log messages of the events produced. The eventstore ingester consumer topic is used to log messages of the events that are consumed. The eventstore ingester error producer logs messages when there is any issue or error when the events are produced. |
|
| Image | |||
| tag | <releaseversion> | Specifies the release version of the image. | |
| pullPolicy | IfNotPresent |
When creating the POD, one can specify the imagePullPolicyspecification, which guides the Kubelet service on how to pull the specified image during an update.
|
|
| Repository | |||
|
eventstoreapi
eventstoreingester
eventstorecatchupprocessor
dbinit_Image |
temenos/ms- eventstore- service
temenos/ms- eventstore- inboxoutbox
temenos/ms- eventstore- inboxoutbox
temenos/ms- eventstore- catchup- processor
temenos/ms- eventstore- initscripts |
Specifies the name of Images for API, ingester, event delivery, scheduler, and dbinit that are pushed to external repositories, Consider external repository as acr.azurecr.io and tag as 21.0.0. | |
| imagePullSecrets |
Docker registry secret contains the Oracle Cloud Infrastructure credentials which the user can use to pull the image. The user needs to specify the image to pull from Container Registry, including the repository location and the docker registry secret to use, in the application manifest file.
kubectl create secret docker-registry <secret-name> --docker-server=<region-key>.o cir.io --docker-username='<tenancy-namespace>/<oci-username>' --docker- password='<oci-auth-token>' --docker-email='<email-address>'. imagePullSecrets specifies the <secret-name>, that can be used in the manifest file to refer to the already created secret. |
||
| Deployment | |||
|
apiname
schedulername
ingestername
deliveryname |
eventstore-api eventstore- catchup eventstore- command- ingester eventstore- event-delivery |
Kubernetes deployment informs Kubernetes how to create or modify pod instances that holds a containerized application. Deployments can scale the number of replica pods, enable the roll out of updated code in a controlled manner, or roll back to an earlier deployment version if necessary. Specifies the name of the image to be loaded for API, scheduler, ingester, and event delivery during deployment. |
|
| Service Name | |||
|
apiname
ingestername |
eventstore-api- svc eventstore- command- ingester-svc |
This service is responsible for enabling network access to a set of pods.
Specifies the Kubernetes service name for API and ingester. |
|
| env | |||
| name | Specifies the environment name which is added as a respective pod prefix. | ||
| hostpath | /c/testing/ | The hostpath variable specifies the path where the file can be loaded inside the k8 pod. The local path can be mounted to specific local path in K8 container, and this path can be mapped inside the camel to load the key store files. | |
| keyfilepath | Specifies the path of keystore.jks file which contains the application server certificate, including its private key. The keystore file is protected with a password. Change the password using the key tool. | ||
| trustfilepath | Specifies the path of .jks file which contains the trusted certificates of Application Sever, including public keys for other entities. For a trusted certificate, the server confirms that the public key in the certificate belongs to the certificate owner. Trusted certificates generally include those of certification authorities. | ||
| keystorepass | The password used to access the Keystore file when SSL is enabled. | ||
| keypass | Keypass is the password of a key in the Keystore. | ||
| truststorepass | Specifies the password used to gain access to the trust store. | ||
| kafkasslenable | Set the following variables to enable SSL connectivity in Kafka | ||
| kafkasslbootstrapservers | This contains a list of host or port pairs for establishing the initial connection to the Kafka cluster. A host and port pair uses : as the separator. To enable kafka_Ssl_Bootstrap_Servers, set kafka_Ssl_Enable variable to Y. | ||
| Database | |||
| database_key | sql | orcl, sql | Specifies the name of the database server. |
| MSSQL_CRED MYSQL_CRED | N |
A Secret is an object that contains a small amount of sensitive data such as a password, a token, or a key which allows to fetch DB username and password through k8s secrets if secret is set to Y.
|
|
| host | eventstore-db- service | Specifies the hostname of the sql server. | |
| database_name |
MSSQL & MYSQL ms_eventstore ORACLE orcl |
Specifies the name of the database used in sql server. | |
|
db_username db_password |
MSSQL
sa Rootroot@12345 MYSQL
root password ORACLE system Oracle_1 |
Indicates the username and password that matches the server login credentials to interact with the database. | |
| driver_name |
MSSQL
com.microsoft. sqlserver.jdbc.SQLS erverDriver
MYSQL com.mysql.jdbc. Driver ORACLE oracle.jdbc.OracleDr iver |
The driver provides Java database connectivity from any Java application, application server, or Java - enabled applet.
MSSQL
MYSQL
ORACLE
|
|
| dialect |
MSSQL org.hibernate.dialect. SQLServer2012Diale ct MYSQL org.hibernate.dialect. MySQL5InnoDBDiale ct ORACLE org.hibernate.dialect. Oracle8iDialect |
For connecting any hibernate application with the database, it is required to provide the configuration of SQL dialect. | |
| db_connection_url |
MSSQL jdbc:sqlserver://event store-db-service: 1433; databaseName=ms_ eventstore MYSQL jdbc:mysql://eventsto re-db-service-np: 3306/ms_eventstore ORACLE jdbc:oracle:thin: @host.docker. internal:1521:orcl |
The general form of the connection URL is as follows:
MSSQL: jdbc:sqlserver://<host_or_ip>:1433;databaseName=<db_name>
MYSQL: jdbc:mysql://<host_or_ip>:3306/<db_name>
ORACLE: jdbc:oracle:thin:@<host_or_ip>:1521:<db_name>
host_or_ip: This is the address (optional) of the server to connect to. This address can be a DNS or IP address, or it can be localhost or 127.0.0.1 for the local computer. If not specified in the connection URL, the server name must be specified in the properties collection.
port number: This is the port (optional) to connect to on serverName. The default is 1433 for mssql , 3306 for mysql , and 1521 for oracle. If the user is using the default, do not specify the port, nor the preceding : in the URL. db_name: This is the name of the database to be used in sql server. |
|
| dbinit_Connection_Url | MSSQL jdbc:sqlserver://event store-db-service. eventstore.svc. cluster.local:1433 MYSQL jdbc:mysql://eventsto re-db-service-np. eventstore.svc. cluster.local:3306 ORACLE: jdbc:oracle:thin: @host.docker. internal:1521:orcl | DB initialization utility URL is used to connect the database. This is used to create tables inside sql database. | |
|
max_pool_size min_pool_size |
MSSQL & MYSQL
5 1
ORACLE
150 10 |
Indicates the maximum number of connections maintained in the pool. Indicates the minimum number of connections maintained in the pool. |
|
|
temn_msf_db_pass_encryption_key
temn_msf_db_pass_encryption_algorithm |
temenos PBEWithMD5A ndTripleDES |
Indicates the key or password and algorithm to encrypt the plain text. These two values are mandatory to encrypt the plain text. |
|
| Kafka | |||
| kafkabootstrapservers | my-cluster-kafka- bootstrap.kafka:9092 |
This contains a list of host or port pairs to establish the initial connection of kafka cluster. A host and port pair uses : as the separator.
|
|
| temnmsfstreamvendor | kafka | Specifies the vendor used to process messages to a topic. | |
| temnqueueimpl | kafka | Specifies the queue used to identify the listener container (processing messages). | |
| schema_registry_url | http://schema- registry-svc.kafka. svc.cluster.local | Schema Registry is an application that resides outside of the Kafka cluster and handles the distribution of schemas to the producer and consumer by storing a copy of the schema in its local cache. Schema registry URL is used to connect schema registry in Kafka. | |
| kafka_Aliases | N | Y, N |
hostAliases are used to overwrite the resolution of the host name and IP at the pod level when adding entries to the /etc/hosts file.
To enable hostAliases, set the below variables to Y. |
|
kafkaip
kafka0ip
kafka1ip kafka2ip |
Set the variables of Kafka IP for hostAliases. | ||
|
kafka_Host_Name
kafka0_Host_Name
kafka1_Host_Name kafka2_Host_Name |
Set the variables of Kafka hostname for hostAliases. | ||
| Security | |||
| temn_msf_security_authz_enabled | false | true, false | Enable or disable the XACML policy authorization. XACML policy authorization is used to configure security policies and access rights to information for APIs. |
| Scheduler Properties | |||
| temn_msf_scheduler_inboxcleanup_schedule | 60 |
Specifies the minutes required to hold the records inside ms_inbox_events table before automatic deletion. |
|
| temn_msf_scheduler_sqlInboxCatchup_cron_trigger_time | 0 0/1 * 1/1 * ? | Indicates the frequency to trigger the scheduler job is set in this property and cron trigger time for sql inbox catchup. | |
| temn_msf_scheduler_sqlOutboxCatchup_cron_trigger_time | 0 0/1 * 1/1 * ? | Indicates the frequency to trigger the scheduler job is set in this property and cron trigger time for sql outbox catchup. | |
| schedule | 5 | Indicates the frequency to trigger the scheduler job is set in this property. | |
| Thread Pool | |||
| tmn_outbox_thread_pool_count | 10 | Indicates the number of thread counts required to process the outbox events. | |
| Logger | |||
| root_level | INFO | DEBUG, INFO, ERROR | Specifies the configuration parameter for logging. INFO logger level is maintained by default. |
| SSL | |||
|
temn_msf_stream_security_kafka_ssl_keystore_location:
temn_msf_stream_security_kafka_ssl_keystore_password:
temn_msf_stream_security_kafka_ssl_key_password:
temn_msf_stream_security_kafka_ssl_truststore_location: temn_msf_stream_security_kafka_ssl_truststore_password: |
Keystore stores private key and identity certificates that a specific program should present to both parties (server or client) for verification. Trust store is used to store certificates from Certified Authorities (CA) that verify the certificate presented by the server in SSL connection which is commented by default. If required, kindly enable them. | ||
| Ports | Specifies the port, name, protocol, target port for the eventstore ingester, and api service. | ||
| name | http | This is a formally defined set of rules for communication between a client (the network resource requesting data or services) and a server (the resource that receives and responds to the request). | |
| port | 8080 | A port number is a way to identify a specific process to which internet or other network message is to be forwarded to a server. | |
| protocol | TCP | The default protocol for services is TCP (other supported protocols can also be used). | |
| target port | 8080 | This service can map any incoming port to a target port. By default, and for convenience, the target port is set to the same value as the port field. | |
| Service | |||
|
eventstoreapi:
type: LoadBalancer
port: 7004
eventstoreingester:
type: LoadBalancer port: 7010 |
This service in Kubernetes is a REST object, similar to a pod. Like all REST objects, the service definition can be posted to the API server to create a new instance.
Specifies the port, type for the eventstore API, and ingester service.
When the Service type is set to load balancer, Kubernetes provides functionality equivalent to type equals ClusterIP to pods within the cluster and extends it by programming the load balancer (external to Kubernetes) with entries for the nodes hosting the relevant Kubernetes pods. |
||
| Logging | |||
|
# socketHost: "192.168.1.37" # socketPort: "4560" exportType: #"socket" protocol: #TCP |
Logging can be collecting, storing, and analyzing distributed logs. Here log stash is used for creating an ingestion pipeline for collecting logs from multiple devices and stored in elastic search for visualizing and analyzing through Grafana.
To enable logging sockethost, SockerPort, and exportType, the protocol should be uncommented, and the required port and host address should be given. |
||
| Metrics | |||
|
#publisherPort: 9091 #publisherHost: 192.168.228.193 metricsDisabled: "true" |
Metrics give us insights into the historical and current state of a system. Data instrumented from microservices are pushed to the PushGateway service. Prometheus collects data from PushGateway at a regular interval. Grafana is used to visualize data in UI.
To enable monitoring in micro services publisherPort, publisherHost can be configured with a push gateway port and host address and metricsDisabled can be changed to false (set to true by default).
Metrics is disabled by default. |
||
| Resources | |||
|
api:
cpulimit: "100m" memorylimit: 1.5G java: heaplimit: ' -XX: +UnlockExperimentalVMOptions - XX: +UseCGroupMemoryLimitForHeap -XX:MaxRAMPercentage=75.0 ' ingester: cpulimit: "100m" memorylimit: 1.5G java: heaplimit: ' -XX: +UnlockExperimentalVMOptions - XX: +UseCGroupMemoryLimitForHeap -XX:MaxRAMPercentage=75.0 ' |
Resource management for pods or containers - Kubernetes uses to control resources such as CPU and memory, can declare limits for CPU and for all pod containers.
|
||
|
Delivery cpulimit: "100m" memorylimit: 1.5G java: heaplimit: ' -XX: +UnlockExperimentalVMOptions - XX: +UseCGroupMemoryLimitForHeap -XX:MaxRAMPercentage=75.0 ' |
|||
|
Scheduler cpulimit: "100m" memorylimit: 1.5G java: heaplimit: ' -XX: +UnlockExperimentalVMOptions - XX: +UseCGroupMemoryLimitForHeap -XX:MaxRAMPercentage=75.0 ' |
In this topic