Helm을 이용해 K8S에 MariaDB Galera 설치하기

테스트 목적
K8S환경에서 MariaDB Galera클러스터의 기능 및 성능 테스트

테스트 내용
1. MariaDB Galera 소개
2. MariaDB Galera With helm 설치
3. 접속 및 ScaleOut 테스트
4. 장애 테스트 (작성예정)
5. 부하 테스트 (작성예정)

---

 

1. MariaDB Galera 소개

https://mariadb.com/kb/en/what-is-mariadb-galera-cluster

MariaDB Galera 클러스터는 MariaDB에서 지원하는 멀티 프라이머리 클러스터입니다. 기본적으로, InnoDB 스토리지 엔진을  지원하며 아래와 같은 특징이 있습니다.

• 액티브-액티브 멀티 프라이머리 토폴로지
• 자동으로 장애가 발생한 노드가 클러스터에서 제외
• 자동으로 노드의 싱크 및 클러스터 참여 : rsync방식, mariabackup방식을 지원.
• 가상 동기방식 리플리케이션 : Galera Cluster는 클러스터간의 데이터 전파 사이에 정합성의 gap이 발생할 수 있음

최근 빠른(?) 버전업그레이드로 안정성이 나날이 좋아지고 있으며, 손쉬운 노드 추가 및 삭제가 가능하여 k8s의 컨셉과 유사한점이 있습니다. 

 

2. MariaDB Galera packaged by Bitnami 설치를 위한 Helm REPO추가 및 업데이트
(https://github.com/bitnami/charts/tree/master/bitnami/mariadb-galera)

$ helm repo add bitnami https://charts.bitnami.com/bitnami
$ helm repo update

 

3.  MariaDB Galera설치

$ helm install mariadb bitnami/mariadb-galera    \
--set rootUser.password=RooTPassw \
--set galera.mariabackup.password=BackPass

## 참고01 
  -- password : RooTPassw

 

4. 모니터링

$ watch kubectl get sts,pod,pvc,svc -n mariadb-galera -o wide

NAME                                      READY   AGE    CONTAINERS       IMAGES
statefulset.apps/mariadb-mariadb-galera   3/3     3m1s   mariadb-galera   docker.io/bitnami/mariadb-galera:10.6.8-debian-11-r3

NAME                           READY   STATUS    RESTARTS   AGE    IP           NODE     NOMINATED NODE   READINESS GATES
pod/mariadb-mariadb-galera-0   1/1     Running   0          3m     172.16.3.3   k8s-w2   <none>           <none>
pod/mariadb-mariadb-galera-1   1/1     Running   0          2m1s   172.16.2.3   k8s-w1   <none>           <none>
pod/mariadb-mariadb-galera-2   1/1     Running   0          61s    172.16.1.5   k8s-w3   <none>           <none>

NAME                                                  STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE    VOLUMEMODE
persistentvolumeclaim/data-mariadb-mariadb-galera-0   Bound    pvc-5334c727-b261-49ef-8478-0e60cc64a77a   8Gi        RWO            local-path     3m     Filesystem
persistentvolumeclaim/data-mariadb-mariadb-galera-1   Bound    pvc-3f06fd10-ba5d-493b-a930-2c657b14698b   8Gi        RWO            local-path     2m1s   Filesystem
persistentvolumeclaim/data-mariadb-mariadb-galera-2   Bound    pvc-97d63b6c-abe8-46cd-9836-5628ec01623c   8Gi        RWO            local-path     61s    Filesystem

NAME                                      TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE     SELECTOR
service/kubernetes                        ClusterIP   10.200.1.1     <none>        443/TCP                      5m10s   <none>
service/mariadb-mariadb-galera            ClusterIP   10.200.1.140   <none>        3306/TCP                     3m1s    app.kubernetes.io/instance=mariadb,app.kubernetes.io/name=mariadb-galera
service/mariadb-mariadb-galera-headless   ClusterIP   None           <none>        4567/TCP,4568/TCP,4444/TCP   3m1s    app.kubernetes.io/instance=mariadb,app.kubernetes.io/name=mariadb-galera

 

5. 접속 방법 확인 : 설치가 진행되면 친절하게 접속방법까지 나온다.

NAME: mariadb
LAST DEPLOYED: Sat Jun 11 22:53:11 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: mariadb-galera
CHART VERSION: 7.3.5
APP VERSION: 10.6.8

** Please be patient while the chart is being deployed **
Tip:

  Watch the deployment status using the command:

    kubectl get sts -w --namespace default -l app.kubernetes.io/instance=mariadb

MariaDB can be accessed via port "3306" on the following DNS name from within your cluster:

    mariadb-mariadb-galera.default.svc.cluster.local

To obtain the password for the MariaDB admin user run the following command:

    echo "$(kubectl get secret --namespace default mariadb-mariadb-galera -o jsonpath="{.data.mariadb-root-password}" | base64 -d)"

To connect to your database run the following command:

    kubectl run mariadb-mariadb-galera-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/mariadb-galera:10.6.8-debian-11-r3 --command \
      -- mysql -h mariadb-mariadb-galera -P 3306 -uroot -p$(kubectl get secret --namespace default mariadb-mariadb-galera -o jsonpath="{.data.mariadb-root-password}" | base64 -d) my_database

To connect to your database from outside the cluster execute the following commands:

    kubectl port-forward --namespace default svc/mariadb-mariadb-galera 3306:3306 &
    mysql -h 127.0.0.1 -P 3306 -uroot -p$(kubectl get secret --namespace default mariadb-mariadb-galera -o jsonpath="{.data.mariadb-root-password}" | base64 -d) my_database

To upgrade this helm chart:

    helm upgrade --namespace default mariadb bitnami/mariadb-galera \
      --set rootUser.password=$(kubectl get secret --namespace default mariadb-mariadb-galera -o jsonpath="{.data.mariadb-root-password}" | base64 -d) \
      --set db.name=my_database \
      --set galera.mariabackup.password=$(kubectl get secret --namespace default mariadb-mariadb-galera -o jsonpath="{.data.mariadb-galera-mariabackup-password}" | base64 -d)

 

6. 접속 및 부하분산 확인

$ kubectl run mariadb-mariadb-galera-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/mariadb-galera:10.6.8-debian-11-r3 --command \
-- mysql -h mariadb-mariadb-galera -P 3306 -uroot -p$(kubectl get secret --namespace default mariadb-mariadb-galera -o jsonpath="{.data.mariadb-root-password}" | base64 -d) -e "show variables like 'wsrep_node_name' ; " 


$ run mariadb-mariadb-galera-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/mariadb-galera:10.6.8-debian-11-r3 --command \
-- mysql -h mariadb-mariadb-galera -P 3306 -uroot -p$(kubectl get secret --namespace default mariadb-mariadb-galera -o jsonpath="{.data.mariadb-root-password}" | base64 -d) -e "show variables like 'wsrep_node_name' ; " 
+-----------------+--------------------------+
| Variable_name   | Value                    |
+-----------------+--------------------------+
| wsrep_node_name | mariadb-mariadb-galera-2 |
+-----------------+--------------------------+
pod "mariadb-mariadb-galera-client" deleted

$ kubectl run mariadb-mariadb-galera-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/mariadb-galera:10.6.8-debian-11-r3 --command \
-- mysql -h mariadb-mariadb-galera -P 3306 -uroot -p$(kubectl get secret --namespace default mariadb-mariadb-galera -o jsonpath="{.data.mariadb-root-password}" | base64 -d) -e "show variables like 'wsrep_node_name' ; " 

+-----------------+--------------------------+
| Variable_name   | Value                    |
+-----------------+--------------------------+
| wsrep_node_name | mariadb-mariadb-galera-1 |
+-----------------+--------------------------+
pod "mariadb-mariadb-galera-client" deleted

$ kubectl run mariadb-mariadb-galera-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/mariadb-galera:10.6.8-debian-11-r3 --command \
-- mysql -h mariadb-mariadb-galera -P 3306 -uroot -p$(kubectl get secret --namespace default mariadb-mariadb-galera -o jsonpath="{.data.mariadb-root-password}" | base64 -d) -e "show variables like 'wsrep_node_name' ; " 

+-----------------+--------------------------+
| Variable_name   | Value                    |
+-----------------+--------------------------+
| wsrep_node_name | mariadb-mariadb-galera-0 |
+-----------------+--------------------------+

 

7. 파드 갯수 조정

$ root@k8s-m:~# kubectl scale sts mariadb-mariadb-galera   --replicas=5
statefulset.apps/mariadb-mariadb-galera scaled

 

8. 파드 증가 확인

$ kubectl get sts,pod,pvc,svc -o wide   
NAME                                      READY   AGE   CONTAINERS       IMAGES
statefulset.apps/mariadb-mariadb-galera   5/5     27m   mariadb-galera   docker.io/bitnami/mariadb-galera:10.6.8-debian-11-r3

NAME                           READY   STATUS    RESTARTS   AGE    IP            NODE     NOMINATED NODE   READINESS GATES
pod/mariadb-mariadb-galera-0   1/1     Running   0          27m    172.16.3.3    k8s-w2   <none>           <none>
pod/mariadb-mariadb-galera-1   1/1     Running   0          26m    172.16.2.3    k8s-w1   <none>           <none>
pod/mariadb-mariadb-galera-2   1/1     Running   0          25m    172.16.1.5    k8s-w3   <none>           <none>
pod/mariadb-mariadb-galera-3   1/1     Running   0          102s   172.16.2.7    k8s-w1   <none>           <none>
pod/mariadb-mariadb-galera-4   1/1     Running   0          57s    172.16.3.12   k8s-w2   <none>           <none>

NAME                                                  STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE    VOLUMEMODE
persistentvolumeclaim/data-mariadb-mariadb-galera-0   Bound    pvc-5334c727-b261-49ef-8478-0e60cc64a77a   8Gi        RWO            local-path     27m    Filesystem
persistentvolumeclaim/data-mariadb-mariadb-galera-1   Bound    pvc-3f06fd10-ba5d-493b-a930-2c657b14698b   8Gi        RWO            local-path     26m    Filesystem
persistentvolumeclaim/data-mariadb-mariadb-galera-2   Bound    pvc-97d63b6c-abe8-46cd-9836-5628ec01623c   8Gi        RWO            local-path     25m    Filesystem
persistentvolumeclaim/data-mariadb-mariadb-galera-3   Bound    pvc-5300a93f-d360-4499-9954-742e027fe138   8Gi        RWO            local-path     102s   Filesystem
persistentvolumeclaim/data-mariadb-mariadb-galera-4   Bound    pvc-8bfff755-6f0f-416f-a4b0-92456c907ecb   8Gi        RWO            local-path     57s    Filesystem

 

9. 노드포트 설정하여 외부에서 접속하기 : helm 설치 시 파라미터 추가로 nodeport설정 가능

$ helm install mariadb bitnami/mariadb-galera    \
--set rootUser.password=RooTPassw54 \
--set galera.mariabackup.password=BakPass93  \
--set service.type=NodePort \
--set service.nodePorts.mysql=32000

 

10. 생성 된 노드포트 확인

$ kubectl get svc mariadb-mariadb-galera
NAME                     TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)          AGE
mariadb-mariadb-galera   NodePort   10.200.1.97   <none>        3306:32000/TCP   19m

 

11. 노드포트를 사용하여 퍼블릭으로 접속하여도 로드밸런싱이 된다.