MinIO tenant

Template version:v24-04-20

MinIO image version:quay.io/minio/minio:RELEASE.2024-04-06T05-26-02Z

MinIO is a Kubernetes-native high performance object store with an S3-compatible API. The MinIO Kubernetes Operator supports deploying MinIO Tenants onto private and public cloud infrastructures ("Hybrid" Cloud).

This service creates a X-node MinIO Tenant using MinIO for object storage.

Template override parameters

File _values-tpl.yaml contains template configuration parameters and their default values:

#
#   _values-tpl.yaml
#
#   cskygen template default values file
#
_tplname: k8s-minio-tenant
_tpldescription: Kubernetes MinIO Tenant
_tplversion: 24-04-20
#
# Values to override
#
## k8s cluster credentials kubeconfig file
kubeconfig: config-k8s-mod
namespace:
  ## k8s namespace name
  name: minio-tenant
publishing:
  ## External api url used by mc
  miniourl: minio-tenant.cskylab.net
  ## External api url used by console
  consoleurl: minio-tenant-console.cskylab.net
credentials:
  # MinIO root user credentials
  minio_accesskey: "admin"
  minio_secretkey: "NoFear21"
certificate:
  ## Cert-manager clusterissuer
  clusterissuer: ca-test-internal
registry:
  ## Proxy Repository for Docker
  proxy: harbor.cskylab.net/dockerhub
localpvnodes:    # (k8s node names)
  srv00: k8s-mod-n1
  srv01: k8s-mod-n2
  srv02: k8s-mod-n3
  srv03: k8s-mod-n4
  # k8s nodes domain name
  domain: cskylab.net
  # k8s nodes local administrator
  localadminusername: kos

TL;DR

Prepare LVM Data services for PV's:

• LVM Data Services

Install namespace and charts:

# Install  
./csdeploy.sh -m install
# Check status
./csdeploy.sh -l

Access:

• Console URL: {{ .publishing.consoleurl }}
• MinIO URL: {{ .publishing.miniourl }}
• AccessKey: {{ .credentials.minio_accesskey }}
• SecretKey: {{ .credentials.minio_secretkey }}

Prerequisites

• MinIO Operator must be installed in K8s Cluster.
• Administrative access to Kubernetes cluster.

Persistence

MinIO Erasure Code Parity (EC:N)

The minimum configuration for a standalone tenant must be 1 server with 4 drives. For a distributed tenant the minimum is 4 servers with 4 drives each one (Total 16 drives).

The reccomended Erasure code is EC:4

Persistent Volumes

Review values in all Persistent volume manifests with the name format ./pv-*.yaml.

The following PersistentVolume & StorageClass manifests are applied:

# PV manifests
pv-minio.yaml

The node assigned in nodeAffinity section of the PV manifest, will be used when scheduling the pod that holds the service.

LVM Data Services

Data services are supported by the following nodes:

To create the corresponding LVM data services, execute from your mcc management machine the following command:

#
# Create LVM data services
#
echo && echo "******** SOE - START of execution ********" && echo \
&& ssh {{ .localpvnodes.localadminusername }}@{{ .localpvnodes.srv00 }}.{{ .localpvnodes.domain }} \
  'sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s00-d00" \
&& sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s00-d01" \
&& sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s00-d02" \
&& sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s00-d03"' \
&& ssh {{ .localpvnodes.localadminusername }}@{{ .localpvnodes.srv01 }}.{{ .localpvnodes.domain }} \
  'sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s01-d00" \
&& sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s01-d01" \
&& sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s01-d02" \
&& sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s01-d03"' \
&& ssh {{ .localpvnodes.localadminusername }}@{{ .localpvnodes.srv02 }}.{{ .localpvnodes.domain }} \
  'sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s02-d00" \
&& sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s02-d01" \
&& sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s02-d02" \
&& sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s02-d03"' \
&& ssh {{ .localpvnodes.localadminusername }}@{{ .localpvnodes.srv03 }}.{{ .localpvnodes.domain }} \
  'sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s03-d00" \
&& sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s03-d01" \
&& sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s03-d02" \
&& sudo cs-lvmserv.sh -m create -qd "/srv/{{ .namespace.name }}-s03-d03"' \
&& echo && echo "******** EOE - END of execution ********" && echo

To delete the corresponding LVM data services, execute from your mcc management machine the following command:

#
# Delete LVM data services
#
echo && echo "******** SOE - START of execution ********" && echo \
&& ssh {{ .localpvnodes.localadminusername }}@{{ .localpvnodes.srv00 }}.{{ .localpvnodes.domain }} \
  'sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s00-d00" \
&& sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s00-d01" \
&& sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s00-d02" \
&& sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s00-d03"' \
&& ssh {{ .localpvnodes.localadminusername }}@{{ .localpvnodes.srv01 }}.{{ .localpvnodes.domain }} \
  'sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s01-d00" \
&& sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s01-d01" \
&& sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s01-d02" \
&& sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s01-d03"' \
&& ssh {{ .localpvnodes.localadminusername }}@{{ .localpvnodes.srv02 }}.{{ .localpvnodes.domain }} \
  'sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s02-d00" \
&& sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s02-d01" \
&& sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s02-d02" \
&& sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s02-d03"' \
&& ssh {{ .localpvnodes.localadminusername }}@{{ .localpvnodes.srv03 }}.{{ .localpvnodes.domain }} \
  'sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s03-d00" \
&& sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s03-d01" \
&& sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s03-d02" \
&& sudo cs-lvmserv.sh -m delete -qd "/srv/{{ .namespace.name }}-s03-d03"' \
&& echo && echo "******** EOE - END of execution ********" && echo

How-to guides

Install

Check versions and update if necesary, for the following image in mod-tenant.yaml file:

  ## Registry location and Tag to download MinIO Server image
  image: quay.io/minio/minio:RELEASE.2024-04-06T05-26-02Z

To check for the latest image version see:

• https://quay.io/minio/minio

To create namespace and install MinIO tenant:

  # Install namespace and tenant
    ./csdeploy.sh -m install

Update

Check image versions and update if necesary.

To reapply manifests and upgrade images:

  # Reapply manifests and update images
    ./csdeploy.sh -m update

Delete

WARNING: This action will delete MinIO tenant. All MinIO Storage Data in K8s Cluster will be erased.

To delete MinIO tenant, remove namespace and PV's run:

  # Delete tenant, PV's and namespace
    ./csdeploy.sh -m delete

Remove

This option is intended to be used only to remove the namespace when tenant deployment is failed. Otherwise, you must run ./csdeploy.sh -m delete.

To remove PV's, namespace and all its contents run:

  # Remove PV's namespace and all its contents
    ./csdeploy.sh -m remove

Display status

To display namespace, persistence and tenant status run:

  # Display namespace, persistence and tenant status
    ./csdeploy.sh -l

Bucket maintenance

Buckets can be created together with users and policies for ReadWrite, ReadOnly and WriteOnly access.

A record file in configuration management ./buckets folder will be created for each bucket in the form bucket_name.config.

Additionally, a source environment file for MinIO bucket access and restic operations will be created in the form source-bucket_name.sh. This file can be used for restic backups operations with the script csrestic-minio.sh. It can be sourced also from a management console to initialize the variables needed to access bucket through MinIO client mc and restic repository through restic commands.

Create bucket, users and policies

To create bucket, users and policies:

# Create Bucket & Users & Policies
  ./csbucket.sh -c mybucket

In this case a file named ./buckets/mybucket.config will be created with the access and secret keys used for the following users:

• mybucket_rw (ReadWrite user)
• mybucket_ro (ReadOnly user)
• mybucket_wo (Write only user)

You can use these keys for specific access to the bucket from any application or user.

Delete bucket, users and policies

To delete bucket, users and policies:

# Delete Bucket & Users & Policies
  ./csbucket.sh -d mybucket

File ./buckets/mybucket.config will also be deleted with access and secret keys.

Display bucket, users and policies

To list current bucket, users and policies:

# List Bucket & Users & Policies
  ./csbucket.sh -l

MinIO Client

Console utility

To acces MinIO console throug web utility:

• Console URL: {{ .publishing.consoleurl }}
• AccessKey: {{ .credentials.console_accesskey }}
• SecretKey: {{ .credentials.console_secretkey }}

Command line utility

If you have minio client installed, you can access mc command line utiliy from the command line.

File .envrc export automatically through "direnv" the environment variable needed to operate mc with minio as hostname from its directory in git repository:

# MinIO host environment variable
export MC_HOST_minio="https://{{ .credentials.minio_accesskey }}:{{ .credentials.minio_secretkey }}@{{ .publishing.miniourl }}"

You can run mc commands to operate from console with buckets and files: Ex mc tree minio.

For more information: https://docs.min.io/docs/minio-client-complete-guide.html

Reference

To learn more see:

• https://github.com/kubernetes/ingress-nginx

Scripts

cs-deploy

Purpose:
  Kubernetes MinIO Tenant.
  
Usage:
  sudo csdeploy.sh [-l] [-m <execution_mode>] [-h] [-q]
Execution modes:
  -l  [list-status]     - List namespace, persistence and tenant status.
  -m  <execution_mode>  - Valid modes are:
      [install]         - Install namespace and MinIO tenant.
      [update]          - Reapply manifests and update images.
      [delete]          - Delete MinIO tenant, remove PV's and namespace.
      [remove]          - Remove PV's namespace and all its contents.
Options and arguments:  
  -h  Help
  -q  Quiet (Nonstop) execution.
Examples:
  # Install namespace and tenant
    ./csdeploy.sh -m install
  # Reapply manifests and update images
    ./csdeploy.sh -m update
  # Delete tenant, PV's and namespace
    ./csdeploy.sh -m delete
  # Remove PV's namespace and all its contents
    ./csdeploy.sh -m remove
  # Display namespace, persistence and tenant status
    ./csdeploy.sh -l

Tasks performed:

csbucket

Purpose:
  Minio Bucket & User & Policy maintenance.
  Use this script to create or delete together a bucket
  with readwrite, readonly and writeonly users and access policies.
  
Usage:
  sudo csdeploy.sh [-l] [-c <bucket_name>] [-d <bucket_name>] [-h] [-q]
Execution modes:
  -l  [list-status]     - List Buckets & Users & Policies.
  -c  <bucket_name>     - Create Bucket & Users & Policies
  -d  <bucket_name>     - Remove Bucket & Users & Policies
Options and arguments:  
  -h  Help
  -q  Quiet (Nonstop) execution.
Examples:
  # Create Bucket & Users & Policies
    ./csbucket.sh -c mybucket
  # Delete Bucket & Users & Policies
    ./csbucket.sh -d mybucket

Tasks performed:

Template values

The following table lists template configuration parameters and their specified values, when machine configuration files were created from the template:

License

Copyright © 2021 cSkyLab.com ™

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.