Организация Kustomize конфигураций и компонентов

Структура Kubernetes инфраструктуры

Обзор

Директория infra/k8s содержит все Kubernetes конфигурации платформы Maniton, организованные по принципам Kustomize.

Структура директорий

infra/k8s/
├── base/                      # Базовые ресурсы
│   ├── kustomization.yaml
│   ├── namespace.yaml
│   └── service-cidr.yaml
├── components/                # Переиспользуемые компоненты
│   ├── apps/                 # Приложения
│   │   ├── auth-service/
│   │   ├── cfa-core/
│   │   ├── ledger-service/
│   │   ├── market-service/
│   │   ├── payments-service/
│   │   └── besu-connector/
│   ├── databases/            # Базы данных
│   │   ├── postgres/
│   │   └── redis/
│   ├── dlt/                  # Блокчейн
│   │   └── besu/
│   ├── messaging/            # Сообщения
│   │   └── kafka/
│   ├── observability/        # Мониторинг
│   │   ├── prometheus/
│   │   ├── grafana/
│   │   └── loki/
│   ├── security/             # Безопасность
│   │   ├── network-policies/
│   │   └── resource-quotas/
│   └── ingress/              # Входящий трафик
│       └── nginx/
├── environments/             # Окружения
│   ├── dev/                  # Разработка
│   ├── dev-full/             # Разработка (полный)
│   └── production/           # Продакшн
├── clusters/                 # Кластеры
│   └── moneyton/
├── helm/                     # Helm чарты
└── Makefile                  # Команды управления

Компоненты

Apps

Микросервисы платформы Maniton.

Auth Service

apiVersion: apps/v1
kind: Deployment
metadata:
  name: auth-service
  namespace: maniton
spec:
  replicas: 2
  selector:
    matchLabels:
      app: auth-service
  template:
    metadata:
      labels:
        app: auth-service
    spec:
      containers:
      - name: auth-service
        image: ghcr.io/maniton/auth-service:latest
        ports:
        - containerPort: 3001
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: auth-service-secrets
              key: database-url
        - name: KAFKA_BROKERS
          value: "kafka:9092"
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "1Gi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /health
            port: 3001
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health
            port: 3001
          initialDelaySeconds: 5
          periodSeconds: 5

CFA Core Service

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cfa-core
  namespace: maniton
spec:
  replicas: 2
  selector:
    matchLabels:
      app: cfa-core
  template:
    metadata:
      labels:
        app: cfa-core
    spec:
      containers:
      - name: cfa-core
        image: ghcr.io/maniton/cfa-core:latest
        ports:
        - containerPort: 3002
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: cfa-core-secrets
              key: database-url
        - name: KAFKA_BROKERS
          value: "kafka:9092"
        - name: BESU_RPC_URL
          value: "http://besu-node1:8545"
        resources:
          requests:
            memory: "1Gi"
            cpu: "1000m"
          limits:
            memory: "2Gi"
            cpu: "2000m"
        livenessProbe:
          httpGet:
            path: /health
            port: 3002
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health
            port: 3002
          initialDelaySeconds: 5
          periodSeconds: 5

Databases

PostgreSQL

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
  namespace: maniton
spec:
  serviceName: postgres
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:16-alpine
        ports:
        - containerPort: 5432
        env:
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secrets
              key: password
        - name: POSTGRES_DB
          value: maniton
        volumeMounts:
        - name: postgres-data
          mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
  - metadata:
      name: postgres-data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 200Gi

Redis

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: maniton
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:7-alpine
        ports:
        - containerPort: 6379
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"

DLT

Hyperledger Besu

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: besu
  namespace: maniton
spec:
  serviceName: besu
  replicas: 4
  selector:
    matchLabels:
      app: besu
  template:
    metadata:
      labels:
        app: besu
    spec:
      containers:
      - name: besu
        image: hyperledger/besu:latest
        ports:
        - containerPort: 8545
        - containerPort: 30303
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        command:
        - /opt/besu/bin/besu
        - --config-file=/config/config.toml
        - --data-path=/data
        volumeMounts:
        - name: besu-data
          mountPath: /data
        - name: besu-config
          mountPath: /config
  volumeClaimTemplates:
  - metadata:
      name: besu-data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 100Gi

Messaging

Kafka

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: kafka
  namespace: maniton
spec:
  serviceName: kafka
  replicas: 3
  selector:
    matchLabels:
      app: kafka
  template:
    metadata:
      labels:
        app: kafka
    spec:
      containers:
      - name: kafka
        image: confluentinc/cp-kafka:latest
        ports:
        - containerPort: 9092
        - containerPort: 29092
        env:
        - name: KAFKA_BROKER_ID
          value: "1"
        - name: KAFKA_ZOOKEEPER_CONNECT
          value: "zookeeper:2181"
        - name: KAFKA_ADVERTISED_LISTENERS
          value: "PLAINTEXT://kafka:29092"
        - name: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR
          value: "1"
        volumeMounts:
        - name: kafka-data
          mountPath: /var/lib/kafka/data
  volumeClaimTemplates:
  - metadata:
      name: kafka-data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 100Gi

Observability

Prometheus

apiVersion: apps/v1
kind: Deployment
metadata:
  name: prometheus
  namespace: maniton
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus
  template:
    metadata:
      labels:
        app: prometheus
    spec:
      containers:
      - name: prometheus
        image: prom/prometheus:latest
        ports:
        - containerPort: 9090
        args:
        - '--config.file=/etc/prometheus/prometheus.yml'
        - '--storage.tsdb.path=/prometheus'
        volumeMounts:
        - name: prometheus-config
          mountPath: /etc/prometheus
        - name: prometheus-data
          mountPath: /prometheus

Grafana

apiVersion: apps/v1
kind: Deployment
metadata:
  name: grafana
  namespace: maniton
spec:
  replicas: 1
  selector:
    matchLabels:
      app: grafana
    template:
    metadata:
      labels:
        app: grafana
    spec:
      containers:
      - name: grafana
        image: grafana/grafana:latest
        ports:
        - containerPort: 3000
        env:
        - name: GF_SECURITY_ADMIN_PASSWORD
          valueFrom:
            secretKeyRef:
              name: grafana-secrets
              key: admin-password
        volumeMounts:
        - name: grafana-data
          mountPath: /var/lib/grafana

Loki

apiVersion: apps/v1
kind: Deployment
metadata:
  name: loki
  namespace: maniton
spec:
  replicas: 1
  selector:
    matchLabels:
      app: loki
    spec:
      containers:
      - name: loki
        image: grafana/loki:latest
        ports:
        - containerPort: 3100
        args:
        - '-config.file=/etc/loki/local-config.yaml'
        volumeMounts:
        - name: loki-config
          mountPath: /etc/loki
        - name: loki-data
          mountPath: /loki

Окружения

Development

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: maniton-dev

resources:
- ../../base/namespace.yaml
- ../../components/databases/postgres/
- ../../components/databases/redis/
- ../../components/messaging/kafka/
- ../../components/dlt/besu/
- ../../components/observability/prometheus/
- ../../components/observability/grafana/
- ../../components/observability/loki/

replicas:
- name: postgres
  count: 1
- name: kafka
  count: 1
- name: besu
  count: 1

Production

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: maniton-prod

resources:
- ../../base/namespace.yaml
- ../../components/databases/postgres/
- ../../components/databases/redis/
- ../../components/messaging/kafka/
- ../../components/dlt/besu/
- ../../components/observability/prometheus/
- ../../components/observability/grafana/
- ../../components/observability/loki/
- ../../components/apps/auth-service/
- ../../components/apps/cfa-core/
- ../../components/apps/ledger-service/
- ../../components/apps/market-service/
- ../../components/apps/payments-service/

replicas:
- name: postgres
  count: 1
- name: kafka
  count: 3
- name: besu
  count: 4
- name: auth-service
  count: 2
- name: cfa-core
  count: 2
- name: ledger-service
  count: 2
- name: market-service
  count: 2
- name: payments-service
  count: 2

images:
- name: auth-service
  newName: ghcr.io/maniton/auth-service
  newTag: v1.2.0
- name: cfa-core
  newName: ghcr.io/maniton/cfa-core
  newTag: v1.2.0
- name: ledger-service
  newName: ghcr.io/maniton/ledger-service
  newTag: v1.2.0
- name: market-service
  newName: ghcr.io/maniton/market-service
  newTag: v1.2.0
- name: payments-service
  newName: ghcr.io/maniton/payments-service
  newTag: v1.2.0

Makefile

Основные команды:

# Установка зависимостей
make install-deps

# Проверка зависимостей
make check-deps

# Валидация манифестов
make validate

# Линтер
make lint

# Проверка качества
make score

# Тесты
make test

# Деплой в dev
make deploy-dev

# Деплой в dev-full
make deploy-dev-apps

# Деплой в prod
make deploy-prod

# Статус
make status

# Логи
make logs APP=auth

# Очистка
make clean

# Полная очистка
make clean-all

# Проброс портов
make port-forward

# Проверка здоровья
make health

# Исправление IP сервиса
make fix-service-ip

# Очистка дубликатов
make cleanup-duplicates

Best Practices

Разделение ответственности: Base, components, environments, clusters
DRY принцип: Переиспользуемые компоненты
Environment-specific patches: Стратегические патчи для различий
Согласованное именование: Следование конвенциям Kustomize
Безопасность: Security contexts, non-root контейнеры, resource limits
Наблюдаемость: Комплексный мониторинг и логирование
Версионирование: Пinned image versions в production
Ресурсы: Resource requests и limits для всех контейнеров
Health checks: Liveness и readiness probes
Persistence: PVC для stateful приложений

Структура Kubernetes инфраструктуры

Структура Kubernetes инфраструктуры

Обзор

Структура директорий

Компоненты

Apps

Databases

DLT

Messaging

Observability

Окружения

Development

Production

Makefile

Best Practices

Дополнительные ресурсы

On this page