Valkey cache server

Valkey can be the cache server of Forgejo

Like with the Ghost platform, you can improve Forgejo’s performance by setting up Valkey as its caching server. Furthermore, deploying Valkey in the Forgejo setup is done like you did for the Ghost platform.

Kustomize project folders for Forgejo and Valkey

First, you have to create a folder for your Forgejo’s main Kustomize project and, within it, a directory tree for the Valkey subproject. You can do this with just one mkdir command:

$ mkdir -p $HOME/k8sprjs/forgejo/components/cache-valkey/{configs,resources,secrets}

Valkey configuration file

Prepare the valkey.conf file specifying the configuration values for your Forgejo’s Valkey server:

1. In the configs subfolder of the Valkey project, create a valkey.conf file:

   $ touch $HOME/k8sprjs/forgejo/components/cache-valkey/configs/valkey.conf

2. Copy in valkey.conf the lines below:

   # Custom Valkey configuration
   bind 0.0.0.0
   protected-mode no
   port 6379
   maxmemory 64mb
   maxmemory-policy allkeys-lru
   aclfile /etc/valkey/users.acl
   dir /data

   The parameters are exactly the same ones set up in the second part of the Ghost deployment procedure.

Valkey secrets

As in the Ghost case, securing the access to this Valkey instance requires setting up two users stored in a secret resource within your Kubernetes cluster.

Valkey ACL user list

The Access Control List declared here configures the default user and one specific for Forgejo:

1. Create a new users.acl file in the configs folder:

   $ touch $HOME/k8sprjs/forgejo/components/cache-valkey/secrets/users.acl

2. In secrets/users.acl enter the ACL rules redefining the default user and specifying the user for Forgejo:

   user default on ~* &* +@all >P4s5W0rd_FOr_7h3_DeF4u1t_uSEr
   user forgejocache on ~forgejo:* &* allcommands >pAS2wOrD_f0r_The_F0rgEJ0_Us3R

   Warning

   The passwords in this secrets/users.acl file are plain unencrypted strings
   Be careful of who can access this users.acl file.

User for Prometheus metrics exporter

The Valkey instance for Forgejo will have its own Prometheus metrics exporter module. This module uses the default Valkey user to access the metrics from the Valkey instance. As it happened in the Ghost’s case, you have to duplicate the default user’s information to make it available as environment variables for this exporter module:

1. Create a default_user_env.properties under the secrets folder:

   $ touch $HOME/k8sprjs/forgejo/components/cache-valkey/secrets/default_user_env.properties

2. Enter the default user’s name and password in secrets/default_user_env.properties as environment variables:

   REDIS_USER=default
   REDIS_PASSWORD=P4s5W0rd_FOr_7h3_DeF4u1t_uSEr

   Warning

   The password in this secrets/default_user_env.properties file is a plain unencrypted string
   Be careful of who can access this default_user_env.properties file.

Valkey persistent storage claim

To link Forgejo’s Valkey instance with the persistent volume that will be declared in the last part of this procedure, you need to declare a PersistentVolumeClaim resource:

1. Create a cache-valkey.persistentvolumeclaim.yaml file under the resources folder:

   $ touch $HOME/k8sprjs/forgejo/components/cache-valkey/resources/cache-valkey.persistentvolumeclaim.yaml

2. Declare Valkey’s PersistentVolumeClaim in the resources/cache-valkey.persistentvolumeclaim.yaml file:

   # Forgejo Valkey claim of persistent storage
   apiVersion: v1
   kind: PersistentVolumeClaim
   
   metadata:
     name: cache-valkey
   spec:
     accessModes:
     - ReadWriteOnce
     storageClassName: local-path
     volumeName: forgejo-ssd-cache
     resources:
       requests:
         storage: 2.8G

Valkey StatefulSet

Since Forgejo’s Valkey instance stores state, it has to be deployed with a StatefulSet:

1. Create a cache-valkey.statefulset.yaml file under the resources subfolder:

   $ touch $HOME/k8sprjs/forgejo/components/cache-valkey/resources/cache-valkey.statefulset.yaml

2. Declare the StatefulSet resource for the Valkey instance in resources/cache-valkey.statefulset.yaml:

   # Forgejo Valkey StatefulSet for a sidecar server pod
   apiVersion: apps/v1
   kind: StatefulSet
   
   metadata:
     name: cache-valkey
   spec:
     replicas: 1
     serviceName: cache-valkey
     template:
       spec:
         containers:
         - name: server
           image: valkey/valkey:9.0-alpine
           command:
           - valkey-server
           - "/etc/valkey/valkey.conf"
           ports:
           - name: server
             containerPort: 6379
           resources:
             requests:
               cpu: "0.5"
               memory: 64Mi
           volumeMounts:
           - name: valkey-storage
             mountPath: /data
           - name: valkey-config
             readOnly: true
             subPath: valkey.conf
             mountPath: /etc/valkey/valkey.conf
           - name: valkey-acl
             readOnly: true
             subPath: users.acl
             mountPath: /etc/valkey/users.acl
         - name: metrics
           image: oliver006/redis_exporter:v1.80.0-alpine
           envFrom:
           - secretRef:
               name: cache-valkey-exporter-user
           resources:
             requests:
               cpu: "0.25"
               memory: 16Mi
           ports:
           - name: metrics
             containerPort: 9121
         volumes:
         - name: valkey-storage
           persistentVolumeClaim:
             claimName: cache-valkey
         - name: valkey-config
           configMap:
             name: cache-valkey-config
             defaultMode: 444
             items:
             - key: valkey.conf
               path: valkey.conf
         - name: valkey-acl
           secret:
             secretName: cache-valkey-acl
             defaultMode: 444
             items:
             - key: users.acl
               path: users.acl

   You may notice that this declaration is identical to the StatefulSet resource declared for the Ghost’s Valkey instance. This is not going to be an issue since the whole Forgejo’s setup will be deployed in its own distinct forgejo namespace in the final part of this deployment procedure.

Valkey Service

Declare the Service object required for Forgejo’s Valkey StatefulSet pod:

1. Generate a new file named cache-valkey.service.yaml, also under the resources subfolder:

   $ touch $HOME/k8sprjs/forgejo/components/cache-valkey/resources/cache-valkey.service.yaml

2. Declare the Valkey Service resource in resources/cache-valkey.service.yaml:

   # Forgejo Valkey headless service
   apiVersion: v1
   kind: Service
   
   metadata:
     name: cache-valkey
     annotations:
       prometheus.io/scrape: "true"
       prometheus.io/port: "9121"
   spec:
     type: ClusterIP
     clusterIP: None
     ports:
     - port: 6379
       targetPort: server
       protocol: TCP
       name: server
     - port: 9121
       targetPort: metrics
       protocol: TCP
       name: metrics

   Again, this Service object is exactly like the one declared for the Ghost Valkey instance.

Valkey Service’s FQDN

Since the whole Forgejo setup will be deployed in the forgejo namespace, the absolute FQDN of the Valkey service is going to be this one:

cache-valkey.forgejo.svc.homelab.cluster.

Valkey Kustomize project

Now declare the main kustomization.yaml file that describes the whole Forgejo’s Valkey Kustomize subproject:

1. In the main cache-valkey folder, create a kustomization.yaml file:

   $ touch $HOME/k8sprjs/forgejo/components/cache-valkey/kustomization.yaml

2. Enter the following Kustomization declaration in the kustomization.yaml file:

   # Forgejo Valkey setup
   apiVersion: kustomize.config.k8s.io/v1beta1
   kind: Kustomization
   
   labels:
   - pairs:
       app: cache-valkey
     includeSelectors: true
     includeTemplates: true
   
   resources:
   - resources/cache-valkey.persistentvolumeclaim.yaml
   - resources/cache-valkey.statefulset.yaml
   - resources/cache-valkey.service.yaml
   
   replicas:
   - name: cache-valkey
     count: 1
   
   images:
   - name: valkey/valkey
     newTag: 9.0-alpine
   - name: oliver006/redis_exporter
     newTag: v1.80.0-alpine
   
   configMapGenerator:
   - name: cache-valkey-config
     files:
     - configs/valkey.conf
   
   secretGenerator:
   - name: cache-valkey-exporter-user
     envs:
     - secrets/default_user_env.properties
   - name: cache-valkey-acl
     files:
     - secrets/users.acl

   This kustomization.yaml file is exactly the same as the one you declared for Ghost’s Valkey deployment.

Validating the Kustomize YAML output

With all the necessary resources declared for your Forgejo Valkey instance, review the YAML resulting from the Forgejo Valkey’s Kustomize subproject:

1. Execute the kubectl kustomize command on the Forgejo Valkey Kustomize subproject’s root folder, piped to less (or your text editor of choice) to get the output paginated:

   $ kubectl kustomize $HOME/k8sprjs/forgejo/components/cache-valkey | less

2. The resulting YAML should be like this one:

   apiVersion: v1
   data:
     valkey.conf: |-
       # Custom Valkey configuration
       bind 0.0.0.0
       protected-mode no
       port 6379
       maxmemory 64mb
       maxmemory-policy allkeys-lru
       aclfile /etc/valkey/users.acl
       dir /data
   kind: ConfigMap
   metadata:
     labels:
       app: cache-valkey
     name: cache-valkey-config-c86dc4fh5d
   ---
   apiVersion: v1
   data:
     users.acl: |
       dXNlciBkZWZhdWx0IG9uIH4qICYqICtAYWxsID5QNHM1VzByZF9GT3JfN2gzX0RlRjR1MX
       RfdVNFcgp1c2VyIGZvcmdlam9jYWNoZSBvbiB+Zm9yZ2VqbzoqICYqIGFsbGNvbW1hbmRz
       ID5wQVMyd09yRF9mMHJfVGhlX0YwcmdFSjBfVXMzUg==
   kind: Secret
   metadata:
     labels:
       app: cache-valkey
     name: cache-valkey-acl-9d8g27k94b
   type: Opaque
   ---
   apiVersion: v1
   data:
     REDIS_PASSWORD: UDRzNVcwcmRfRk9yXzdoM19EZUY0dTF0X3VTRXI=
     REDIS_USER: ZGVmYXVsdA==
   kind: Secret
   metadata:
     labels:
       app: cache-valkey
     name: cache-valkey-exporter-user-6mdd99ft8d
   type: Opaque
   ---
   apiVersion: v1
   kind: Service
   metadata:
     annotations:
       prometheus.io/port: "9121"
       prometheus.io/scrape: "true"
     labels:
       app: cache-valkey
     name: cache-valkey
   spec:
     clusterIP: None
     ports:
     - name: server
       port: 6379
       protocol: TCP
       targetPort: server
     - name: metrics
       port: 9121
       protocol: TCP
       targetPort: metrics
     selector:
       app: cache-valkey
     type: ClusterIP
   ---
   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     labels:
       app: cache-valkey
     name: cache-valkey
   spec:
     accessModes:
     - ReadWriteOnce
     resources:
       requests:
         storage: 2.8G
     storageClassName: local-path
     volumeName: forgejo-ssd-cache
   ---
   apiVersion: apps/v1
   kind: StatefulSet
   metadata:
     labels:
       app: cache-valkey
     name: cache-valkey
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: cache-valkey
     serviceName: cache-valkey
     template:
       metadata:
         labels:
           app: cache-valkey
       spec:
         containers:
         - command:
           - valkey-server
           - /etc/valkey/valkey.conf
           image: valkey/valkey:9.0-alpine
           name: server
           ports:
           - containerPort: 6379
             name: server
           resources:
             requests:
               cpu: "0.5"
               memory: 64Mi
           volumeMounts:
           - mountPath: /data
             name: valkey-storage
           - mountPath: /etc/valkey/valkey.conf
             name: valkey-config
             readOnly: true
             subPath: valkey.conf
           - mountPath: /etc/valkey/users.acl
             name: valkey-acl
             readOnly: true
             subPath: users.acl
         - envFrom:
           - secretRef:
               name: cache-valkey-exporter-user-6mdd99ft8d
           image: oliver006/redis_exporter:v1.80.0-alpine
           name: metrics
           ports:
           - containerPort: 9121
             name: metrics
           resources:
             requests:
               cpu: "0.25"
               memory: 16Mi
         volumes:
         - name: valkey-storage
           persistentVolumeClaim:
             claimName: cache-valkey
         - configMap:
             defaultMode: 444
             items:
             - key: valkey.conf
               path: valkey.conf
             name: cache-valkey-config-c86dc4fh5d
           name: valkey-config
         - name: valkey-acl
           secret:
             defaultMode: 444
             items:
             - key: users.acl
               path: users.acl
             secretName: cache-valkey-acl-9d8g27k94b

Do not deploy this Valkey project on its own

This Valkey setup is missing one critical element, the persistent volume it needs to store its working directory data. Do not confuse it with the claim you have configured for your Valkey cache server. That persistent volume and other elements are going to be declared in the main Kustomize project you will declare in the final part of this Forgejo deployment procedure. Until then, do not deploy this Valkey subproject.

Relevant system paths

Folders in kubectl client system

• $HOME/k8sprjs/forgejo
• $HOME/k8sprjs/forgejo/components
• $HOME/k8sprjs/forgejo/components/cache-valkey
• $HOME/k8sprjs/forgejo/components/cache-valkey/configs
• $HOME/k8sprjs/forgejo/components/cache-valkey/resources
• $HOME/k8sprjs/forgejo/components/cache-valkey/secrets

Files in kubectl client system

• $HOME/k8sprjs/forgejo/components/cache-valkey/kustomization.yaml
• $HOME/k8sprjs/forgejo/components/cache-valkey/configs/valkey.conf
• $HOME/k8sprjs/forgejo/components/cache-valkey/resources/cache-valkey.persistentvolumeclaim.yaml
• $HOME/k8sprjs/forgejo/components/cache-valkey/resources/cache-valkey.service.yaml
• $HOME/k8sprjs/forgejo/components/cache-valkey/resources/cache-valkey.statefulset.yaml
• $HOME/k8sprjs/forgejo/components/cache-valkey/secrets/default_user_env.properties
• $HOME/k8sprjs/forgejo/components/cache-valkey/secrets/users.acl

References

Valkey

• Documentation by Topic

  • The Valkey server
  • ACL
  • Cluster tutorial

• Docker Hub. Valkey

• Docker Hub. Prometheus Valkey & Redis Metrics Exporter

• GitHub. Valkey. Example valkey.conf for Valkey 9.0

Redis

• Docs

Other Redis-related contents

• rpi4cluster. K3s Kubernetes. Redis
• Daniel Cushing. Simple Redis Cache on Kubernetes with Prometheus Metrics
• Mark Lu. Deploy and Operate a Redis Cluster in Kubernetes
• Suse Rancher Blog. Deploying Redis Cluster on Top of Kubernetes
• StackOverflow. Redis sentinel vs clustering

Kubernetes

• Kubernetes Documentation

Pods and containers

• Kubernetes Documentation. Concepts. Scheduling, Preemption and Eviction

  • Assigning Pods to Nodes

• Kubernetes Documentation. Tasks. Configure Pods and Containers

  • Assign Pods to Nodes using Node Affinity
  • Configure a Pod to Use a ConfigMap

• Kubernetes Documentation. Tasks. Inject Data Into Applications

  • Define Dependent Environment Variables
  • Define Environment Variables for a Container

• Kubernetes Documentation. Reference. Kubernetes API. Workload Resources

  • Pod
    • Scheduling

ConfigMaps

• Kubernetes Documentation. Concepts. Configuration

  • ConfigMaps

• Kubernetes Documentation. Tutorials. Configuration

  • Configuring Redis using a ConfigMap

Labels

• Kubernetes Documentation. Concepts. Overview. Objects in Kubernetes
  • Labels and Selectors

Services

• Kubernetes Documentation. Concepts. Services, Load Balancing, and Networking
  • Service. Headless Services

Other Kubernetes-related contents

About services

• GeeksForGeeks. Kubernetes Headless Service

About pod scheduling

• TheNewStack. Strategies for Kubernetes Pod Placement and Scheduling
• TheNewStack. Implement Node and Pod Affinity/Anti-Affinity in Kubernetes: A Practical Example
• TheNewStack. Tutorial: Apply the Sidecar Pattern to Deploy Redis in Kubernetes

About port names

• StackOverflow. Is there any way to disable or increase port name length in Kubernetes?

About ConfigMaps and Secrets

• Opensource.com. An Introduction to Kubernetes Secrets and ConfigMaps
• Dev. Kubernetes - Using ConfigMap SubPaths to Mount Files
• GoLinuxCloud. Kubernetes Secrets | Declare confidential data with examples
• StackOverflow. Import data to config map from kubernetes secret

About CPU requests and limits

• Baeldung. CPU Requests and Limits in Kubernetes
• DEV. Kubernetes CPU Limits: The Silent Killer of Performance (And How to Fix It)