Gitea on Kubernetes on One IP

For some reason I wanted to try to run my own Git server at home. I’m always looking for ways to automate the infrastructure in my basement (VMs, Ansible, Kubernetes, etc) among other things, and having a centrally-located place for storage of configuration data would help me get one step closer to my goal.

I tried out Gitlab, but the resource requirements were super high, and I didn’t really need all the features. I just need a place to create repositories, add users/roles (with SSH keys), and display changes in a web browser without necessarily checking out the repository. And I wanted to run it on my Kubernetes cluster, for fun, profit, and generally playing around.

In comes Gitea, and their Helm charts. So how did I get it running?

The main accomplishments I wanted to achieve:

• Access the Gitea web UI via SSL, with certificates via Lets Encrypt.
• Both Git (over SSH) and the web UI accessed via the same hostname/IP. No seperate hostnames for SSH and Web access.

Pre-existing configuration I already had running:

• Metallb: Being able to serve Kubernetes services from a range of IP addresses from bare metal Kubernetes.
• Lets Encrypt via Cert Manager
• ingress-nginx hooked up to cert-manager: Providing HTTP/TCP proxying for services with the ability to generate and serve SSL certificates for those services.

So how did I tie this all together into a self-hosted local Git server?

Some yaml configuration, and then a hopefully-easier-to-understand RPC diagram

Helm values.yaml for gitea

#

---
gitea:
  config:
    APP_NAME: "Gitea: git.internal.domain.com"
service:
  http:
    type: ClusterIP
    port: 3000
  ssh:
    type: ClusterIP
    port: 22

Services

#

---
apiVersion: v1
kind: Service
metadata:
  name: gitea-metallb-https
  annotations:
    metallb.universe.tf/allow-shared-ip: "foobarbaz1"
spec:
  type: LoadBalancer
  loadBalancerIP: 192.168.1.250
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx-ingress
    app.kubernetes.io/component: controller
    release: nginx-ingress-release
---
apiVersion: v1
kind: Service
metadata:
  name: gitea-metallb-ssh
  annotations:
    metallb.universe.tf/allow-shared-ip: "foobarbaz1"
spec:
  type: LoadBalancer
  loadBalancerIP: 192.168.1.250
  ports:
  - port: 22
    protocol: TCP
    targetPort: 22
  selector:
    app: gitea

Ingress

#

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
    name: gitea
    annotations:
        cert-manager.io/cluster-issuer: "letsencrypt-issuer-prod"
spec:
    rules:
    - host: git.local.lan
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gitea-http
              port:
                number: 3000
    tls:
    - hosts:
        - git.local.lan
      secretName: gitea-tls-secret

RPC

#

graph TD;
    User-- 192.168.1.250:ssh -->service:gitea-metallb-ssh;
    User-- 192.168.1.250:https -->service:gitea-metallb-https;
    service:gitea-metallb-https --> service:nginx-ingress;
    service:nginx-ingress --> gitea:http;
    service:gitea-metallb-ssh -->  gitea:ssh
           
    subgraph "kubernetes";
      service:nginx-ingress
      subgraph "metallb";
        service:gitea-metallb-https
        service:gitea-metallb-ssh
      end
      subgraph "gitea";
        gitea:http
        gitea:ssh
      end
    end

Both SSH and HTTPS requests for the git service’s IP address (192.168.1.250) are sent to the physical (or in my case, VM) node which currently attracts traffic for that IP address.

This metallb node is listening on ports 22 and 443. Port 22 traffic is directed at the node whose labels match “app=gitea”, while port 443 traffic is directed at the service with the labels app: nginx-ingress, app.kubernetes.io/component: controller, and release:nginx-ingress-release.

In this case, port 22 (git+ssh) traffic goes directly to the Gitea pod, while port 443 (https) traffic is sent to the nginx-ingress controller. This is where SSL termination takes place, where the TLS certificate provided by certbot and Lets Encrypt is served. It is at this point that HTTPS traffic is turned into HTTP traffic and forwarded to port 3000 on the gitea-http service.

Victory.