Actions Runner Controller

Introduction

Actions Runner Controller (ARC) is a Kubernetes operator that orchestrates and scales self-hosted runners for Gitea Actions. For more information, see Operator pattern in the Kubernetes documentation.

With ARC, you can create runner scale sets that automatically scale based on the number of workflows running in your repository, organization, or enterprise. Because controlled runners can be ephemeral and based on containers, new runner instances can scale up or down rapidly and cleanly.

Prerequisites

In order to use ARC, ensure you have the following.

Gitea Enterprise Edition version number greater than 23.8.0, activated
A Kubernetes cluster

Local Persistent Volume

kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.31/deploy/local-path-storage.yaml

A successful output will look similar to the following.

namespace/local-path-storage created
serviceaccount/local-path-provisioner-service-account created
role.rbac.authorization.k8s.io/local-path-provisioner-role created
Warning: resource clusterroles/local-path-provisioner-role is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrole.rbac.authorization.k8s.io/local-path-provisioner-role configured
rolebinding.rbac.authorization.k8s.io/local-path-provisioner-bind created
Warning: resource clusterrolebindings/local-path-provisioner-bind is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
clusterrolebinding.rbac.authorization.k8s.io/local-path-provisioner-bind configured
deployment.apps/local-path-provisioner created
Warning: resource storageclasses/local-path is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
storageclass.storage.k8s.io/local-path configured
configmap/local-path-config created

For more information about local-path-provisioner, see local-path-provisioner README

Installing Actions Runner Controller

cat > install.yml << EOF
apiVersion: v1
kind: Namespace
metadata:
  labels:
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/name: runner-controller
    control-plane: controller-manager
  name: runner-controller-system
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.16.1
  name: runners.gitea.com
spec:
  group: gitea.com
  names:
    kind: Runner
    listKind: RunnerList
    plural: runners
    singular: runner
  scope: Namespaced
  versions:
  - name: v1
    schema:
      openAPIV3Schema:
        description: Runner is the Schema for the runners API
        properties:
          apiVersion:
            description: |-
              APIVersion defines the versioned schema of this representation of an object.
              Servers should convert recognized schemas to the latest internal value, and
              may reject unrecognized values.
              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
            type: string
          kind:
            description: |-
              Kind is a string value representing the REST resource this object represents.
              Servers may infer this from the endpoint the client submits requests to.
              Cannot be updated.
              In CamelCase.
              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
            type: string
          metadata:
            type: object
          spec:
            description: RunnerSpec defines the desired state of Runner
            properties:
              gitea:
                description: Gitea is the Gitea spec
                properties:
                  instanceUrl:
                    description: InstanceUrl is the URL of the Gitea instance
                    format: url
                    type: string
                  registrationToken:
                    description: RegistrationToken is the registration token of the
                      runner
                    type: string
                required:
                - instanceUrl
                - registrationToken
                type: object
              runtime:
                default:
                  image: gitea/act_runner:nightly-dind
                  labels:
                    ubuntu-20.04: docker://gitea/runner-images:ubuntu-20.04
                    ubuntu-22.04: docker://gitea/runner-images:ubuntu-22.04
                    ubuntu-latest: docker://gitea/runner-images:ubuntu-latest
                description: Runtime is the runtime spec
                properties:
                  image:
                    default: gitea/act_runner:nightly-dind
                    description: Image is the image of the runner
                    type: string
                  labels:
                    additionalProperties:
                      type: string
                    default:
                      ubuntu-20.04: docker://gitea/runner-images:ubuntu-20.04
                      ubuntu-22.04: docker://gitea/runner-images:ubuntu-22.04
                      ubuntu-latest: docker://gitea/runner-images:ubuntu-latest
                    description: Labels is the labels of the runner
                    type: object
                required:
                - labels
                type: object
              scalability:
                default:
                  cpuRequest: "2"
                  maxSize: 10
                  memoryRequest: 4Gi
                description: Scalability is the scalability spec
                properties:
                  cpuRequest:
                    anyOf:
                    - type: integer
                    - type: string
                    default: "2"
                    description: CpuRequest is the CPU request of each runner
                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                    x-kubernetes-int-or-string: true
                  maxSize:
                    default: 10
                    description: MaxSize is the maximum size of the runner
                    maximum: 9999
                    minimum: 1
                    type: integer
                  memoryRequest:
                    anyOf:
                    - type: integer
                    - type: string
                    default: 4Gi
                    description: MemoryRequest is the memory request of each runner
                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                    x-kubernetes-int-or-string: true
                type: object
              storage:
                default:
                  cacheServerVolumeSize: 500Gi
                  cacheVolumeSize: 100Gi
                  className: standard
                  dataVolumeSize: 100Mi
                description: Storage is the storage spec
                properties:
                  cacheServerVolumeSize:
                    anyOf:
                    - type: integer
                    - type: string
                    default: 500Gi
                    description: CacheServerVolumeSize is the size of the cache server
                      volume
                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                    x-kubernetes-int-or-string: true
                  cacheVolumeSize:
                    anyOf:
                    - type: integer
                    - type: string
                    default: 100Gi
                    description: CacheVolumeSize is the size of the cache volume
                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                    x-kubernetes-int-or-string: true
                  className:
                    default: standard
                    description: ClassName is the storage class name
                    type: string
                  dataVolumeSize:
                    anyOf:
                    - type: integer
                    - type: string
                    default: 100Mi
                    description: DataVolumeSize is the size of the data volume
                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                    x-kubernetes-int-or-string: true
                type: object
            required:
            - gitea
            type: object
          status:
            description: RunnerStatus defines the observed state of Runner
            properties:
              targetSize:
                type: integer
              targetSizeUpdatedAt:
                format: date-time
                type: string
            required:
            - targetSize
            - targetSizeUpdatedAt
            type: object
        type: object
    served: true
    storage: true
    subresources:
      status: {}
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/name: runner-controller
  name: runner-controller-controller-manager
  namespace: runner-controller-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/name: runner-controller
  name: runner-controller-leader-election-role
  namespace: runner-controller-system
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - patch
  - delete
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - patch
  - delete
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: runner-controller-manager-role
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - events
  - persistentvolumeclaims
  - persistentvolumes
  - pods
  - services
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - gitea.com
  resources:
  - runners
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - gitea.com
  resources:
  - runners/finalizers
  verbs:
  - update
- apiGroups:
  - gitea.com
  resources:
  - runners/status
  verbs:
  - get
  - patch
  - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: runner-controller-metrics-auth-role
rules:
- apiGroups:
  - authentication.k8s.io
  resources:
  - tokenreviews
  verbs:
  - create
- apiGroups:
  - authorization.k8s.io
  resources:
  - subjectaccessreviews
  verbs:
  - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: runner-controller-metrics-reader
rules:
- nonResourceURLs:
  - /metrics
  verbs:
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/name: runner-controller
  name: runner-controller-runner-editor-role
rules:
- apiGroups:
  - gitea.com
  resources:
  - runners
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - gitea.com
  resources:
  - runners/status
  verbs:
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/name: runner-controller
  name: runner-controller-runner-viewer-role
rules:
- apiGroups:
  - gitea.com
  resources:
  - runners
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - gitea.com
  resources:
  - runners/status
  verbs:
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/name: runner-controller
  name: runner-controller-leader-election-rolebinding
  namespace: runner-controller-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: runner-controller-leader-election-role
subjects:
- kind: ServiceAccount
  name: runner-controller-controller-manager
  namespace: runner-controller-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/name: runner-controller
  name: runner-controller-manager-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: runner-controller-manager-role
subjects:
- kind: ServiceAccount
  name: runner-controller-controller-manager
  namespace: runner-controller-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: runner-controller-metrics-auth-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: runner-controller-metrics-auth-role
subjects:
- kind: ServiceAccount
  name: runner-controller-controller-manager
  namespace: runner-controller-system
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/name: runner-controller
    control-plane: controller-manager
  name: runner-controller-controller-manager-metrics-service
  namespace: runner-controller-system
spec:
  ports:
  - name: https
    port: 8443
    protocol: TCP
    targetPort: 8443
  selector:
    control-plane: controller-manager
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/name: runner-controller
    control-plane: controller-manager
  name: runner-controller-controller-manager
  namespace: runner-controller-system
spec:
  replicas: 1
  selector:
    matchLabels:
      control-plane: controller-manager
  template:
    metadata:
      annotations:
        kubectl.kubernetes.io/default-container: manager
      labels:
        control-plane: controller-manager
    spec:
      containers:
      - args:
        - --metrics-bind-address=:8443
        - --leader-elect
        - --health-probe-bind-address=:8081
        command:
        - /manager
        image: commitgo/runner-controller:v0.3.1
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8081
          initialDelaySeconds: 15
          periodSeconds: 20
        name: manager
        readinessProbe:
          httpGet:
            path: /readyz
            port: 8081
          initialDelaySeconds: 5
          periodSeconds: 10
        resources:
          limits:
            cpu: 500m
            memory: 128Mi
          requests:
            cpu: 10m
            memory: 64Mi
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
      securityContext:
        runAsNonRoot: true
      serviceAccountName: runner-controller-controller-manager
      terminationGracePeriodSeconds: 10
EOF

kubectl apply -f install.yml

A successful output will look similar to the following.

namespace/runner-controller-system created
customresourcedefinition.apiextensions.k8s.io/runners.gitea.com created
serviceaccount/runner-controller-controller-manager created
role.rbac.authorization.k8s.io/runner-controller-leader-election-role created
clusterrole.rbac.authorization.k8s.io/runner-controller-manager-role created
clusterrole.rbac.authorization.k8s.io/runner-controller-metrics-auth-role created
clusterrole.rbac.authorization.k8s.io/runner-controller-metrics-reader created
clusterrole.rbac.authorization.k8s.io/runner-controller-runner-editor-role created
clusterrole.rbac.authorization.k8s.io/runner-controller-runner-viewer-role created
rolebinding.rbac.authorization.k8s.io/runner-controller-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/runner-controller-manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/runner-controller-metrics-auth-rolebinding created
service/runner-controller-controller-manager-metrics-service created
deployment.apps/runner-controller-controller-manager created

Configuration

cat > runner.yml << EOF
apiVersion: gitea.com/v1
kind: Runner
metadata:
  labels:
    app.kubernetes.io/name: runner-controller
    app.kubernetes.io/managed-by: kustomize
  name: "gitea-runner" # TODO, like my-runner
spec:
  gitea:
    instanceUrl: "Your Gitea instanlce URL" # TODO, like https://mygitea.com/
    registrationToken: "<Your registration token>" # TODO, it's the same token to register runners
  runtime:
    labels:
      ubuntu-latest: docker://gitea/runner-images:ubuntu-latest
      ubuntu-22.04: docker://gitea/runner-images:ubuntu-22.04
      ubuntu-20.04: docker://gitea/runner-images:ubuntu-20.04
  storage:
    className: "local-path" # TODO, it should be real storage class in your k8s cluster
  scalability:
    maxSize: 10
    cpuRequest: 4
    memoryRequest: 8Gi
EOF

kubectl apply -f runner.yml

A successful output will look similar to the following.

runner.gitea.com/gitea-runner created

runners

Get the runner opration pods

NAME                                                  READY   STATUS    RESTARTS   AGE
runner-controller-controller-manager-ff9cbc7b-kbxzh   1/1     Running   0          14m

Get the runner opration logs

kubectl -n runner-controller-system logs -f runner-controller-controller-manager-ff9cbc7b-kbxzh

log-1

Using runner scale sets

Now you will create and run a simple test workflow that uses the runner scale set runners.

In a repository, create a workflow similar to the following example. The runs-on value should match the runner runtime labels name you used when you installed the autoscaling runner set.

For more information on adding workflows to a repository, see Quickstart for Gitea Actions.

name: Gitea Actions Demo
run-name: ${{ gitea.actor }} is testing out Gitea Actions
on: [push]

jobs:
Explore-Gitea-Actions:
    runs-on: ubuntu-latest
    steps:
    - run: echo " The job was automatically triggered by a ${{ gitea.event_name }} event."
    - run: echo " This job is now running on a ${{ runner.os }} server hosted by Gitea!"
    - run: echo " The name of your branch is ${{ gitea.ref }} and your repository is ${{ gitea.repository }}."
    - name: Check out repository code
        uses: actions/checkout@v4
    - run: echo " The ${{ gitea.repository }} repository has been cloned to the runner."
    - run: echo " The workflow is now ready to test your code on the runner."
    - name: List files in the repository
        run: |
        ls ${{ gitea.workspace }}
    - run: echo " This job's status is ${{ job.status }}."

actions-1

actions-2

To view the runner pods being created while the workflow is running, run the following command from your terminal.

kubectl get pods

A successful output will look similar to the following.

NAME                        READY   STATUS    RESTARTS   AGE
gitea-runner-0001           1/1     Running   0          56s
gitea-runner-cache-server   1/1     Running   0          26s