k8s-aur-builder/README.md
2026-10-12 09:03:53 +02:00

14 KiB

k8s-aur-builder

A distributed Arch Linux AUR package builder with a built-in Pacman repository. Build AUR, Git, and ZIP-sourced packages across multiple agents, store artifacts in S3-compatible storage, and serve them as a standard pacman-compatible repository.

Features

  • Multiple source types — AUR, arbitrary Git repos, or uploaded ZIP archives
  • Multi-arch builds — amd64 and arm64 (cross-compiled via Rust/clang in the build image)
  • Distributed agents — pull-based model; add as many agents as needed
  • AUR dependency resolution — automatically detects and registers AUR deps before building
  • Built-in Pacman repo — packages served as a fully compatible repository; no extra tooling required
  • S3 registry — all packages and logs stored in S3-compatible object storage (MinIO, AWS S3, Cloudflare R2, etc.)
  • Scheduled auto-updates — daily AUR version checks; new versions trigger builds automatically
  • Web UI — dashboard, package management, live build logs, agent monitoring
  • RBAC — three roles (viewer, editor, admin) with optional OIDC/SSO login

Architecture overview

┌─────────────┐   REST/SSE   ┌─────────────┐   poll   ┌─────────────┐
│   Frontend  │◄────────────►│   Backend   │◄─────────│    Agent    │
│  (React UI) │              │  (Go/Gin)   │          │  (Go daemon)│
└─────────────┘              └──────┬──────┘          └──────┬──────┘
                                    │ GORM                   │ Docker SDK /
                                    ▼                        │ Kubernetes API
                             ┌─────────────┐                 ▼
                             │  PostgreSQL │          ┌─────────────┐
                             └─────────────┘          │ Build image │
                                    │                 │(Arch+paru)  │
                                    ▼                 └─────────────┘
                             ┌─────────────┐
                             │  S3 storage │
                             └─────────────┘

The backend generates the Pacman repository database on the fly from package metadata stored in PostgreSQL. Package files are served by redirecting clients to pre-signed S3 URLs — the Go backend is never in the data path for downloads.


Quick start (Docker Compose)

Prerequisites

  • Docker with BuildKit enabled
  • Docker Compose v2

1. Build the build image

The agent needs the Arch Linux build image locally before it can run builds:

docker compose --profile build build buildimage

This produces aur-buildimage:latest. It takes a few minutes the first time (compiles paru from source).

2. Start all services

docker compose up -d
Service URL Default credentials
Frontend http://localhost:3000 admin / admin
Backend http://localhost:8080 —
MinIO UI http://localhost:9001 minioadmin / minioadmin

3. Create the MinIO bucket

Open http://localhost:9001, log in, and create a bucket named aur-packages (or whatever you set S3_BUCKET to).

4. Add a package

Log in to the UI, go to Packages → Add Package, search AUR, and click Add & Build.

5. Configure pacman

The Dashboard shows the exact lines to add to /etc/pacman.conf:

[aur-custom]
SigLevel = Never
Server = http://localhost:8080/repo/$arch

Then run sudo pacman -Sy to sync.


Configuration

Backend environment variables

Variable Default Description
PORT 8080 HTTP listen port
DB_DRIVER sqlite sqlite, mysql, or postgres
DB_HOST — Database host (postgres/mysql only)
DB_PORT 5432 Database port
DB_NAME — Database name
DB_USER — Database user
DB_PASSWORD — Database password
DB_SSLMODE disable PostgreSQL SSL mode
JWT_SECRET — Secret for signing JWT tokens (required in production)
CORS_ORIGIN http://localhost:3000 Allowed CORS origin
S3_BUCKET aur-packages S3 bucket name
S3_ENDPOINT — S3 endpoint URL (leave empty for AWS)
S3_REGION us-east-1 S3 region
AWS_ACCESS_KEY_ID — S3 access key
AWS_SECRET_ACCESS_KEY — S3 secret key
BUILD_IMAGE aur-buildimage:latest Docker image used by the Docker agent runner
REPO_NAME aur-custom Pacman repository name
REPO_URL — Public base URL for the repo (used in dashboard snippet)
AUR_USERNAME — aur.archlinux.org username (used for auto-vote)
AUR_PASSWORD — aur.archlinux.org password
AUR_AUTO_VOTE false Set to true to vote for AUR packages after a successful build
OIDC_ISSUER — OIDC issuer URL (leave empty to disable)
OIDC_CLIENT_ID — OIDC client ID
OIDC_CLIENT_SECRET — OIDC client secret
OIDC_REDIRECT_URL — OIDC redirect URL
INITIAL_ADMIN_USERNAME admin Username of the first admin user
INITIAL_ADMIN_PASSWORD admin Password of the first admin user
INITIAL_AGENT_ID — UUID for the seeded agent
INITIAL_AGENT_TOKEN — Token for the seeded agent
INITIAL_AGENT_NAME — Display name for the seeded agent
INITIAL_AGENT_ARCHITECTURE — Architecture(s) for the seeded agent

S3 credentials can also be configured through the UI under Settings and are stored encrypted in the database, overriding environment variables.

Agent environment variables

Variable Default Description
BACKEND_URL — Backend API base URL, e.g. http://backend:8080/api/v1
AGENT_ID — Agent UUID (must match a registered agent in the backend)
AGENT_TOKEN — Agent secret token
AGENT_MODE docker docker or k8s
MAX_PARALLEL_JOBS 1 Maximum concurrent builds
BUILD_TIMEOUT 2h Per-build timeout (Go duration string, e.g. 90m, 3h)
K8S_NAMESPACE aur-builder Namespace for build Jobs (k8s mode)
K8S_ARCHITECTURES — Comma-separated architectures, e.g. amd64,arm64 (k8s mode)

Kubernetes deployment (Helm)

Prerequisites

  • Kubernetes 1.24+
  • Helm 3.x
  • An S3-compatible bucket reachable from the cluster
  • An ingress controller (chart defaults to nginx)

Install

helm dependency update helm/aur-builder

helm install aur-builder helm/aur-builder \
  --namespace aur-builder --create-namespace \
  --set ingress.host=aur-builder.example.com \
  --set backend.env.S3_ENDPOINT=https://s3.example.com \
  --set backend.env.S3_BUCKET=aur-packages \
  --set backend.secrets.AWS_ACCESS_KEY_ID.value=<key> \
  --set backend.secrets.AWS_SECRET_ACCESS_KEY.value=<secret> \
  --set backend.secrets.JWT_SECRET.value=<random-secret>

JWT_SECRET is auto-generated on first install if omitted and preserved across upgrades.

Enable the in-cluster agent

The agent is disabled by default. Enable it to have it spawn Kubernetes Jobs for builds:

helm upgrade aur-builder helm/aur-builder \
  --reuse-values \
  --set agent.enabled=true \
  --set agent.mode=k8s \
  --set agent.architectures=amd64 \
  --set agent.secrets.agentID.value=<uuid> \
  --set agent.secrets.agentToken.value=<token>

Key Helm values

Value Default Description
ingress.host aur-builder.example.com Ingress hostname
ingress.tls false Enable TLS on the ingress
ingress.tlsSecretName — TLS secret name
ingress.annotations {} Extra ingress annotations (cert-manager, etc.)
agent.enabled false Deploy the agent
agent.mode k8s k8s or docker
agent.architectures amd64 Architectures handled by this agent
agent.buildNamespace "" Namespace for build Jobs (defaults to release namespace)
agent.nodeSelector {} Node selector for build Jobs
agent.tolerations [] Tolerations for build Jobs
agent.maxParallelJobs 1 Concurrent builds per agent
agent.terminationGracePeriodSeconds 3600 Grace period during rolling updates
backend.persistence.enabled false Persist uploaded ZIP sources
postgresql.enabled true Deploy PostgreSQL sub-chart
postgresql.primary.persistence.size 10Gi Database volume size

For a full reference see helm/aur-builder/values.yaml.

Using an external database

Set postgresql.enabled=false and provide connection details:

--set postgresql.enabled=false \
--set backend.env.DB_HOST=pg.example.com \
--set backend.env.DB_NAME=aurbuilder \
--set backend.env.DB_USER=aurbuilder \
--set backend.secrets.DB_PASSWORD.value=<password>

Referencing existing Secrets

Instead of passing credentials as plain values you can reference existing Kubernetes Secrets:

backend:
  secrets:
    AWS_ACCESS_KEY_ID:
      valueFrom:
        secret: my-s3-secret
        key: access-key
    AWS_SECRET_ACCESS_KEY:
      valueFrom:
        secret: my-s3-secret
        key: secret-key

Adding your first package

AUR package

  1. Go to Packages → Add Package
  2. Select the AUR tab and search for the package
  3. If AUR dependencies are detected, select which ones to register automatically
  4. Choose architectures and optionally pin a preferred agent
  5. Click Add & Build

Git repository

  1. Select the Git Repo tab
  2. Enter the Git URL and branch containing the PKGBUILD
  3. Click Add & Build

ZIP upload

  1. Select the ZIP Upload tab
  2. Upload a .zip containing a PKGBUILD at the root
  3. Click Add & Build

Pacman repository

The repository is served unauthenticated at /repo/{arch}/:

URL Description
/repo/x86_64/aur-custom.db Repository database (pacman sync DB)
/repo/x86_64/aur-custom.db.tar.gz Same, explicit extension
/repo/x86_64/<package>.pkg.tar.zst Redirects to a pre-signed S3 URL
/repo/aarch64/aur-custom.db Same for ARM64

Add to /etc/pacman.conf:

[aur-custom]
SigLevel = Never
Server = https://aur-builder.example.com/repo/$arch

Then sync: sudo pacman -Sy

The repository database is generated on the fly from PostgreSQL metadata and cached via Last-Modified / If-Modified-Since headers.


User roles

Role Capabilities
viewer View packages, builds, agents
editor All viewer permissions + add/edit packages, trigger builds
admin All editor permissions + delete packages/builds, manage agents, users, and settings

OIDC login maps external identities to local user accounts. Configure the provider under Settings → OIDC.


Building images

All images are multi-arch (amd64 + arm64) when built through CI. For local builds:

# Backend
docker build -f Dockerfile.backend -t aur-builder/backend:latest .

# Frontend
docker build -f Dockerfile.frontend -t aur-builder/frontend:latest .

# Agent
docker build -f agent/Dockerfile -t aur-builder/agent:latest .

# Build image (takes ~10 minutes — compiles paru from source)
docker build -f buildimage/Dockerfile -t aur-buildimage:latest buildimage/

For multi-arch builds use docker buildx:

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -f Dockerfile.backend \
  -t aur-builder/backend:latest \
  --push .

API

The backend exposes a REST API under /api/v1. All endpoints except auth and the repo route require a JWT bearer token obtained from POST /api/v1/auth/login.

Interactive docs are served at /swagger/index.html when the backend is running.

Endpoint overview

Method Path Auth Description
POST /auth/login — Password login, returns tokens
POST /auth/refresh — Refresh access token
GET /auth/me JWT Current user
GET /packages JWT List packages
POST /packages editor+ Create package
PUT /packages/:id editor+ Update package
DELETE /packages/:id admin Delete package
POST /packages/:id/build editor+ Trigger build
POST /packages/upload editor+ Upload ZIP source
GET /builds JWT List builds
GET /builds/:id/log JWT Full build log
GET /agents JWT List agents
POST /agents admin Create agent
PUT /agents/:id admin Update agent
DELETE /agents/:id admin Delete agent
GET /settings admin Get settings
PUT /settings admin Update settings
GET /repo/:arch/*filename — Pacman repo (public)

Troubleshooting

error: database is inconsistent: version mismatch on package X The package has an epoch in its version (e.g. 1:2.0-1). Rebuild the package to regenerate its entry in the repo database, or update to the latest backend which handles epochs correctly.

Agent stuck in pending state Check that the agent is running (docker ps / kubectl get pods), its AGENT_ID and AGENT_TOKEN match what is registered in the backend, and that it can reach BACKEND_URL.

Build fails immediately with unknown SOURCE_TYPE The SOURCE_TYPE env var was not set. This is always set by the backend — check that the agent and backend versions match.

MinIO bucket does not exist The bucket must be created manually. Open the MinIO console and create it, or use mc mb myminio/aur-packages.

Packages not showing in pacman -Ss Run sudo pacman -Sy to force a sync. If the DB endpoint returns 404, verify REPO_NAME matches the [section] name in pacman.conf.