Distbuild

Distbuild is a distributed build system, it is a set of Baserock systems working together to satisfy a set of build requests. System builds are naturally parallelisable, assuming the dependencies for a, b and c are satisfied, then a Distbuild network with 3 workers can build a, b and c in parallel.

There are 3 components:

• Initiators
• Controllers
• Workers

Initiators are Baserock devel machines, they submit build requests to Controllers. Controllers manage build requests, they decide what needs to be built and tell Workers what to build. Controllers also perform some basic build scheduling to avoid wasting build resources. All components of the distbuild network make use of a single trove, this trove is used to share the definitions for the systems being built as well as the artifacts produced by the workers. Once a worker completes a build the resultant artifacts are transferred to the trove's artifact cache.

Deploying a KVM based distbuild network

Note: This guide assumes you have some experience deploying KVM virtual machines, if not please read Deploy to KVM

First, build the appropriate Baserock build-system-xxx variant for your architecture. (Note that in releases up to and including Baserock 14.40, these was called distbuild-system-xxx.)

$ morph build systems/build-system-x86_64.morph

The following example cluster morph deploys an x86-64 distbuild network with one controller node and two worker nodes.

Note: To deploy a distbuild network you will need the trove's worker keys, deployment will copy these keys onto the nodes to allow them to access all trove content.

In this cluster morph we set ATTACH_DISK to attach artifact storage space to the distbuild workers and FSTAB_HOMES to mount the storage into the right place. It's up to you where this storage comes from, typically you will want at least 50G. In this example we have allocated a logical volume for each worker.

name: x86_64_distbuild_network
kind: cluster
description: A cluster morph for an x86_64 distbuild network
systems:
- morph: systems/build-system-x86_64.morph
  deploy-defaults:
    TROVE_ID: $trove_id
    TROVE_HOST: $trove_url
    CONTROLLERHOST: &controllerhost $controller_url
    DISTBUILD_CONTROLLER: no
    DISTBUILD_WORKER: yes
    INSTALL_FILES: distbuild/manifest
    WORKER_SSH_PUBKEY: worker.key.pub
    WORKER_SSH_KEY: worker.key
    VERSION_LABEL: distbuild
    DISK_SIZE: 5G
    RAM_SIZE: 2G
    VCPUS: 2
    FSTAB_HOMES: LABEL=src /srv/distbuild auto defaults 0 0
  deploy:
    controller:
      type: extensions/kvm
      location: kvm+ssh://user@vmhost/distbuild-controller/path/to/vm/controller_vm.img
      DISTBUILD_CONTROLLER: yes
      DISTBUILD_WORKER: no
      HOSTNAME: *controllerhost
      WORKERS: $worker0_url, $worker1_url
      FSTAB_HOMES: no
    worker0:
      type: extensions/kvm
      location: kvm+ssh://user@vmhost/distbuild-worker0/path/to/vm/worker0_vm.img
      HOSTNAME: $worker0_url
      ATTACH_DISKS: /dev/vg/worker0_src
    worker1:
      type: extensions/kvm
      location: kvm+ssh://user@vmhost/distbuild-worker1/path/to/vm/worker1_vm.img
      HOSTNAME: $worker1_url
      ATTACH_DISKS: /dev/vg/worker1_src

Then deploy:

$ morph deploy distbuild-cluster.morph

Using a distbuild network

To run a build using your distbuild network:

$ morph distbuild --controller-initiator-address=<controllerhost> <system>

Distbuild places build logs for each chunk in the current working directory, for example the log for a build of pango-misc would be stored in build-step-pango-misc.log, so you could view build progress using:

$ tailf build-step-pango-misc.log