GuidesDeterminate in Semaphore

Determinate in Semaphore

Semaphore is a powerful CI/CD platform that is compatible with FlakeHub and FlakeHub Cache.

Onboarding

To allow authentication from Semaphore to FlakeHub, you must:

  1. Create an organization on Semaphore and make note of the organization ID.
  2. Contact our support with your Semaphore organization ID and your FlakeHub organization name.
⚠️

The OpenID Connect (OIDC) feature and options are only available to organizations on Semaphore’s Scaleup plan. If your organization isn’t on this plan, your Semaphore pipelines won’t be able to authenticate with FlakeHub.

OIDC configuration for Semaphore

Semaphore allows workloads to request OIDC tokens, which those workloads can then use to authenticate to FlakeHub. To enable OIDC tokens to be passed to FlakeHub:

  1. Go to the OIDC Token Configuration page in Semaphore, which you can find on the OIDC Token tab in your Semaphore organization’s settings.
  2. Ensure that these claims are enabled:
    • org_id
    • org
    • prj
    • prj_id

Example pipeline

This Semaphore pipeline configuration:

  • Installs Determinate Nix
  • Authenticates with FlakeHub to publish flakes and FlakeHub Cache to push to and pull from the organization’s cache
  • Builds a package using Nix (which is automatically pushed to FlakeHub Cache)
  • Publishes the flake to FlakeHub only if a Git tag has been pushed
.semaphore/semaphore.yml
version: v1.0
name: Determinate Nix publish and cache pipeline
agent:
  machine:
    type: f1-standard-4
    os_image: ubuntu2404
blocks:
  - name:
    dependencies: []
    task:
      prologue:
        commands:
          # Get the latest version of the repository's source code from GitHub
          - checkout
 
      jobs:
        - name: Publish flake and cache package
          commands:
          # The flake's repository
          - export FLAKEHUB_PUSH_REPOSITORY="$(echo "${SEMAPHORE_ORGANIZATION_URL}" | cut -d "." -f1 | cut -d '/' -f3)/${SEMAPHORE_PROJECT_NAME}"
 
          # Environment variables for Magic Nix Cache, which automatically pushes Nix artifacts to FlakeHub Cache
          - export MAGIC_NIX_CACHE_CLOSURE_URL="https://install.determinate.systems/magic-nix-cache-closure/branch/main/X64-Linux?ci=semaphore"
          - export MNC_LISTEN="127.0.0.1:37515"
 
          # Install Determinate Nix and start the Nix daemon
          - curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install linux --determinate --no-confirm --init systemd
          - . /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh
 
          # Log in using the `determinate-nixd login` command (used by magic-nix-cache, substitutions)
          - echo "${SEMAPHORE_OIDC_TOKEN}" | determinate-nixd login token --token-file /dev/stdin
 
          # Acquire the `flakehub-push` executable
          - curl -L "${FLAKEHUB_PUSH_BINARY_URL}" | sudo tee /usr/bin/flakehub-push &>/dev/null
          - sudo chmod +x /usr/bin/flakehub-push
 
          # Acquire the `magic-nix-cache` executable
          - export MNC_CLSR="$(curl -L "${MAGIC_NIX_CACHE_CLOSURE_URL}" | xz -d | sudo "$(which nix-store)" --import | tail -n1 | head -n1)"
          - sudo ln -sf "${MNC_CLSR}/bin/magic-nix-cache" /usr/bin/magic-nix-cache
          - magic-nix-cache --help
 
          # Stage login credentials for `flakehub-push`
          - export FLAKEHUB_PUSH_OIDC_TOKEN="${SEMAPHORE_OIDC_TOKEN}"
 
          # Start Magic Nix Cache
          - export MNC_STARTUP_FILE="/tmp/mnc-startup"
          - nohup magic-nix-cache --listen "${MNC_LISTEN}" --startup-notification-file "${MNC_STARTUP_FILE}" &>/tmp/mnc.log &
          - |
              (
              STARTED=0
              for n in {1..6}; do
                if [ -e "${MNC_STARTUP_FILE}" ]; then
                  echo "magic-nix-cache daemon has successfully started up after ${n} attempt(s)"
                  STARTED=1
                  break
                else
                  echo "waiting on magic-nix-cache daemon; on attempt ${n}"
                  sleep 2
                fi
              done
              if [[ "${STARTED}" != "1" ]]; then
                echo "The daemon did not start up within 60 seconds; exiting"
                exit 1
              fi
              ) || true
 
          # Build a package output by the repository's Nix flake
          - nix build ".#packages.x86_64-linux.default"
 
          # Publish a flake release to FlakeHub if and only if it's a tag reference
          - |
            if [[ "${SEMAPHORE_GIT_REF_TYPE}" == "tag" ]]; then
              flakehub-push \
                --tag "$(cat "${SEMAPHORE_GIT_REF}" | cut -d '/' -f2)" \
                --visibility private \
                --include-output-paths
            fi
 
          # Stop Magic Nix Cache
          - curl -XPOST "http://${MNC_LISTEN}/api/workflow-finish"
Easy caching with Determinate CIMigrating to FlakeHub Cache