#Pipeline Model

This document describes how Opal interprets .gitlab-ci.yml and schedules jobs locally.

#Parsing

• Supported include: directives are resolved recursively from the local filesystem. Opal currently handles string paths plus local:, file:, and files: include entries. Local include paths are resolved from the repository root, local include wildcards such as configs/*.yml are supported, include paths can use parse-time environment expansion, and included files must be .yml or .yaml. Plain file: / files: entries are still local-only conveniences rather than full GitLab include:project semantics. include:project is available as a partial local approximation when --gitlab-token is configured, including nested direct local includes inside the fetched project; remote:, template:, and component: still fail explicitly.
• default.* values merge into jobs for the subset Opal models today: image, before_script, after_script, variables, cache, services, timeout, retry, and interruptible. For retry, Opal models max, when, and exit_codes for local rerun decisions.
• inherit:default can now disable or selectively retain the modeled default-key subset: image, before_script, after_script, cache, services, timeout, retry, and interruptible.
• image supports string form, image.name, image:entrypoint, and image:docker:platform / image:docker:user for local engines that can express those options. image strings support shell-style variable expansion from configured variables.
• services supports string form plus mapping entries with name, alias, entrypoint, command, variables, and services:docker:platform / services:docker:user. Mapping-form services must use name for the image; image is rejected. Service image names also support shell-style variable expansion (for example ${MY_ENV_DOCKER_REGISTRY}).
• Job environments include GitLab-style predefined metadata such as CI_JOB_NAME, CI_JOB_NAME_SLUG, CI_JOB_STAGE, CI_PROJECT_DIR, and CI_PIPELINE_ID.
• When you override language-specific home directories through variables, Opal passes them through exactly as configured. That includes paths like CARGO_HOME and RUSTUP_HOME, so a workspace-local override can intentionally or accidentally replace what the base image already provides.
• [[jobs]] runtime overrides from .opal/config.toml can target exact job names to adjust local engine behavior like architecture selection or Linux capability flags without editing the pipeline itself.
• Hidden/template jobs (names beginning with .) may be referenced via extends. Cycles are detected and reported.
• workflow:rules, rules, only, and except are partially supported. See docs/gitlab-parity.md for the exact supported forms and known divergences.

#Local ref context

Opal chooses local branch/tag context conservatively.

• plain opal run and opal plan behave like a normal local push/branch pipeline
• Opal does not infer CI_COMMIT_TAG just because HEAD happens to be tagged
• this matches GitLab's model more closely: branch pipelines and tag pipelines are different pipeline contexts for the same commit

If you want local tag-pipeline behavior, set it explicitly:

CI_COMMIT_TAG=v0.1.0-rc3 opal run --no-tui

or:

GIT_COMMIT_TAG=v0.1.0-rc3 opal plan --no-pager

Behavior summary:

• explicit CI_COMMIT_TAG wins
• GIT_COMMIT_TAG is mapped into CI_COMMIT_TAG when present
• if no explicit tag variable is set, Opal uses branch context instead of guessing from local tags on HEAD
• if you explicitly request tag context and local git state is ambiguous, Opal fails fast instead of guessing

#Graph construction

• Each job becomes a node in a DAG.
• Scheduler dependencies come from explicit needs: or the implicit stage ordering (stages: list) when needs: is not present.
• dependencies: affects which artifacts are mounted/restored for a job; it does not create scheduler edges.
• needs:project can download cross-project artifacts when GitLab credentials are configured.
• Parallel jobs (parallel:<count> or parallel:matrix) expand into individual job instances so you can examine each variant independently.

#Scheduling

• The executor keeps a ready queue and launches up to --max-parallel-jobs at a time.
• Jobs run through the selected local engine:
  • docker, podman, container, orbstack, or sandbox for the supported local engine set.
  • nerdctl remains available as a Linux-oriented option when the underlying containerd environment is directly usable.
  • auto picks a sane default for the current platform.
  • sandbox uses the local Anthropic srt CLI path, so container-image-specific runtime flags are not applied directly by Opal for that job execution path.
  • sandbox network/filesystem/policy behavior is controlled via .opal/config.toml sandbox settings, including per-job generated srt settings profiles.
• For sandbox jobs that launch nested containerized runs, expected sandbox privileges usually include:
  • outbound domains for dependency/image pulls (for example crates.io, index.crates.io, static.crates.io, docker.io, registry-1.docker.io, ghcr.io, quay.io, plus org registries)
  • unix socket access for the engine backends in use (for example /var/run/docker.sock, $HOME/.docker/run/docker.sock, $HOME/.orbstack/run/docker.sock, $HOME/.local/share/containers/podman/machine/podman.sock, $HOME/.containerd-rootless/grpc.sock)
  • write access to workspace, temp, and runtime state paths (for example ., /tmp/**, $HOME/.local/share/opal, engine state/cache directories under $HOME)
  • allow_local_binding = true when local bind/listen is required for service checks or runtime probes
  • enable_weaker_nested_sandbox = true when sandboxed jobs invoke nested sandboxed tooling
• Use home-relative patterns ($HOME/... or ~...) and engine-specific globs instead of machine-specific absolute user paths in shared docs/config examples.
• If Apple container networking in your environment needs a custom resolver, set [container].dns so Opal forwards --dns on container runs.
• Job services start as sibling containers on a per-job network, and Opal performs a readiness gate before running the job script when service inspection is available. On Apple’s container engine, Opal now fails fast if the underlying container network create call stalls instead of hanging indefinitely.
  • Services still run on the run/global engine even when a job-level .opal/config.toml engine override is set.
  • Jobs that use the sandbox engine and also declare services fail fast today because that cross-engine connectivity path is not wired yet.
• Manual jobs (when: manual) appear in the UI and can be started interactively.
• resource_group serializes matching jobs within a local run.
• When a job fails, downstream jobs that depend on it are cancelled, and no new work starts (fail-fast semantics). Use r to restart a failed job after fixing the issue.

This is intentionally a local-runner approximation, not a full reproduction of GitLab Runner orchestration semantics. In particular, service networking, interruptible, and cross-pipeline coordination remain partial.

#Workspace Semantics

• Opal does not do a fresh Git clone/fetch for every local job the way GitLab Runner normally does.
• Instead, Opal prepares a per-job workspace snapshot from your current working tree so local development can run against dirty tracked edits and in-progress source changes.
• The snapshot includes the repository's .git directory so Git-aware local behavior still works inside jobs and during local rule/ref evaluation.
• When the source checkout is a linked worktree (.git is a gitdir pointer file), Opal does not copy that pointer into the snapshot.
• Opal does not create synthetic Git commits while preparing job workspaces.
• This is intentional: Opal is meant to help you debug the pipeline you are actively editing, not force a clean remote-style checkout before every job.
• The snapshot is still Git-aware enough to avoid leaking obvious runtime/build garbage into jobs:
  • top-level and nested Git-ignored paths are excluded
  • runtime-heavy directories such as target/, tests-temp/, .opal/, node_modules/, .svelte-kit/, .wrangler/, and similar generated directories are excluded from the copied workspace
• In practice, the goal is:
  • include source files and dirty local edits you actually want to test
  • exclude generated junk that GitLab Runner would not treat as meaningful source input
• This means Opal is deliberately closer to “run my current local pipeline against my working tree” than to “recreate GitLab’s clone/fetch/clean lifecycle exactly.”

#Artifacts & logs

• Each run gets a session directory under $XDG_DATA_HOME/opal/<run-id>/ (default ~/.local/share/opal/<run-id>/). Job artifacts are stored under $XDG_DATA_HOME/opal/<run-id>/<job>/artifacts/.
• Declared artifacts.paths are copied into that directory and can be consumed by downstream jobs that request needs: { artifacts: true }. artifacts:untracked is also collected.
• dependencies: can mount a narrower artifact subset from earlier jobs, and needs:project can fetch artifacts from GitLab when --gitlab-token is configured.
• Logs are archived under $XDG_DATA_HOME/opal/<run-id>/logs/. The TUI also keeps an in-memory buffer and highlights diff-like lines (+/-).

#Customization

#Config-level env defaults (.opal/config.toml)

• Use a root-level [env] table in .opal/config.toml to inject Opal-only runner defaults into all jobs without modifying .gitlab-ci.yml:

  [env]
  RUNNER_BOOTSTRAP = "enabled"
  RUNNER_INIT_SCRIPT = "/opal/bootstrap/init.sh"

• Values support shell-style expansion against available runtime environment values (for example $HOME).

• These keys are local Opal runtime augmentation, not GitLab YAML syntax.

#Runner bootstrap pre-step (.opal/config.toml)

• Use [bootstrap] when you need Opal to prepare runner-like state before any jobs execute (for example generating helper scripts or runtime env files):

  [bootstrap]
  command = "bash .opal/bootstrap/prepare-runner.sh"
  env_file = "bootstrap/generated.env"
  
  [bootstrap.env]
  RUNNER_HELPER = "/opal/bootstrap/scripts/helper.sh"
  
  [[bootstrap.mounts]]
  host = "bootstrap/scripts"
  container = "/opal/bootstrap/scripts"
  read_only = true

• The bootstrap command runs once per Opal run before job execution.

• env_file (dotenv) and bootstrap.env entries are injected into every job environment.

• bootstrap.mounts are mounted into every job container, which is useful to mimic GitLab runner-provided scripts/tools.

• env_file and bootstrap.mounts.host resolve relative to .opal/config.toml.

• This is Opal-only runtime augmentation; no new .gitlab-ci.yml syntax is introduced.

#Forwarding host env vars

• Use --env GLOB (repeat) to forward selected host environment variables into every job. The glob uses globset syntax, so * and ? work the way they do in shells:

  opal run --env CI_* --env 'AWS_ACCESS_KEY_ID' --env 'APP_??'

  The example above forwards everything starting with CI_, both AWS credentials, and any APP_ var with exactly two characters after the underscore. Patterns are evaluated against the host’s environment, and the matches are injected before job-level variables, so jobs can override them if needed. These forwarded variables are also available for shell-style expansion in image references, including service images. When the same key exists in both --env and .opal/config.toml [env], the --env value wins.

  Repeat --env for each glob you need. Use quotes when your pattern includes characters your shell might expand (e.g., ?).

#Repository secrets (.opal/env)

• Add plain files under .opal/env in your repo; the filename becomes the secret name. For example:

  .opal/env/
  ├─ GITLAB_TOKEN        # contains the token text
  └─ MY_CERT.crt         # binary cert, can be any name

• During a run Opal:

  • Copies the whole directory into the container at /opal/secrets/…, mounted read-only.
  • Sets $GITLAB_TOKEN to the file contents (if UTF‑8) and $GITLAB_TOKEN_FILE=/opal/secrets/GITLAB_TOKEN.
  • Always sets $<NAME>_FILE so you can read binary data even when the value isn’t UTF‑8.
  • Masks the plaintext values in logs (anything matching the file contents is replaced with [MASKED] before printing).

  This mirrors GitLab’s _FILE behavior, so jobs that already expect _FILE env vars work unchanged. Keep .opal/env out of version control (the default .gitignore already ignores it).

• Pass --base-image to supply a default container when jobs do not specify one.

#Tracing job scripts

• Use opal run --trace-scripts … when you want every job to echo its commands as they execute. The flag makes Opal write each generated script with set -x, so you will see the shell-expanded command stream (+ cargo fmt, etc.) in the log.
• Alternatively set OPAL_DEBUG=1 in the environment to enable the same behavior without touching CLI flags (useful when scripting or when you need trace logs globally).
• The tracing flag stacks with any verbosity coming from the job itself; Opal still forces set -eu so jobs fail fast while showing the debug output.

#Planning pipelines

Run opal plan when you want to inspect the DAG without touching containers. The command parses .gitlab-ci.yml, evaluates top-level filters plus workflow/rules, and prints each stage with:

• Job order plus dependency list (depends on shows implicit stage ordering when no explicit needs exist).
• Manual/delayed gates, retry counts, timeouts, and whether a job may fail without stopping the pipeline.
• Artifact paths, environments, services, tags, and resource groups.

It is the fastest way to understand why a job is (or is not) scheduled for the current branch/tag, and it surfaces external needs so you can adjust credentials/infrastructure before running the pipeline for real.

Use --job <name> (repeatable) with either opal plan or opal run when you want to focus on one part of the pipeline. Opal keeps the selected jobs and automatically includes the required upstream dependency closure so the resulting subgraph remains runnable.

This model mirrors a useful local subset of GitLab while remaining deterministic and debuggable on a single machine. When in doubt, compare the DAG produced by opal plan with GitLab’s pipeline graph and consult docs/gitlab-parity.md for known differences.