<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Sachin on CI/CD]]></title><description><![CDATA[Sachin on CI/CD]]></description><link>https://sachinwarp.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Sat, 21 Nov 2026 15:14:41 GMT</lastBuildDate><atom:link href="https://sachinwarp.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[A Queued GitHub Actions Job Is Waiting on One of Five Things]]></title><description><![CDATA[Why GitHub Actions jobs sit in Queued: measure the wait from the API, match the pattern to one of five causes, and fix it.]]></description><link>https://sachinwarp.hashnode.dev/a-queued-github-actions-job-is-waiting-on-one-of-five-things</link><guid isPermaLink="true">https://sachinwarp.hashnode.dev/a-queued-github-actions-job-is-waiting-on-one-of-five-things</guid><category><![CDATA[GitHub]]></category><category><![CDATA[Devops]]></category><category><![CDATA[Devops articles]]></category><category><![CDATA[CI/CD]]></category><category><![CDATA[github-actions]]></category><dc:creator><![CDATA[Sachin]]></dc:creator><pubDate>Thu, 19 Nov 2026 15:38:00 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6aff16450bddce7c88f112e1/2161ccc6-3709-45bd-9743-5d4cea3e18fb.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>A GitHub Actions job only leaves "Queued" when five things are true at the same time. If it sits there for minutes, one of them is false, and only some of them get fixed by adding runners.</p>
<p>For a sense of normal: on a busy private monorepo running ephemeral Linux VMs, I measured 105 jobs over about 16 hours. The median wait was 13 seconds and the longest was 21. If your median is in minutes, this post shows how to find which of the five is holding your jobs, and what to change.</p>
<p>You need the <a href="https://cli.github.com">GitHub CLI</a> with read access to the repository's Actions. Its built-in <code>--jq</code> handles the JSON.</p>
<h2>The five things a job waits on</h2>
<ol>
<li><strong>Your account is under its concurrent job limit</strong> for that runner type.</li>
<li><strong>A runner with every one of the job's</strong> <code>runs-on</code> <strong>labels is online and free.</strong></li>
<li><strong>The job's</strong> <code>concurrency</code> <strong>group is free</strong>, if the workflow uses one.</li>
<li><strong>Everything before it is done</strong>: the jobs it <code>needs:</code>, environment approvals, and first-time contributor approval on public repos.</li>
<li><strong>GitHub's runner service is healthy.</strong></li>
</ol>
<h2>Measure the wait</h2>
<p>The Actions UI shows how long a job ran, not how long it waited. The API has both timestamps. <code>created_at</code> is when the job was queued and <code>started_at</code> is when a runner picked it up.</p>
<p>This prints the 20 longest waits across your last 30 runs:</p>
<pre><code class="language-bash">OWNER=your-org
REPO=your-repo

gh api "repos/$OWNER/$REPO/actions/runs?per_page=30" --jq '.workflow_runs[].id' |
while read -r run; do
  gh api "repos/$OWNER/$REPO/actions/runs/$run/jobs?per_page=100" --jq '
    .jobs[] | select(.started_at != null and .runner_name != null)
    | [((.started_at | fromdate) - (.created_at | fromdate)), (.labels | join(",")), .name]
    | @tsv'
done | sort -rn | head -20
</code></pre>
<p>The output looks something like this (seconds waited, labels, job name):</p>
<pre><code class="language-text">725   macos-15,xlarge   build-ios
270   ubuntu-latest     test (ubuntu)
</code></pre>
<blockquote>
<p><strong>Keep the</strong> <code>runner_name</code> <strong>filter.</strong> Skipped jobs (from an <code>if:</code> that was false) never get a runner and come back as 0 seconds, which makes the queue look healthier than it is. Jobs still waiting also have an empty <code>runner_name</code>.</p>
</blockquote>
<h2>Match the pattern</h2>
<p>Two things in the output point to the cause: which labels wait, and when.</p>
<table>
<thead>
<tr>
<th>What you see</th>
<th>Which of the five</th>
</tr>
</thead>
<tbody><tr>
<td>Everything waits at busy times, then clears</td>
<td>1. Concurrent job limit</td>
</tr>
<tr>
<td>Only macOS jobs wait</td>
<td>1. The macOS limit, or GitHub's macOS capacity (5)</td>
</tr>
<tr>
<td>One label never starts</td>
<td>2. Labels or runner group</td>
</tr>
<tr>
<td>Self-hosted jobs wait in bursts</td>
<td>2. Not enough runners</td>
</tr>
<tr>
<td>A run is "pending" behind another run of the same workflow</td>
<td>3. Concurrency group</td>
</tr>
<tr>
<td>Waiting with an environment or approval message</td>
<td>4. Something before it</td>
</tr>
<tr>
<td>Every repository waits at once</td>
<td>5. GitHub incident</td>
</tr>
</tbody></table>
<h2>1. The concurrent job limit</h2>
<p>GitHub-hosted runners cap how many jobs an account runs at once. From GitHub's <a href="https://docs.github.com/en/actions/reference/limits">Actions limits</a> page as of November 2026:</p>
<table>
<thead>
<tr>
<th>Plan</th>
<th>Concurrent jobs</th>
<th>Concurrent macOS jobs</th>
</tr>
</thead>
<tbody><tr>
<td>Free</td>
<td>20</td>
<td>5</td>
</tr>
<tr>
<td>Pro</td>
<td>40</td>
<td>5</td>
</tr>
<tr>
<td>Team</td>
<td>60</td>
<td>5</td>
</tr>
<tr>
<td>Enterprise</td>
<td>500</td>
<td>50</td>
</tr>
</tbody></table>
<p>A matrix of 4 Node versions and 5 test shards is 20 jobs. With lint and build, one push takes 22 slots, so two pushes at once on Team get close to 60. Coding agents that open pull requests on their own fill it faster.</p>
<p>macOS is tighter. Free, Pro and Team get 5 concurrent macOS jobs, and GitHub says larger macOS runners share that same limit. One iOS push with a build, unit tests and two simulator jobs fills it. GitHub's macOS capacity also runs short at times, including for images close to retirement (macOS 14 right now), so macOS jobs can wait even under your own limit.</p>
<p>What brings the wait down:</p>
<ul>
<li>Cancel superseded pull request runs (YAML under section 3).</li>
<li>Run one Node version on pull requests and the full matrix on <code>main</code> or nightly.</li>
<li>Merge very short jobs. A 10 second lint job still takes a slot and a runner boot.</li>
<li>Keep work that doesn't need Xcode off Macs, and run the full simulator suite before merge, not on every push.</li>
<li>Ask GitHub Support, which can raise limits on paid plans.</li>
<li>Move heavy jobs to runners outside this cap. Self-hosted and third-party runners have their own limits, and some don't cap concurrency at all (<a href="https://www.warpbuild.com/docs/ci/cloud-runners#concurrency">WarpBuild</a> doesn't on Linux and Windows). When comparing runners, check concurrency before price. It's often not on the pricing page.</li>
</ul>
<h2>2. No runner with the right labels is free</h2>
<p>A job only starts on a runner that has <strong>all</strong> of its <code>runs-on</code> labels. If none is online, the job doesn't fail. It shows "Waiting for a runner to pick up this job...", and for self-hosted runners GitHub cancels it after 24 hours.</p>
<p>Label problems usually look like this:</p>
<ul>
<li>A typo or a renamed label (<code>linux-arm64</code> in the workflow, <code>arm64-linux</code> on the runner).</li>
<li>One extra label no online runner has, like <code>[self-hosted, linux, gpu]</code> while the GPU machine is down.</li>
<li>A runner group that doesn't allow this repository, or a larger runner label copied from another organization's workflow.</li>
</ul>
<p>Compare the labels from the script with <strong>Settings &gt; Actions &gt; Runners</strong> and check the runner group's repository access.</p>
<p>With self-hosted runners, a wait in bursts means your own capacity is short. Common reasons are too few runners at peak, long-lived runners stuck on an old job or offline after an OS update, and autoscaling that only starts a machine after a job is queued. Keep a few warm runners during working hours, use ephemeral runners so one broken machine doesn't block the next job, and rerun the script after each change.</p>
<h2>3. The concurrency group is busy</h2>
<p>A job that joins a <code>concurrency</code> group with a run already in progress waits, and GitHub shows it as "pending". By default only one run waits per group, and a newer one replaces it.</p>
<p>For pull requests, put the branch in the group and cancel the old run:</p>
<pre><code class="language-yaml">concurrency:
  group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
  cancel-in-progress: true
</code></pre>
<p><code>github.head_ref</code> only exists on <code>pull_request</code> events. The <code>run_id</code> fallback gives each push to <code>main</code> its own group, so those never cancel each other.</p>
<p>For deploys, let runs wait their turn:</p>
<pre><code class="language-yaml">concurrency:
  group: deploy-production
  queue: max
</code></pre>
<p><code>queue: max</code> lets up to 100 runs wait. GitHub rejects the workflow if you combine it with <code>cancel-in-progress: true</code>.</p>
<blockquote>
<p><strong>A group with no branch in it holds everything.</strong> <code>group: ci</code> makes every branch wait for every other branch. It looks like a capacity problem but isn't.</p>
</blockquote>
<h2>4. Something before it isn't done</h2>
<p>These look like a queue, but more runners won't help:</p>
<ul>
<li><code>needs:</code> is waiting for another job to finish.</li>
<li>An environment has required reviewers or a wait timer.</li>
<li>On public repositories, workflows from first-time contributors can need a maintainer's approval.</li>
</ul>
<p>Check the job's environment and the approval banner on the run first.</p>
<h2>5. GitHub is having an incident</h2>
<p>If every repository waits at once and nothing changed on your side, check <a href="https://www.githubstatus.com">githubstatus.com</a> before changing anything.</p>
<h2>Trade-offs you accept</h2>
<ul>
<li>Cancelling superseded runs saves slots, but you lose results for the commits in between. If you need a result for every commit, use <code>queue: max</code> or skip the concurrency group.</li>
<li>GitHub changes these limits from time to time. Check the limits page before planning around the table.</li>
</ul>
<h2>What to check</h2>
<ul>
<li>Queue time from the API, before and after every change.</li>
<li>Which labels wait, and whether it's only at busy hours.</li>
<li>Concurrency group names include the branch.</li>
<li>Runner labels and runner group access match the workflow.</li>
<li>The concurrency limit of any runner you're comparing, not just the price.</li>
</ul>
]]></content:encoded></item></channel></rss>