Keep the progress bar moving for the whole run

Three stretches of a run had no way to report anything, and the bar
handled each of them badly.

pacman prints nothing at all between "starting full system upgrade" and
the transaction it eventually prepares. On a 161-package backlog that
silence ran to three minutes and nineteen seconds, and the bar spent all
of it frozen on "0 of 161" - a counter seeded from checkupdates before
there was anything to count. Working out the upgrade is now a step of its
own, with a label and deliberately no item count, because the number was
the part that was lying.

A step that turns out to have no work is dropped from the bar instead of
being handed its share for nothing. Packages already in the cache are
never announced by pacman, so a run that only has to unpack used to jump
thirty points the moment unpacking started; the same went for AUR with
nothing pending and for machines with no Gear Lever or no Flatpaks.

pacman's output is line-buffered through stdbuf. Writing to a log rather
than a terminal, libc released it in 4KB blocks - around a hundred and
sixty "upgrading foo..." lines at a time - so the bar sat still and then
leapt to the end of the step in one poll.

What is left is work whose length genuinely cannot be known: resolving a
transaction, and an AUR helper compiling for a quarter of an hour. Those
now creep along a curve that approaches the end of their step without
reaching it. The item counter stays put throughout - it is the field that
would be lying if it moved - and any real report overtakes the creep. A
bar that has not moved since it appeared is read as a hang, and somebody
who reads it that way reaches for the power button mid-update.

The bar is also monotonic now. Dropping a step rescales the run, and the
conflict-recovery loop restarts pacman and its tally from the top; both
are honest, neither is a reason to show a bar that retreats.

README.md

@@ -214,19 +214,33 @@ Two things about how it is put together:
   the length of the update, holding the connection open and taking instructions
   on stdin. It needs **python-gobject**; without it there is simply no bar and
   nothing else changes.
-- Downloading and unpacking are two separate steps on the bar. On a domestic
-  line the download is the longer of the two, and calling the whole thing
-  "installing" leaves the bar sitting at 4% for six minutes, which reads as a
-  hang rather than as progress.
-- Neither phase carries a counter on an unattended run, so both are counted a
-  line at a time — `foo-1.2-1-x86_64 downloading...` and `upgrading foo...`.
-  The database sync just before prints the same shape (` core downloading...`)
-  with the suffix that would give it away already stripped, so counting starts
-  only after pacman's `:: Retrieving packages...` header. pacman's other
-  `(n/m)` sequences — checking keys, package integrity, loading files — each
-  count to the same total, so only the transaction verbs are followed;
-  otherwise the bar would reach the end three times before the first package
-  was unpacked.
+- Working out the upgrade, downloading it and unpacking it are three separate
+  steps. The first is the one that used to look broken: between
+  `:: Starting full system upgrade...` and the transaction it eventually
+  prepares, pacman prints nothing at all, and on a large backlog that silence
+  runs to minutes. A counter frozen at "0 of 161" reads as a stuck update, so
+  that stretch carries a label and deliberately no counter. On a domestic line
+  the download is then the longest of the three, and calling the whole thing
+  "installing" would leave the bar at 4% for six minutes.
+- A step that turns out to have no work is dropped from the bar rather than
+  handed its share for nothing. Packages already in the cache are never
+  announced, so a run that only has to unpack skips the download step outright
+  instead of leaping 30% the moment unpacking starts — and likewise for AUR
+  with nothing pending, or a machine with no Flatpaks. The weights only ever
+  have to be right about the steps that actually run.
+- pacman's output is line-buffered through `stdbuf`. Writing to a log rather
+  than a terminal, libc would hand it over in 4 KB blocks instead, and 4 KB of
+  `upgrading foo...` is on the order of a hundred and sixty packages arriving
+  at once — which is how a bar comes to sit still and then jump to the end.
+- Neither counted phase gets a counter from pacman on an unattended run, so
+  both are counted a line at a time — `foo-1.2-1-x86_64 downloading...` and
+  `upgrading foo...`. The database sync just before prints the same shape
+  (` core downloading...`) with the suffix that would give it away already
+  stripped, so counting starts only after pacman's `:: Retrieving packages...`
+  header. pacman's other `(n/m)` sequences — checking keys, package integrity,
+  loading files — each count to the same total, so only the transaction verbs
+  are followed; otherwise the bar would reach the end three times before the
+  first package was unpacked.
 
 This is Plasma's job interface. On a desktop that does not implement it the
 helper exits quietly and the ordinary notifications carry on as before.

doc/cachy-auto-update.1.scd

@@ -122,18 +122,29 @@ of the update and holds that connection open. It requires _python-gobject_.
 Where that is missing, or on a desktop with no job interface, there is no
 progress entry and nothing else is affected.
 
-Fetching the packages and unpacking them are two steps rather than one. On a
-domestic line the download is the longer of the two, and a bar that called the
-whole thing "installing" would sit near its beginning for minutes at a time
-looking stuck.
+Working out the upgrade, fetching it and unpacking it are three steps rather
+than one. Between "Starting full system upgrade" and the transaction it
+eventually prepares, pacman prints nothing at all, and on a large backlog that
+silence runs to minutes; that stretch therefore carries a label of its own and
+deliberately no item count, because a counter frozen at "0 of 161" reads as a
+stuck update. On a domestic line the download is then the longest of the three,
+and a bar that called the whole thing "installing" would sit near its beginning
+for minutes at a time looking stuck.
 
-Neither phase gets a counter from pacman on an unattended run, so both are
-counted here, a line at a time: "foo-1.2-1-x86_64 downloading..." for the
-first, "upgrading foo..." for the second. Packages already in the cache are
-never announced, so the download step regularly ends short of its total and
-gives up the rest of its share when unpacking begins. pacman's other (n/m)
-sequences - checking keys, package integrity, loading package files - each
-count up to the same total and are deliberately ignored.
+A step that turns out to have no work is dropped from the bar instead of being
+handed its share for nothing. Packages already in the cache are never
+announced, so a run with everything already fetched skips the download step
+outright rather than jumping when unpacking starts; the same applies to AUR
+with nothing pending, or a machine with no Flatpaks installed.
+
+Neither counted phase gets a counter from pacman on an unattended run, so both
+are counted here, a line at a time: "foo-1.2-1-x86_64 downloading..." for the
+first, "upgrading foo..." for the second. pacman's output is line-buffered
+through *stdbuf*(1) so those lines arrive as they happen - writing to a log
+rather than a terminal, libc would otherwise release them in 4KB blocks, around
+a hundred and sixty packages at a time. pacman's other (n/m) sequences -
+checking keys, package integrity, loading package files - each count up to the
+same total and are deliberately ignored.
 
 # HOW LONG NOTIFICATIONS STAY
 

po/cachy-auto-update.pot

@@ -234,6 +234,12 @@ msgstr ""
 #. looking for it - it appears next to whatever they were doing - so each one
 #. says outright that this is an update running, rather than naming the kind of
 #. package on its own.
+msgid "Checking for updates"
+msgstr ""
+
+msgid "Preparing the update"
+msgstr ""
+
 msgid "Downloading updates"
 msgstr ""
 

po/de.po

@@ -235,6 +235,12 @@ msgstr "Ohne Befehl wird ein interaktives Menü angezeigt."
 #. looking for it - it appears next to whatever they were doing - so each one
 #. says outright that this is an update running, rather than naming the kind of
 #. package on its own.
+msgid "Checking for updates"
+msgstr "Nach Updates wird gesucht"
+
+msgid "Preparing the update"
+msgstr "Update wird vorbereitet"
+
 msgid "Downloading updates"
 msgstr "Updates werden heruntergeladen"
 

src/cachy-auto-update-run

@@ -185,7 +185,7 @@ failed=0
 # Open the desktop's progress bar, told up front which steps this run will
 # perform. Only those count towards the bar, so a machine with no Flatpaks
 # does not sit at 85% for the last second of the run.
-progress_steps=(download repo)
+progress_steps=(resolve download repo)
 [[ $CFG_AUR == yes ]]      && progress_steps+=(aur)
 [[ $CFG_FLATPAK == yes ]]  && progress_steps+=(flatpak)
 [[ $CFG_APPIMAGE == yes ]] && progress_steps+=(appimage)

src/lib/pkg_appimage.sh

@@ -43,6 +43,26 @@ cau_appimage_update() {
 	local user uid count rc=0
 	local -a cmd
 
+	# Is there a Gear Lever on this machine at all? Asked before the step is
+	# announced rather than discovered inside the loop: on a machine without
+	# one - the common case, it is an optional dependency - a step that exists
+	# only to hand its share of the bar straight to the next one is a jump the
+	# bar does not need. Stops at the first user who has it, so the extra probe
+	# costs anything only in the case it is there to remove.
+	local found=0
+	while read -r user uid; do
+		[[ -n $user ]] || continue
+		if _cau_gearlever_cmd "$user" "$uid" > /dev/null; then
+			found=1
+			break
+		fi
+	done < <(cau_active_session_users)
+
+	if (( ! found )); then
+		cau_progress_drop appimage
+		return 0
+	fi
+
 	cau_progress_step appimage "Updating AppImages"
 
 	while read -r user uid; do

src/lib/pkg_aur.sh

@@ -124,7 +124,9 @@ cau_aur_update() {
 	local pending failures
 	local -a args
 
-	cau_aur_ready || return 0
+	# No helper, no base-devel, no bar: a step that cannot run should not be
+	# holding a share of it.
+	cau_aur_ready || { cau_progress_drop aur; return 0; }
 
 	cau_progress_step aur "Updating AUR packages"
 
@@ -132,23 +134,30 @@ cau_aur_update() {
 	if (( pending == 0 )); then
 		cau_info "No AUR updates pending"
 		cau_state_clear aur_failures
+		cau_progress_drop aur
 		return 0
 	fi
 
 	cau_info "Updating $pending AUR package(s) with $CAU_AUR_HELPER"
 
-	# The helper builds each package from source with no counter this side of
-	# its output, so the bar sits at the start of the step until it is done.
+	# The helper builds each package from source and prints plenty about it,
+	# none of it countable from this side. A single large package can take ten
+	# minutes, so the bar creeps through the step rather than sitting at its
+	# start for all of them; the item count stays where it is, because that is
+	# the number that would be lying if it moved.
 	cau_progress_item 0 "$pending"
 	mapfile -t args < <(cau_aur_helper_args)
+	cau_progress_creep_start
 
 	if cau_run_logged cau_as_build_user "$CAU_AUR_HELPER" "${args[@]}"; then
+		cau_progress_creep_stop
 		CAU_AUR_COUNT="$pending"
 		cau_progress_item "$pending"
 		cau_state_clear aur_failures
 		return 0
 	fi
 
+	cau_progress_creep_stop
 	CAU_AUR_COUNT=0
 	failures="$(cau_state_read aur_failures 0)"
 	[[ $failures =~ ^[0-9]+$ ]] || failures=0

src/lib/pkg_flatpak.sh

@@ -26,21 +26,31 @@ cau_flatpak_pending_system() {
 cau_flatpak_update() {
 	local rc=0 pending user uid home count
 
-	cau_have flatpak || return 0
+	cau_have flatpak || { cau_progress_drop flatpak; return 0; }
 
 	cau_progress_step flatpak "Updating Flatpak apps"
 
-	# refresh appstream metadata first so remote-ls sees current versions
+	# refresh appstream metadata first so remote-ls sees current versions.
+	# Nothing is countable until that has finished, so the bar creeps rather
+	# than waiting at the start of the step for it.
+	cau_progress_creep_start
 	cau_run_logged flatpak update --appstream --system --noninteractive || true
+	cau_progress_creep_stop
 
 	pending="$(cau_flatpak_pending_system)"
 	if (( pending > 0 )); then
 		cau_info "Updating $pending system Flatpak(s)"
 		cau_progress_item 0 "$pending"
+		# Started after the count above and stopped before the one below, so the
+		# only reports this side makes while a ticker is running are further
+		# along than the ticker ever gets.
+		cau_progress_creep_start
 		if cau_run_logged flatpak update --system --noninteractive --assumeyes; then
+			cau_progress_creep_stop
 			CAU_FLATPAK_COUNT=$(( CAU_FLATPAK_COUNT + pending ))
 			cau_progress_item "$pending"
 		else
+			cau_progress_creep_stop
 			cau_warn "System Flatpak update failed"
 			rc=1
 		fi

src/lib/pkg_pacman.sh

@@ -52,11 +52,18 @@ CAU_PACMAN_DL_AWK='
 	END { print n + 0, name }'
 
 # _cau_pacman_progress_watch <logfile>
-# Feeds the desktop's progress bar by watching pacman work, through both of the
-# phases a pacman run has: first everything is fetched, then everything is
-# unpacked. They are two steps on the bar rather than one, because they are two
-# steps to sit through - a run that has been "installing updates" at 4% for six
-# minutes has not hung, it is still downloading, and the bar should say so.
+# Feeds the desktop's progress bar by watching pacman work, through the three
+# phases a pacman run has: first it works out what the upgrade consists of,
+# then everything is fetched, then everything is unpacked. Three steps on the
+# bar rather than one, because they are three stretches to sit through - and
+# each one is silent in its own way.
+#
+# The first is the one that used to look like a hang. Between "starting full
+# system upgrade" and the transaction it eventually prepares, pacman says
+# nothing whatsoever, and on a large backlog that silence is minutes long. The
+# only honest thing to show there is a label and no counter at all: a tally
+# frozen at "0 of 161" reads as a stuck update, where "Preparing the update"
+# with no number reads as what it actually is.
 #
 # In the transaction, pacman announces each package twice over, in one of two
 # shapes, and which one depends on a flag this program sets itself:
@@ -83,7 +90,8 @@ CAU_PACMAN_DL_AWK='
 _cau_pacman_progress_watch() {
 	local log="$1"
 	local total="${CAU_PACMAN_COUNT:-0}" announced processed line pkg last=''
-	local phase=download fetched shown=''
+	local phase=resolve fetched shown='' labelled=''
+	local t0=$SECONDS
 
 	while :; do
 		sleep 1
@@ -94,26 +102,15 @@ _cau_pacman_progress_watch() {
 
 		line="$(grep -aoE "$CAU_PACMAN_OP_RE" "$log" 2>/dev/null | tail -n1)"
 
-		# Nothing unpacked yet, so this is still the download - or the database
-		# sync ahead of it, which the awk above declines to count.
-		if [[ -z $line ]]; then
-			read -r fetched pkg < <(awk "$CAU_PACMAN_DL_AWK" "$log" 2>/dev/null)
-			[[ $fetched =~ ^[0-9]+$ ]] && (( fetched > 0 )) || continue
-			[[ $fetched != "$shown" ]] || continue
-			shown="$fetched"
-
-			cau_progress_item "$fetched" "$total"
-			# Down to the bare name, as the transaction reports it: the file
-			# pacman names here carries version, release and architecture.
-			cau_progress_detail "Package" "${pkg%-*-*-*}"
-			continue
-		fi
-
-		# The first package being unpacked ends the download step. Its share of
-		# the bar is given up wherever it had got to - packages already in the
-		# cache are fetched in no time at all and never print a line, so the
-		# tally regularly stops short of the total it was promised.
-		if [[ $phase == download ]]; then
+		if [[ -n $line ]]; then
+			# Unpacking has started, so whatever came before it is over. If
+			# nothing was ever retrieved - every package already sitting in the
+			# cache, which is the ordinary state of affairs after a run that was
+			# interrupted once already - then the download step never happened,
+			# and it is dropped rather than handed its whole share of the bar in
+			# exchange for no work at all.
+			if [[ $phase != install ]]; then
+				[[ $phase == download ]] || cau_progress_drop download
 				phase=install
 				cau_progress_step repo "Updating system packages" "$total"
 			fi
@@ -126,8 +123,8 @@ _cau_pacman_progress_watch() {
 			processed="$(grep -acE "$CAU_PACMAN_OP_RE" "$log" 2>/dev/null)"
 			[[ $processed =~ ^[0-9]+$ ]] || continue
 
-		# Where pacman does carry a counter, believe it over the tally: it is
-		# the same number, but it also knows the true total.
+			# Where pacman does carry a counter, believe it over the tally: it
+			# is the same number, but it also knows the true total.
 			if [[ $line =~ ^\([[:space:]]*([0-9]+)/([0-9]+)\) ]]; then
 				processed="${BASH_REMATCH[1]}"
 				total="${BASH_REMATCH[2]}"
@@ -137,6 +134,41 @@ _cau_pacman_progress_watch() {
 
 			pkg="${line##* }"
 			cau_progress_detail "Package" "${pkg%...}"
+			continue
+		fi
+
+		# Fetching. The header is what tells this apart from the database sync
+		# a few lines earlier, which prints the very same shape.
+		if grep -qa '^:: Retrieving packages' "$log" 2>/dev/null; then
+			if [[ $phase == resolve ]]; then
+				phase=download
+				cau_progress_step download "Downloading updates" "$total"
+			fi
+
+			read -r fetched pkg < <(awk "$CAU_PACMAN_DL_AWK" "$log" 2>/dev/null)
+			[[ $fetched =~ ^[0-9]+$ ]] || continue
+			[[ $fetched != "$shown" ]] || continue
+			shown="$fetched"
+
+			cau_progress_item "$fetched" "$total"
+			# Down to the bare name, as the transaction reports it: the file
+			# pacman names here carries version, release and architecture.
+			[[ -n $pkg ]] && cau_progress_detail "Package" "${pkg%-*-*-*}"
+			continue
+		fi
+
+		# Still resolving. pacman does mark the point where it stops syncing
+		# databases and starts working out the upgrade, and that is the half
+		# worth naming, because it is the half that takes the minutes.
+		if [[ $phase == resolve && $labelled != upgrade ]] \
+			&& grep -qa '^:: Starting full system upgrade' "$log" 2>/dev/null; then
+			labelled=upgrade
+			cau_progress_step resolve "Preparing the update"
+		fi
+
+		# Nothing countable happens in here at all, so the bar creeps instead.
+		# This is the stretch that used to look like a hung update.
+		cau_progress_creep $(( SECONDS - t0 ))
 	done
 }
 
@@ -156,11 +188,21 @@ _cau_pacman_exec() {
 		watcher=$!
 	fi
 
+	# Line-buffered on purpose. pacman writes to a file or through a pipe here,
+	# never to a terminal, so libc buffers it in 4KB blocks - and 4KB of
+	# "upgrading foo..." is on the order of a hundred and sixty packages. The
+	# watcher would see nothing at all, then a hundred and sixty lines at once,
+	# which is exactly how a bar comes to sit still and then leap to the end.
+	# Guarded rather than assumed: without coreutils there is no bar to feed
+	# either, but there is still an update to run.
+	local -a buffered=()
+	cau_have stdbuf && buffered=(stdbuf -oL)
+
 	if [[ -n $CAU_INTERACTIVE ]]; then
-		pacman "$@" 2>&1 | tee "$log"
+		"${buffered[@]}" pacman "$@" 2>&1 | tee "$log"
 		rc="${PIPESTATUS[0]}"
 	else
-		pacman "$@" > "$log" 2>&1
+		"${buffered[@]}" pacman "$@" > "$log" 2>&1
 		rc=$?
 	fi
 
@@ -245,12 +287,18 @@ cau_pacman_update() {
 	local log kind
 	local -a flags
 
-	# The download comes first and the watcher moves on to the repo step once
-	# pacman starts unpacking.
-	cau_progress_step download "Downloading updates"
+	# checkupdates goes first, against its own private database, and the bar
+	# says so rather than naming a step that has not begun. The watcher takes
+	# over from here and moves on to the download and repo steps as pacman
+	# actually reaches them.
+	cau_progress_step resolve "Checking for updates"
 
 	if ! cau_pacman_pending; then
 		cau_info "No repository updates pending"
+		# Neither of the two steps this would have led to is going to happen,
+		# so the rest of the run gets their share of the bar instead of
+		# watching it jump 70% the moment Flatpaks start.
+		cau_progress_drop download repo
 		return 0
 	fi
 
@@ -258,7 +306,10 @@ cau_pacman_update() {
 		CAU_PACMAN_COUNT="$(grep -c . <<< "$CAU_PACMAN_PENDING")"
 		[[ $CAU_PACMAN_COUNT =~ ^[0-9]+$ ]] || CAU_PACMAN_COUNT=0
 		cau_info "Updating $CAU_PACMAN_COUNT repository package(s)"
-		cau_progress_item 0 "$CAU_PACMAN_COUNT"
+		# Deliberately no item count yet. Until pacman has prepared a
+		# transaction there is nothing being worked through, and "0 of 161"
+		# against a bar that cannot move for the next few minutes is the exact
+		# impression the resolve step exists to avoid.
 	else
 		# checkupdates is unavailable, so the list is unknown and pacman is
 		# asked to work it out itself.
@@ -291,6 +342,13 @@ cau_pacman_update() {
 
 	while true; do
 		if _cau_pacman_exec "$log" -Syu "${flags[@]}" "${extra[@]}"; then
+			# The watcher decided the same thing in a subshell, so a step it
+			# dropped is still in the plan out here. Same question, same answer,
+			# and the two copies agree on what the rest of the run is scaled
+			# against. Only on the way out: a failed attempt is about to be
+			# retried, and that retry may well download after all.
+			grep -qa '^:: Retrieving packages' "$log" 2>/dev/null \
+				|| cau_progress_drop download
 			cat "$log" >> "$CAU_RUNLOG" 2>/dev/null
 			grep -E '^(removing|replacing) ' "$log" 2>/dev/null \
 				| while read -r line; do cau_info "  $line"; done

src/lib/progress.sh

@@ -30,8 +30,15 @@ CAU_PROGRESS_LOCALES=()
 # them about equally, on a domestic line - and the cleanup is a rounding error.
 # They do not have to add up to 100 - only the steps a given run will actually
 # perform are counted, and the total is normalised against those.
+#
+# "resolve" is everything pacman does before it has a transaction: syncing the
+# databases and working out what the upgrade actually consists of. It is
+# usually seconds, which is why it is worth so little - but on a large backlog
+# it is minutes, and those minutes used to be spent looking at a bar that had
+# not moved yet.
 declare -A CAU_PROGRESS_WEIGHTS=(
-	[download]=30 [repo]=40 [aur]=15 [flatpak]=10 [appimage]=3 [cleanup]=2
+	[resolve]=10 [download]=30 [repo]=40 [aur]=15 [flatpak]=10 [appimage]=3
+	[cleanup]=2
 )
 
 CAU_PROGRESS_PLAN=()
@@ -133,6 +140,49 @@ cau_progress_active() {
 	(( ${#CAU_PROGRESS_FDS[@]} ))
 }
 
+# cau_progress_drop <step-id...>
+# Takes steps out of the plan and rescales the bar to what is left.
+#
+# Which steps a run will perform is only half known up front. The other half
+# turns up while it runs: nothing to download because every package was already
+# in the cache, no AUR updates pending, no Flatpaks installed. A step like that
+# keeps its whole share of the bar and then hands it over in a single jump the
+# moment the next one starts - which is precisely the stutter this is here to
+# remove. Dropping it hands its share to the steps that do have work instead,
+# so the bar advances at a steady pace rather than leaping across the gaps.
+#
+# It is also what lets the weights above stay rough: they never have to be
+# right about a step that does not run, only about the ones that do.
+#
+# Only ever called for a step that has not started, so nothing already behind
+# the bar is rescaled and the bar does not travel backwards.
+cau_progress_drop() {
+	local drop step
+	local -a kept=()
+
+	(( ${#CAU_PROGRESS_FDS[@]} )) || return 0
+
+	for step in "${CAU_PROGRESS_PLAN[@]}"; do
+		for drop in "$@"; do
+			[[ $step == "$drop" ]] && continue 2
+		done
+		kept+=("$step")
+	done
+
+	(( ${#kept[@]} == ${#CAU_PROGRESS_PLAN[@]} )) && return 0
+
+	CAU_PROGRESS_PLAN=("${kept[@]}")
+	CAU_PROGRESS_SCALE=0
+	for step in "${CAU_PROGRESS_PLAN[@]}"; do
+		CAU_PROGRESS_SCALE=$(( CAU_PROGRESS_SCALE + ${CAU_PROGRESS_WEIGHTS[$step]:-0} ))
+	done
+
+	# Nothing left to weigh against would divide by zero further down. Cannot
+	# happen while cleanup is unconditional, but this is cheaper than relying
+	# on that staying true.
+	(( CAU_PROGRESS_SCALE > 0 )) || CAU_PROGRESS_SCALE=1
+}
+
 # cau_progress_step <step-id> <label-msgid> [item-count]
 # Moves on to the next step. The bar jumps to where that step begins, so a step
 # that reported fewer items than it promised still completes rather than
@@ -143,11 +193,17 @@ cau_progress_step() {
 
 	(( ${#CAU_PROGRESS_FDS[@]} )) || return 0
 
+	local found=0
 	for step in "${CAU_PROGRESS_PLAN[@]}"; do
-		[[ $step == "$id" ]] && break
+		[[ $step == "$id" ]] && { found=1; break; }
 		base=$(( base + ${CAU_PROGRESS_WEIGHTS[$step]:-0} ))
 	done
 
+	# A step that was dropped for having no work is not a step to move to.
+	# Without this the loop above would fall off the end of the plan and hand
+	# back the sum of every weight, i.e. send the bar straight to 100%.
+	(( found )) || return 0
+
 	CAU_PROGRESS_BASE=$base
 	CAU_PROGRESS_SPAN=${CAU_PROGRESS_WEIGHTS[$id]:-0}
 	CAU_PROGRESS_TOTAL=$total
@@ -170,10 +226,38 @@ cau_progress_step() {
 	cau_progress_item 0
 }
 
+# _cau_progress_pct <numerator> <denominator>
+# How far through the current step we are, as a share of its span, turned into
+# one number for the whole run and sent on if it has moved.
+_cau_progress_pct() {
+	local num="$1" den="$2" pct scaled
+
+	if (( den > 0 )); then
+		scaled=$(( CAU_PROGRESS_BASE * 100 + CAU_PROGRESS_SPAN * 100 * num / den ))
+	else
+		scaled=$(( CAU_PROGRESS_BASE * 100 ))
+	fi
+
+	pct=$(( scaled / CAU_PROGRESS_SCALE ))
+	(( pct > 100 )) && pct=100
+
+	# Never backwards. Two honest things can ask for that: dropping a step
+	# rescales the run against a smaller total, and the conflict-recovery loop
+	# restarts pacman - and with it the item tally - from the top. Both are
+	# real, neither is a reason to show somebody a bar that retreats.
+	(( pct < CAU_PROGRESS_SHOWN )) && pct=$CAU_PROGRESS_SHOWN
+
+	# Only when the whole number changes. Percent is the one field the runner
+	# would otherwise rewrite for every package on a 500-package upgrade.
+	(( pct == CAU_PROGRESS_SHOWN )) && return 0
+	CAU_PROGRESS_SHOWN=$pct
+	_cau_progress_line 'percent\t%s' "$pct"
+}
+
 # cau_progress_item <processed> [total]
 # How far through the current step we are.
 cau_progress_item() {
-	local processed="$1" total="${2:-$CAU_PROGRESS_TOTAL}" pct scaled
+	local processed="$1" total="${2:-$CAU_PROGRESS_TOTAL}"
 
 	(( ${#CAU_PROGRESS_FDS[@]} )) || return 0
 	[[ $processed =~ ^[0-9]+$ ]] || return 0
@@ -185,19 +269,86 @@ cau_progress_item() {
 			_cau_progress_line 'total\t%s' "$total"
 		fi
 		_cau_progress_line 'done\t%s' "$processed"
-		scaled=$(( CAU_PROGRESS_BASE * 100 + CAU_PROGRESS_SPAN * 100 * processed / total ))
+		_cau_progress_pct "$processed" "$total"
 	else
-		scaled=$(( CAU_PROGRESS_BASE * 100 ))
+		_cau_progress_pct 0 0
 	fi
+}
 
-	pct=$(( scaled / CAU_PROGRESS_SCALE ))
-	(( pct > 100 )) && pct=100
+# cau_progress_creep <seconds-elapsed>
+# Moves the bar through a step whose length cannot be known in advance.
+#
+# Some of a run has no counter to offer and never will. pacman prints nothing
+# whatsoever between "starting full system upgrade" and the transaction it
+# eventually prepares; an AUR helper compiling a package prints plenty, none of
+# it countable. On a large backlog either is minutes. There is no honest number
+# to show for that - but a bar that has not moved since it appeared is read as
+# a hang, and somebody who reads it that way reaches for the power button in
+# the middle of an update. That is the failure this is here to prevent.
+#
+# So it creeps, along a curve that approaches the end of the step without ever
+# reaching it: half the step's share after HALFLIFE seconds, three quarters
+# after three times that, the whole of it never. Nothing is claimed that is not
+# known - the item counter stays empty throughout, which is the field that
+# would be lying if it moved - and the step still finishes the instant real
+# work reports in, because every real report is further along than the creep.
+#
+# Confined to the step's own span, so a creep can never overtake the step that
+# comes after it however long it is left running.
+CAU_PROGRESS_CREEP_HALFLIFE=45
 
-	# Only when the whole number changes. Percent is the one field the runner
-	# would otherwise rewrite for every package on a 500-package upgrade.
-	(( pct == CAU_PROGRESS_SHOWN )) && return 0
-	CAU_PROGRESS_SHOWN=$pct
-	_cau_progress_line 'percent\t%s' "$pct"
+cau_progress_creep() {
+	local elapsed="$1"
+
+	(( ${#CAU_PROGRESS_FDS[@]} )) || return 0
+	[[ $elapsed =~ ^[0-9]+$ ]] || return 0
+
+	_cau_progress_pct "$elapsed" $(( elapsed + CAU_PROGRESS_CREEP_HALFLIFE ))
+}
+
+# cau_progress_creep_start / cau_progress_creep_stop
+# The same, for a step that blocks in one long call instead of polling: the
+# ticker runs alongside it and is stopped when it returns. Only one at a time,
+# and starting a second one replaces the first.
+CAU_PROGRESS_CREEP_PID=0
+CAU_PROGRESS_CREEP_T0=0
+
+cau_progress_creep_start() {
+	cau_progress_creep_stop
+	(( ${#CAU_PROGRESS_FDS[@]} )) || return 0
+
+	CAU_PROGRESS_CREEP_T0=$SECONDS
+	local t0=$SECONDS
+	{
+		# Waiting without forking a sleep every two seconds, for the same
+		# reason cau_progress_begin opens its fifo read-write: a pipe held open
+		# at both ends never reports end-of-file, so a timed read on it blocks
+		# for exactly the timeout and nothing else. A forked sleep would also
+		# survive the kill below - it is a child of this subshell, not this
+		# subshell - and inherit the fifo's write end, which would keep the
+		# helper from seeing the end of its input until the sleep ran out.
+		local nap
+		exec {nap}<> <(:)
+		while :; do
+			read -r -t 2 -u "$nap" _ || true
+			cau_progress_creep $(( SECONDS - t0 ))
+		done
+	} &
+	CAU_PROGRESS_CREEP_PID=$!
+}
+
+cau_progress_creep_stop() {
+	(( CAU_PROGRESS_CREEP_PID )) || return 0
+	kill "$CAU_PROGRESS_CREEP_PID" 2>/dev/null
+	wait "$CAU_PROGRESS_CREEP_PID" 2>/dev/null
+	CAU_PROGRESS_CREEP_PID=0
+
+	# The ticker moved the bar from inside a subshell, so this side never saw
+	# it happen and still believes the bar is where it was left. Catching up
+	# costs one recomputation - the curve is a function of elapsed time and
+	# nothing else - and without it the next ordinary report from here would be
+	# measured against a stale percentage and send the bar backwards.
+	cau_progress_creep $(( SECONDS - CAU_PROGRESS_CREEP_T0 ))
 }
 
 # cau_progress_detail <label-msgid> <value>
@@ -235,6 +386,10 @@ cau_progress_end() {
 	local outcome="${1:-ok}" msgid="${2:-}"
 	local i fd
 
+	# Before the descriptors go: a ticker still running would be writing into
+	# a pipe whose reader is about to be waited on.
+	cau_progress_creep_stop
+
 	# The last step never consumes its own share - nothing reports items for
 	# the cleanup - so the bar would stop a few percent short of the end and
 	# vanish there. Only on the way out of a run that actually worked, though: