diff --git a/README.md b/README.md index a12841d..febc2b3 100644 --- a/README.md +++ b/README.md @@ -214,19 +214,33 @@ Two things about how it is put together: the length of the update, holding the connection open and taking instructions on stdin. It needs **python-gobject**; without it there is simply no bar and nothing else changes. -- Downloading and unpacking are two separate steps on the bar. On a domestic - line the download is the longer of the two, and calling the whole thing - "installing" leaves the bar sitting at 4% for six minutes, which reads as a - hang rather than as progress. -- Neither phase carries a counter on an unattended run, so both are counted a - line at a time — `foo-1.2-1-x86_64 downloading...` and `upgrading foo...`. - The database sync just before prints the same shape (` core downloading...`) - with the suffix that would give it away already stripped, so counting starts - only after pacman's `:: Retrieving packages...` header. pacman's other - `(n/m)` sequences — checking keys, package integrity, loading files — each - count to the same total, so only the transaction verbs are followed; - otherwise the bar would reach the end three times before the first package - was unpacked. +- Working out the upgrade, downloading it and unpacking it are three separate + steps. The first is the one that used to look broken: between + `:: Starting full system upgrade...` and the transaction it eventually + prepares, pacman prints nothing at all, and on a large backlog that silence + runs to minutes. A counter frozen at "0 of 161" reads as a stuck update, so + that stretch carries a label and deliberately no counter. On a domestic line + the download is then the longest of the three, and calling the whole thing + "installing" would leave the bar at 4% for six minutes. +- A step that turns out to have no work is dropped from the bar rather than + handed its share for nothing. Packages already in the cache are never + announced, so a run that only has to unpack skips the download step outright + instead of leaping 30% the moment unpacking starts — and likewise for AUR + with nothing pending, or a machine with no Flatpaks. The weights only ever + have to be right about the steps that actually run. +- pacman's output is line-buffered through `stdbuf`. Writing to a log rather + than a terminal, libc would hand it over in 4 KB blocks instead, and 4 KB of + `upgrading foo...` is on the order of a hundred and sixty packages arriving + at once — which is how a bar comes to sit still and then jump to the end. +- Neither counted phase gets a counter from pacman on an unattended run, so + both are counted a line at a time — `foo-1.2-1-x86_64 downloading...` and + `upgrading foo...`. The database sync just before prints the same shape + (` core downloading...`) with the suffix that would give it away already + stripped, so counting starts only after pacman's `:: Retrieving packages...` + header. pacman's other `(n/m)` sequences — checking keys, package integrity, + loading files — each count to the same total, so only the transaction verbs + are followed; otherwise the bar would reach the end three times before the + first package was unpacked. This is Plasma's job interface. On a desktop that does not implement it the helper exits quietly and the ordinary notifications carry on as before. diff --git a/doc/cachy-auto-update.1.scd b/doc/cachy-auto-update.1.scd index a65867c..a210bfd 100644 --- a/doc/cachy-auto-update.1.scd +++ b/doc/cachy-auto-update.1.scd @@ -122,18 +122,29 @@ of the update and holds that connection open. It requires _python-gobject_. Where that is missing, or on a desktop with no job interface, there is no progress entry and nothing else is affected. -Fetching the packages and unpacking them are two steps rather than one. On a -domestic line the download is the longer of the two, and a bar that called the -whole thing "installing" would sit near its beginning for minutes at a time -looking stuck. +Working out the upgrade, fetching it and unpacking it are three steps rather +than one. Between "Starting full system upgrade" and the transaction it +eventually prepares, pacman prints nothing at all, and on a large backlog that +silence runs to minutes; that stretch therefore carries a label of its own and +deliberately no item count, because a counter frozen at "0 of 161" reads as a +stuck update. On a domestic line the download is then the longest of the three, +and a bar that called the whole thing "installing" would sit near its beginning +for minutes at a time looking stuck. -Neither phase gets a counter from pacman on an unattended run, so both are -counted here, a line at a time: "foo-1.2-1-x86_64 downloading..." for the -first, "upgrading foo..." for the second. Packages already in the cache are -never announced, so the download step regularly ends short of its total and -gives up the rest of its share when unpacking begins. pacman's other (n/m) -sequences - checking keys, package integrity, loading package files - each -count up to the same total and are deliberately ignored. +A step that turns out to have no work is dropped from the bar instead of being +handed its share for nothing. Packages already in the cache are never +announced, so a run with everything already fetched skips the download step +outright rather than jumping when unpacking starts; the same applies to AUR +with nothing pending, or a machine with no Flatpaks installed. + +Neither counted phase gets a counter from pacman on an unattended run, so both +are counted here, a line at a time: "foo-1.2-1-x86_64 downloading..." for the +first, "upgrading foo..." for the second. pacman's output is line-buffered +through *stdbuf*(1) so those lines arrive as they happen - writing to a log +rather than a terminal, libc would otherwise release them in 4KB blocks, around +a hundred and sixty packages at a time. pacman's other (n/m) sequences - +checking keys, package integrity, loading package files - each count up to the +same total and are deliberately ignored. # HOW LONG NOTIFICATIONS STAY diff --git a/po/cachy-auto-update.pot b/po/cachy-auto-update.pot index 34312e5..4931b69 100644 --- a/po/cachy-auto-update.pot +++ b/po/cachy-auto-update.pot @@ -234,6 +234,12 @@ msgstr "" #. looking for it - it appears next to whatever they were doing - so each one #. says outright that this is an update running, rather than naming the kind of #. package on its own. +msgid "Checking for updates" +msgstr "" + +msgid "Preparing the update" +msgstr "" + msgid "Downloading updates" msgstr "" diff --git a/po/de.po b/po/de.po index 9e09aa4..7ee7fc8 100644 --- a/po/de.po +++ b/po/de.po @@ -235,6 +235,12 @@ msgstr "Ohne Befehl wird ein interaktives Menü angezeigt." #. looking for it - it appears next to whatever they were doing - so each one #. says outright that this is an update running, rather than naming the kind of #. package on its own. +msgid "Checking for updates" +msgstr "Nach Updates wird gesucht" + +msgid "Preparing the update" +msgstr "Update wird vorbereitet" + msgid "Downloading updates" msgstr "Updates werden heruntergeladen" diff --git a/src/cachy-auto-update-run b/src/cachy-auto-update-run index 61c20cc..3bb2eb4 100644 --- a/src/cachy-auto-update-run +++ b/src/cachy-auto-update-run @@ -185,7 +185,7 @@ failed=0 # Open the desktop's progress bar, told up front which steps this run will # perform. Only those count towards the bar, so a machine with no Flatpaks # does not sit at 85% for the last second of the run. -progress_steps=(download repo) +progress_steps=(resolve download repo) [[ $CFG_AUR == yes ]] && progress_steps+=(aur) [[ $CFG_FLATPAK == yes ]] && progress_steps+=(flatpak) [[ $CFG_APPIMAGE == yes ]] && progress_steps+=(appimage) diff --git a/src/lib/pkg_appimage.sh b/src/lib/pkg_appimage.sh index b968f70..c7e52b4 100644 --- a/src/lib/pkg_appimage.sh +++ b/src/lib/pkg_appimage.sh @@ -43,6 +43,26 @@ cau_appimage_update() { local user uid count rc=0 local -a cmd + # Is there a Gear Lever on this machine at all? Asked before the step is + # announced rather than discovered inside the loop: on a machine without + # one - the common case, it is an optional dependency - a step that exists + # only to hand its share of the bar straight to the next one is a jump the + # bar does not need. Stops at the first user who has it, so the extra probe + # costs anything only in the case it is there to remove. + local found=0 + while read -r user uid; do + [[ -n $user ]] || continue + if _cau_gearlever_cmd "$user" "$uid" > /dev/null; then + found=1 + break + fi + done < <(cau_active_session_users) + + if (( ! found )); then + cau_progress_drop appimage + return 0 + fi + cau_progress_step appimage "Updating AppImages" while read -r user uid; do diff --git a/src/lib/pkg_aur.sh b/src/lib/pkg_aur.sh index 8a13d38..6f4c37c 100644 --- a/src/lib/pkg_aur.sh +++ b/src/lib/pkg_aur.sh @@ -124,7 +124,9 @@ cau_aur_update() { local pending failures local -a args - cau_aur_ready || return 0 + # No helper, no base-devel, no bar: a step that cannot run should not be + # holding a share of it. + cau_aur_ready || { cau_progress_drop aur; return 0; } cau_progress_step aur "Updating AUR packages" @@ -132,23 +134,30 @@ cau_aur_update() { if (( pending == 0 )); then cau_info "No AUR updates pending" cau_state_clear aur_failures + cau_progress_drop aur return 0 fi cau_info "Updating $pending AUR package(s) with $CAU_AUR_HELPER" - # The helper builds each package from source with no counter this side of - # its output, so the bar sits at the start of the step until it is done. + # The helper builds each package from source and prints plenty about it, + # none of it countable from this side. A single large package can take ten + # minutes, so the bar creeps through the step rather than sitting at its + # start for all of them; the item count stays where it is, because that is + # the number that would be lying if it moved. cau_progress_item 0 "$pending" mapfile -t args < <(cau_aur_helper_args) + cau_progress_creep_start if cau_run_logged cau_as_build_user "$CAU_AUR_HELPER" "${args[@]}"; then + cau_progress_creep_stop CAU_AUR_COUNT="$pending" cau_progress_item "$pending" cau_state_clear aur_failures return 0 fi + cau_progress_creep_stop CAU_AUR_COUNT=0 failures="$(cau_state_read aur_failures 0)" [[ $failures =~ ^[0-9]+$ ]] || failures=0 diff --git a/src/lib/pkg_flatpak.sh b/src/lib/pkg_flatpak.sh index 3e2aa3a..77731ee 100644 --- a/src/lib/pkg_flatpak.sh +++ b/src/lib/pkg_flatpak.sh @@ -26,21 +26,31 @@ cau_flatpak_pending_system() { cau_flatpak_update() { local rc=0 pending user uid home count - cau_have flatpak || return 0 + cau_have flatpak || { cau_progress_drop flatpak; return 0; } cau_progress_step flatpak "Updating Flatpak apps" - # refresh appstream metadata first so remote-ls sees current versions + # refresh appstream metadata first so remote-ls sees current versions. + # Nothing is countable until that has finished, so the bar creeps rather + # than waiting at the start of the step for it. + cau_progress_creep_start cau_run_logged flatpak update --appstream --system --noninteractive || true + cau_progress_creep_stop pending="$(cau_flatpak_pending_system)" if (( pending > 0 )); then cau_info "Updating $pending system Flatpak(s)" cau_progress_item 0 "$pending" + # Started after the count above and stopped before the one below, so the + # only reports this side makes while a ticker is running are further + # along than the ticker ever gets. + cau_progress_creep_start if cau_run_logged flatpak update --system --noninteractive --assumeyes; then + cau_progress_creep_stop CAU_FLATPAK_COUNT=$(( CAU_FLATPAK_COUNT + pending )) cau_progress_item "$pending" else + cau_progress_creep_stop cau_warn "System Flatpak update failed" rc=1 fi diff --git a/src/lib/pkg_pacman.sh b/src/lib/pkg_pacman.sh index cb9b7ef..f0433f7 100644 --- a/src/lib/pkg_pacman.sh +++ b/src/lib/pkg_pacman.sh @@ -52,11 +52,18 @@ CAU_PACMAN_DL_AWK=' END { print n + 0, name }' # _cau_pacman_progress_watch -# Feeds the desktop's progress bar by watching pacman work, through both of the -# phases a pacman run has: first everything is fetched, then everything is -# unpacked. They are two steps on the bar rather than one, because they are two -# steps to sit through - a run that has been "installing updates" at 4% for six -# minutes has not hung, it is still downloading, and the bar should say so. +# Feeds the desktop's progress bar by watching pacman work, through the three +# phases a pacman run has: first it works out what the upgrade consists of, +# then everything is fetched, then everything is unpacked. Three steps on the +# bar rather than one, because they are three stretches to sit through - and +# each one is silent in its own way. +# +# The first is the one that used to look like a hang. Between "starting full +# system upgrade" and the transaction it eventually prepares, pacman says +# nothing whatsoever, and on a large backlog that silence is minutes long. The +# only honest thing to show there is a label and no counter at all: a tally +# frozen at "0 of 161" reads as a stuck update, where "Preparing the update" +# with no number reads as what it actually is. # # In the transaction, pacman announces each package twice over, in one of two # shapes, and which one depends on a flag this program sets itself: @@ -83,7 +90,8 @@ CAU_PACMAN_DL_AWK=' _cau_pacman_progress_watch() { local log="$1" local total="${CAU_PACMAN_COUNT:-0}" announced processed line pkg last='' - local phase=download fetched shown='' + local phase=resolve fetched shown='' labelled='' + local t0=$SECONDS while :; do sleep 1 @@ -94,49 +102,73 @@ _cau_pacman_progress_watch() { line="$(grep -aoE "$CAU_PACMAN_OP_RE" "$log" 2>/dev/null | tail -n1)" - # Nothing unpacked yet, so this is still the download - or the database - # sync ahead of it, which the awk above declines to count. - if [[ -z $line ]]; then + if [[ -n $line ]]; then + # Unpacking has started, so whatever came before it is over. If + # nothing was ever retrieved - every package already sitting in the + # cache, which is the ordinary state of affairs after a run that was + # interrupted once already - then the download step never happened, + # and it is dropped rather than handed its whole share of the bar in + # exchange for no work at all. + if [[ $phase != install ]]; then + [[ $phase == download ]] || cau_progress_drop download + phase=install + cau_progress_step repo "Updating system packages" "$total" + fi + + # Nothing new since the last look. Checked before the counting grep + # because on a large upgrade this loop spends most of its life here. + [[ $line != "$last" ]] || continue + last="$line" + + processed="$(grep -acE "$CAU_PACMAN_OP_RE" "$log" 2>/dev/null)" + [[ $processed =~ ^[0-9]+$ ]] || continue + + # Where pacman does carry a counter, believe it over the tally: it + # is the same number, but it also knows the true total. + if [[ $line =~ ^\([[:space:]]*([0-9]+)/([0-9]+)\) ]]; then + processed="${BASH_REMATCH[1]}" + total="${BASH_REMATCH[2]}" + fi + + cau_progress_item "$processed" "$total" + + pkg="${line##* }" + cau_progress_detail "Package" "${pkg%...}" + continue + fi + + # Fetching. The header is what tells this apart from the database sync + # a few lines earlier, which prints the very same shape. + if grep -qa '^:: Retrieving packages' "$log" 2>/dev/null; then + if [[ $phase == resolve ]]; then + phase=download + cau_progress_step download "Downloading updates" "$total" + fi + read -r fetched pkg < <(awk "$CAU_PACMAN_DL_AWK" "$log" 2>/dev/null) - [[ $fetched =~ ^[0-9]+$ ]] && (( fetched > 0 )) || continue + [[ $fetched =~ ^[0-9]+$ ]] || continue [[ $fetched != "$shown" ]] || continue shown="$fetched" cau_progress_item "$fetched" "$total" # Down to the bare name, as the transaction reports it: the file # pacman names here carries version, release and architecture. - cau_progress_detail "Package" "${pkg%-*-*-*}" + [[ -n $pkg ]] && cau_progress_detail "Package" "${pkg%-*-*-*}" continue fi - # The first package being unpacked ends the download step. Its share of - # the bar is given up wherever it had got to - packages already in the - # cache are fetched in no time at all and never print a line, so the - # tally regularly stops short of the total it was promised. - if [[ $phase == download ]]; then - phase=install - cau_progress_step repo "Updating system packages" "$total" + # Still resolving. pacman does mark the point where it stops syncing + # databases and starts working out the upgrade, and that is the half + # worth naming, because it is the half that takes the minutes. + if [[ $phase == resolve && $labelled != upgrade ]] \ + && grep -qa '^:: Starting full system upgrade' "$log" 2>/dev/null; then + labelled=upgrade + cau_progress_step resolve "Preparing the update" fi - # Nothing new since the last look. Checked before the counting grep - # because on a large upgrade this loop spends most of its life here. - [[ $line != "$last" ]] || continue - last="$line" - - processed="$(grep -acE "$CAU_PACMAN_OP_RE" "$log" 2>/dev/null)" - [[ $processed =~ ^[0-9]+$ ]] || continue - - # Where pacman does carry a counter, believe it over the tally: it is - # the same number, but it also knows the true total. - if [[ $line =~ ^\([[:space:]]*([0-9]+)/([0-9]+)\) ]]; then - processed="${BASH_REMATCH[1]}" - total="${BASH_REMATCH[2]}" - fi - - cau_progress_item "$processed" "$total" - - pkg="${line##* }" - cau_progress_detail "Package" "${pkg%...}" + # Nothing countable happens in here at all, so the bar creeps instead. + # This is the stretch that used to look like a hung update. + cau_progress_creep $(( SECONDS - t0 )) done } @@ -156,11 +188,21 @@ _cau_pacman_exec() { watcher=$! fi + # Line-buffered on purpose. pacman writes to a file or through a pipe here, + # never to a terminal, so libc buffers it in 4KB blocks - and 4KB of + # "upgrading foo..." is on the order of a hundred and sixty packages. The + # watcher would see nothing at all, then a hundred and sixty lines at once, + # which is exactly how a bar comes to sit still and then leap to the end. + # Guarded rather than assumed: without coreutils there is no bar to feed + # either, but there is still an update to run. + local -a buffered=() + cau_have stdbuf && buffered=(stdbuf -oL) + if [[ -n $CAU_INTERACTIVE ]]; then - pacman "$@" 2>&1 | tee "$log" + "${buffered[@]}" pacman "$@" 2>&1 | tee "$log" rc="${PIPESTATUS[0]}" else - pacman "$@" > "$log" 2>&1 + "${buffered[@]}" pacman "$@" > "$log" 2>&1 rc=$? fi @@ -245,12 +287,18 @@ cau_pacman_update() { local log kind local -a flags - # The download comes first and the watcher moves on to the repo step once - # pacman starts unpacking. - cau_progress_step download "Downloading updates" + # checkupdates goes first, against its own private database, and the bar + # says so rather than naming a step that has not begun. The watcher takes + # over from here and moves on to the download and repo steps as pacman + # actually reaches them. + cau_progress_step resolve "Checking for updates" if ! cau_pacman_pending; then cau_info "No repository updates pending" + # Neither of the two steps this would have led to is going to happen, + # so the rest of the run gets their share of the bar instead of + # watching it jump 70% the moment Flatpaks start. + cau_progress_drop download repo return 0 fi @@ -258,7 +306,10 @@ cau_pacman_update() { CAU_PACMAN_COUNT="$(grep -c . <<< "$CAU_PACMAN_PENDING")" [[ $CAU_PACMAN_COUNT =~ ^[0-9]+$ ]] || CAU_PACMAN_COUNT=0 cau_info "Updating $CAU_PACMAN_COUNT repository package(s)" - cau_progress_item 0 "$CAU_PACMAN_COUNT" + # Deliberately no item count yet. Until pacman has prepared a + # transaction there is nothing being worked through, and "0 of 161" + # against a bar that cannot move for the next few minutes is the exact + # impression the resolve step exists to avoid. else # checkupdates is unavailable, so the list is unknown and pacman is # asked to work it out itself. @@ -291,6 +342,13 @@ cau_pacman_update() { while true; do if _cau_pacman_exec "$log" -Syu "${flags[@]}" "${extra[@]}"; then + # The watcher decided the same thing in a subshell, so a step it + # dropped is still in the plan out here. Same question, same answer, + # and the two copies agree on what the rest of the run is scaled + # against. Only on the way out: a failed attempt is about to be + # retried, and that retry may well download after all. + grep -qa '^:: Retrieving packages' "$log" 2>/dev/null \ + || cau_progress_drop download cat "$log" >> "$CAU_RUNLOG" 2>/dev/null grep -E '^(removing|replacing) ' "$log" 2>/dev/null \ | while read -r line; do cau_info " $line"; done diff --git a/src/lib/progress.sh b/src/lib/progress.sh index aa1499d..477e836 100644 --- a/src/lib/progress.sh +++ b/src/lib/progress.sh @@ -30,8 +30,15 @@ CAU_PROGRESS_LOCALES=() # them about equally, on a domestic line - and the cleanup is a rounding error. # They do not have to add up to 100 - only the steps a given run will actually # perform are counted, and the total is normalised against those. +# +# "resolve" is everything pacman does before it has a transaction: syncing the +# databases and working out what the upgrade actually consists of. It is +# usually seconds, which is why it is worth so little - but on a large backlog +# it is minutes, and those minutes used to be spent looking at a bar that had +# not moved yet. declare -A CAU_PROGRESS_WEIGHTS=( - [download]=30 [repo]=40 [aur]=15 [flatpak]=10 [appimage]=3 [cleanup]=2 + [resolve]=10 [download]=30 [repo]=40 [aur]=15 [flatpak]=10 [appimage]=3 + [cleanup]=2 ) CAU_PROGRESS_PLAN=() @@ -133,6 +140,49 @@ cau_progress_active() { (( ${#CAU_PROGRESS_FDS[@]} )) } +# cau_progress_drop +# Takes steps out of the plan and rescales the bar to what is left. +# +# Which steps a run will perform is only half known up front. The other half +# turns up while it runs: nothing to download because every package was already +# in the cache, no AUR updates pending, no Flatpaks installed. A step like that +# keeps its whole share of the bar and then hands it over in a single jump the +# moment the next one starts - which is precisely the stutter this is here to +# remove. Dropping it hands its share to the steps that do have work instead, +# so the bar advances at a steady pace rather than leaping across the gaps. +# +# It is also what lets the weights above stay rough: they never have to be +# right about a step that does not run, only about the ones that do. +# +# Only ever called for a step that has not started, so nothing already behind +# the bar is rescaled and the bar does not travel backwards. +cau_progress_drop() { + local drop step + local -a kept=() + + (( ${#CAU_PROGRESS_FDS[@]} )) || return 0 + + for step in "${CAU_PROGRESS_PLAN[@]}"; do + for drop in "$@"; do + [[ $step == "$drop" ]] && continue 2 + done + kept+=("$step") + done + + (( ${#kept[@]} == ${#CAU_PROGRESS_PLAN[@]} )) && return 0 + + CAU_PROGRESS_PLAN=("${kept[@]}") + CAU_PROGRESS_SCALE=0 + for step in "${CAU_PROGRESS_PLAN[@]}"; do + CAU_PROGRESS_SCALE=$(( CAU_PROGRESS_SCALE + ${CAU_PROGRESS_WEIGHTS[$step]:-0} )) + done + + # Nothing left to weigh against would divide by zero further down. Cannot + # happen while cleanup is unconditional, but this is cheaper than relying + # on that staying true. + (( CAU_PROGRESS_SCALE > 0 )) || CAU_PROGRESS_SCALE=1 +} + # cau_progress_step [item-count] # Moves on to the next step. The bar jumps to where that step begins, so a step # that reported fewer items than it promised still completes rather than @@ -143,11 +193,17 @@ cau_progress_step() { (( ${#CAU_PROGRESS_FDS[@]} )) || return 0 + local found=0 for step in "${CAU_PROGRESS_PLAN[@]}"; do - [[ $step == "$id" ]] && break + [[ $step == "$id" ]] && { found=1; break; } base=$(( base + ${CAU_PROGRESS_WEIGHTS[$step]:-0} )) done + # A step that was dropped for having no work is not a step to move to. + # Without this the loop above would fall off the end of the plan and hand + # back the sum of every weight, i.e. send the bar straight to 100%. + (( found )) || return 0 + CAU_PROGRESS_BASE=$base CAU_PROGRESS_SPAN=${CAU_PROGRESS_WEIGHTS[$id]:-0} CAU_PROGRESS_TOTAL=$total @@ -170,10 +226,38 @@ cau_progress_step() { cau_progress_item 0 } +# _cau_progress_pct +# How far through the current step we are, as a share of its span, turned into +# one number for the whole run and sent on if it has moved. +_cau_progress_pct() { + local num="$1" den="$2" pct scaled + + if (( den > 0 )); then + scaled=$(( CAU_PROGRESS_BASE * 100 + CAU_PROGRESS_SPAN * 100 * num / den )) + else + scaled=$(( CAU_PROGRESS_BASE * 100 )) + fi + + pct=$(( scaled / CAU_PROGRESS_SCALE )) + (( pct > 100 )) && pct=100 + + # Never backwards. Two honest things can ask for that: dropping a step + # rescales the run against a smaller total, and the conflict-recovery loop + # restarts pacman - and with it the item tally - from the top. Both are + # real, neither is a reason to show somebody a bar that retreats. + (( pct < CAU_PROGRESS_SHOWN )) && pct=$CAU_PROGRESS_SHOWN + + # Only when the whole number changes. Percent is the one field the runner + # would otherwise rewrite for every package on a 500-package upgrade. + (( pct == CAU_PROGRESS_SHOWN )) && return 0 + CAU_PROGRESS_SHOWN=$pct + _cau_progress_line 'percent\t%s' "$pct" +} + # cau_progress_item [total] # How far through the current step we are. cau_progress_item() { - local processed="$1" total="${2:-$CAU_PROGRESS_TOTAL}" pct scaled + local processed="$1" total="${2:-$CAU_PROGRESS_TOTAL}" (( ${#CAU_PROGRESS_FDS[@]} )) || return 0 [[ $processed =~ ^[0-9]+$ ]] || return 0 @@ -185,19 +269,86 @@ cau_progress_item() { _cau_progress_line 'total\t%s' "$total" fi _cau_progress_line 'done\t%s' "$processed" - scaled=$(( CAU_PROGRESS_BASE * 100 + CAU_PROGRESS_SPAN * 100 * processed / total )) + _cau_progress_pct "$processed" "$total" else - scaled=$(( CAU_PROGRESS_BASE * 100 )) + _cau_progress_pct 0 0 fi +} - pct=$(( scaled / CAU_PROGRESS_SCALE )) - (( pct > 100 )) && pct=100 +# cau_progress_creep +# Moves the bar through a step whose length cannot be known in advance. +# +# Some of a run has no counter to offer and never will. pacman prints nothing +# whatsoever between "starting full system upgrade" and the transaction it +# eventually prepares; an AUR helper compiling a package prints plenty, none of +# it countable. On a large backlog either is minutes. There is no honest number +# to show for that - but a bar that has not moved since it appeared is read as +# a hang, and somebody who reads it that way reaches for the power button in +# the middle of an update. That is the failure this is here to prevent. +# +# So it creeps, along a curve that approaches the end of the step without ever +# reaching it: half the step's share after HALFLIFE seconds, three quarters +# after three times that, the whole of it never. Nothing is claimed that is not +# known - the item counter stays empty throughout, which is the field that +# would be lying if it moved - and the step still finishes the instant real +# work reports in, because every real report is further along than the creep. +# +# Confined to the step's own span, so a creep can never overtake the step that +# comes after it however long it is left running. +CAU_PROGRESS_CREEP_HALFLIFE=45 - # Only when the whole number changes. Percent is the one field the runner - # would otherwise rewrite for every package on a 500-package upgrade. - (( pct == CAU_PROGRESS_SHOWN )) && return 0 - CAU_PROGRESS_SHOWN=$pct - _cau_progress_line 'percent\t%s' "$pct" +cau_progress_creep() { + local elapsed="$1" + + (( ${#CAU_PROGRESS_FDS[@]} )) || return 0 + [[ $elapsed =~ ^[0-9]+$ ]] || return 0 + + _cau_progress_pct "$elapsed" $(( elapsed + CAU_PROGRESS_CREEP_HALFLIFE )) +} + +# cau_progress_creep_start / cau_progress_creep_stop +# The same, for a step that blocks in one long call instead of polling: the +# ticker runs alongside it and is stopped when it returns. Only one at a time, +# and starting a second one replaces the first. +CAU_PROGRESS_CREEP_PID=0 +CAU_PROGRESS_CREEP_T0=0 + +cau_progress_creep_start() { + cau_progress_creep_stop + (( ${#CAU_PROGRESS_FDS[@]} )) || return 0 + + CAU_PROGRESS_CREEP_T0=$SECONDS + local t0=$SECONDS + { + # Waiting without forking a sleep every two seconds, for the same + # reason cau_progress_begin opens its fifo read-write: a pipe held open + # at both ends never reports end-of-file, so a timed read on it blocks + # for exactly the timeout and nothing else. A forked sleep would also + # survive the kill below - it is a child of this subshell, not this + # subshell - and inherit the fifo's write end, which would keep the + # helper from seeing the end of its input until the sleep ran out. + local nap + exec {nap}<> <(:) + while :; do + read -r -t 2 -u "$nap" _ || true + cau_progress_creep $(( SECONDS - t0 )) + done + } & + CAU_PROGRESS_CREEP_PID=$! +} + +cau_progress_creep_stop() { + (( CAU_PROGRESS_CREEP_PID )) || return 0 + kill "$CAU_PROGRESS_CREEP_PID" 2>/dev/null + wait "$CAU_PROGRESS_CREEP_PID" 2>/dev/null + CAU_PROGRESS_CREEP_PID=0 + + # The ticker moved the bar from inside a subshell, so this side never saw + # it happen and still believes the bar is where it was left. Catching up + # costs one recomputation - the curve is a function of elapsed time and + # nothing else - and without it the next ordinary report from here would be + # measured against a stale percentage and send the bar backwards. + cau_progress_creep $(( SECONDS - CAU_PROGRESS_CREEP_T0 )) } # cau_progress_detail @@ -235,6 +386,10 @@ cau_progress_end() { local outcome="${1:-ok}" msgid="${2:-}" local i fd + # Before the descriptors go: a ticker still running would be writing into + # a pipe whose reader is about to be waited on. + cau_progress_creep_stop + # The last step never consumes its own share - nothing reports items for # the cleanup - so the bar would stop a few percent short of the end and # vanish there. Only on the way out of a run that actually worked, though: