Add a separate function inside dietpi-globals to generate troubleshoot template for the forum

[JappeHallunken]

Users then have to only run G_BUG_REPORT to get the informationen needed, they can just copy and paste it from terminal into the forum.

G_EXEC error handler should work like before.

I still need to figure out way to trigger an error in one of dietpi-* functions to test this properly. I tried dietpi-update without internet access, but this triggers not the full error handler

[JappeHallunken]

Alright, I learned a bit more bash and I think I got it now.

A function G_BUG_REPORT() generates info for the forums troubleshoot template, which the can be copied and pasted into the forum by the users.
G_BUG_REPORT() is also executed inside G_EXEC() to inject the necessary data needed for the github bug report template.

I tested with some preset conditions (see code below, it has the test conditions in it) and I also sent some bug reports for testing ( sry ;) )
G_BUG_REPORT() works as intended and I also get now the blue-backgrounded template after leaving the interactive menu.

I will commit right away.

G_BUG_REPORT() {
    local dietpi_version="v$G_DIETPI_VERSION_CORE.$G_DIETPI_VERSION_SUB.$G_DIETPI_VERSION_RC ($G_GITOWNER/$G_GITBRANCH)"
    local image_creator preimage_name

    if [[ -f '/boot/dietpi/.prep_info' ]]; then
        image_creator=$(mawk 'NR==1' /boot/dietpi/.prep_info)
        [[ $image_creator == 0 ]] && image_creator='DietPi Core Team'
        preimage_name=$(mawk 'NR==2' /boot/dietpi/.prep_info)
    fi

    cat <<EOF
#### Required Information
- DietPi version | $dietpi_version
- Distro version | $G_DISTRO_NAME (ID=$G_DISTRO${G_RASPBIAN:+,RASPBIAN=$G_RASPBIAN})
- Kernel version | \`$(uname -a)\`
- Architecture	 | \`$(dpkg --print-architecture)\`
- SBC model 	 | $G_HW_MODEL_NAME (ID=$G_HW_MODEL)
- Image creator  | $image_creator
- Pre-image      | $preimage_name
EOF
}

	#-----------------------------------------------------------------------------------
	# Error handled command execution wrapper
	#-----------------------------------------------------------------------------------
	# IMPORTANT:
	# - Never pipe G_EXEC! "G_EXEC command | command" leads to G_EXEC not being able to unset G_EXEC_* variables and functions from originating shell or kill the originating script in case of error.
	# Required input:
	# - $@=<command>		| Command to execute
	# Optional input:
	# - $G_EXEC_DESC=<text>		| Command description to print instead of raw command string
	# - $G_EXEC_RETRIES=<int>	| Amount of non-interactive retries in case of error, before doing interactive error prompt
	# - G_EXEC_PRE_FUNC(){}		| Function to call before every input command attempt, e.g. to re-evaluate variables
	# - G_EXEC_POST_FUNC(){}	| Function to call after every input command attempt, e.g. to handle errors without error exit code
	# - $G_EXEC_OUTPUT=1		| Print full command output instead of animated processing message
	# - $G_EXEC_OUTPUT_COL='\e[90m'	| Override colour of command output via console colour code, requires $G_EXEC_OUTPUT=1
	# - $G_EXEC_NOFAIL=1		| On error, override as success, only useful to replace verbose output by animated processing message, inherits $G_EXEC_NOHALT=1 and $G_EXEC_NOEXIT=1
	# - $G_EXEC_NOHALT=1		| On error, print short error message only, skip error handler menu and do not exit script, inherits $G_EXEC_NOEXIT=1
	# - $G_EXEC_NOEXIT=1		| On error, do not exit script, inherited by $G_EXEC_NOHALT=1
	# - $G_EXEC_ARRAY_TEXT[]	| Add additional entries to error handler menu
	# - $G_EXEC_ARRAY_ACTION[]	| Associative array, containing uneven $G_EXEC_ARRAY_TEXT[] values as keys and related commands as values
	G_EXEC(){

		local exit_code fp_log='/tmp/G_EXEC_LOG' attempt=1 acommand=("$@")
		local ecommand=${acommand[*]//\\/\\\\}

		# Enter retry loop
		while :
		do
			declare -F G_EXEC_PRE_FUNC &> /dev/null && G_EXEC_PRE_FUNC
			# Exit immediately if exit_code=0 was set by G_EXEC_PRE_FUNC
			[[ $exit_code == 0 ]] && break

			# Execute command, store output to $fp_log file and store exit code to $exit_code variable
			# - Print full command output if $G_EXEC_OUTPUT=1 is given
			if [[ $G_EXEC_OUTPUT == 1 ]]; then

				# Print $G_EXEC_DESC if given, else raw input command string and show current non-interactive attempt count if $G_EXEC_RETRIES is given
				G_DIETPI-NOTIFY 2 "${G_EXEC_DESC:-$ecommand}, please wait...${G_EXEC_RETRIES:+ ($attempt/$((G_EXEC_RETRIES+1)))}"
				[[ $G_EXEC_OUTPUT_COL ]] && echo -ne "$G_EXEC_OUTPUT_COL"
				"${acommand[@]}" 2>&1 | tee "$fp_log"
				exit_code=${PIPESTATUS[0]}
				[[ $G_EXEC_OUTPUT_COL ]] && echo -ne '\e[0m'

			# - Else print animated processing message only
			else

				G_DIETPI-NOTIFY -2 "${G_EXEC_DESC:-$ecommand}${G_EXEC_RETRIES:+ ($attempt/$((G_EXEC_RETRIES+1)))}"
				"${acommand[@]}" &> "$fp_log"
				exit_code=$?

			fi

			declare -F G_EXEC_POST_FUNC &> /dev/null && G_EXEC_POST_FUNC

			# Override exit code if $G_EXEC_NOFAIL=1 is given
			[[ $G_EXEC_NOFAIL == 1 ]] && exit_code=0

			### Success: Print OK and exit retry loop
			[[ $exit_code == 0 ]] && { G_DIETPI-NOTIFY 0 "${G_EXEC_DESC:-$ecommand}"; break; }

			### Error

			# Retry non-interactively if current $attempt is <= $G_EXEC_RETRIES
			[[ $attempt -le $G_EXEC_RETRIES ]] && { ((attempt++)) && continue; }

			# Print FAILED, append raw command string if $G_EXEC_DESC is given
			G_DIETPI-NOTIFY 1 "${G_EXEC_DESC:+$G_EXEC_DESC\n - Command: }$ecommand"

			# Exit retry loop if $G_EXEC_NOHALT=1 is given
			[[ $G_EXEC_NOHALT == 1 ]] && break

			   # Prepare error handler menu and GitHub issue template
        local fp_error_report='/tmp/G_EXEC_ERROR_REPORT' log_content=$(<"$fp_log") last_whip_menu_item sent_bug_report
        if [[ -f '/boot/dietpi/.prep_info' ]]; then
            image_creator=$(mawk 'NR==1' /boot/dietpi/.prep_info)
            [[ $image_creator == 0 ]] && image_creator='DietPi Core Team'
            preimage_name=$(mawk 'NR==2' /boot/dietpi/.prep_info)
        fi

        local dietpi_version="v$G_DIETPI_VERSION_CORE.$G_DIETPI_VERSION_SUB.$G_DIETPI_VERSION_RC ($G_GITOWNER/$G_GITBRANCH)"
        local error_details=$(G_BUG_REPORT)
G_PROGRAM_NAME='dietpi-testERRORhandler'
        [[ ${G_PROGRAM_NAME,,} == 'dietpi-'* ]] && echo -e "\e[41m
\e[41m
---------------------------------------------------------------------
- DietPi has encountered an error                                   -
- Please create a ticket: https://github.com/MichaIng/DietPi/issues -
- Copy and paste only the BLUE lines below into the ticket          -
---------------------------------------------------------------------\e[44m
#### Details:
- Date           | $(date)
- Command        | \`$ecommand\`
- Exit code      | $exit_code
- Software title | $G_PROGRAM_NAME
$error_details
#### Steps to reproduce:
<!-- Explain how to reproduce the issue -->
1. ...
2. ...
#### Expected behaviour:
<!-- What SHOULD happen? -->
- ...
#### Actual behaviour:
<!-- What IS happening? -->
- ...
#### Extra details:
<!-- Please post any extra details that might help solve the issue -->
- ...
#### Additional logs:
\`\`\`
$log_content
\`\`\`\e[41m
---------------------------------------------------------------------\e[0m" > "$fp_error_report"

G_INTERACTIVE=1 ####### REMOVE ME AFTER TESTING
			# Enter error handler menu loop in interactive mode
			[[ $G_INTERACTIVE == 1 ]] && while :
			do
				G_WHIP_MENU_ARRAY=('Retry' ': Re-run the last command that failed')
				# Add targeted solution suggestions, passed via $G_EXEC_ARRAY_TEXT[] and $G_EXEC_ARRAY_ACTION[${G_EXEC_ARRAY_TEXT[]}]
				[[ $G_EXEC_ARRAY_TEXT ]] && G_WHIP_MENU_ARRAY+=("${G_EXEC_ARRAY_TEXT[@]}")
				# Allow to open DietPi-Config if this error was not produced within DietPi-Config
				pgrep -cf 'dietpi-config' &> /dev/null || G_WHIP_MENU_ARRAY+=('DietPi-Config' ': Edit network, APT/NTP mirror settings etc')
				G_WHIP_MENU_ARRAY+=('Open subshell' ': Open a subshell to investigate or solve the issue')
 ####### REMOVE AFTER TESTING
G_PROGRAMM_NAME='dietpi-test'  ####### REMOVE ME AFTER TESTING
                # Allow to send bug report, if it was produced by one of our scripts
				[[ ${G_PROGRAM_NAME,,} == 'dietpi-'* && $G_PROGRAM_NAME != 'DietPi-Installer' ]] && G_WHIP_MENU_ARRAY+=('Send report' ': Uploads bugreport containing system info to DietPi')
				G_WHIP_MENU_ARRAY+=('' '●─ Devs only ')
				G_WHIP_MENU_ARRAY+=('Change command' ': Adjust and rerun the command')
 ####### REMOVE AFTER TESTING
G_EXEC_NOEXIT=1  ####### REMOVE AFTER TESTING
				# Show "Ignore" on cancel button if $G_EXEC_NOEXIT=1 is given, else "Exit"
				[[ $G_EXEC_NOEXIT == 1 ]] && G_WHIP_BUTTON_CANCEL_TEXT='Ignore' || G_WHIP_BUTTON_CANCEL_TEXT='Exit'
				G_WHIP_DEFAULT_ITEM=${last_whip_menu_item:-Retry}
				G_WHIP_MENU "${G_EXEC_DESC:+$(mawk '{gsub("\\\e[[0-9][;0-9]*m","");print}' <<< "$G_EXEC_DESC")\n} - Command: ${acommand[*]}
 - Exit code: $exit_code
 - DietPi version: $dietpi_version | HW_MODEL: $G_HW_MODEL | HW_ARCH: $G_HW_ARCH | DISTRO: $G_DISTRO
${image_creator:+ - Image creator: $image_creator\n}${preimage_name:+ - Pre-image: $preimage_name\n} - Error log:
$log_content" || break # Exit error handler menu loop on cancel

 ####### REMOVE AFTER TESTING
				last_whip_menu_item=$G_WHIP_RETURNED_VALUE  ####### REMOVE AFTER TESTING

				if [[ $G_WHIP_RETURNED_VALUE == 'Retry' ]]; then

					# Reset current $attempt and continue retry loop
					attempt=1
					continue 2

				elif [[ $G_WHIP_RETURNED_VALUE == 'DietPi-Config' ]]; then

					/boot/dietpi/dietpi-config

				elif [[ $G_WHIP_RETURNED_VALUE == 'Open subshell' ]]; then

					G_WHIP_MSG 'A bash subshell will now open which allows you to investigate and/or fix the issue.
\nPlease use the "exit" command when you are finished, to return to this error handler menu.'
					# Prevent dietpi-login call in subshell
					local reallow_dietpi_login=1
					[[ $G_DIETPI_LOGIN ]] && reallow_dietpi_login=0
					export G_DIETPI_LOGIN=1
					bash &> /dev/tty < /dev/tty
					(( $reallow_dietpi_login )) && unset -v G_DIETPI_LOGIN
G_WHIP_RETURNED_VALUE='Send report'
				elif [[ $G_WHIP_RETURNED_VALUE == 'Send report' ]]; then

					/boot/dietpi/dietpi-bugreport 1 && sent_bug_report=1
					read -rp '
Press any key to continue...'

				elif [[ $G_WHIP_RETURNED_VALUE == 'Change command' ]]; then

					G_WHIP_DEFAULT_ITEM=${acommand[*]}
					if G_WHIP_INPUTBOX 'Please enter/alter the command to be executed.
\nNB: Please only use this solution if you know for sure that it will not cause follow up issues from the originating script. It will e.g. allow you to continue a certain software install, but if you edit the download link, the originating script might expect files which are not present.
\nUse this with caution!'; then

						G_DIETPI-NOTIFY 2 "Executing alternative command: ${G_WHIP_RETURNED_VALUE//\\/\\\\}"
						$G_WHIP_RETURNED_VALUE
						exit_code=$?
						G_DIETPI-NOTIFY -1 "$exit_code" 'Alternative command execution'
						# Exit retry loop if alternative command succeeded, else stay in menu loop and wait for key press to allow reviewing alternative command output
						# shellcheck disable=SC2015
						[[ $exit_code == 0 ]] && break 2 || read -rp 'Press any key to return to error handler menu...'

					fi

				# Attempt targeted solution, passed via $G_EXEC_ARRAY_TEXT[] and $G_EXEC_ARRAY_ACTION[${G_EXEC_ARRAY_TEXT[]}]
				elif [[ $G_WHIP_RETURNED_VALUE ]]; then

					eval "${G_EXEC_ARRAY_ACTION[$G_WHIP_RETURNED_VALUE]}"

				fi
			done

			# Error has not been solved, print GitHub issue template if it was produced and exit error handler menu loop
			if [[ -f $fp_error_report ]]; then

				# Add bug report ID if it was sent
				[[ $sent_bug_report == 1 ]] && sed --follow-symlinks -i "/^- Date           | /a\- Bug report     | $G_HW_UUID" "$fp_error_report"
				cat "$fp_error_report"

			fi
			break
		done

		# Do not exit originating script if $G_EXEC_NOEXIT=1 or $G_EXEC_NOHALT=1 is given
		local noexit
		[[ $G_EXEC_NOEXIT == 1 || $G_EXEC_NOHALT == 1 ]] && noexit=1

		# Cleanup
		rm -f "$fp_log" "$fp_error_report"
		unset -v G_EXEC_DESC G_EXEC_RETRIES G_EXEC_OUTPUT G_EXEC_OUTPUT_COL G_EXEC_NOFAIL G_EXEC_NOHALT G_EXEC_NOEXIT G_EXEC_ARRAY_TEXT G_EXEC_ARRAY_ACTION
		unset -f G_EXEC_PRE_FUNC G_EXEC_POST_FUNC

		# In case of unresolved error when exiting originating script, inform user and kill via SIGINT to prevent exiting from interactive shell session
		[[ $exit_code == 0 || $noexit == 1 ]] || { G_DIETPI-NOTIFY 1 "Unable to continue, ${G_PROGRAM_NAME:-command} will now terminate."; kill -INT $$; }

		# Else return exit code
		return "$exit_code"

	}

[MichaIng]

I still wonder whether it would be easier to have the whole template pre-filled as output, instead of only the first block. So one can copy&paste everything into an empty GitHub issue or forum topic. What you think?

The lines "Command"/"Exit code"/"Software title" (which we should rename to "Script" or "Program") could be added dynamically, e.g. given as input parameters for G_BUG_REPORT.

What I do not like is the doubled generation of $image_creator etc. Two options:

• Generate them as as non-local variables, so they can be re-used in G_EXEC. At best keep them local when $G_PROGRAM_NAME has not been set, hence when this has not been executed from one of our scripts and would leave variables in login shell. But they would then be missing in the error prompt from G_EXEC calls outside of our script.
• Just remove this info form the error prompt. It is probably pretty unimportant for users, respectively they know when they generated the image themselves. It is more important for us, and still included in the bug report.

[MichaIng]

Sorry that it took so long. I did the adjustments suggested above: G_BUG_REPORT prints now a complete template, which can be copy&pasted as a whole as GitHub issue or forum topic. Also, what I wanted to do earlier already, the template is not always printed anymore, but instead there is a menu option to do so. It spams the console quite much, and the raw command output was often duplicated.

[JappeHallunken]

I think I should apologize, because I started this and did not continue to work on it.

[MichaIng]

From console:

root@VM-Bookworm:~# G_BUG_REPORT
#### Details:
- Date           | 2024-07-07 19:00:15
- DietPi version | v9.6.0 (JappeHallunken/dev)
- Distro version | bookworm (ID=7)
- Kernel version | `Linux VM-Bookworm 6.1.0-22-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.94-1 (2024-06-21) x86_64 GNU/Linux`
- Architecture   | `amd64`
- Hardware model | Virtual Machine (x86_64) (ID=20)
- Power supply   | (EG: RAVPower 5V 1A)
- SD card        | (EG: SanDisk Ultra 16 GB)
#### Steps to reproduce:
<!-- Explain how to reproduce the issue -->
1. ...
2. ...
#### Expected behaviour:
<!-- What SHOULD happen? -->
- ...
#### Actual behaviour:
<!-- What IS happening? -->
- ...
#### Extra details:
<!-- Please post any extra details that might help solve the issue -->
- ...
#### Additional logs:
```
<!-- Please paste logs or related console output here -->
```

New menu entry in error handler menu:

Which prints the report as before, with program/script name, failed command, exit code and logs added:

I added back the PSU and SD card line after this screenshot, which is still valuable info, IMO.

Also pinging @Joulinar, if one of you has another suggestion what to further add (or remove) from the printed info? I'll also align the GitHub and forum templates then, to ask for the same info.

One more nice thing would be to add the software title automatically, depending on at which step dietpi-software failed. Since we do set a software_id variable, it is quite easy to pass the currently installing software name via variable to the error handler. Currently, people often e.g. say "Nextcloud", while the error happened during the "PHP" installation, which sometimes requires another Q/A iteration to find out. But something for another PR, as it requires a change in dietpi-software as well.

[MichaIng]

@JappeHallunken

I think I should apologize, because I started this and did not continue to work on it.

No problem, and while it was good already, I had further ideas but did not find/take the time do merge them. I hope you find it beneficial as well the way it works now, as a complete template replacement.

[MichaIng]

Merging this now for release. We can fine tune it later, when needed.