Unfortunately, it is not easy to selectively capture stderr output from an external utility (console application) in a variable.
The most concise, though somewhat obscure approach is (PSv4+):
# First, make sure that $ErrorActionPreference is at its default,
# 'Continue'
$stdout, $stderr = (./testrunner.bat 2>&1).Where({
$_ -is [string]
}, 'Split')
Note: If you're content to collect the combined stdout/stderr output in a single variable, the task simplifies to:
$output = ./testrunner.bat 2>&1
Note that the variables in both scenarios will contain an array of objects representing the collection of the individual output lines: stdout lines become (as onw would expect) [string] array elements, whereas PS converts stderr lines to [System.Management.Automation.ErrorRecord] instances.
You can then inspect the $stderr (or $output) variable for the success/failure indicator substring - note the use of enclosing wildcard characters *, because -like always matches against the entire string (unlike the regex-matching -match operator):
if ($stderr -like '*(0 failed)*') {
# ignore errors
} else {
# don't ignore errors
}
As for what you experienced:
when the batch file throws errors, the powershell script throws an error.
External programs, including batch files cannot throw errors in a PowerShell sense; they integrate as follows:
Their exit code is recorded in $LASTEXITCODE, and $? is set to $True if
$LASTEXITCODE is 0, and $False otherwise.
In the regular console, stderr output is passed through to the console - not colored in red - and not logged in $Error (the in-memory log of errors). In other words: stdout and stderr lines are formatted the same.
Unfortunately, the ISE behaves differently and does print stderr lines as if they were PowerShell errors and does log them in $Error - see this GitHub issue.
Therefore, it sounds like you were running from the ISE.
Additionally, if $ErrorActionPreference = 'Stop' is in effect when you run your code from the ISE, not only would the first stderr line present as a PowerShell error, but it would trigger termination of your entire PowerShell script.