Check if a string does not exist in multiple arrays

Question

Trying to check and see if a string value is NOT a member of two different arrays. If the string value does not exists in any of them, then i need to perform a statement. I was able to do it with an if statement....

if [[ $REQ_FIELDS_LIST != *"$XFR_FIELD"* && $NON_INPUT_FIELDS_LIST != *"$XFR_FIELD"* ]];then

But the asterisk is causing substrings to return false positives. Unfortunately, removing the "*" and the if statement just doesn't work at all. It seems, checking this site, that the only SAFE way to do it in bash is by doing a for loop. But what is the most efficient way of doing it for two different arrays. Also, the string value is a member of an array itself. So we are looping through an array already. Loop through array of string values and for each string value check to see if that string is not a member of the other two arrays. If so, then perform a statement.

So i need...

for XFR_FIELD in $INPUT_FIELDS_LIST
do
    if XFR field is not a member of REQ_FIELDS_LIST AND is not a member of NON_INPUT_FIELDS_LIST then
        "return 0"

Are $REQ_FIELDS_LIST && $NON_INPUT_FIELDS_LIST arrays or Variables? They are written as Variables, but you mention them as arrays. Also - how many space or other separated values are there in $INPUT_FIELDS_LIST, and is that an array? — itChi
– itChi, Commented Jan 17, 2019 at 16:20
@BrianGurka If REQ_FIELDS_LIST is an array, then $REQ_FIELDS_LIST only gets its first element; you need to use something like "$REQ_FIELDS_LIST[@]" to get all elements. But that won't work right in the test either; you really need to loop over the elements. — Gordon Davisson
– Gordon Davisson, Commented Jan 17, 2019 at 16:26
Have you considered using Associative arrays instead? That way the string value can become a key and membership is a simple test. — cdarke
– cdarke, Commented Jan 17, 2019 at 16:49
(checking associative-array key membership is also an amortized-constant-time operation, so it doesn't get slower as your arrays get longer; so it's not just less code, it's faster). — Charles Duffy
– Charles Duffy, Commented Jan 17, 2019 at 17:08
Not related to your issue, but in general you should avoid using uppercase variable names. This is because there are many uppercase names used by bash and you could get a name collision with one of them. — cdarke
– cdarke, Commented Jan 17, 2019 at 17:18

Charles Duffy · Accepted Answer · 2019-01-17 17:12:25Z

3

The efficient approach is to use bash 4.0's associative arrays:

#!/usr/bin/env bash
case $BASH_VERSION in ''|[123].*) echo "ERROR: Requires bash 4.0+" >&2; exit 1;; esac

declare -A input_fields_list=( [one]=1 [two]=1 [three]=1 [four]=1 [five]=1 )
declare -A req_fields_list=( [one]=1 [six]=1 [seven]=1 [eight]=1 )
declare -A non_input_fields_list=( [two]=1 [three]=1 [seven]=1 [eight]=1 [nine]=1 )

for xfr in "${!input_fields_list[@]}"; do
  [[ ${req_fields_list[$xfr]} ]] && continue
  [[ ${non_input_fields_list[$xfr]} ]] && continue
  echo "$xfr not found" >&2
  exit 1
done

echo "All input fields are valid" >&2
exit 0

As you can see at https://ideone.com/IhmVKy, this correctly exits with five not found.

answered Jan 17, 2019 at 17:12

Charles Duffy

299k43 gold badges442 silver badges498 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

cdarke Over a year ago

Doh! You beat me to it!

itChi Over a year ago

Very neat as a solution, nice use of keys!

Bsquare ℬℬ Over a year ago

Why did you output the informative message "All input fields are valid" in the error output?

Charles Duffy Over a year ago

@Bsquare, because it's an informational/logging message, which is what stderr is for (per POSIX definition). Putting your logs on stdout makes them pollute output. There's a reason shells (without a separate TTY fd) write prompts to stderr, even though those aren't errors either.

itChi · Accepted Answer · 2019-01-17 16:58:00Z

1

As an example, you would iterate like this:

#!/bin/bash

INPUT_FIELDS_LIST=( one two three four five)
REQ_FIELDS_LIST=( one six seven eight )
NON_INPUT_FIELDS_LIST=( two three seven eight nine )

for ifl in "${INPUT_FIELDS_LIST[@]}"
do

        for rfl in "${REQ_FIELDS_LIST[@]}"
        do
                myvar=0
                if [[ $ifl == "$rfl" ]]
                then myvar=1; break
                else continue
                fi
        done

        for nifl in "${NON_INPUT_FIELDS_LIST[@]}"
        do
                myvar2=0
                if [[ $ifl == "$nifl" ]]
                then myvar2=1; break
                else continue
                fi
        done

        if [[ $myvar == 1 ]] || [[ $myvar2 == 1 ]]
        then continue
        else echo "$ifl"
        fi

done

Iterate through secondary loops manually, and then "Flag" them as invalid.

the if statement at the end checks if either value has been matched, if it has it moves on to the next primary iteration.

Output:

four
five

edited Jan 17, 2019 at 16:58

answered Jan 17, 2019 at 16:49

itChi

6827 silver badges20 bronze badges

2 Comments

Brian Gurka Over a year ago

I chose to use this method. Thanks a ton!! Exactly what I was looking for. You rock

itChi Over a year ago

@BrianGurka I'm glad it worked out for you :) If this suits your needs, please consider marking this answered :)

Diego Torres Milano · Accepted Answer · 2019-01-17 18:48:23Z

0

You can check if an element is in an array with something like this, no loops and not affected by whitespace in elements

#! /bin/bash
function elem_in_array() {
    local e="$1"
    shift
    local a=("$@")
    [[ $(printf '\x01%s\x01' "${a[@]}") =~ $(printf '\x01%s\x01' "$e") ]]
}

a1=(A B "C D" E F)
elem_in_array "A" "${a1[@]}" && echo Y || echo N
elem_in_array "B" "${a1[@]}" && echo Y || echo N
elem_in_array "C D" "${a1[@]}" && echo Y || echo N
elem_in_array "AB" "${a1[@]}" && echo Y || echo N

edited Jan 17, 2019 at 18:48

answered Jan 17, 2019 at 17:14

Diego Torres Milano

69.9k9 gold badges116 silver badges145 bronze badges

3 Comments

Charles Duffy Over a year ago

[[ ]] parses its values as C strings. It can't possibly be honoring NUL literals (as opposed to, say, silently discarding them). Indeed, see the "ignoring NUL byte literal" warnings in the stderr log at ideone.com/USLNi9

Charles Duffy Over a year ago

(And the command substitutions aren't free -- real David Korn ksh93 optimizes $(printf ...) to not fork, but bash isn't that smart).

Charles Duffy Over a year ago

Try running elem_in_array "AB" "${a1[@]}" && echo BUG || echo OK with this code; you'll see it emits BUG, thinking that AB is a string present in your content.

Collectives™ on Stack Overflow

Check if a string does not exist in multiple arrays

3 Answers 3

4 Comments

2 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

2 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related