0

In bash I have the following array with data:

(
"01.05.2022 01:01:35 [ERROR] test
 error message"
"01.05.2022 00:00:55 [SUCCESS] test
 success message"
"02.05.2022 00:00:35 [ERROR] test
 error message"
"01.05.2022 00:05:00 [WARNING] test
 warning message"
)

Expected Result:

(
"01.05.2022 00:00:55 [SUCCESS] test
 success message"
"01.05.2022 00:05:00 [WARNING] test
 warning message"
"01.05.2022 01:01:35 [ERROR] test
 error message"
"02.05.2022 00:00:35 [ERROR] test
 error message"
)

How can I sort a given array by dates only directly in bash and save it to a file?

6
  • 2
    Please, post the related expected output. Don't post is as a comment, an image, a table or a link to an off-site service but use text and include it to your original question. Thanks Commented Jun 9, 2022 at 9:07
  • Complemented by the expected result @JamesBrown Commented Jun 9, 2022 at 9:11
  • so you want to sort a bash array? and the elements in the bash array contain newlines? Commented Jun 9, 2022 at 9:11
  • Yes, I want to sort an array in bash. Yes, the elements in the bash array contain newlines. @Fravadona Commented Jun 9, 2022 at 9:16
  • please update the question to indicate the date format ... M/D/Y or D/M/Y? perhaps also add an entry that makes this clear and also has a different year (eg, 12/22/2019) Commented Jun 9, 2022 at 15:03

3 Answers 3

2

How can I sort a given array by dates only directly in bash and save it to a file?

I’m assuming that “only directly in Bash” implies that additional processes and external tools such as sort are disallowed.

A solution in pure Bash can be based around the fact that indexed “arrays” in Bash are sparse and automatically sorted by (possibly sparse) indices.

That said, the problem translates into a choice of sparse indices such that their sorting order corresponds to the sorting order of dates. For example, one can pick indices in which four highest decimal orders represent years, lowest two decimal orders represent seconds and everything else is in between, rather obviously:

sort_my_special_array() {
  local -n array="$1"  # passing array by reference
  local -a reindexed   # sparse array indexed by YYYYMMDDhhmmss
  local -i idx         # index in the above
  local a year month day hour minute second drop
  for a in "${array[@]}"; do
    IFS='.: ' read day month year hour minute second drop <<< "$a"
    idx="10#${year}${month}${day}${hour}${minute}${second}"
    reindexed[idx]="$a"
  done
  array=("${reindexed[@]}")
}

A simple test for the code above:

my_bash_array=(
    '01.05.2022 01:01:35 [ERROR] test
 error message'
    '01.05.2022 00:00:55 [SUCCESS] test
 success message'
    '02.05.2022 00:00:35 [ERROR] test
 error message'
    '01.05.2022 00:05:00 [WARNING] test
 warning message'
)

echo -e 'Before:'
(IFS=$'\n'; echo "${my_bash_array[*]@Q}";)

sort_my_special_array my_bash_array  # Here!

echo -e '\nAfter:'
(IFS=$'\n'; echo "${my_bash_array[*]@Q}";)

The output of the test is (…drumroll…):

Before:
$'01.05.2022 01:01:35 [ERROR] test\n error message'
$'01.05.2022 00:00:55 [SUCCESS] test\n success message'
$'02.05.2022 00:00:35 [ERROR] test\n error message'
$'01.05.2022 00:05:00 [WARNING] test\n warning message'

After:
$'01.05.2022 00:00:55 [SUCCESS] test\n success message'
$'01.05.2022 00:05:00 [WARNING] test\n warning message'
$'01.05.2022 01:01:35 [ERROR] test\n error message'
$'02.05.2022 00:00:35 [ERROR] test\n error message'

To store a Bash array into a file in a parsable format, you can use the A modifier:

echo "${my_bash_array[@]@A}" > /path/to/some/file

This↑↑↑ yields the following output:

declare -a my_bash_array=([0]=$'01.05.2022 00:00:55 [SUCCESS] test\n success message' [1]=$'01.05.2022 00:05:00 [WARNING] test\n warning message' [2]=$'01.05.2022 01:01:35 [ERROR] test\n error message' [3]=$'02.05.2022 00:00:35 [ERROR] test\n error message')
Sign up to request clarification or add additional context in comments.

3 Comments

Not really, the sort can be used.
What's the purpose of 10# in "idx="10#..."?
To prevent numbers starting with 0 from being interpreted as octal (example: declare -i a=08 causes an error). Admittedly, years don’t start with zeros, so in this specific case it is unnecessary to specify the base. It would only matter if the highest order was e.g. hours.
2

Your dates appear at the start of each string and are in a format that allows to sort them alphabetically (i.e. without the need of conversion to EPOCH time).
update: I was wrong. As @markp-fuso noticed, the dates are not in a sortable format so you'll have to decorate | sort | undecorate them

So, given the bash array:

my_bash_array=(
"01.05.2022 01:01:35 [ERROR] test
 error message"
"01.05.2022 00:00:55 [SUCCESS] test
 success message"
"02.05.2022 00:00:35 [ERROR] test
 error message"
"01.05.2022 00:05:00 [WARNING] test
 warning message"
)

Here's a simple way to sort this array by date (that is, if your sort command supports the -z option):

# for bash >= 4.3:
readarray -d '' my_bash_array < <(
    printf '%s\0' "${my_bash_array[@]}" |
    sed -z -E 's/^((..)\.(..)\.(....) (..):(..):(..) )/\4\3\2\5\6\7 \1/' |
    sort -z -n -k1,1 |
    sed -z 's/^[^ ]* //'
)
# for older bash:
i=0
while IFS='' read -r -d '' element
do
   my_bash_array[i++]=$element
done < <(
    printf '%s\0' "${my_bash_array[@]}" |
    sed -z -nE 's/^((..)\.(..)\.(....) (..):(..):(..) )/\4\3\2\5\6\7 \1/p' |
    sort -z -n -k1,1 |
    sed -z 's/^[^ ]* //'
)

5 Comments

This is a great solution, it worked for me. The only clarification is that files of large sizes (more than 200mb) are sorted for several minutes ... Are there any faster sorting methods?
How to you fill your bash array initially? can't you sort it while building it?
Initially, I fill my array according to the algorithm that I wrote in advance. And the array assembly script itself is fast, but it slows down a lot at the sorting stage.
I would rewrite the array assembly function/script for it to output each element followed by a null-byte, then I would fill the bash array after sorting the outputed elements; something along the lines of readarray -d '' my_bash_array < <(assembly_script | sed ... | sort ... | sed ...) . That will avoid the big array expansion done by printf '%s\0' "${my_bash_array[@]}", which is pointlessly time consuming
Are there any faster sorting methods? For something so big, do not use Bash arrays then. Do not use Bash then. Use python or perl. Reading readarray and printing printf ... ${my_bash_array are very expensive operations for Bash, consider a different langauge.
0

Let's sort the items as strings first.

$ cat dates.txt | tr -d '"' | sort -k1,2
01.05.2022 00:00:55 [SUCCESS] test success message
01.05.2022 00:05:00 [WARNING] test warning message
01.05.2022 01:01:35 [ERROR] test error message
02.05.2022 00:00:35 [ERROR] test error message

As you can see this works.

Now if we want to process the array:

array=(                                                                                                                                          
    "01.05.2022 01:01:35 [ERROR] test error message"                                                                                            \
    "01.05.2022 00:00:55 [SUCCESS] test success message"                                                                                        \
    "02.05.2022 00:00:35 [ERROR] test error message"                                                                                            \
    "01.05.2022 00:05:00 [WARNING] test warning message"                                                                                         
)                                                                                                                                                
                                                                                                                                                 
str=""                                                                                                                                           
for a in "${array[@]}"; do                                                                                                                       
    str+="${a}\n"                                                                                                                                
done                                                                                                                                             
                                                                                                                                                 
str2=$(sort -k1,2 < <(echo -e "$str") | sed -E '/^$/d')                                                                                          
array2=()                                                                                                                                        
                                                                                                                                                 
while read line; do                                                                                                                              
    array2+=("$line")                                                                                                                            
done < <(echo -e "$str2")                                                                                                                        
                                                                                                                                                 
#########################################                                                                                                        
# array2 is what we want to write to file                                                                                                        
#########################################    

You can now extract elements from array2 and write to a file as you wish.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.