Issue with Bash reading file and building string

Question

I'm attempting to read all the lines of a file and concatenate them into a string. After Googling extensively, here's what I have:

#!/bin/bash

filename='file.txt'
files=''

while read p; do
    files="$files $p"
done < $filename

echo $files

The problem is that it seems the files variable seems to get overwritten each time. Shouldn't it just be appending it?

Example file.txt:

test1
test2
test3

Output is:

 test2

(Note that I intentially posted this as is - there is a space prior to the word test2 in the output)

@heemayl Hmm. I am on Windows using Git Bash. Posted exactly what I'm running — General_Twyckenham
– General_Twyckenham, Commented Sep 23, 2016 at 20:55
@ThomasBPreusser I tried that, it just gave me the last line — General_Twyckenham
– General_Twyckenham, Commented Sep 23, 2016 at 20:55
Could it be that the CR in your Windows text files just hides the rest? It would still be there. You just do not see it on the screen. — Thomas B Preusser
– Thomas B Preusser, Commented Sep 23, 2016 at 21:00

chepner · Accepted Answer · 2016-09-23 21:00:29Z

3

Your input file has DOS line endings, and does not end with a newline. This means that the value of files is really test1\r test2\r. test3 is never added because the last call to read, which sets p to test3, has an exit status of 1 resulting from its failure to find a final newline character.

The terminal treats a carriage return as an instruction to move the cursor to the beginning of the line, so everything following a \r overwrites whatever was previously written on the line.

The right thing to do is to fix your input file so that it conforms to the POSIX standard for text files: all lines, including the last one, are terminated by a single \n character.

answered Sep 23, 2016 at 21:00

chepner

538k77 gold badges594 silver badges747 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Gordon Davisson Over a year ago

You could also hack the loop to deal with DOS format, with while IFS=$'\r' read p || [[ -n "$p" ]]; do. The IFS setting will make it trim trailing carriage return(s), and the || clause will continue the loop for the last (not-properly-terminated) line.

strobelight · Accepted Answer · 2016-09-23 21:00:32Z

1

The issue is that your file.txt has carriage returns.

To check this out, change your last line:

echo $files | cat -v

answered Sep 23, 2016 at 21:00

strobelight

2772 silver badges7 bronze badges

Collectives™ on Stack Overflow

Issue with Bash reading file and building string

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related