How do you send commands with arguments already inside them to GNU parallel?

Question

I have a bash array:

nodes=(
    "command"
    "command arg"
    ...
    )

and I want to run all the commands with all the arguments that are already attached to them using GNU parallel.

I've tried

printf '%s\n' "${nodes[@]}" | parallel python

and

parallel python ::: "${nodes[@]}"

The output command is

python path_to_file\ arg

and the error it gives is "can't open file 'path_to_file arg'"

I think the problem has to do that backslash - I get the same error when I run the command without parallel.

How do I prevent it from putting the backslash in?

You need to re-split your input.

KamilCuk
– KamilCuk

2020-01-23 02:20:05 +00:00
Commented Jan 23, 2020 at 2:20 — KamilCuk
– KamilCuk, Commented Jan 23, 2020 at 2:20

Ole Tange · Accepted Answer · 2020-01-24 03:10:40Z

2

Try:

printf '%s\n' "${nodes[@]}" | parallel eval python

eval is a shell command that evals the string as shell expression. I typically used it to "de-quote" a string.

or:

printf '%s\n' "${nodes[@]}" | parallel python {=uq=}

Newer versions of GNU Parallel have uq() which leaves the value unquoted. Normally GNU Parallel will quote values.

or:

printf '%s\n' "${nodes[@]}" | parallel

The exception to the rule above is when there is no command. Then the value is unquoted.

edited Jan 24, 2020 at 3:10

answered Jan 23, 2020 at 11:47

Ole Tange

34.2k9 gold badges93 silver badges112 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Florian van Kampen-Wright Over a year ago

first one works - nice What would the second and third one do? And why does this first one work? I don't know if I've ever used eval

KamilCuk · Accepted Answer · 2020-01-23 02:33:35Z

1

You need to split you input on spaces into arguments. You can for example (ab-)use word splitting expansion by the shell. parallel passed your arguments as they are, so you run python "command arg" - with one argument with literal space preserved. Yet I guess you want python command arg - pass two arguments and the space acts as a separator.

parallel 'python $*' ::: "${nodes[@]}"

Using xargs that would look like:

printf "%s\n" "${nodes[@]}" | xargs -l sh -c 'python $*' --

answered Jan 23, 2020 at 2:33

KamilCuk

146k8 gold badges85 silver badges154 bronze badges

Collectives™ on Stack Overflow

How do you send commands with arguments already inside them to GNU parallel?

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related