0

I am trying to execute the shell script in Python using subprocess module.

Below is my shell script which is called as testing.sh.

#!/bin/bash

hello=$jj1

echo $hello

echo $jj1
echo $jj2

for el1 in $jj3
do
    echo "$el1"
done

for el2 in $jj4
do
    echo "$el2"
done

Now I am trying to execute the above shell script in Python so I did like this - 

subprocess.call(['./testing.sh'])

and it works fine. Now I am thinking to add the above script in a JSON document like this and then execute it - 

json_script = '{"script":"above testing.sh script here"}'
j = json.loads(json_script)
shell_script = j['script']
subprocess.call(shell_script, shell=True)

But everytime I am trying, it is giving me an error -

Below is my full Python script which contains the above testing.sh shell script in a JSON document - 

#!/usr/bin/python

import subprocess
import json
import socket
import os

jsonData = '{"pp": [0,3,5,7,9], "sp": [1,2,4,6,8]}'
jj = json.loads(jsonData)

print jj['pp']
print jj['sp']

os.putenv( 'jj1',  'Hello World 1')
os.putenv( 'jj2',  'Hello World 2')
os.putenv( 'jj3', ' '.join( str(v) for v in jj['pp']  ) )
os.putenv( 'jj4', ' '.join( str(v) for v in jj['sp']  ) )

print "start"

jsonDataaa = '{"script":"#!/bin/bash \\n hello=$jj1 \\n echo $hello \\n echo $jj1 \\n echo $jj2 \\n for el1 in $jj3 \\n do \\n echo "$el1" \\n done \\n for el2 in $jj4 \\n do \\n echo "$el2" \\n done"}'
j = json.loads(jsonDataaa)

shell_script = j['script']
print "start"
subprocess.call(shell_script, shell=True)
print "end"

And below is the error I am getting - 

 File "C:\Python27\lib\json\__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "C:\Python27\lib\json\decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Python27\lib\json\decoder.py", line 381, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting , delimiter: line 1 column 113 (char 112)

And the expected output I should be getting like this - 

[0, 3, 5, 7, 9]
[1, 2, 4, 6, 8]
start
Hello World 1
Hello World 2
0
3
5
7
9
1
2
4
6
8
end

UPDATE:-

If I have my jsonDataaa like this 

jsonDataaa = '{"script":"#!/bin/bash \\n hello=$jj1 \\n echo $hello \\n echo $jj1 \\n echo $jj2 \\n"}'

then it works fine... And I am able to execute it properly.. But if my jsonDataaa is as I mentioned in my question, then only it gives me an error. I am thinking there might be some syntax error which I am not able to understand.

Improve this question

3 Answers 3

Reset to default
2

It gives you this error because your json string is invalid. Specifically, it contains unescaped quotes.

It works if you replace your jsonDataaa assignment with:

jsonDataaa = '{"script":"#!/bin/bash \\n hello=$jj1 \\n echo $hello \\n echo $jj1 \\n echo $jj2 \\n for el1 in $jj3 \\n do \\n echo \\"$el1\\" \\n done \\n for el2 in $jj4 \\n do \\n echo \\"$el2\\" \\n done"}'
Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

Aah silly mistake.. Thanks for pointing out.. One quick question. With the way I am doing currently, I always need to make the shell script in this format only? Meaning a new line will be added like this \\n and then followed by some command and escaped double quotes etc.
And also is there something wrong with this json document? jsonDataaa = '{"script":"#!/bin/bash \\n hello=$jj1 \\n echo $hello \\n echo $jj1 \\n echo $jj2 \\n for el1 in $jj3 \\n do \\n echo \\"$el1\\" \\n done \\n for el2 in $jj4 \\n do \\n echo \\"$el2\\" \\n done \\n for i in $( ls ); do \\n echo item: $i \\n done"}'
You do not need \\n you can use triple-quoted strings.
@JohnZwinck: Can you provide an example for that?
@SSH I don't know why you'd ever want to embed a bash script in a json string in a python file, but if you from the python repl do import json and json.dumps({ "script": open("yourfile").read() }), you'll get a correctly escaped string that can be pasted into the python file.
|
1

This part doesn't quite make sense:

jsonDataaa = '{"script":"#!/bin/bash \\n hello=$jj1 \\n echo $hello \\n echo $jj1 \\n echo $jj2 \\n for el1 in $jj3 \\n do \\n echo "$el1" \\n done \\n for el2 in $jj4 \\n do \\n echo "$el2" \\n done"}'
j = json.loads(jsonDataaa)

shell_script = j['script']
print "start"
subprocess.call(shell_script, shell=True)

What you've done here is pass a literal string as the argument to subprocess.call(), but it expects a program name, not program text. So you have a couple choices: you could write the contents of shell_script to a NamedTemporaryFile and execute that, or you could launch a piped bash in subprocess and feed its stdin from shell_script as a string. That latter approach is what I'd prefer, and you can get more help to do it here: Python - How do I pass a string into subprocess.Popen (using the stdin argument)?

P.S.: if you use a triple-quoted string you can make your JSON look a lot nicer, on multiple lines and whatnot.

Improve this answer

5 Comments

If I have my jsonDataaa like this jsonDataaa = '{"script":"#!/bin/bash \\n hello=$jj1 \\n echo $hello \\n echo $jj1 \\n echo $jj2 \\n"}' then it works fine... And I am able to execute it properly.. But if my jsonDataaa is as I mentioned in my question, then only it gives me an error. I am thinking there might be some syntax error which I am not able to understand..
I don't understand. Perhaps you can place a clear, detailed comparison of the exact minimal Python scripts that work and do not work, side-by-side, in the question. And change it to use triple-quoted block strings so we can read it.
See the edited question.. If you replace the jsonDataaa string in my question with my updated jsonDataaa string, then the whole program gets executed but with the original jsonDataaa string, it doesn't works fine.. So there is some syntax error in my json document
Actually, with 'shell=True' the first parameter is a complete command line which is to be passed to /bin/sh. It could be "(rm *.h; ls|grep spider>arach.txt)" for instance.
So I now think my answer shouldn't help, and I'm not sure why the different jsonDataaa makes a difference. Perhaps there's an error in the formatting of the original that we are missing; try inserting open("script_test","w").write(shell_script) and see what happens.
0

You could also do:

subprocess.call(["/bin/sh", "-c", shell_script])

You are passing the whole script as a single command-line parameter to sh. This should work even if there are newlines in shell_script; but if it's more than a certain size (16K bytes? something like that) it won't fit in a command line.

EDIT - by using subprocess.call(shell_script, shell=True) it actually amounts to the same as what I've suggested, more or less (perhaps the effect of other options might be different). So now I'm guessing the quoting of the \n etc may be incorrect as you've hinted at - see comments below.

Improve this answer

4 Comments

So you are saying, in your example, I don't need to add new line characters as \\n and then followed by some command? I can directly put them in a single line with a space in between and it should work?
Wait - not so sure, see other answer.
Actually, I'm relying here that the newlines are actually newlines (not \ actual n, but the actual newline char) and that's ok in my example (since it still appears as a single parm to /bin/sh)
So, I think maybe the formatting of JsonDataa (through 2 levels of quoting) may be the issue. Follow suggestion to use tripe-quotes, and my suggestion about writing the string to a file to see what's really there (or just 'print shell_script' and print repr(shell_script) and examine carefully)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.