input to C++ executable python subprocess

Question

I have a C++ executable which has the following lines of code in it

/* Do some calculations */
.
.
for (int i=0; i<someNumber; i++){
   int inputData;
   std::cin >> inputData;
   std::cout<<"The data sent from Python is :: "<<inputData<<std::endl;
   .
   .
   /* Do some more calculations with inputData */
}

and this is called in a loop. I want to call this executable in python subprocess like

p = Popen(['./executable'], shell=True, stdout=PIPE, stderr=PIPE, stdin=PIPE)

I could get the output from the executable using

p.server.stdout.read()

But I am not able to send data (integers) from python using

p.stdin.write(b'35')

Since cin is called in a loop, the stdin.write should also be called multiple times (in a loop). Is this above possible .. ?

Any hints and suggestion how I could do it ? Thanks in advance.

omg ... i hope your calculations are very expensive other wise this is probably faster to just do in python or just do in C ... if they are expensive you should look at compiling your c++ into a dll or even a python library ... (and certainly take the user input out of the c code if you can) — Joran Beasley
– Joran Beasley, Commented Sep 14, 2015 at 17:04
@JoranBeasley yes my computations are expensive to do them in python. That is the reason I chose to do it like this. Also I need to communicate small data with python. — AdityaG
– AdityaG, Commented Sep 14, 2015 at 19:46

Community · Accepted Answer · 2017-05-23 12:33:24Z

19

Here's minimalistic example of how to call a C++ executable from Python, and communicate with it from Python.

1) Note that you must add \n when writing to input stream (i.e. stdin) of the subprocess (just like you would hit Rtn if running the program manually).

2) Also note the flushing of the streams, so that the receiving program doesn't get stuck waiting for the whole buffer to fill before printing the result.

3) And if running Python 3, be sure to convert the streaming value from string to bytes (see https://stackoverflow.com/a/5471351/1510289).

Python:

from subprocess import Popen, PIPE

p = Popen(['a.out'], shell=True, stdout=PIPE, stdin=PIPE)
for ii in range(10):
    value = str(ii) + '\n'
    #value = bytes(value, 'UTF-8')  # Needed in Python 3.
    p.stdin.write(value)
    p.stdin.flush()
    result = p.stdout.readline().strip()
    print(result)

C++:

#include <iostream>

int main(){
    for( int ii=0; ii<10; ++ii ){
        int input;
        std::cin >> input;
        std::cout << input*2 << std::endl;
        std::cout.flush();
    }
}

Output of running Python:

edited May 23, 2017 at 12:33

CommunityBot

11 silver badge

answered Sep 14, 2015 at 17:38

Velimir Mlaker

11k4 gold badges48 silver badges60 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

AdityaG Over a year ago

thank you very much for your help. But unfortunately I get error saying TypeError: 'str' does not support the buffer interface ....

Joran Beasley Over a year ago

@AdityaG just change to p.stdin.write(b'35\n') ... that use bytes instead of str (@Velimir is using str , maybe different python versions)

Velimir Mlaker Over a year ago

@Joran Beasley, you're right. AdityaG, you must be running Python 3. In that case, you must convert the string to bytes. I added a comment that does just that.

AdityaG Over a year ago

@Velimir ... one more small question, do you think we can send numpy array like that ..? (I will experiment but am just curious).

Velimir Mlaker Over a year ago

Well, using this methodology you can stream anything you want from Python, so long as your C++ executable properly parses standard input. But ideally you'd call your C++ computation routines directly in Python, by wrapping your C++ code (e.g. using SWIG) and generating Python callable functions -- creating subprocesses and streaming bytes is not optimal.

Collectives™ on Stack Overflow

input to C++ executable python subprocess

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related