Running Python script from Cocoa application using GCD

Question

I'm trying to run a Python script from a Cocoa app. It's working just fine on the main thread, but I'd like to have it running in the background, on a concurrent GCD queue.

I'm using the following method to setup a manager class that runs the Python script:

- (BOOL)setupPythonEnvironment {
    if (Py_IsInitialized()) return YES;

    Py_SetProgramName("/usr/bin/python");
    Py_Initialize();

    NSString *scriptPath = [[NSBundle mainBundle] pathForResource:@"MyScript"     ofType:@"py"];

    FILE *mainFile = fopen([scriptPath UTF8String], "r");
    return (PyRun_SimpleFile(mainFile, (char *)[[scriptPath lastPathComponent] UTF8String]) == 0);
}

After which the script is (repeatedly) called from the following instance method, using a shared singleton instance of the manager class:

- (id)runScriptWithArguments:(NSArray *)arguments {
    return [NSClassFromString(@"MyScriptExecutor") runWithArguments:arguments];
}

The above Objective-C code hooks into the following Python code:

from Foundation import *

def run_with_arguments(arguments):
#    ...a long-running script

class MyScriptExecutor(NSObject):
    @classmethod
    def runWithArguments_(self, arguments):
        return run_with_arguments(arguments)

This works when I always run the above Objective-C methods from the main queue, but the script returns null when run from any other queue. Could someone explain me if what I'm trying to do is just not supported, and whether there's a good way around it?

The Python scripts is called often and runs long, so doing that on the main thread would be too slow, a would be running it form a serial queue. In addition, I'd like to contain the concurrency code within Objective-C as much as possible.

Thanks,

Not enough information in this example to really know what you're doing. — jkh
– jkh, Commented Jan 15, 2012 at 6:58
Would you tell me how you integrated the Python binary with xcode? — bijan
– bijan, Commented Oct 22, 2012 at 12:10

ipmcc · Accepted Answer · 2012-01-15 23:00:25Z

14

From this page, it looks like there are some some pretty complex threading concerns specific to embedding python. Is there a reason you couldn't just run these scripts in a separate process? For instance, the following -runBunchOfScripts method would run the script ten times (by calling -runPythonScript) on a parallel background queue, collecting the resulting outputs into an array of strings, and then calling your object back on the main thread once all the scripts has completed:

- (NSString*)runPythonScript
{
    NSTask* task = [[[NSTask alloc] init] autorelease];
    task.launchPath = @"/usr/bin/python";  
    NSString *scriptPath = [[NSBundle mainBundle] pathForResource:@"MyScript" ofType:@"py"];
    task.arguments = [NSArray arrayWithObjects: scriptPath, nil];

    // NSLog breaks if we don't do this...
    [task setStandardInput: [NSPipe pipe]];

    NSPipe *stdOutPipe = nil;
    stdOutPipe = [NSPipe pipe];
    [task setStandardOutput:stdOutPipe];

    NSPipe* stdErrPipe = nil;
    stdErrPipe = [NSPipe pipe];
    [task setStandardError: stdErrPipe];

    [task launch];        

    NSData* data = [[stdOutPipe fileHandleForReading] readDataToEndOfFile];

    [task waitUntilExit];

    NSInteger exitCode = task.terminationStatus;

    if (exitCode != 0)
    {
        NSLog(@"Error!");
        return nil;
    }

    return [[[NSString alloc] initWithBytes: data.bytes length:data.length encoding: NSUTF8StringEncoding] autorelease];
}

- (void)runBunchOfScripts
{
    dispatch_group_t group = dispatch_group_create();
    NSMutableArray* results = [[NSMutableArray alloc] init];
    for (NSUInteger i = 0; i < 10; i++)
    {
        dispatch_group_async(group, dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
            NSString* result = [self runPythonScript];
            @synchronized(results)
            {
                [results addObject: result];
            }
        });
    }

    dispatch_group_notify(group, dispatch_get_main_queue(), ^{
        [self scriptsDidFinishWithResults: results];
        dispatch_release(group);
        [results release];
    });
}

- (void)scriptsDidFinishWithResults: (NSArray*)results
{
    NSLog(@"Do something with the results...");
}

Naturally the approach of using separate processes has it's limitations, not the least of which being the hard limit on the number of processes you can launch, but it seems a lot less fraught with peril than embedding the entire interpreter. I would say that unless you need to interact chattily between the scripts and the hosting environment, this would be a better approach.

answered Jan 15, 2012 at 23:00

ipmcc

30k5 gold badges89 silver badges154 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Bastiaan M. van de Weerd Over a year ago

Thanks, @ipmcc. Unfortunately, scripts will have to run multiple times. Enough to hit that hard limitation all too soon. That's why I had hoped to setup the Python environment once, leaving only minimal overhead for every time a script is run. In fact, the script is sent to the Python executor as an NSString, to be evaluated using a "safe" version of eval.

ipmcc Over a year ago

You could do a similar solution with a NSTask; Start one (or n) long-running Python interpreter task and have it read scripts from stdin and eval them, and then you can collect the output from standard out. If you can interact with your scripts using only strings, this would still make more sense than embedding the interpreter (at least from where I'm sitting.)

Bastiaan M. van de Weerd Over a year ago

Brilliant. That should definitely do it!

Collectives™ on Stack Overflow

Running Python script from Cocoa application using GCD

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related