Python recursion limit

Question

So I understand the reason for the recursion limit of 1000. I want to run a script continuously, but am I right understanding that eventually the recursion limit will be reached (even if I set it higher) and Python will break?

In the scheme of things, its not a big deal, because I could get the OS to keep re-starting the script, but I thought there may be a more elegant solution I can employ within the script itself (swapping threads??).

My script:

import os
import subprocess
import time
import logging
import datetime
from sys import argv

if len(argv) < 3:
    exit('Please provide two arguments - Source Destination')


LOC_DIR = argv[1]
REM_DIR = argv[2]

POLL_INT = 10
RUN_INT = 60
FILE_EXT = '.mov'


# logging setup
logging.basicConfig(filename='%s' % os.path.join(LOC_DIR, '%s the_log.log' % datetime.datetime.now()),level=logging.DEBUG)

# make an easy print and logging function
def printLog(string):
    print '%s %s' % (datetime.datetime.now(), string)
    logging.info('%s %s' % (datetime.datetime.now(), string))


# get the files with absolute paths
def getFiles(path):
    return [os.path.join(path, entry) for entry in os.listdir(path)]


# check if file is still being copied (file size has changed within the poll interval)
def checkSize(path):
    same = False
    while same is False:
        printLog("Processing '%s'" % os.path.basename(path))
        printLog('Waiting %s seconds for any filesize change' % POLL_INT)
        size1 = os.path.getsize(path)
        time.sleep(POLL_INT)
        size2 = os.path.getsize(path)
        if size1 == size2:
            same = True
            printLog('File size stayed the same for %s seconds' % POLL_INT)
            return same
        else:
            printLog('File size change detected. Waiting a further %s seconds' % POLL_INT)


# check if correct file extension
def checkExt(path):
    if path.endswith(FILE_EXT):
        return True


# rsync subprocess
def rsyncFile(path):
    printLog("Syncing file '%s'" % os.path.basename(path))
    try:
        command = ['rsync', '-a', '--remove-source-files', path, REM_DIR]
        p = subprocess.Popen(command, stdout=subprocess.PIPE)
        for line in p.stdout:
            printLog("rsync: '%s'" %line)
        p.wait()
        if p.returncode == 0:
            printLog('<<< File synced successfully :) >>>')
        elif p.returncode == 10:
            printLog('****** Please check your internet connection!! ******  Rsync error code: %s' % p.returncode)
        else:
            printLog('There was a problem. Error code: %s' % p.returncode)
    except Exception as e:
        logging.debug(e)


# main logic
def main():
    all_files = getFiles(LOC_DIR)
    files = []
    for f in all_files:
        if checkExt(f):
            files.append(f)
    if len(files) == 1:
        printLog('<<< Found %s matching file >>>' % len(files))
    elif len(files) > 1:
        printLog('<<< Found %s matching files >>>' % len(files))
    for f in files:
        if checkSize(f):
            rsyncFile(f)
    printLog('No files found.  Checking again in %s seconds' % RUN_INT)
    time.sleep(RUN_INT)
    printLog('Checking for files')
    main()

if __name__ == "__main__":


    main()

Computer science tells us that any recursion can be rewritten as a loop. — 9000
– 9000, Commented Apr 23, 2011 at 5:27

Blender · Accepted Answer · 2013-12-19 10:08:08Z

7

CPython has no optimizations for recursion, so you really want to avoid deeply-recursive code in favor of regular loops:

def main():
    while True:
        all_files = getFiles(LOC_DIR)
        files = []
        for f in all_files:
            if checkExt(f):
                files.append(f)
        if len(files) == 1:
            printLog('<<< Found %s matching file >>>' % len(files))
        elif len(files) > 1:
            printLog('<<< Found %s matching files >>>' % len(files))
        for f in files:
            if checkSize(f):
                rsyncFile(f)
        printLog('No files found.  Checking again in %s seconds' % RUN_INT)
        time.sleep(RUN_INT)
        printLog('Checking for files')

if __name__ == "__main__":    
    main()

edited Dec 19, 2013 at 10:08

answered Apr 23, 2011 at 5:12

Blender

300k55 gold badges462 silver badges512 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

MFB Over a year ago

Thanks Blender, I didn't know I was recursing unnecessarily. Much obliged!

Nick ODell · Accepted Answer · 2011-04-23 05:12:04Z

1

You're going about this in the wrong way.

Replace the main loop with a loop.

# main logic
def main():
    while True:
        all_files = getFiles(LOC_DIR)
        files = []
        for f in all_files:
            if checkExt(f):
                files.append(f)
        if len(files) == 1:
            printLog('<<< Found %s matching file >>>' % len(files))
        elif len(files) > 1:
            printLog('<<< Found %s matching files >>>' % len(files))
        for f in files:
            if checkSize(f):
                rsyncFile(f)
        printLog('No files found.  Checking again in %s seconds' % RUN_INT)
        time.sleep(RUN_INT)
        printLog('Checking for files')

answered Apr 23, 2011 at 5:12

community wiki

Nick ODell

1 Comment

MFB Over a year ago

Thanks Nick, I'll give this a try too. Cheers!

Mahdi Yusuf · Accepted Answer · 2011-04-23 05:19:26Z

0

The recursion limit is only set with recursive functions from my understanding, so If you really want to run something repeatedly, you can simply run.

while True:
   #repeated stuff goes here

Recursion is an amazing tool, but handle with care, it often can end up burning you. You were right in the fact that python can only go 1000 calls deep recursively, so if you recursive method doesn't finish by then the exception gets thrown.

Goodluck.

edited Apr 23, 2011 at 5:19

answered Apr 23, 2011 at 5:13

Mahdi Yusuf

21.2k29 gold badges77 silver badges102 bronze badges

1 Comment

MFB Over a year ago

Thanks Dusty, I didn't even know I was using recursion.. completely unintentional! Cheers

Collectives™ on Stack Overflow

Python recursion limit

3 Answers 3

1 Comment

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related