Setting a limit for running time with a Python 'while' loop

Question

I have some questions related to setting the maximum running time in Python. In fact, I would like to use pdfminer to convert the PDF files to .txt. The problem is that very often, some files are not possible to decode and take an extremely long time. So I want to set time.time() to limit the conversion time for each file to 20 seconds. In addition, I run under Windows so I cannot use signal function.

I succeeded in running the conversion code with pdfminer.convert_pdf_to_txt() (in my code it is "c"), but I could not integrate the time.time() in the while loop. It seems to me that in the following code, the while loop and time.time() do not work.

In summary, I want to:

Convert the PDf file to a .txt file
The time limit for each conversion is 20 seconds. If it runs out of time, throw an exception and save an empty file
Save all the txt files under the same folder
If there are any exceptions/errors, still save the file, but with empty content.

Here is the current code:

import converter as c
import os
import timeit
import time

yourpath = 'D:/hh/'

for root, dirs, files in os.walk(yourpath, topdown=False):

    for name in files:

        t_end = time.time() + 20

        try:
            while time.time() < t_end:

                c.convert_pdf_to_txt(os.path.join(root, name))

                t = os.path.split(os.path.dirname(os.path.join(root, name)))[1]
                a = str(os.path.split(os.path.dirname(os.path.join(root, name)))[0])

                g = str(a.split("\\")[1])
                with open("D:/f/" + g + "&" + t + "&" + name + ".txt", mode="w") as newfile:
                    newfile.write(c.convert_pdf_to_txt(os.path.join(root, name)))
                    print "yes"

            if time.time() > t_end:

                print "no"

                with open("D:/f/" + g + "&" + t + "&" + name + ".txt", mode="w") as newfile:
                    newfile.write("")

        except KeyboardInterrupt:
           raise

        except:
            for name in files:
                t = os.path.split(os.path.dirname(os.path.join(root, name)))[1]
                a = str(os.path.split(os.path.dirname(os.path.join(root, name)))[0])

                g = str(a.split("\\")[1])
                with open("D:/f/" + g + "&" + t + "&" + name + ".txt", mode="w") as newfile:
                    newfile.write("")

@Stormvirux Yes I read this post before completing the above code...I still could not figure out how to integrate in my code ;( — SXC88
– SXC88, Commented Nov 22, 2016 at 14:30
No version of this will work since there is nothing in here that will interrupt an ongoing conversion that takes longer than 20s. — pvg
– pvg, Commented Nov 22, 2016 at 14:55

Peter Mortensen · Accepted Answer · 2021-01-05 16:35:03Z

1

You have the wrong approach.

You define the end time and immediately enter the while loop if the current timestamp is lower than the end timestamp (will be always True). So the while loop is entered and you get stuck at the converting function.

I would suggest the signal module, which is already included in Python. It allows you to quit a function after n seconds. A basic example can be seen in this Stack Overflow answer.

Your code would be like this:

return astring
import converter as c
import os
import timeit
import time
import threading
import thread

yourpath = 'D:/hh/'

for root, dirs, files in os.walk(yourpath, topdown=False):
    for name in files:
        try:
            timer = threading.Timer(5.0, thread.interrupt_main)
            try:
                c.convert_pdf_to_txt(os.path.join(root, name))
            except KeyboardInterrupt:
                 print("no")

                 with open("D:/f/" + g + "&" + t + "&" + name + ".txt", mode="w") as newfile:
                     newfile.write("")
            else:
                timer.cancel()
                t = os.path.split(os.path.dirname(os.path.join(root, name)))[1]
                a = str(os.path.split(os.path.dirname(os.path.join(root, name)))[0])
                g = str(a.split("\\")[1])

                print("yes")

                with open("D:/f/" + g + "&" + t + "&" + name + ".txt", mode="w") as newfile:
                    newfile.write(c.convert_pdf_to_txt(os.path.join(root, name)))

        except KeyboardInterrupt:
           raise

        except:
            for name in files:
                t = os.path.split(os.path.dirname(os.path.join(root, name)))[1]
                a = str(os.path.split(os.path.dirname(os.path.join(root, name)))[0])

                g = str(a.split("\\")[1])
                with open("D:/f/"+g+"&"+t+"&"+name+".txt", mode="w") as newfile:
                    newfile.write("")

Just for the future: Four spaces indentation and not too much whitespace ;)

edited Jan 5, 2021 at 16:35

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Nov 22, 2016 at 14:47

linusg

6,4395 gold badges32 silver badges81 bronze badges

Sign up to request clarification or add additional context in comments.

15 Comments

SXC88 Over a year ago

Hi thank you for your comment! But the problem is that I use the module pdfminer which can only be used under python 2.X so I don't think signal function is available here (It always throw an error actually). In addition, with your code an error message pops up and indicates that there is a syntax error with "except KeyboardInterrupt"...I am so confused...;((

linusg Over a year ago

@SXC88 - Sorry, I had some wrong indentation. Is fixed now. The signal module is available under Python 2 too: docs.python.org/2/library/signal.html. Please run again, tell me if the syntax error is gone an what error you get about the signal module.

SXC88 Over a year ago

Than you! But it always throws me this message "line 14, in <module> signal.signal(signal.SIGALRM, timeout_handler) AttributeError: 'module' object has no attribute 'SIGALRM'" ...

linusg Over a year ago

Ah ok. So its not the signal module being not available, but an not existing variable. Wait a sec, I'll fix that.

linusg Over a year ago

@SXC88 - I'm very sorry, but I guess there's no fast solution for this. The signal module has some Unix-only parts, and I'm runing Ubuntu. I assume you have windows, so I can't test it for you :( However, hopefully you know what was wrong with your code now and I could help at least a bit!

|

Collectives™ on Stack Overflow

Setting a limit for running time with a Python 'while' loop

1 Answer 1

15 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

15 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related