waiting to connect

[Skaperen]

i am calling a library that connects to some site and gets some data. sometimes it is slow. i would like for my code to output a message to stderr if this takes longer than N seconds to complete. if it completes in less than N seconds then i want no message to be output. this seems like it should be simple. i'd like to see the simplest pythonic code that can do this without using threads or processes.

[bowlofred]

Do you want to cancel the call at that point? I don't see how you can continue the call and do something else while wanting to use no threads or processes. It's pretty easy if the library owner writes a non-blocking call. But if the call is blocking, you're blocked.

Why the restriction on threads or processes?

[ndc85430]

It would probably help if you said which library. In any case, they often let you set timeouts for various things. Did you read the documentation?

[Skaperen]

i don't want to cancel it. the message will just be saying what the delay is for.

the library is botocore.

[ndc85430]

As bowlofred suggests, it doesn't seem to be possible to do what you want without threads/processes/coroutines or something. If the call blocks, then the main thread of your program is going to be blocked waiting for that to finish and so you won't be able to do anything else.

[Skaperen]

would such a block stop the handling of a signal? if not, what about scheduling a timer/alarm signal? or would that mess up the library's connection timeout?

[bowlofred]

Depends. Might work for a python library, if you don't do any library calls in the signal handler (since that might not be consistent). Potentially you could do a print in a signal handler.

If the library is written in C though, the signal may not be delivered to the python execution VM until after the C code finishes.

[Skaperen]

a print is all i need if the signal won't repeat. otherwise, i also need t cancel the repeating signal. if the library call returns fast, i need to either cancel the signal or set a flag that tells the signal handler to not do the print.

[DeaD_EyE]

The problem is, that the connection is closed after the timeout and the socket is unusable.

Two different possible solutions:

from socket import (
    socket,
    AF_INET,
    SOCK_STREAM,
    timeout as SocketTimeout,
)

import backoff
# https://pypi.org/project/backoff/


def connect(ip, port):
    addr = ip, port
    while True:
        sock = socket(AF_INET, SOCK_STREAM)
        sock.settimeout(1)
        try:
            sock.connect(addr)
        except SocketTimeout:
            print("Got a timeout. Trying again")
        else:
            return sock


@backoff.on_exception(wait_gen=backoff.expo, exception=SocketTimeout, max_tries=5, max_time=60, factor=0.5)
def connect_backoff(ip, port):
    addr = ip, port
    sock = socket(AF_INET, SOCK_STREAM)
    sock.settimeout(1)
    try:
        sock.connect(addr)
    except SocketTimeout:
        print("Got a timeout on addr", addr)
        # re raise the exception, that it can bubble up and
        # catched with backoff.on_exception
        raise
    return sock

You have to fit the timeout setting of the socket with your requirements.
The timout can only be a int, floats are not allowed for socket.settimeout().
The second function waits after an exception, to call it again.
If max_tries or max_time were hit, it will stop calling the function.
Async seems also to work with the backoff module.

[Skaperen]

i got it working.

def handler(signum,frame):
    print('waiting for list of regions from AWS',file=stderr,flush=True)
    return

signal.signal(signal.SIGALRM,handler)
signal.setitimer(signal.ITIMER_REAL,3,60)

valid_regions[:] = get_regions_from_aws(primary_region)

signal.setitimer(signal.ITIMER_REAL,0)
signal.signal(signal.SIGALRM,signal.SIG_DFL)

it will output the message if the call takes 3 seconds to complete and repeat every minute.