Switch from raising a timeout on EINTR to retrying until the timeout has elapsed

Earlier I had submitted #250 to handle select being interrupted, and had changed this behavior to return as if it was a timeout. Now that we've been running this for a while we are seeing some occasional issues with that patch. Primarily that the remaining kazoo codebase assumes that a timeout from select is an actual timeout (which is reasonable)-- except that the previous patch changed that-- so you actually have to check the time elapsed in addition to the return. Instead of doing that (which seems like a mess, and error prone) this patch simply retries until the given timeout (assuming one was given). This does take into account that you may experience more than one interrupt in a given select() call (and we adjust the timeout accordingly for each iteration).

Author: jacksontj

kazoo/handlers/threading.py

@@ -143,17 +143,24 @@ def stop(self):
             python2atexit.unregister(self.stop)
 
     def select(self, *args, **kwargs):
-        try:
-            return select.select(*args, **kwargs)
-        except select.error as ex:
-            # if the system call was interrupted, we'll return as a timeout
-            # in Python 3, system call interruptions are a native exception
-            # in Python 2, they are not
-            errnum = ex.errno if isinstance(ex, OSError) else ex[0]
-            # to mimic a timeout, we return the same thing select would
-            if errnum == errno.EINTR:
-                return ([], [], [])
-            raise
+        # select() takes no kwargs, so it will be in args
+        timeout = args[3] if len(args) == 4 else None
+        # either the time to give up, or None
+        end = (time.time() + timeout) if timeout else None
+        while timeout is None or time.time() < end:
+            if timeout:
+                args = list(args)  # make a list, since tuples aren't mutable
+                args[3] = end - time.time()  # set the timeout to the remaining time
+            try:
+                return select.select(*args, **kwargs)
+            except select.error as ex:
+                # if the system call was interrupted, we'll retry until timeout
+                # in Python 3, system call interruptions are a native exception
+                # in Python 2, they are not
+                errnum = ex.errno if isinstance(ex, OSError) else ex[0]
+                if errnum == errno.EINTR:
+                    continue
+                raise
 
     def socket(self):
         return utils.create_tcp_socket(socket)