Python Forum
Thread Rating:
  • 2 Vote(s) - 3.5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
pyutils.py
#1
the original thread is quite old and now closed.

here is an update ... still a work in progress (and probably forever so):

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from __future__ import absolute_import,division,print_function,unicode_literals
"""
file          pyutils.py
purpose       collection of my useful python stuff
email         10054452614123394844460370234029112340408691

The intent is that this module works correctly under both Python 2 and
Python 3.  Please report failures or code improvement to the author.
"""

__license__ = """
Copyright (C) 2016, 2017 by Phil D. Howard - all other rights reserved

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA, OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE, OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

The author may be contacted by decoding the number
10054452614123394844460370234029112340408691
"""

import  base64
import  codecs
import  datetime
import  decimal
import  fcntl
import  hashlib
import  json
import  numbers
import  os
import  pickle
import  random
import  signal
import  struct
import  subprocess
import  sys
import  termios
import  time

# assign some shortcuts
now = datetime.datetime.now

b16decode = base64.b16decode
b16encode = base64.b16encode
b32decode = base64.b32decode
b32encode = base64.b32encode
b64decode = base64.b64decode
b64encode = base64.b64encode

getencoder = codecs.getencoder

dec              = decimal.Decimal
InvalidOperation = decimal.InvalidOperation

sha224 = hashlib.sha224
sha256 = hashlib.sha256
sha384 = hashlib.sha384
sha512 = hashlib.sha512

Number = numbers.Number

chdir  = os.chdir
env    = os.environ
getcwd = os.getcwd
getuid = os.getuid
link   = os.link
mkdir  = os.mkdir
rename = os.rename
remove = os.remove
rmdir  = os.rmdir

randint = random.randint

SIGKILL = signal.SIGKILL
SIGTERM = signal.SIGTERM

call  = subprocess.call
ocall = subprocess.check_output
PIPE  = subprocess.PIPE
Popen = subprocess.Popen

argv         = sys.argv
stderr       = sys.stderr
stdin        = sys.stdin
stdout       = sys.stdout
version      = sys.version
version_info = sys.version_info

sleep    = time.sleep
timesecs = time.time

stformat = '%Y-%m-%d-%H:%M:%S.%f'
tfnformat = stformat.replace(':','-').replace('.','-')

__y__=('ano','bai','da','ie','ja','jah','jes','kylla','oui','si','sim','taip','tak','yes')
__n__=('dim','ei','ez','ingen','inno','nao','ne','nee','nei','nein','nej','neniu','nie','no','non','nu')
__t__=('cierto','egia','ekte','ikke','istina','pravi','rigtigt','sann','satt','totta','true','vera','veraj','verdade','veritable','vero','vrai','waar','wahr','wier')
__f__=('falsa','falsch','false','falsk','falso','faux','lazan','rangar','sbagliate','vaara','vals')
__yt__=set(__y__+__t__+('j','y','1'))
__nf__=set(__n__+__f__+('n','0'))

xrangemax = 0x7fffffffffffffff # 9223372036854775807

if version_info[0] > 2:
    xrange = range
    long   = int

    
def get_terminal_geometry(fd=0):
    """Get terminal geometry (width,heght)

function        get_terminal_geometry
purpose         get terminal geometry (width,heght)
"""
    return struct.unpack('4H',fcntl.ioctl(fd,termios.TIOCGWINSZ,struct.pack('4H',0,0,0,0)))[1::-1]


def str_to_bytes(s):
    """Convert from str to bytes without any encoding changes

function        str_to_bytes
purpose         convert from str to bytes without any encoding changes
warning         this conversion can have unexpected effects
"""
    if version_info[0] < 3:
        return s
    return b''.fromhex(''.join([hex(ord(c))[2:] for c in s]))


def bytes_to_str(b):
    """Convert from bytes to str without any decoding changes

function        bytes_to_str
purpose         convert from bytes to str without any decoding changes
warning         this conversion can have unexpected effects
"""
    if version_info[0] < 3:
        return b
    return ''.join([chr(x) for x in b])


def hexify(o):
    """Convert strings,byte,ints,longs tohex

function        hexify
purpose         convert strings,byte,ints,longs to hex
"""
    if isinstance(o,str):
        return ''.join([hex(ord(c)) for c in o])
    if isinstance(o,bytes):
        return ''.join([hex(b) for b in o])
    if isinstance(o,(int,long)):
        return hex(o)
    return hexify(repr(o))


_rot13 = getencoder('rot-13')
def rot13(x):
    """Do rot13 conversion in a simple API way

function        rot13
purpose         do rot13 conversion in a simple API way
"""
    return _rot13(x)[0]


def chunks(s,l):
    """Split a sequence into fixed size chunks

function        chunks
purpose         split a sequence into fixed size chunks
    note          the last chunk is often shorter
    arguments     1 sequence to be split
                  2 length of each chunk (except last)
    thanks        j.crater,wavic,Mekire (python-forum.io)
"""
    return [s] if l<1 else (s[0+i:l+i] for i in range(0,len(s),l))


def timeyears():
    """Yield the current time scaled in average years (365.2425 days) of the proleptic Gregorian calendar

function        timeyears
purpose         yield the current time scaled in average years (365.2425 days) of the proleptic Gregorian calendar
"""
    return timesecs()/31556952.0


def timemonths():
    """Yield the current time scaled in average months (30.436875 days) of the proleptic Gregorian calendar

function        timemonths
purpose         yield the current time scaled in average months (30.436875 days) of the proleptic Gregorian calendar
"""
    return timesecs()/2629746.0


def timeweeks():
    """Yield the current time scaled in weeks

function        timeweeks
purpose         yield the current time scaled in weeks
"""
    return timesecs()/604800.0

    
def timedays():
    """Yield the current time scaled in days

function        timedays
purpose         yield the current time scaled in days
"""
    return timesecs()/86400.0


def timehours():
    """Yield the current time scaled in hours

function        timehours
purpose         yield the current time scaled in hours
"""
    return timesecs()/3600.0


def timemins():
    """Yield the current time scaled in minutes

function            timemins
purpose         yield the current time scaled in minutes
"""
    return timesecs()/60.0


def timemsecs():
    """Yield the current time scaled in milliseconds

function        timemsecs
purpose         yield the current time scaled in milliseconds
"""
    return timesecs()*1000.0


def timeusecs():
    """Yield the current time scaled in microseconds

function        timeusecs
purpose         yield the current time scaled in microseconds
"""
    return timesecs()*1000000.0


def timensecs():
    """Yield the current time scaled in nanoseconds

function        timensecs
purpose         yield the current time scaled in nanoseconds
"""
    return timesecs()*1000000000.0


def fint(x):
    """Convert x to an int only if x is a float with an int value

function        fint
purpose         convert x to an int only if x is a float with an int value
"""
    if isinstance(x,float):
        if x.is_integer():
            return int(x)
    return x


def tfn():
    """Return a timestamp string for a filename

function        tfn
purpose         return a timestamp string for a filename
"""
    return now().strftime(tfnformat)


def pv(*names,**opts):
    """Print a variable (name and value)
#       #       #       #       #       #       #       #       #       #       #
function        pv
purpose         print a variable (name and value)
"""
    p = '==>'
    for x in names:
        if not isinstance(x,(list,tuple)):
            x = [ x ]
        for name in x:
            name = str(name)
            d=inspect.currentframe().f_back.f_locals
            if name not in d:
                xprint(p,name,'not assigned',**opts)
            else:
                xprint(p,name,'=',repr(d[name]),**opts)
    return


def xprint(*msg,**opt):
    """Print, avoiding exceptions such as broken pipes.

function        xprint
purpose         print, avoiding exceptions such as broken pipes.
                also extends file= option to support str and int values
                file=str is a file name to be opened, printed to, then closed
                file=int is a file descriptor to be used
"""
    doclose = False
    if 'file' in opt:
        ofile = opt['file']
        try:
            if isinstance(ofile,str):
                ofile = open(ofile,'w')
                doclose = True
            if isinstance(ofile,int) and ofile > 0:
                ofile = os.fdopen(ofile,'w')
        except IOError:
            return
        opt['file'] = ofile
    try:
        otime = opt.pop('time',False)
        if otime:
            r = print(now().strftime(stformat),*msg,**opt)
        else:
            r = print(*msg,**opt)
    except IOError:
        r = None
    if doclose:
        ofile.close()
    return r
    
def tprint(*msg,**opt):
    """Like xprint but defaults to time=True

function        tprint
purpose         like xprint but defaults to time=True
"""
    opt['time'] = opt.pop('time',True)
    return xprint(*msg,**opt)


def eprint(*msg,**opt):
    """Like xprint() but defaults to file=stderr.

function        eprint
purpose         like xprint() but defaults to file=stderr.
"""
    opt['file'] = opt.pop('file',sys.stderr)
    return xprint(*msg,**opt)


def teprint(*msg,**opt):
    """Like tprint() but defaults file= to stderr.

function        teprint
purpose         like eprint() but defaults to time=True
"""
    opt['time'] = opt.pop('time',True)
    return eprint(*msg,**opt)


def vcall(*a,**aa):
    """Execute the given command after outputting the command.

function        vcall
purpose         execute the given command after outputting the command
                to stdout or where specified with cmdfile= or
                cmdfilename=
"""
    doclose = None
    if len(a) < 1:
        return -1
    if 'cmdfilename' in aa and isinstance(aa['cmdfilename'],str):
        try:
            cmdfile = open(aa['cmdfilename'],'a')
            doclose = cmdfile
            aa['cmdfile'] = cmdfile
        except IOError:
            pass
        aa.discard('cmdfilename')
    if 'cmdfile' in aa:
        xprint(repr(a[0]),file=aa['cmdfile'])
        if doclose != None:
            doclose.close()
    else:
        xprint(repr(a[0]))
        stdout.flush()
    return call(*a,**aa)


def vecall(*a,**aa):
    """Like vcall but defaults output to stderr.

function        vecall
purpose         like vcall but defaults output to stderr.
"""
    doclose = None
    if len(a) < 1:
        return -1
    if 'cmdfilename' in aa and isinstance(aa['cmdfilename'],str):
        try:
            cmdfile = open(aa['cmdfilename'],'a')
            doclose = cmdfile
            aa['cmdfile'] = cmdfile
        except IOError:
            pass
        aa.discard('cmdfilename')
    if 'cmdfile' in aa:
        eprint(repr(a[0]),file=aa['cmdfile'])
        if doclose != None:
            doclose.close()
    else:
        eprint(repr(a[0]))
        stderr.flush()
    return call(*a,**aa)
    

def ecall(args,**opt):
    """Like vcall() but defaults to file=sys.stderr

function        ecall
purpose         like vcall() but defaults to file=sys.stderr
"""
    if 'file' not in opt and 'filename' not in opt:
        opt['file'] = stderr
    return vcall(args,**opt)


def bcall(s,a):
    """Execute the given command in a named screen background session.

function        bcall
purpose         execute the given command in a named screen background session
note            due to a misfeature of screen the process started this
                way cannot be SIGSTOPped (SCREEN does SIGCONT)
"""
    if len(a) < 1:
        return -1
    if 'DISPLAY' in os.environ:
        d = os.environ['DISPLAY']
    else:
        d = ''
    return 0 if 0 == call(['/usr/bin/env','DISPLAY='+d,'SESSION_NAME='+s,'/usr/bin/screen','-S',s,'-dm']+a) else -2


def acall(*args,**opts):
    """Like call but all the positional arguments form the command.

function        acall
purpose         like call but all the positional arguments form the command
"""
    if len(args) < 1:
        return -1
    return call(args,**opts)


def xcall(args,**opt):
    """Like call but executes in the current process.

function        xcall
purpose         like call but executes in the current process replacing
                the python interpreter running this code and its caller.
"""
    if len(args) < 1:
        return -1
    return os.execvp(args[0],args)


def xacall(*args,**opts):
    """Like acall but executes in the current process.

function        xacall
purpose         like acall but executes in the current process
                replacing the python interpreter running this
                code and its caller.
"""
    if len(args) < 1:
        return -1
    return os.execvp(args[0],args)


def q_chdir(*a):
    """

function        q_chdir
purpose         do chdir() without raising an exception
    usage         this is convenient for use cases where an error can be ignored
"""
    result = None
    try:
        result = os.chdir(*a)
    except OSError:
        pass
    return result


cd = q_chdir


def q_mkdir(*a):
    """

function        q_mkdir
purpose         do mkdir() without raising an exception
    usage         this is convenient for use cases where an error can be ignored
"""
    result = None
    try:
        result = os.mkdir(*a)
    except OSError:
        pass
    return result


def q_link(*a):
    """

function        q_link
purpose         do link() without raising an exception
    usage         this is convenient for use cases where an error can be ignored
"""
    result = None
    try:
        result = os.link(*a)
    except OSError:
        pass
    return result


def q_symlink(*a):
    """

function        q_symlink
purpose         do symlink() without raising an exception
    usage         this is convenient for use cases where an error can be ignored
"""
    result = None
    try:
        result = os.symlink(*a)
    except OSError:
        pass
    return result


def q_rename(*a):
    """

function        q_rename
purpose         do rename() without raising an exception
    usage         this is convenient for use cases where an error can be ignored
"""
    result = None
    try:
        result = os.rename(*a)
    except OSError:
        pass
    return result


def q_remove(*a):
    """

function        q_remove
purpose         do remove() without raising an exception
    usage         this is convenient for use cases where an error can be ignored
"""
    result = None
    try:
        result = os.remove(*a)
    except OSError:
        pass
    return result


rm = q_remove


def q_rmdir(*a):
    """

function        q_rmdir
purpose         do rmdir() without raising an exception
    usage         this is convenient for use cases where an error can be ignored
"""
    result = None
    try:
        result = os.rmdir(*a)
    except OSError:
        pass
    return result


def b16to32(x):
    """

function        b16to32
purpose         convert base 16 encoding to base 32 encoding
"""
    return b32encode(b16decode(x.upper())).decode('ascii')


def b16to64(x,*a):
    """

function        b16to64
purpose         convert base 16 encoding to base 64 encoding
"""
    return b64encode(b16decode(x.upper()),*a).decode('ascii')


def b32to16(x):
    """

function        b32to16
purpose         convert base 32 encoding to base 16 encoding
"""
    return b16encode(b32decode(x)).decode('ascii').lower()


def b32to64(x,*a):
    """

function        b32to64
purpose         convert base 32 encoding to base 64 encoding
"""
    return b64encode(b32decode(x),*a).decode('ascii')


def b64to16(x,*a):
    """

function        b64to16
purpose         convert base 64 encoding to base 16 encoding
"""
    return b16encode(b64decode(x,*a)).decode('ascii').lower()


def b64to32(x,*a):
    """

function        b64to32
purpose         convert base 64 encoding to base 32 encoding
"""
    return b32encode(b64decode(x,*a)).decode('ascii')


def mc_env(name):
    """

function        mc_env
purpose            mixed case environment lookup (UPPER CASE PREVAILS)
"""
    if name.upper() in env:
        return env[ name.upper() ]
    if name.lower() in env:
        return env[ name.lower() ]
    if name in env:
        return env[ name ]
    return None


def valid_decimal(decarg):
    """

function        valid_decimal
purpose         non-exception test if given value can be decimal.Decimal (dec)
"""
    try:
        r = dec(decarg)
    except InvalidOperation:
        r = None
    return r


def unmod(n,m):
    """

function        unmod
purpose         Return the remainder after subtracting the modulus
    note          If a list or tuple of numbers is given,return in kind
    arguments     1 (num) number
                  2 (num) modulus
    returns       (num) number - (number % modulus)
    example       unmod(456789,1000) -> 456000
                  unmod(256,15) -> 240
    usage         unmod(seconds,3600) -> start of this hour in seconds
                  unmod(seconds,86400) -> start of this day in seconds
                  unmod(minutes,1440) -> start of this day in minutes
"""
    if not isinstance(m,Number):
        raise TypeError('Not a number passed to unmod() arg2: %s (modulus)' % repr(m))
    if isinstance(n,(tuple,list)):
        ns = [unmod(x,m) for x in n]
        if isinstance(n,tuple):
            return tuple(ns)
        return ns
    if not isinstance(n,Number):
        raise TypeError('Not a number passed to unmod() arg1: %s (value)' % repr(n))
    if m == 0:
        raise ValueError('Zero passed to unmod() arg2 (modulus)')
    if m < 0:
        raise ValueError('Negative value passed to unmod() arg2: %s (modulus)' % repr(m))
    return n - (n % m)


def sleep_to_interval(i):
    """

function        sleep_to_interval
purpose         sleep to the next interval boundary
"""
    stdout.flush()
    return sleep(i-timesecs()%i)


def find_unicode(u,notfound=-1):
    """

function        find_unicode
purpose         In a unicode string,find the position of the first character
                  that requires the unicode type and cannot be represented in
                  mere string type.
    returns       (int) position or the value specified by notfound=
"""
    l = len(u)
    for n in xrange(l):
        if ord(u[n]) > 127:
            return n
    return notfound


def scale(new1,new2,old1,old2,value):
    """

function        scale
purpose         change the scale of a given number
    arguments     1 (num) the beginning of the new scale
                  2 (num) the ending of the new scale
                  3 (num) the beginning of the old scale
                  4 (num) the ending of the old scale
                  5 (num or tuple/list/dict of nums) value(s) to be rescaled
    returns       (like arg 5) rescaled value(s) or None
"""
    delta = old2 - old1
    if isinstance(value,tuple):
        t = ()
        for v in value:
            t = t + (((new2 - new1) * (v - old1) + (new1 * delta)) / delta,)
        return t
    if isinstance(value,(list,tuple)):
        l = []
        for v in value:
            l += [ ((new2 - new1) * (v - old1) + (new1 * delta)) / delta ]
        return l
    if isinstance(value,dict):
        d = {}
        for k in value.keys():
            d[k] = ((new2 - new1) * (value[k] - old1) + (new1 * delta)) / delta
        return d
    if isinstance(value,Number):
        return ((new2 - new1) * (value - old1) + (new1 * delta)) / delta
    return None


def plural(value,one='',not_one='s'):
    """

function        plural
purpose         Syntactic sugar to yield 's' for plural cases else ''
                  and allow substituting other strings.
    usage         print('I see %d thing%s here'%(n,plural(n)))
                  print('il y a %d chose%s ici'%(n,plural(n)))
                  print('Det er %d element%s her'%(n,plural(n,'','er')))
                  print('There %s %d thing%s here'%(plural(n,'is','are'),n,plural(n)))
"""
    return one if value==1 else not_one


def yes_or_no(x,no=False,yes=True,unknown=None):
    """

function        yes_or_no
    aliases       oui_ou_non,ja_oder_nein,ja_of_nee,ja_eller_nej,ja_eller_nei
purpose         Determine if a given string means yes/true or no/false
    arguments     1: the string or value to check
                  no= alternate value to return for no
                  yes= alternate value to return for yes
                  unknown= alternate value to return for unknown
    returns       False or no= if string/value indicates no/false
                  True or yes= if string/value indicates yes/true
                  None or unknwon= if string/value cannot be determined
    note          Only a few European languages are represented here.
"""
    if x == None:
        return unknown
    if isinstance(x,Number): # isnum also matches bool
        if x:
            return yes
        else:
            return no
    elif isinstance(x,str):
        if x.lower() in __nf__:
            return no
        elif x.lower() in __yt__:
            return yes
    return unknown

oui_ou_non   = yes_or_no
ja_oder_nein = yes_or_no
ja_of_nee    = yes_or_no
ja_eller_nej = yes_or_no
ja_eller_nei = yes_or_no


def test_pyutils(args):
    """

function        test_pyutils
purpose         perform tests of functions in this module with results output to stdout
"""

    alpha = '0123456789abcdefghijklmnopqrstuvwxyz' + 'abcdefghijklmnopqrstuvwxyz'.upper()

    tprint('Python',version)

    tprint('\nTesting scale')
    for c in (-40,0,10,15.20,25,100):
        f = scale(32,212,0,100,c)
        c = scale(0,100,32,212,f)
        tprint(str(f)+'F','==',str(c)+'C')

    tprint('\nTesting plural')
    for n in (1,2,3):
        xprint()
        tprint('Il y a %d chose%s ici'%(n,plural(n)))
        tprint('Det er %d element%s her'%(n,plural(n,'','er')))
        tprint('There %s %d thing%s here'%(plural(n,'is','are'),n,plural(n)))

    tprint('\nTesting oui_ou_non')
    for f in ('oui','non','yes','no','True','False','None','unknown',True,False,None,-1,0,1,2):
        if oui_ou_non(f):
            e = True
        else:
            e = False
        tprint('for',repr(f),'the result is',repr(e))

    xprint()

    for x in range(0,12):
        print(x,repr(list(chunks(alpha,x))))
    for x in (18,36,99,0,-1,-2,-3):
        print(x,repr(list(chunks(alpha,x))))

    xprint()

    sleep_to_interval(2.0)
    tprint('time is',repr(timeyears()),'years')
    sleep_to_interval(2.0)
    tprint('time is',repr(timemonths()),'months')
    sleep_to_interval(2.0)
    tprint('time is',repr(timeweeks()),'weeks')
    sleep_to_interval(2.0)
    tprint('time is',repr(timedays()),'days')
    sleep_to_interval(2.0)
    tprint('time is',repr(timehours()),'hours')
    sleep_to_interval(2.0)
    tprint('time is',repr(timemins()),'minutes')
    sleep_to_interval(2.0)
    tprint('time is',repr(timesecs()),'seconds')
    sleep_to_interval(2.0)
    tprint('time is',repr(timemsecs()),'milliseconds')
    sleep_to_interval(2.0)
    tprint('time is',repr(timeusecs()),'microseconds')
    sleep_to_interval(2.0)
    tprint('time is',repr(timensecs()),'nanoseconds')

    xprint()

    eprint('Python',version)
    xprint('DONE',time=True)

    return 0


def main(args):
    """main"""
    return test_pyutils(args)


if __name__ == '__main__':
    result = main(argv)
    stdout.flush()
    try:
        exit(int(result))
    except ValueError:
        print(str(result),file=stderr)
        exit(1)
    except TypeError:
        if result == None:
            exit(0)
        exit(255)
                                                        
# EOF
this is something i use in a lot of scripts i write or convert.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
Github?
"As they say in Mexico 'dosvidaniya'. That makes two vidaniyas."
https://freedns.afraid.org
Reply
#3
(Mar-07-2017, 10:28 AM)wavic Wrote: Github?

sorry, i won't be signing up there.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#4
update...

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from __future__ import absolute_import,division,print_function,unicode_literals
"""
file          pyutils.py
purpose       collection of my useful python stuff
email         10054452614123394844460370234029112340408691

The intent is that this module works correctly under both Python 2 and
Python 3.  Please report failures or code improvement to the author.
"""

__license__ = """
Copyright (C) 2016, 2017 by Phil D. Howard - all other rights reserved

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA, OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE, OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

The author may be contacted by decoding the number
10054452614123394844460370234029112340408691
(provu igi la numeron al duuma)
"""

import  base64
import  codecs
import  datetime
import  decimal
import  fcntl
import  hashlib
import  json
import  numbers
import  os
import  pickle
import  random
import  signal
import  struct
import  subprocess
import  sys
import  termios
import  time

# assign some shortcuts
now = datetime.datetime.now

b16decode = base64.b16decode
b16encode = base64.b16encode
b32decode = base64.b32decode
b32encode = base64.b32encode
b64decode = base64.b64decode
b64encode = base64.b64encode

getencoder = codecs.getencoder

dec              = decimal.Decimal
InvalidOperation = decimal.InvalidOperation

sha224 = hashlib.sha224
sha256 = hashlib.sha256
sha384 = hashlib.sha384
sha512 = hashlib.sha512

Number = numbers.Number

chdir  = os.chdir
env    = os.environ
getcwd = os.getcwd
getuid = os.getuid
link   = os.link
mkdir  = os.mkdir
rename = os.rename
remove = os.remove
rmdir  = os.rmdir

randint = random.randint

SIGKILL = signal.SIGKILL
SIGTERM = signal.SIGTERM

call  = subprocess.call
ocall = subprocess.check_output
PIPE  = subprocess.PIPE
Popen = subprocess.Popen

argv         = sys.argv
stderr       = sys.stderr
stdin        = sys.stdin
stdout       = sys.stdout
version      = sys.version
version_info = sys.version_info

sleep    = time.sleep
timesecs = time.time

stformat = '%Y-%m-%d-%H:%M:%S.%f'
tfnformat = stformat.replace(':','-').replace('.','-')

__y__=('ano','bai','da','ie','ja','jah','jes','kylla','oui','si','sim','taip','tak','yes')
__n__=('dim','ei','ez','ingen','inno','nao','ne','nee','nei','nein','nej','neniu','nie','no','non','nu')
__t__=('cierto','egia','ekte','ikke','istina','pravi','rigtigt','sann','satt','totta','true','vera','veraj','verdade','veritable','vero','vrai','waar','wahr','wier')
__f__=('falsa','falsch','false','falsk','falso','faux','lazan','rangar','sbagliate','vaara','vals')
__yt__=set(__y__+__t__+('j','y','1'))
__nf__=set(__n__+__f__+('n','0'))

xrangemax = 0x7fffffffffffffff # 9223372036854775807


if version_info[0] < 3:
    testint = (int,long,)
    teststr = (str,unicode,)


if version_info[0] > 2:
    testint = (int,)
    teststr = (str,)
    xrange = range
    long   = int

    
def get_terminal_geometry(fd=0):
    """Get terminal geometry (width,heght)

function        get_terminal_geometry
purpose         get terminal geometry (width,heght)
"""
    return struct.unpack('4H',fcntl.ioctl(fd,termios.TIOCGWINSZ,struct.pack('4H',0,0,0,0)))[1::-1]


def str_to_bytes(s):
    """Convert from str to bytes without any encoding changes

function        str_to_bytes
purpose         convert from str to bytes without any encoding changes
warning         this conversion can have unexpected effects
"""
    if version_info[0] > 2:
        return s
    return b''.fromhex(''.join([hex(ord(c))[2:] for c in s]))


def bytes_to_str(b):
    """Convert from bytes to str without any decoding changes

function        bytes_to_str
purpose         convert from bytes to str without any decoding changes
warning         this conversion can have unexpected effects
"""
    return ''.join([chr(x) for x in b])


def hexify(o):
    """Convert strings,byte,ints,longs tohex

function        hexify
purpose         convert strings,byte,ints,longs to hex
"""
    if isinstance(o,str):
        return ''.join([hex(ord(c)) for c in o])
    if isinstance(o,bytes):
        return ''.join([hex(b) for b in o])
    if isinstance(o,(int,long)):
        return hex(o)
    return hexify(repr(o))


_rot13 = getencoder('rot-13')
def rot13(x):
    """Do rot13 conversion in a simple API way

function        rot13
purpose         do rot13 conversion in a simple API way
"""
    return _rot13(x)[0]


def chunks(s,l):
    """Split a sequence into fixed size chunks

function        chunks
purpose         split a sequence into fixed size chunks
    note          the last chunk is often shorter
    arguments     1 sequence to be split
                  2 length of each chunk (except last)
    thanks        j.crater,wavic,Mekire (python-forum.io)
"""
    return [s] if l<1 else (s[0+i:l+i] for i in range(0,len(s),l))


def timeyears():
    """Return the current time scaled in average years (365.2425 days) of the proleptic Gregorian calendar

function        timeyears
purpose         return the current time scaled in average years
                (365.2425 days) of the proleptic Gregorian
                calendar
"""
    return timesecs()/31556952.0


def timemonths():
    """Return the current time scaled in average months (30.436875 days) of the proleptic Gregorian calendar

function        timemonths
purpose         return the current time scaled in average months
                (30.436875 days) of the proleptic Gregorian
                calendar
"""
    return timesecs()/2629746.0


def timeweeks():
    """Return the current time scaled in weeks

function        timeweeks
purpose         return the current time scaled in weeks
"""
    return timesecs()/604800.0

    
def timedays():
    """Return the current time scaled in days

function        timedays
purpose         return the current time scaled in days
"""
    return timesecs()/86400.0


def timehours():
    """Return the current time scaled in hours

function        timehours
purpose         return the current time scaled in hours
"""
    return timesecs()/3600.0


def timemins():
    """Return the current time scaled in minutes

function        timemins
purpose         return the current time scaled in minutes
"""
    return timesecs()/60.0


def timemsecs():
    """Return the current time scaled in milliseconds

function        timemsecs
purpose         return the current time scaled in milliseconds
"""
    return timesecs()*1000.0


def timeusecs():
    """Return the current time scaled in microseconds

function        timeusecs
purpose         return the current time scaled in microseconds
"""
    return timesecs()*1000000.0


def timensecs():
    """Return the current time scaled in nanoseconds

function        timensecs
purpose         return the current time scaled in nanoseconds
"""
    return timesecs()*1000000000.0


def fint(x):
    """Convert x to an int only if x is a float with an int value

function        fint
purpose         convert x to an int only if x is a float with an int
                value
"""
    if isinstance(x,float):
        if x.is_integer():
            return int(x)
    return x


def tfn():
    """Return a timestamp string for a filename

function        tfn
purpose         return a timestamp string for a filename
"""
    return now().strftime(tfnformat)


def pv(*names,**opts):
    """Print a variable (name and value)
function        pv
purpose         print a variable (name and value)
"""
    p = '==>'
    for x in names:
        if not isinstance(x,(list,tuple)):
            x = [ x ]
        for name in x:
            name = str(name)
            d=inspect.currentframe().f_back.f_locals
            if name not in d:
                xprint(p,name,'not assigned',**opts)
            else:
                xprint(p,name,'=',repr(d[name]),**opts)
    return


def xprint(*msg,**opt):
    """Print, avoiding exceptions such as broken pipes.

function        xprint
purpose         print, avoiding exceptions such as broken pipes.
                also extends file= option to support str and int values
                file=str is a file name to be opened, printed to, then closed
                file=int is a file descriptor to be used
"""
    doclose = False
    if 'file' in opt:
        ofile = opt['file']
        try:
            if isinstance(ofile,str):
                ofile = open(ofile,'w')
                doclose = True
            if isinstance(ofile,int) and ofile > 0:
                ofile = os.fdopen(ofile,'w')
        except IOError:
            return
        opt['file'] = ofile
    try:
        otime = opt.pop('time',False)
        if otime:
            r = print(now().strftime(stformat),*msg,**opt)
        else:
            r = print(*msg,**opt)
    except IOError:
        r = None
    if doclose:
        ofile.close()
    return r
    
def tprint(*msg,**opt):
    """Like xprint but defaults to time=True

function        tprint
purpose         like xprint but defaults to time=True
"""
    opt['time'] = opt.pop('time',True)
    return xprint(*msg,**opt)


def eprint(*msg,**opt):
    """Like xprint() but defaults to file=stderr.

function        eprint
purpose         like xprint() but defaults to file=stderr.
"""
    opt['file'] = opt.pop('file',sys.stderr)
    return xprint(*msg,**opt)


def teprint(*msg,**opt):
    """Like tprint() but defaults file= to stderr.

function        teprint
purpose         like eprint() but defaults to time=True
"""
    opt['time'] = opt.pop('time',True)
    return eprint(*msg,**opt)


def vcall(*a,**aa):
    """Execute the given command after outputting the command.

function        vcall
purpose         execute the given command after outputting the command
                to stdout or where specified with cmdfile= or
                cmdfilename=
"""
    doclose = None
    if len(a) < 1:
        return -1
    if 'cmdfilename' in aa and isinstance(aa['cmdfilename'],str):
        try:
            cmdfile = open(aa['cmdfilename'],'a')
            doclose = cmdfile
            aa['cmdfile'] = cmdfile
        except IOError:
            pass
        aa.discard('cmdfilename')
    if 'cmdfile' in aa:
        xprint(repr(a[0]),file=aa['cmdfile'])
        if doclose != None:
            doclose.close()
    else:
        xprint(repr(a[0]))
        stdout.flush()
    return call(*a,**aa)


def vecall(*a,**aa):
    """Like vcall but defaults output to stderr.

function        vecall
purpose         like vcall but defaults output to stderr.
"""
    doclose = None
    if len(a) < 1:
        return -1
    if 'cmdfilename' in aa and isinstance(aa['cmdfilename'],str):
        try:
            cmdfile = open(aa['cmdfilename'],'a')
            doclose = cmdfile
            aa['cmdfile'] = cmdfile
        except IOError:
            pass
        aa.discard('cmdfilename')
    if 'cmdfile' in aa:
        eprint(repr(a[0]),file=aa['cmdfile'])
        if doclose != None:
            doclose.close()
    else:
        eprint(repr(a[0]))
        stderr.flush()
    return call(*a,**aa)
    

def ecall(args,**opt):
    """Like vcall() but defaults to file=sys.stderr

function        ecall
purpose         like vcall() but defaults to file=sys.stderr
"""
    if 'file' not in opt and 'filename' not in opt:
        opt['file'] = stderr
    return vcall(args,**opt)


def bcall(s,a):
    """Execute the given command in a named screen background session.

function        bcall
purpose         execute the given command in a named screen background session
note            due to a misfeature of screen the process started this
                way cannot be SIGSTOPped (SCREEN does SIGCONT)
"""
    if len(a) < 1:
        return -1
    if 'DISPLAY' in os.environ:
        d = os.environ['DISPLAY']
    else:
        d = ''
    return 0 if 0 == call(['/usr/bin/env','DISPLAY='+d,'SESSION_NAME='+s,'/usr/bin/screen','-S',s,'-dm']+a) else -2


def acall(*args,**opts):
    """Like call but all the positional arguments form the command.

function        acall
purpose         like call but all the positional arguments form the command
"""
    if len(args) < 1:
        return -1
    return call(args,**opts)


def xcall(args,**opt):
    """Like call but executes in the current process.

function        xcall
purpose         like call but executes in the current process replacing
                the python interpreter running this code and its caller.
"""
    if len(args) < 1:
        return -1
    return os.execvp(args[0],args)


def xacall(*args,**opts):
    """Like acall but executes in the current process.

function        xacall
purpose         like acall but executes in the current process
                replacing the python interpreter running this
                code and its caller.
"""
    if len(args) < 1:
        return -1
    return os.execvp(args[0],args)


def q_chdir(*a):
    """Do chdir() without raising an exception

function        q_chdir
purpose         do chdir() without raising an exception
usage           this is convenient for use cases where an error can be ignored
"""
    result = None
    try:
        result = os.chdir(*a)
    except OSError:
        pass
    return result


cd = q_chdir


def q_mkdir(*a):
    """Do mkdir() without raising an exception

function        q_mkdir
purpose         do mkdir() without raising an exception
usage           this is convenient for use cases where an error can be ignored
"""
    result = None
    try:
        result = os.mkdir(*a)
    except OSError:
        pass
    return result


def q_link(*a):
    """Do link() without raising an exception

function        q_link
purpose         do link() without raising an exception
usage           this is convenient for use cases where an error can be ignored
"""
    result = None
    try:
        result = os.link(*a)
    except OSError:
        pass
    return result


def q_symlink(*a):
    """Do symlink() without raising an exception

function        q_symlink
purpose         do symlink() without raising an exception
usage           this is convenient for use cases where an error can be ignored
"""
    result = None
    try:
        result = os.symlink(*a)
    except OSError:
        pass
    return result


def q_rename(*a):
    """Do rename() without raising an exception

function        q_rename
purpose         do rename() without raising an exception
usage           this is convenient for use cases where an error can be ignored
"""
    result = None
    try:
        result = os.rename(*a)
    except OSError:
        pass
    return result


def q_remove(*a):
    """Do remove() without raising an exception

function        q_remove
purpose         do remove() without raising an exception
usage           this is convenient for use cases where an error can be ignored
"""
    result = None
    try:
        result = os.remove(*a)
    except OSError:
        pass
    return result
rm = q_remove


def q_rmdir(*a):
    """Do rmdir() without raising an exception

function        q_rmdir
purpose         do rmdir() without raising an exception
usage           this is convenient for use cases where an error can be ignored
"""
    result = None
    try:
        result = os.rmdir(*a)
    except OSError:
        pass
    return result


def b16to32(x):
    """Convert base 16 encoding to base 32 encoding

function        b16to32
purpose         convert base 16 encoding to base 32 encoding
"""
    return b32encode(b16decode(x.upper())).decode('ascii')


def b16to64(x,*a):
    """convert base 16 encoding to base 64 encoding

function        b16to64
purpose         convert base 16 encoding to base 64 encoding
"""
    return b64encode(b16decode(x.upper()),*a).decode('ascii')


def b32to16(x):
    """convert base 32 encoding to base 16 encoding

function        b32to16
purpose         convert base 32 encoding to base 16 encoding
"""
    return b16encode(b32decode(x)).decode('ascii').lower()


def b32to64(x,*a):
    """convert base 32 encoding to base 64 encoding

function        b32to64
purpose         convert base 32 encoding to base 64 encoding
"""
    return b64encode(b32decode(x),*a).decode('ascii')


def b64to16(x,*a):
    """convert base 64 encoding to base 16 encoding

function        b64to16
purpose         convert base 64 encoding to base 16 encoding
"""
    return b16encode(b64decode(x,*a)).decode('ascii').lower()


def b64to32(x,*a):
    """convert base 64 encoding to base 32 encoding

function        b64to32
purpose         convert base 64 encoding to base 32 encoding
"""
    return b32encode(b64decode(x,*a)).decode('ascii')


def lcset(s):
    """make a copy set with strings converted to lower case

function        lcset
purpose         make a copy set with strings converted to lower case
"""
    new = set()
    for x in s:
        if isinstance(x,str):
            new += x.lower()
        else:
            new += x
    return new


def ucset(s):
    """make a copy set with strings converted to upper case

function        lcset
purpose         make a copy set with strings converted to upper case
"""
    new = set()
    for x in s:
        if isinstance(x,str):
            new += x.upper()
        else:
            new += x
    return new


def mcenv(name):
    """Mixed case environment lookup (UPPER CASE PREVAILS)

function        mcenv
purpose         mixed case environment lookup (UPPER CASE PREVAILS)
"""
    if name.upper() in env:
        return env[ name.upper() ]
    if name.lower() in env:
        return env[ name.lower() ]
    if name in env:
        return env[ name ]
    return None


def valid_decimal(decarg):
    """non-exception test if given value can be decimal.Decimal (dec)

function        valid_decimal
purpose         non-exception test if given value can be decimal.Decimal (dec)
"""
    try:
        r = dec(decarg)
    except InvalidOperation:
        r = None
    return r


def unmod(n,m):
    """Return the remainder after subtracting the modulus

function        unmod
purpose         return the remainder after subtracting the modulus
note            if a list or tuple of numbers is given, return in kind
arguments       1 (num) number
                2 (num) modulus
returns         (num) number - (number % modulus)
example         unmod(456789,1000) -> 456000
usage           unmod(256,15) -> 240
                unmod(seconds,3600) -> start of this hour in seconds
                unmod(seconds,86400) -> start of this day in seconds
                unmod(minutes,1440) -> start of this day in minutes
"""
    if not isinstance(m,Number):
        raise TypeError('Not a number passed to unmod() arg2: %s (modulus)' % repr(m))
    if isinstance(n,list):
        return [unmod(x,m) for x in n]
    if isinstance(n,set):
        return set([unmod(x,m) for x in n])
    if isinstance(n,tuple):
        return tuple([unmod(x,m) for x in n])
    if not isinstance(n,Number):
        raise TypeError('Not a number passed to unmod() arg1: %s (value)' % repr(n))
    if m == 0:
        raise ValueError('Zero passed to unmod() arg2 (modulus)')
    if m < 0:
        raise ValueError('Negative value passed to unmod() arg2: %s (modulus)' % repr(m))
    return n - (n % m)


def sleep_to_interval(i):
    """sleep to the next interval boundary

function        sleep_to_interval
purpose         sleep to the next interval boundary
"""
    stdout.flush()
    return sleep(i-timesecs()%i)


def scale(n,o,value):
    """Change the scale of given number(s)

function        scale
purpose         change the scale of given number(s)
arguments       1 (num) the new scale (from,to)
                2 (num) the old scale (from,to)
                3 (num or list/set/tuple of nums) value(s) to be rescaled
returns         (like arg 5) rescaled value(s) or None
"""
    if isinstance(value,list):
        return [scale(n,o,x) for x in value]
    elif isinstance(value,set):
        return set([scale(n,o,x) for x in value])
    elif isinstance(value,tuple):
        return tuple([scale(n,o,x) for x in value])
    d = o[1] - o[0]
    return ((n[1] - n[0]) * (value - o[0]) + (n[0] * d)) / d


def plural(value,one='',not_one='s'):
    """

function        plural
purpose         Syntactic sugar to yield 's' for plural cases else ''
                and allow substituting other strings.
examples        print('I see %d thing%s here'%(n,plural(n)))
                print('il y a %d chose%s ici'%(n,plural(n)))
                print('Det er %d element%s her'%(n,plural(n,'','er')))
                print('There %s %d thing%s here'%(plural(n,'is','are'),n,plural(n)))
"""
    return one if value==1 else not_one


def yes_or_no(x,no=False,yes=True,unknown=None):
    """Determine if a given string means yes/true or no/false

function        yes_or_no
aliases         oui_ou_non,ja_oder_nein,ja_of_nee,ja_eller_nej,ja_eller_nei
purpose         determine if a given string means yes/true or no/false
arguments       1: the string or value to check
                no= alternate value to return for no/false
                yes= alternate value to return for yes/true
                unknown= alternate value to return for unknown
returns         False or the value of no= if string/value indicates
                no/false
                True or the value of yes= if string/value indicates
                yes/true
                None or the value of unknwon= if string/value cannot
                be determined
note            Only a few European languages are represented here.
"""
    if isinstance(x,teststr):
        if x.lower() in __nf__:
            return no
        elif x.lower() in __yt__:
            return yes
    elif isinstance(x,Number):
        if x:
            return yes
        else:
            return no
    return unknown
oui_ou_non   = yes_or_no
ja_oder_nein = yes_or_no
ja_of_nee    = yes_or_no
ja_eller_nej = yes_or_no
ja_eller_nei = yes_or_no


def test_pyutils(args):
    """

function        test_pyutils
purpose         perform tests of functions in this module with results output to stdout
"""

    alpha = '0123456789abcdefghijklmnopqrstuvwxyz' + 'abcdefghijklmnopqrstuvwxyz'.upper()

    tprint('Python',version)

    tprint('\nTesting scale')
    for c in (-40,0,10,15.20,25,100):
        f = scale((32,212),(0,100),c)
        c = scale((0,100),(32,212),f)
        tprint(str(f)+'F','==',str(c)+'C')

    tprint('\nTesting plural')
    for n in (1,2,3):
        xprint()
        tprint('Il y a %d chose%s ici'%(n,plural(n)))
        tprint('Det er %d element%s her'%(n,plural(n,'','er')))
        tprint('There %s %d thing%s here'%(plural(n,'is','are'),n,plural(n)))

    tprint('\nTesting oui_ou_non')
    for f in ('oui','non','yes','no','True','False','None','unknown',True,False,None,-1,0,1,2):
        if oui_ou_non(f):
            e = True
        else:
            e = False
        tprint('for',repr(f),'the result is',repr(e))

    xprint()

    for x in range(0,12):
        print(x,repr(list(chunks(alpha,x))))
    for x in (18,36,99,0,-1,-2,-3):
        print(x,repr(list(chunks(alpha,x))))

    xprint()

    sleep_to_interval(2.0)
    tprint('time is',repr(timeyears()),'years')
    sleep_to_interval(2.0)
    tprint('time is',repr(timemonths()),'months')
    sleep_to_interval(2.0)
    tprint('time is',repr(timeweeks()),'weeks')
    sleep_to_interval(2.0)
    tprint('time is',repr(timedays()),'days')
    sleep_to_interval(2.0)
    tprint('time is',repr(timehours()),'hours')
    sleep_to_interval(2.0)
    tprint('time is',repr(timemins()),'minutes')
    sleep_to_interval(2.0)
    tprint('time is',repr(timesecs()),'seconds')
    sleep_to_interval(2.0)
    tprint('time is',repr(timemsecs()),'milliseconds')
    sleep_to_interval(2.0)
    tprint('time is',repr(timeusecs()),'microseconds')
    sleep_to_interval(2.0)
    tprint('time is',repr(timensecs()),'nanoseconds')

    xprint()

    eprint('Python',version)
    xprint('DONE',time=True)

    return 0


def main(args):
    """main"""
    return test_pyutils(args)


if __name__ == '__main__':
    result = main(argv)
    stdout.flush()
    try:
        exit(int(result))
    except ValueError:
        print(str(result),file=stderr)
        exit(1)
    except TypeError:
        if result == None:
            exit(0)
        exit(255)
                                                        
# EOF
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#5
Looks like a very interesting script. So being a nubie let me ask a dumb question? Is there any samples of script usage with the functions that it includes?
Reply
#6
only in my private scripts, for now. also, i have ended this project and started a replacement named sfc (skaperen's function collection). it's probably too big to post here (over 2000 lines). i probably should make a set of scripts to test all the functions as well as a set of simple scripts to illustrate expected usage.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#7
Just curious... why assign all the shortcuts?
ie:
b64encode = base64.b64encode
Why not just directly import?
from base64 import b64encode
although I didn't read over the whole thing... maybe I'm missing something
Reply
#8
Seems like quite a hodgepodge of different things. Why would one want to depend on all of it, or none of it, rather than just having the bits they needed?
Reply
#9
(May-07-2020, 06:45 AM)rootVIII Wrote: Just curious... why assign all the shortcuts?
ie:
b64encode = base64.b64encode
Why not just directly import?
from base64 import b64encode
although I didn't read over the whole thing... maybe I'm missing something

i usually do that. but, i also wanted to make it clear that the module itself had been imported. and the file is a bit smaller this way. things ca look ugly with lots of from ... import ... lines unless i pad and align them.

(May-07-2020, 12:34 PM)ndc85430 Wrote: Seems like quite a hodgepodge of different things. Why would one want to depend on all of it, or none of it, rather than just having the bits they needed?

the alternative is having, and keeping track of, lots of modules. and for me it's worse maintaining this.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  pyutils module Skaperen 15 14,556 Oct-08-2016, 04:48 PM
Last Post: Mekire

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020