Python Forum

Full Version: Annotations
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
I think annotations are a little special, so I want to try to organize this a little differently than how a tutorial is normally structured.  That is to say:
1) What they are.  A description of them, with basic code examples showing what they look like.
2) Why they are.  Possible uses for them, the reason they were added to Python, the problem they try to solve, and what people did before they were available.


So what's this annotation thing I keep talking about?  It's a note, which is built into a function definition.
It looks a little like this:
def some_function(param: str, param2: str = "") -> str:
   return 42
So let's break this down...
- param: str - param should be a str.  It isn't forced to be a str, and python will make no attempt at all to warn you if you do differently.  str is just a note to yourself or others, that param should be a str, and you possibly haven't tested it with other types.

- param2: str = "" - same as before, except this is how you'd annotate a parameter while also providing a default value.  Again, python makes no attempt to check that the default value is the same as annotation (I'll explain later why that might be a feature).

- -> str: - the function supposedly returns a str.  How you prefer to code could change how you use this... for example, I find that having a tuple as the return type, with the first element being possible Error types and the second element being the actual return type, being an easy way to know which errors are possible from different parts of the code.  ie: for this example: -> (None, str):, though I'd just do -> str: and elide the None unless there was a reason to specify it.


But why is any of this useful?  Isn't the point of python to be dynamically typed, and to not care what type things actually are as long as they act as we expect?

I'd give that a resounding maybe.  Types DO matter, sometimes at least.  Look at this, for example:
>>> "fifty" / 2
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for /: 'str' and 'int'
Just because types don't ALWAYS matter, doesn't mean that they NEVER matter.  Sometimes you care that something is iterable, but you don't care if it's a list/tuple/dict/generator.  An annotation could provide a lowest common denominator of what could work with your function.

But, just because most annotations will probably be an actual type, doesn't mean they HAVE to be.  An annotation can be any legal python identifier.  For example, you could do this:
def wait_for_server(ip_addr: "XXX.XXX.XXX.XXX"):
This information is normally provided via comments or doc-strings, but binding formatting hints or type notes directly to an argument could help clean up your doc string so it only contains actual documentation.  I'd like to make a note, that I think (without external tooling) that this is the most useful part of annotations.

...and by external tooling, I mean an ide that parses your program, so code-completion can be type-aware, or provide warnings without your code even needing to run.  Other languages call this "static analysis", and if we can get it sometimes working for python, you could find yourself more willing to work on old code that other things depend on.


LET'S BUILD SOMETHING!
Ok, so this is going to be a pretty dumb example, but deal with it.  This was originally very simple, but I kept wanting to add more and more until now it's almost a better example of the inspect module than it is of annotations :/  Anyway, this simple script will read whatever python source file you pass it, and parse any functions contained within, and then emit those functions as bbcode that our forum will understand.

Here's the sample input file I used to test various annotation features:
Once run, here's the output:

Documentation for function 'temp.test' Wrote:Raise x to the yth power.
Defaults to squaring x.
Arguments:
  • x int
  • ?y int = 2
Return Type: int

Documentation for function 'temp.test2' Wrote:No documentation provided.
Arguments:
None
Return Type: Unknown

Documentation for function 'temp.test3' Wrote:I don't do anything meaningful.
But I return a mapping of indices to string values
Arguments:
  • first
  • second float
  • ?third = 'spam'
  • ?fourth (str, 4) = 'eggs'
Return Type: (int, str)



And finally, here's the source in all it's glory.  Hopefully you can divine some meaning from this mess, and hopefully annotations aren't as mysterious as before.  (there's a closing python tag contained within.  I added a space in it so it can actually be posted to the forum.  If you want to run it, take that space out)

import inspect
import re


def type_name(obj):
   try:
       # test if iterable
       vals = ", ".join(map(type_name, obj))
       return "({0})".format(vals)
   except TypeError:
       # not iterable
       if hasattr(obj, "__name__"):
           return obj.__name__
       return repr(obj)


def parse_function(func):
   # output is bbcode compatable

   # start by getting the function's docstring
   doc = inspect.getdoc(func)
   if doc:
       # items wrapped in back-ticks should be assumed to be variables
       # so replace the back-ticks with [icode] tags
       doc = re.sub(r"`([^`]+)`", r"[icode]\1[/icode]", doc)
   else:
       doc = "No documentation provided."

   # get the name of the function, and spit out a quote block with it and the
   # doc string
   module = inspect.getmodule(func)
   yield '
[quote='Documentation for function \'{0}.{1}'']
{2}
[/quote]
'.format(module.__name__, func.__name__, doc)

   # build an argument list, including annotations and default values
   # we could also just use func.__annotations__ directly
   sig = inspect.signature(func)
   params = sig.parameters
   yield "[size=large]Arguments:[/size]"
   if params:
       yield "
[list]
"        for name in params:            param = params[name]            param_type = ""            if param.annotation is not param.empty:                param_type = " " + type_name(param.annotation)            default = " = {0}".format(                repr(param.default)) if param.default is not param.empty else ""            # if there's a default value, add a ? to the argument to indicate            # it's optional            name = "?{0}".format(name) if default else name            name = "[icode]{0}[/icode]".format(name)            yield "[*] {0}{1}{2}".format(name, param_type, default)
        yield "
[/list]"
   else:
       yield "None"

   returns = "Unknown"
   if sig.return_annotation is not sig.empty:
       returns = type_name(sig.return_annotation)

   yield "[size=large]Return Type: {0}[/size]".format(returns)

   # now that the details have been pulled out, spit out the whole function
   # source in a spoiler tag
   obj = inspect.getsource(func)
   # if the code contains either a closing python or spoiler bb tag,
   # add a space so it doesn't distort our output
   obj = re.sub(r"\[/(python|spoiler)\]", r"[/\1 ]", obj)
   yield "[spoiler=Source][python]{0}[/ python][/spoiler]".format(obj)
   #
[hr]
between functions
   yield "
[hr]
"


if __name__ == "__main__":
   import importlib
   import sys

   files = sys.argv[1:]
   # scan self if nothing passed
   if not files:
       files = sys.argv[:1]

   for src in files:
       if ".py" == src[-3:]:
           src = src[:-3]

       module = importlib.import_module(src)
       for name, obj in inspect.getmembers(module):
           if inspect.isfunction(obj):
               for chunk in parse_function(obj):
                   print(chunk)