Creating C extensions

[metulburr]

Originally posted by Casevh...
If You want to repost this, ill delete it
@[casevh]

This is part 1 of several short tutorials on creating C extensions for Python. In this post, I'll describe what is possibly the shortest, and most useless, C extension possible.

A few caveats:

• All examples will focus on Python 3.x. It is possible to write C extensions that compile on both 2.x and 3.x but supporting both major versions makes the C code more complicated.
  
  
• My primary development platform is Linux but I will also cover Windows. I don't have access to a Mac so I can't assist with OSX.
  
  
• The Windows binary for Python 3.3 was compiled with Visual Studio 2010. You will need to install Visual Studio 2010 Express to compile extensions for Python 3.3 on Windows. For earlier versions of Python, you will need Visual Studio 2008 Express (if you can still find it on Microsoft's web site) or you can install the Windows 7.0 SDK (aka Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1). I will cover the use of the SDK compilers in another post.
  

A simple extension consists of two files: a 'setup.py' file written in Python and C source file written in (you guessed it) C.

Here is a simple setup.py file:

from distutils.core import setup, Extension

case_module = Extension('case', sources = ['case.c'])

setup (name = 'case',
       version = '0.0',
       description = 'Collection of Arbitrary Simple Extensions',
       ext_modules = [case_module])

Here is corresponding C file (case.c):

#include "Python.h"

PyDoc_STRVAR(case_doc,
"Collection of Arbitrary Simple Extensions.");

static struct PyModuleDef case_module = {
    PyModuleDef_HEAD_INIT,
    "case",           /* m_name */
    case_doc,         /* m_doc */
    -1,               /* m_size */
    NULL,             /* m_methods */
    NULL,             /* m_reload */
    NULL,             /* m_traverse */
    NULL,             /* m_clear */
    NULL              /* m_free */
};

PyMODINIT_FUNC
PyInit_case(void)
{
    return PyModule_Create(&case_module);
}

Save these two files to a directory on your computer and enter "python3 setup.py install" and the extension should be compiled and automatically installed. Note: replace python3 by whatever command you need to use on your computer to invoke a Python 3.x interpreter.

So what's in case.c?

The line #include "Python.h" gives the extension access to the C-API of the Python interpreter. If you are running Linux and get an error message stating that Python.h cannot be found, then you will need to install the development libraries for your version of Python.

Then we create a docstring for the module. It is called case_doc.

Next, we create a Python Module Definition (PyModuleDef) structure that describes the module we are creating. The module structure is called case_module. PyModuleDef_HEAD_INIT is used by Python for internal bookkeeping. We then provide the name of the module ("case"), the docstring (case_doc) that provides help for the module, and the remaining fields are given default values.

Lastly, we define a function PyInit_case. The Python interpreter calls this function after loading the extension. The purpose of this function is to initialize a module (based on the values given in case_module) and return it to the Python interpreter.

You can "import case" and read the docstring via "help(case)" and that's about it.  :lol:

Now that we've created the most useless extension module, we'll add what is probably the most useless object in Part 2.

Enjoy,
casevh

[metulburr]

-

[metulburr]

Originally posted by Casevh

Sorry about the long delay between Part 1 and Part 2. In Part 1, we created the simplest possible Python extension. In Part 2, we'll create what may be the most useless Python object (but it still has a purpose).

The "None" object in Python is very basic. Is it possible to create an object that does even less than "None"? Yes, it si. The new object "Undefined" has even fewer capabilities than "None":

• "Undefined" does not have a boolean value. It can't be converted to True/False.
  
  
• "Undefined" can't be compared to any other object.
  
  
• "Undefined" can't be used as a dictionary key.
  

Here is the C code.

#include "Python.h"

/* The Undefined object is an attempt to make the most useless object possible.
 * It is intended to be used as a default value for a keyword argument that
 * could not be *reasonably* passed as a usable value. None is normally used
 * but None can be a useful value.
 *
 * Undefined does not support conversion to a boolean value so "if Undefined:"
 * will raise an exception. Comparisons are not supported, either. Hashing is
 * not supported so Undefined can't be used as a dictionary key. */

/* Create a varible to store the single instance of Undefined. */

static PyObject *case_Undefined = NULL;

/* Create the C structure to store the internals that are used by Python.
 * PyObject_HEAD is Python macro that defines variable to hold:
 *   - the reference count
 *   - pointers to other structures that define supported methods
 *   - etc. */

typedef struct {
    PyObject_HEAD
} UndefinedObject;

/* Create a corresponding type... */

static PyTypeObject Undefined_Type;

/* Define a macro to check the type of an object. */

#define UndefinedObject_Check(v) (Py_TYPE(v) == &Undefined_Type)

/* Create a new instance of an UndefinedObject. This is equivalent to the
 * __new__ special method of Python objects.*/

static UndefinedObject *
UndefinedObject_new(PyObject *arg)
{
    return PyObject_New(UndefinedObject, &Undefined_Type);
}

/* Delete an instance of an UndefinedObject. This is equivalent to the
 * __del__ special method of Python objects.*/

static void
UndefinedObject_dealloc(UndefinedObject *self)
{
    PyObject_Del(self);
}

/* Define a replacement for __bool__ (or __nonzero__ for Python 2.x) to
 * prevent converting an instance of Undefined into a boolean value. bool()
 * actually checks __bool__, then __len__, and if neither are defined, the
 * object is considered True. The replacment function sets an exception and
 * then returns a special value (-1) to indicate an exception has been set. */

static int
UndefinedObject_bool(UndefinedObject *self)
{
    PyErr_SetString(PyExc_TypeError,
                    "UndefinedObject does not have a boolean value");
    return -1;
}

/* Define a replacement for the comparison methods. In C, all siz of the
 * special methods (__lt__, __le__, etc.) are implemented by a single function.
 * Note that this function returns NULL to indicate that an exception has
 * been set. */

static PyObject *
Undefined_richcompare(PyObject *self, PyObject *other, int op)
{
    PyErr_SetString(PyExc_TypeError,
                    "UndefinedObject can not be compared");
    return NULL;
}

static PyObject *
Undefined_str(PyObject *self)
{
    return Py_BuildValue("s", "Undefined");
}

PyDoc_STRVAR(undefined_doc,
"The Undefined object can be used as a default value instead of None.\n"
"To minimize chances of using Undefined as useful Python object, it\n"
"is designed to be as unusable as possible:\n"
"  - Undefined cannot be a dictionary key\n"
"  - Undefined does not have a boolean value\n"
"  - Undefined cannot be compared to anything, even itself");

/* Define a table that has pointers to the C functions that implement the
 * various __XXX__ special methods that support numerical capabilities. The
 * __bool__ method slot is replaced by the function defined above. */

static PyNumberMethods undefined_number_methods =
{
    0,                              /* nb_add */
    0,                              /* nb_subtract */
    0,                              /* nb_multiply */
    0,                              /* nb_remainder */
    0,                              /* nb_divmod */
    0,                              /* nb_power */
    0,                              /* nb_negative */
    0,                              /* nb_positive */
    0,                              /* nb_absolute */
    (inquiry)UndefinedObject_bool,  /* nb_bool */
    0,                              /* nb_invert */
    0,                              /* nb_lshift */
    0,                              /* nb_rshift */
    0,                              /* nb_and */
    0,                              /* nb_xor */
    0,                              /* nb_or */
    0,                              /* nb_int */
    0,                              /* nb_reserved */
    0,                              /* nb_float */
    0,                              /* nb_inplace_add */
    0,                              /* nb_inplace_subtract */
    0,                              /* nb_inplace_multiply */
    0,                              /* nb_inplace_remainder */
    0,                              /* nb_inplace_power */
    0,                              /* nb_inplace_lshift */
    0,                              /* nb_inplace_rshift */
    0,                              /* nb_inplace_and */
    0,                              /* nb_inplace_xor */
    0,                              /* nb_inplace_or */
    0,                              /* nb_floor_divide */
    0,                              /* nb_true_divide */
    0,                              /* nb_inplace_floor_divide */
    0,                              /* nb_inplace_true_divide */
    0,                              /* nb_index */
};

/* Define the details of Undefined_Type. The following slots (and their Python
 * equivalent) are implemented by functions defined above.
 *   tp_dealloc     (__del__)
 *   tp_repr        (__repr__)
 *   tp_str         (__str__)
 *   tp_richcompare (__let__, etc.)
 *
 *   tp_as_number points to the custom table of numerical methods defined
 *   above. This overwrites __bool__. */

static PyTypeObject Undefined_Type = {
    PyVarObject_HEAD_INIT(NULL, 0)
    "UndefinedObject",                       /* tp_name */
    sizeof(UndefinedObject),                 /* tp_basicsize */
    0,                                       /* tp_itemsize */
                        /* methods */
    (destructor)UndefinedObject_dealloc,     /* tp_dealloc */
    0,                                       /* tp_print */
    0,                                       /* tp_getattr */
    0,                                       /* tp_setattr */
    0,                                       /* tp_reserved */
    (reprfunc)Undefined_str,                 /* tp_repr */
    &undefined_number_methods,               /* tp_as_number */
    0,                                       /* tp_as_sequence */
    0,                                       /* tp_as_mapping */
    0,                                       /* tp_hash */
    0,                                       /* tp_call */
    (reprfunc)Undefined_str,                 /* tp_str */
    0,                                       /* tp_getattro */
    0,                                       /* tp_setattro */
    0,                                       /* tp_as_buffer */
    Py_TPFLAGS_DEFAULT,                      /* tp_flags */
    undefined_doc,                           /* tp_doc */
    0,                                       /* tp_traverse */
    0,                                       /* tp_clear */
    (richcmpfunc)&Undefined_richcompare,     /* tp_richcompare */
    0,                                       /* tp_weaklistoffset */
    0,                                       /* tp_iter */
    0,                                       /* tp_iternext */
    0,                                       /* tp_methods */
    0,                                       /* tp_members */
    0,                                       /* tp_getset */
    0,                                       /* tp_base */
    0,                                       /* tp_dict */
    0,                                       /* tp_descr_get */
    0,                                       /* tp_descr_set */
    0,                                       /* tp_dictoffset */
    0,                                       /* tp_init */
    0,                                       /* tp_alloc */
    0,                                       /* tp_new */
    0,                                       /* tp_free */
    0,                                       /* tp_is_gc */
};

/* End of UndefinedObject. */

PyDoc_STRVAR(case_doc,
"A collection of arbitrary simple extensions.");

static struct PyModuleDef case_module = {
    PyModuleDef_HEAD_INIT,
    "case",           /* m_name */
    case_doc,         /* m_doc */
    -1,               /* m_size */
    NULL,             /* m_methods */
    NULL,             /* m_reload */
    NULL,             /* m_traverse */
    NULL,             /* m_clear */
    NULL              /* m_free */
};

PyMODINIT_FUNC
PyInit_case(void)
{
    PyObject *m = NULL;

    /* Initialize the new types. */

    if (PyType_Ready(&Undefined_Type) < 0)
        goto fail;

    /* Create an empty module. */

    m = PyModule_Create(&case_module);
    if (m == NULL)
        goto fail;

    /* Create the singleton instances and add them to the module. */

    case_Undefined = (PyObject*)UndefinedObject_new(NULL);
    if (case_Undefined == NULL)
        goto fail;

    Py_INCREF(case_Undefined);
    PyModule_AddObject(m, "Undefined", case_Undefined);

    return m;
 fail:
    Py_XDECREF(m);
    Py_XDECREF(case_Undefined);
    return NULL;
}

And here are some examples:

>>> from case import Undefined
>>> bool(Undefined)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: UndefinedObject does not have a boolean value
>>> Undefined < True
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: UndefinedObject can not be compared
>>> Undefined == Undefined
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: UndefinedObject can not be compared
>>> a=Undefined
>>> a is Undefined
True
>>> 

The "is" keyword compares the memory locations of two objects. Since only one Undefined object is created, all references to Undefined point to the same memory location so all instances are the same (according to "is").

Is there a practical use for Undefined? "None" is frequently used as a default value for a keyword argument in a function definition. Since "None" has some practical uses, it is impossible to distinguish between None-as-default and None-as-specified. Undefined can be used as the default value. Undefined has such limited use that it is doubtful that anyone would want to specify it as a value.

>>> def f(val=Undefined):
...   if val is Undefined:
...     print("No argument passed.")
...   else:
...     print("An argument was passed.")
... 
>>> f()
No argument passed.
>>> f(None)
An argument was passed.
>>> 

Please let me know if you would like additional comments or explanations. 

casevh

[metulburr]

Embedding python in C can be found here