Input Arrays

Input arrays are defined as arrays of data that are passed into a routine but are not altered in-place or returned to the user. The python input array is therefore allowed to be almost any python sequence (such as a list) that can be converted to the requested type of array. The input array signatures are

1D:

• ( DATA_TYPE IN_ARRAY1[ANY] )
• ( DATA_TYPE* IN_ARRAY1, int DIM1 )
• ( int DIM1, DATA_TYPE* IN_ARRAY1 )

2D:

• ( DATA_TYPE IN_ARRAY2[ANY][ANY] )
• ( DATA_TYPE* IN_ARRAY2, int DIM1, int DIM2 )
• ( int DIM1, int DIM2, DATA_TYPE* IN_ARRAY2 )
• ( DATA_TYPE* IN_FARRAY2, int DIM1, int DIM2 )
• ( int DIM1, int DIM2, DATA_TYPE* IN_FARRAY2 )

3D:

• ( DATA_TYPE IN_ARRAY3[ANY][ANY][ANY] )
• ( DATA_TYPE* IN_ARRAY3, int DIM1, int DIM2, int DIM3 )
• ( int DIM1, int DIM2, int DIM3, DATA_TYPE* IN_ARRAY3 )
• ( DATA_TYPE* IN_FARRAY3, int DIM1, int DIM2, int DIM3 )
• ( int DIM1, int DIM2, int DIM3, DATA_TYPE* IN_FARRAY3 )

The first signature listed, ( DATA_TYPE IN_ARRAY[ANY] ) is for one-dimensional arrays with hard-coded dimensions. Likewise, ( DATA_TYPE IN_ARRAY2[ANY][ANY] ) is for two-dimensional arrays with hard-coded dimensions, and similarly for three-dimensional.

In-Place Arrays

In-place arrays are defined as arrays that are modified in-place. The input values may or may not be used, but the values at the time the function returns are significant. The provided python argument must therefore be a NumPy array of the required type. The in-place signatures are

1D:

• ( DATA_TYPE INPLACE_ARRAY1[ANY] )
• ( DATA_TYPE* INPLACE_ARRAY1, int DIM1 )
• ( int DIM1, DATA_TYPE* INPLACE_ARRAY1 )

2D:

• ( DATA_TYPE INPLACE_ARRAY2[ANY][ANY] )
• ( DATA_TYPE* INPLACE_ARRAY2, int DIM1, int DIM2 )
• ( int DIM1, int DIM2, DATA_TYPE* INPLACE_ARRAY2 )
• ( DATA_TYPE* INPLACE_FARRAY2, int DIM1, int DIM2 )
• ( int DIM1, int DIM2, DATA_TYPE* INPLACE_FARRAY2 )

3D:

• ( DATA_TYPE INPLACE_ARRAY3[ANY][ANY][ANY] )
• ( DATA_TYPE* INPLACE_ARRAY3, int DIM1, int DIM2, int DIM3 )
• ( int DIM1, int DIM2, int DIM3, DATA_TYPE* INPLACE_ARRAY3 )
• ( DATA_TYPE* INPLACE_FARRAY3, int DIM1, int DIM2, int DIM3 )
• ( int DIM1, int DIM2, int DIM3, DATA_TYPE* INPLACE_FARRAY3 )

These typemaps now check to make sure that the INPLACE_ARRAY arguments use native byte ordering. If not, an exception is raised.

Argout Arrays

Argout arrays are arrays that appear in the input arguments in C, but are in fact output arrays. This pattern occurs often when there is more than one output variable and the single return argument is therefore not sufficient. In python, the convential way to return multiple arguments is to pack them into a sequence (tuple, list, etc.) and return the sequence. This is what the argout typemaps do. If a wrapped function that uses these argout typemaps has more than one return argument, they are packed into a tuple or list, depending on the version of python. The python user does not pass these arrays in, they simply get returned. For the case where a dimension is specified, the python user must provide that dimension as an argument. The argout signatures are

1D:

• ( DATA_TYPE ARGOUT_ARRAY1[ANY] )
• ( DATA_TYPE* ARGOUT_ARRAY1, int DIM1 )
• ( int DIM1, DATA_TYPE* ARGOUT_ARRAY1 )

2D:

• ( DATA_TYPE ARGOUT_ARRAY2[ANY][ANY] )

3D:

• ( DATA_TYPE ARGOUT_ARRAY3[ANY][ANY][ANY] )

These are typically used in situations where in C/C++, you would allocate a(n) array(s) on the heap, and call the function to fill the array(s) values. In python, the arrays are allocated for you and returned as new array objects.

Note that we support DATA_TYPE* argout typemaps in 1D, but not 2D or 3D. This is because of a quirk with the SWIG typemap syntax and cannot be avoided. Note that for these types of 1D typemaps, the python function will take a single argument representing DIM1.

Argoutview Arrays

Argoutview arrays are for when your C code provides you with a view of its internal data and does not require any memory to be allocated by the user. This can be dangerous. There is almost no way to guarantee that the internal data from the C code will remain in existence for the entire lifetime of the NumPy array that encapsulates it. If the user destroys the object that provides the view of the data before destroying the NumPy array, then using that array my result in bad memory references or segmentation faults. Nevertheless, there are situations, working with large data sets, where you simply have no other choice.

The C code to be wrapped for argoutview arrays are characterized by pointers: pointers to the dimensions and double pointers to the data, so that these values can be passed back to the user. The argoutview typemap signatures are therefore

1D:

• ( DATA_TYPE** ARGOUTVIEW_ARRAY1, DIM_TYPE* DIM1 )
• ( DIM_TYPE* DIM1, DATA_TYPE** ARGOUTVIEW_ARRAY1 )

2D:

• ( DATA_TYPE** ARGOUTVIEW_ARRAY2, DIM_TYPE* DIM1, DIM_TYPE* DIM2 )
• ( DIM_TYPE* DIM1, DIM_TYPE* DIM2, DATA_TYPE** ARGOUTVIEW_ARRAY2 )
• ( DATA_TYPE** ARGOUTVIEW_FARRAY2, DIM_TYPE* DIM1, DIM_TYPE* DIM2 )
• ( DIM_TYPE* DIM1, DIM_TYPE* DIM2, DATA_TYPE** ARGOUTVIEW_FARRAY2 )

3D:

• ( DATA_TYPE** ARGOUTVIEW_ARRAY3, DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3)
• ( DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3, DATA_TYPE** ARGOUTVIEW_ARRAY3)
• ( DATA_TYPE** ARGOUTVIEW_FARRAY3, DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3)
• ( DIM_TYPE* DIM1, DIM_TYPE* DIM2, DIM_TYPE* DIM3, DATA_TYPE** ARGOUTVIEW_FARRAY3)

Note that arrays with hard-coded dimensions are not supported. These cannot follow the double pointer signatures of these typemaps.

Output Arrays

The numpy.i interface file does not support typemaps for output arrays, for several reasons. First, C/C++ return arguments are limited to a single value. This prevents obtaining dimension information in a general way. Second, arrays with hard-coded lengths are not permitted as return arguments. In other words:

double[3] newVector(double x, double y, double z);

is not legal C/C++ syntax. Therefore, we cannot provide typemaps of the form:

%typemap(out) (TYPE[ANY]);

If you run into a situation where a function or method is returning a pointer to an array, your best bet is to write your own version of the function to be wrapped, either with %extend for the case of class methods or %ignore and %rename for the case of functions.

Other Common Types: bool

Note that C++ type bool is not supported in the list in the Available Typemaps section. NumPy bools are a single byte, while the C++ bool is four bytes (at least on my system). Therefore:

%numpy_typemaps(bool, NPY_BOOL, int)

will result in typemaps that will produce code that reference improper data lengths. You can implement the following macro expansion:

%numpy_typemaps(bool, NPY_UINT, int)

to fix the data length problem, and Input Arrays will work fine, but In-Place Arrays might fail type-checking.

Other Common Types: complex

Typemap conversions for complex floating-point types is also not supported automatically. This is because python and NumPy are written in C, which does not have native complex types. Both python and NumPy implement their own (essentially equivalent) struct definitions for complex variables:

/* Python */
typedef struct {double real; double imag;} Py_complex;

/* NumPy */
typedef struct {float  real, imag;} npy_cfloat;
typedef struct {double real, imag;} npy_cdouble;

We could have implemented:

%numpy_typemaps(Py_complex , NPY_CDOUBLE, int)
%numpy_typemaps(npy_cfloat , NPY_CFLOAT , int)
%numpy_typemaps(npy_cdouble, NPY_CDOUBLE, int)

which would have provided automatic type conversions for arrays of type Py_complex, npy_cfloat and npy_cdouble. However, it seemed unlikely that there would be any independent (non-python, non-NumPy) application code that people would be using SWIG to generate a python interface to, that also used these definitions for complex types. More likely, these application codes will define their own complex types, or in the case of C++, use std::complex. Assuming these data structures are compatible with python and NumPy complex types, %numpy_typemap expansions as above (with the user's complex type substituted for the first argument) should work.

Why is There a Second File?

The SWIG type checking and conversion system is a complicated combination of C macros, SWIG macros, SWIG typemaps and SWIG fragments. Fragments are a way to conditionally insert code into your wrapper file if it is needed, and not insert it if not needed. If multiple typemaps require the same fragment, the fragment only gets inserted into your wrapper code once.

There is a fragment for converting a python integer to a C long. There is a different fragment that converts a python integer to a C int, that calls the rountine defined in the long fragment. We can make the changes we want here by changing the definition for the long fragment. SWIG determines the active definition for a fragment using a "first come, first served" system. That is, we need to define the fragment for long conversions prior to SWIG doing it internally. SWIG allows us to do this by putting our fragment definitions in the file pyfragments.swg. If we were to put the new fragment definitions in numpy.i, they would be ignored.

Macros

is_array(a)

Evaluates as true if a is non-NULL and can be cast to a PyArrayObject*.

array_type(a)

Evaluates to the integer data type code of a, assuming a can be cast to a PyArrayObject*.

array_numdims(a)

Evaluates to the integer number of dimensions of a, assuming a can be cast to a PyArrayObject*.

array_dimensions(a)

Evaluates to an array of type npy_intp and length array_numdims(a), giving the lengths of all of the dimensions of a, assuming a can be cast to a PyArrayObject*.

array_size(a,i)

Evaluates to the i-th dimension size of a, assuming a can be cast to a PyArrayObject*.

array_data(a)

Evaluates to a pointer of type void* that points to the data buffer of a, assuming a can be cast to a PyArrayObject*.

array_is_contiguous(a)

Evaluates as true if a is a contiguous array. Equivalent to (PyArray_ISCONTIGUOUS(a)).

array_is_native(a)

Evaluates as true if the data buffer of a uses native byte order. Equivalent to (PyArray_ISNOTSWAPPED(a)).

array_is_fortran(a)

Evaluates as true if a is FORTRAN ordered.

Routines

pytype_string()

Return type: char*

Arguments:

• PyObject* py_obj, a general python object.

Return a string describing the type of py_obj.

typecode_string()

Return type: char*

Arguments:

• int typecode, a NumPy integer typecode.

Return a string describing the type corresponding to the NumPy typecode.

type_match()

Return type: int

Arguments:

• int actual_type, the NumPy typecode of a NumPy array.
• int desired_type, the desired NumPy typecode.

Make sure that actual_type is compatible with desired_type. For example, this allows character and byte types, or int and long types, to match. This is now equivalent to PyArray_EquivTypenums().

obj_to_array_no_conversion()

Return type: PyArrayObject*

Arguments:

• PyObject* input, a general python object.
• int typecode, the desired NumPy typecode.

Cast input to a PyArrayObject* if legal, and ensure that it is of type typecode. If input cannot be cast, or the typecode is wrong, set a python error and return NULL.

obj_to_array_allow_conversion()

Return type: PyArrayObject*

Arguments:

• PyObject* input, a general python object.
• int typecode, the desired NumPy typecode of the resulting array.
• int* is_new_object, returns a value of 0 if no conversion performed, else 1.

Convert input to a NumPy array with the given typecode. On success, return a valid PyArrayObject* with the correct type. On failure, the python error string will be set and the routine returns NULL.

make_contiguous()

Return type: PyArrayObject*

Arguments:

• PyArrayObject* ary, a NumPy array.
• int* is_new_object, returns a value of 0 if no conversion performed, else 1.
• int min_dims, minimum allowable dimensions.
• int max_dims, maximum allowable dimensions.

Check to see if ary is contiguous. If so, return the input pointer and flag it as not a new object. If it is not contiguous, create a new PyArrayObject* using the original data, flag it as a new object and return the pointer.

obj_to_array_contiguous_allow_conversion()

Return type: PyArrayObject*

Arguments:

• PyObject* input, a general python object.
• int typecode, the desired NumPy typecode of the resulting array.
• int* is_new_object, returns a value of 0 if no conversion performed, else 1.

Convert input to a contiguous PyArrayObject* of the specified type. If the input object is not a contiguous PyArrayObject*, a new one will be created and the new object flag will be set.

require_contiguous()

Return type: int

Arguments:

• PyArrayObject* ary, a NumPy array.

Test whether ary is contiguous. If so, return 1. Otherwise, set a python error and return 0.

require_native()

Return type: int

Arguments:

• PyArray_Object* ary, a NumPy array.

Require that ary is not byte-swapped. If the array is not byte-swapped, return 1. Otherwise, set a python error and return 0.

require_dimensions()

Return type: int

Arguments:

• PyArrayObject* ary, a NumPy array.
• int exact_dimensions, the desired number of dimensions.

Require ary to have a specified number of dimensions. If the array has the specified number of dimensions, return 1. Otherwise, set a python error and return 0.

require_dimensions_n()

Return type: int

Arguments:

• PyArrayObject* ary, a NumPy array.
• int* exact_dimensions, an array of integers representing acceptable numbers of dimensions.
• int n, the length of exact_dimensions.

Require ary to have one of a list of specified number of dimensions. If the array has one of the specified number of dimensions, return 1. Otherwise, set the python error string and return 0.

require_size()

Return type: int

Arguments:

• PyArrayObject* ary, a NumPy array.
• npy_int* size, an array representing the desired lengths of each dimension.
• int n, the length of size.

Require ary to have a specified shape. If the array has the specified shape, return 1. Otherwise, set the python error string and return 0.

require_fortran()

Return type: int

Arguments:

• PyArrayObject* ary, a NumPy array.

Require the given PyArrayObject to to be FORTRAN ordered. If the the PyArrayObject is already FORTRAN ordered, do nothing. Else, set the FORTRAN ordering flag and recompute the strides.

A Common Example

Consider a reasonable prototype for a dot product function:

double dot(int len, double* vec1, double* vec2);

The python interface that we want is:

def dot(vec1, vec2):
    """
    dot(PyObject,PyObject) -> double
    """

The problem here is that there is one dimension argument and two array arguments, and our typemaps are set up for dimensions that apply to a single array (in fact, SWIG does not provide a mechanism for associating len with vec2 that takes two python input arguments). The recommended solution is the following:

%apply (int DIM1, double* IN_ARRAY1) {(int len1, double* vec1),
                                      (int len2, double* vec2)}
%rename (dot) my_dot;
%exception my_dot {
    $action
    if (PyErr_Occurred()) SWIG_fail;
}
%inline %{
double my_dot(int len1, double* vec1, int len2, double* vec2) {
    if (len1 != len2) {
        PyErr_Format(PyExc_ValueError,
                     "Arrays of lengths (%d,%d) given",
                     len1, len2);
        return 0.0;
    }
    return dot(len1, vec1, vec2);
}
%}

If the header file that contains the prototype for double dot() also contains other prototypes that you want to wrap, so that you need to %include this header file, then you will also need a %ignore dot; directive, placed after the %rename and before the %include directives. Or, if the function in question is a class method, you will want to use %extend rather than %inline in addition to %ignore.

A note on error handling: Note that my_dot returns a double but that it can also raise a python error. The resulting wrapper function will return a python float representation of 0.0 when the vector lengths do not match. Since this is not NULL, the python interpreter will not know to check for an error. For this reason, we add the %exception directive above for my_dot to get the behavior we want (note that $action is a macro that gets expanded to a valid call to my_dot). In general, you will probably want to write a SWIG macro to perform this task.

Other Situations

There are other wrapping situations in which numpy.i may be helpful when you encounter them.

• In some situations, it is possible that you could use the %numpy_templates macro to implement typemaps for your own types. See the Other Common Types: bool or Other Common Types: complex sections for examples. Another situation is if your dimensions are of a type other than int (say long for example):

  %numpy_typemaps(double, NPY_DOUBLE, long)
  

• You can use the code in numpy.i to write your own typemaps. For example, if you had a four-dimensional array as a function argument, you could cut-and-paste the appropriate three-dimensional typemaps into your interface file. The modifications for the fourth dimension would be trivial.

• Sometimes, the best approach is to use the %extend directive to define new methods for your classes (or overload existing ones) that take a PyObject* (that either is or can be converted to a PyArrayObject*) instead of a pointer to a buffer. In this case, the helper routines in numpy.i can be very useful.

• Writing typemaps can be a bit nonintuitive. If you have specific questions about writing SWIG typemaps for NumPy, the developers of numpy.i do monitor the Numpy-discussion and Swig-user mail lists.

A Final Note

When you use the %apply directive, as is usually necessary to use numpy.i, it will remain in effect until you tell SWIG that it shouldn't be. If the arguments to the functions or methods that you are wrapping have common names, such as length or vector, these typemaps may get applied in situations you do not expect or want. Therefore, it is always a good idea to add a %clear directive after you are done with a specific typemap:

%apply (double* IN_ARRAY1, int DIM1) {(double* vector, int length)}
%include "my_header.h"
%clear (double* vector, int length);

In general, you should target these typemap signatures specifically where you want them, and then clear them after you are done.