Basic functionality

Warning

Note that astropy-helpers is deprecated. As justified and described in APE17, the astropy-helpers infrastructure is no longer used for Astropy coordinated or infrastructure packages, and hence is no longer maintained.

The big-picture purpose of astropy-helpers is to provide customization to Python’s packaging infrastructure process in ways that the Astropy Project has found to help simplifying the developing and releasing packages. This is primarily built around setup.py commands, as outlined below, as well as code to help manage version numbers and better control the build process of larger packages.

Custom setup.py commands

The main part of astropy-helpers is to provide customized setuptools commands. For example, in a package that uses astropy-helpers, the following command will be available:

python setup.py build_docs

and this command is implemented in astropy-helpers. To use the custom commands described here, you can either use the simplified method of opting in to astropy-helpers described in Opting in to all basic functionality, or if you want more control, use the register_commands() function by adding:

from astropy_helpers.setup_helpers import register_commands

to your setup.py file, then doing:

# Create a dictionary with setup command overrides. Note that this gets
# information about the package (name and version) from the setup.cfg file.
cmdclassd = register_commands()

This function requires that the package name and full version are set in the setup.cfg file in the [metadata] section, e.g.:

[metadata]
name = mypackage
version = 0.4.dev

Then, pass cmdclassd to the setup function in setup.py:

setup(...,
      cmdclass=cmdclassd)

The commands we provide or customize are:

python setup.py test

This command will automatically build the package, install it to a temporary directory, and run the tests using pytest on this installed version. Note that the bulk of this command is actually defined in astropy.tests.command.AstropyTest, because that allows the test machinery to operate outside a setuptools command. This, here we simply define the custom setuptools command.

python setup.py sdist

We redefine sdist to use the version from distutils rather than from setuptools, as the setuptools version requires duplication of information in MANIFEST.in.

python setup.py build_docs

This command will automatically build the package, then run sphinx to build the documentation. This makes development much easier because it ensures sphinx extensions that use the package’s code to make documentation are actually using the in-development version of the code. Sphinx itself provides a custom setuptools command, which we expand with the following options:

  • -w: set the return code to 1 if there are any warnings during the build process.

  • -l: completely clean previous builds, including files generated by the sphinx-automodapi package (which creates API pages for different functions/classes).

  • -n: disable the intersphinx option.

  • -o: open the documentation in a browser if a build finishes successfully.

In addition, build_docs will automatically download and temporarily install sphinx-astropy (which is a meta-package that provides standardized configuration and documentation dependencies for astropy packages) if it isn’t already installed. Temporary installation means that the package will be installed into an .eggs directory in the current working directory, and it will only be available for the duration of the call to build_docs.

python setup.py build_ext

This is also used when running build or install. We add several features compared to the default build_ext command:

  • For packages with C/Cython extensions, we create a packagename.compiler_version submodule and a packagename.cython_version submodule that contain information about the compilers used.

  • Packages that need to build C extensions using the Numpy C API, we allow those packages to define the include path as 'numpy' as opposed to having to import Numpy and call get_include. The goal is to solve the issue that if one has to import Numpy to define extensions, then Numpy has to be installed/available before the package is installed, which means that one needs to install Numpy in a separate installation step.

  • We detect broken compilers and replace them with other compilers on-the-fly unless the compiler is explicitly specified with the CC environment variable.

  • If Cython is not installed, then we automatically check for generated C files (which are normally present in the stable releases) and give a nice error if these are not found.

Version helpers

Another piece of functionality we provide in astropy-helpers is the ability to generate a packagename.version module that includes functions that automatically set the version string for developer versions, to e.g. 3.2.dev22213 so that each developer version has a unique number (although note that branches an equal number of commits away from the master branch will share the same version number).

In addition, this module contains variables such as major, minor, and bugfix, as well as version_info (a tuple of the previous three values), a release flag that indicates whether we are using a stable release, and several other complementary variables. To use the version helpers, you can either use the simplified method of opting in to astropy-helpers described in Opting in to all basic functionality, or if you want more control, use the generate_version_py(), import:

from astropy_helpers.version_helpers import generate_version_py

in your setup.py file, and call:

# Freeze build information in version.py. Note that this gets information
# about the package (name and version) from the setup.cfg file.
version = generate_version_py()

The version variable will be set to the version number of your package including any developer suffix. Note that this requires that the package name and version are set in the setup.cfg file in the [metadata] section, e.g.:

[metadata]
name = mypackage
version = 0.4.dev

Then, pass version to the setup function in setup.py:

setup(...,
      version=version)

Note that if you want to be able to generate developer versions such as 3.2.dev22213 without having to use the generate_version_py machinery, you can instead just import get_git_devstr():

from astropy_helpers.git_helpers import get_git_devstr

and you will then be able to use e.g.:

version += get_git_devstr()

to add the developer suffix to the version string.

Collecting package information

The setup function from setuptools can take a number of options that indicate for example what extensions to build, and what package data to include. However, for large packages this can become cumbersome. We therefore provide a mechanism for defining extensions and package data inside individual sub-packages. To do this, you can create setup_package.py files anywhere in your package, and these files can include one or more of the following functions:

  • get_package_data:

    This function, if defined, should return a dictionary mapping the name of the subpackage(s) that need package data to a list of data file paths (possibly including wildcards) relative to the path of the package’s source code. e.g. if the source distribution has a needed data file astropy/wcs/tests/data/3d_cd.hdr, this function should return {'astropy.wcs.tests':['data/3d_cd.hdr']}. See the package_data option of the distutils.core.setup() function.

    It is recommended that all such data be in a directory named data inside the package within which it is supposed to be used. This package data should be accessed via the astropy.utils.data.get_pkg_data_filename and astropy.utils.data.get_pkg_data_fileobj functions.

  • get_extensions:

    This provides information for building C or Cython extensions. If defined, it should return a list of distutils.core.Extension objects.

  • get_build_options:

    This function allows a package to add extra build options. It should return a list of tuples, where each element has:

    • name: The name of the option as it would appear on the commandline or in the setup.cfg file.

    • doc: A short doc string for the option, displayed by setup.py build --help.

    • is_bool (optional): When True, the option is a boolean option and doesn’t have an associated value.

    Once an option has been added, its value can be looked up using astropy_helpers.setup_helpers.get_distutils_build_option.

  • get_external_libraries:

    This function declares that the package uses libraries that are included in the astropy distribution that may also be distributed elsewhere on the users system. It should return a list of library names. For each library, a new build option is created, '--use-system-X' which allows the user to request to use the system’s copy of the library. The package would typically call astropy_helpers.setup_helpers.use_system_library from its get_extensions function to determine if the package should use the system library or the included one.

With these files in place, you can either use the simplified method of opting in to astropy-helpers described in Opting in to all basic functionality, or if you want more control, use theyou can then make use of the get_package_info() function in your setup.py file with:

from astropy_helpers.setup_helpers import get_package_info

...

package_info = get_package_info()

...

setup(..., **package_info)

Opting in to all basic functionality

If you are happy to opt in to all the functionality described on this page, you can make use of the setup() function from astropy_helpers.setup_helpers which wraps the function of the same name from setuptools and automatically runs register_commands(), generate_version_py(), and get_package_info(). If you want to do this, make sure the package name and version number are defined in setup.cfg:

[metadata]
name = mypackage
version = 0.4.dev

then use the setup() function from astropy-helpers in your setup.py file as follows:

import ah_bootstrap
from astropy_helpers.setup_helpers import setup
setup()

We recommend that you also include a comment along the following lines in your setup.py file:

# The configuration for the package, including the name, version, and other
# information are set in the setup.cfg file.