How to create a Python package - Part 2

This is the second part of How to create a Python package. Up to this point we created a GitHub repository ad connected it to Travis CI. We synced the content with our local repository and created a file structure with cookiecutter. In this blog post I will show you how to add logging and testing functionality to your package and how to put it on PyPi.

Step 6: Logging

One thing you need to have in mind is that if someone uses your package in their code you shouldn't spam on the command line. The output of your module should go where the user wants it to go. Luckily it is not that hard to implement such a solution because there is a logging module in the standard library. The solution that worked best for me can be found here on the Hitchhikers Guide to Python. What you basically have to do is put the following code into the __init__.py file in the root folder of your package:

import logging
logging.getLogger(__name__).addHandler(logging.NullHandler())

This adds the NullHandler to your logger. This means, that the output will lead nowhere. However a user of your package can add another handler to the logger which will the log to a file or stdout. More about the logging package and handlers can be found here. Now let us implement that logger in a new file in our package that contains the core. Let us call it core.py and fill it with the following code:

import logging

class MyClass(object):
    def __init__(self):
        self.logger = logging.getLogger(__name__)
        self.logger.info("package initialized")

As you can see we initialize the logger in the constructor of our class and log our first info message. There are several levels of logging. This helps your users to decide what information they want to see. Maybe they are just interested in error messages or they want to see a very verbose log and choose the debug level. More about the logging levels can be found here

Step 7: Testing

If you are developing software you should always think about testing as soon as possible. You should define tests that cover your whole interface. This way you can make sure that new code does not destroy your module. The tool that we will by using is called pytest. What this tool does is the following: it looks for python files that start with test_ or end with _test. Inside these files it will call methods that start with test_. More about the test discovery can be found here. Our cookiecutter structure already created a tests folder including a test_sample.py file. It contains the following code:

def test_pass():
    assert True, "dummy sample test"

Since the condition True will always be true the test will never fail. Good for us. Just navigate to the folder of your new package and call pytest. You should see output similar to this:

$ pytest
============================= test session starts =============================
platform win32 -- Python 3.6.1, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
rootdir: C:\Users\hilleckesh\git\package, inifile:
collected 1 item

tests\test_sample.py .

========================== 1 passed in 0.05 seconds ===========================

Now let us add a more useful test. We add a method call get_5 to our class. It will just return 5. Then we can create a test that checks if the method returns 5. Here is the code:

# in package/core.py
def get_5(self):
    return 5

# in tests/test_sample.py
def test_get_5():
    assert myclass.get_5() == 5, "MyClass get_5 returns 5"

Now there is just puzzle piece missing and that is something where I struggled a lot with. How can I reach the following two goals:

  • run tests locally
  • run tests with Travis CI globally

Turns out that there is a solution that works and I will stick with it: create a file called context.py in your tests folder and put the following code into it:

import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

from package.core import MyClass

Now we can import our class in the test_sample.py file:

from .context import MyClass

def test_get_5():
    myclass = MyClass()
    assert myclass.get_5() == 5, "MyClass get_5 returns 5"

I found the code in the this GitHub repository. It is accompanied by a lot of explanations and I recommend reading the whole thing. If you call pytest again you should get the following output that shows that there a two successful tests now.

$ pytest
============================= test session starts =============================
platform win32 -- Python 3.6.1, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
rootdir: C:\Users\hilleckesh\git\package, inifile:
collected 2 items

tests\test_sample.py ..

========================== 2 passed in 0.07 seconds ===========================

This confirms that you can test locally. Now let us check if Travis CI works, too. Push your changes to your GitHub repository. Travis CI will automatically start a new build. If not you can trigger it manually in the options. If you click on your package on the left side you can see the log of the current build and since we configured Travis CI to call pytest after installing our package we will see the pytest output in this log, too.

 Travis CI calls pytest after installing the package. This gets triggered after every push to your GitHub repository. 

Travis CI calls pytest after installing the package. This gets triggered after every push to your GitHub repository. 

STEP 8: Uploading your package to PYpi

Now that you have your development and testing up and running you want to upload your package to pypi. As mentioned in the first post you need to create or own an account on pypi. You can create it easily here

Uploading your package to pypi contains of two steps:

  • create the source distribution files
  • uploading the source distribution files

More about source distribution files here. Create the source distribution files by calling

python setup.py sdist

You will see that a new folder dist will be created which contains a file called package-0.0.1.tar.gz. This file contains your package. Now we use the tool twine to upload it:

$ twine upload dist/package-0.0.1.tar.gz
Uploading distributions to https://upload.pypi.org/legacy/
Enter your username: hhllcks
Enter your password:
Uploading package-0.0.1.tar.gz

Of course it is not possible for me to upload a package called 'package'. It already exists and has a really great introduction in the readme. But if you have found a new name for your package that does not exist yet you should be able to upload it. You can also use the TestPyPI server to validate your package: 

$ twine upload --repository-url https://test.pypi.org/legacy/ dist/*

You can check if everything has worked by installing your package using pip install in a new environment or a new machine. 

Summary

This concludes my two-part tutorial on how to create a python package. If you have any comments, questions or additions feel free to comment or tweet at me.