A Look at two Python Plugin Managers: Stevedore and Pike

Post to Twitter

If you’ve worked with OpenStack code for even a minimal amount of time you’ve probably come across a Python plugin library called stevedore (you can find the latest docs here). The second Python plugin manager I’m going to cover is one called Pike (latest docs can be found here) which had it’s beginning in a project I blogged about (here and here) a while back called Pynsive. Let’s go over some example code to get a basic idea how each of these work.

Note: For each of these examples I’m using Python 3.4.x.

I’ve setup the first example application in the following structure:

stevedore app structure

Stevedore supports a few kinds of plugins: drivers, hooks, and extensions. Our example will utilize a driver and a couple extensions.

First, install stevedore:

$ pip install -U stevedore

In the setup.py file add the following code:

from setuptools import setup, find_packages

setup(
    name='myexample',
    version='1.0',

    classifiers=['Development Status :: 3 - Alpha',
                 'License :: OSI Approved :: Apache Software License',
                 'Programming Language :: Python :: 3',
                 'Programming Language :: Python :: 3.4',
                 'Intended Audience :: Developers',
                 'Environment :: Console',
                 ],

    platforms=['Any'],
    scripts=[],
    packages=find_packages(),
    include_package_data=True,
    entry_points={
        'mydrivers': [
            'simple_driver1 = drivers.my_driver:SimpleDriver1',
            'simple_driver2 = drivers.my_driver:SimpleDriver2',
        ],
        'myextensions': [
            'simple_ext1 = extensions.my_ext:SimpleExtension1',
            'simple_ext2 = extensions.my_ext:SimpleExtension2',
        ],
    },
    zip_safe=False,
)

The interesting part is of course the entry_points section:

entry_points={
    'mydrivers': [
        'simple_driver1 = drivers.my_driver:SimpleDriver1',
        'simple_driver2 = drivers.my_driver:SimpleDriver2',
    ],
    'myextensions': [
        'simple_ext1 = extensions.my_ext:SimpleExtension1',
        'simple_ext2 = extensions.my_ext:SimpleExtension2',
    ],
}

Stevedore builds on top of the setuptools entry point. If you look at OpenStack projects you can see how its being used heavily in some cases. Projects outside of OpenStack are also using stevedore and a good project that is easy to grasp and understand is pifpaf, check out the setup.cfg.

So with my example code above you can see I have two drivers configured (we will only load one) and two extensions. Let’s move onto the my_driver.py file:

# Drivers

class SimpleDriver1(object):

    def get_name(self):
        return "This is SimpleDriver1"


class SimpleDriver2(object):

    def get_name(self):
        return "This is SimpleDriver2"

Two classes in the same file which is okay just as an example but a real project you would split these out into their own driver files. All the get_name function does is return a string which happens to be the driver name.

Let’s move onto the extension code which for this example will look very similar to the driver code. Inside the my_ext.py file add the following code:

# Extensions

class SimpleExtension1(object):

    def get_name(self):
        return "This is SimpleExtension1"


class SimpleExtension2(object):

    def get_name(self):
        return "This is SimpleExtension2"

Inside the main.py add the following code:

from stevedore import driver, extension


if __name__ == "__main__":
    mydriver = driver.DriverManager(namespace="mydrivers",
                                    name='simple_driver1')
    print(mydriver.driver)
    print(mydriver.driver().get_name())

    extension = extension.ExtensionManager(namespace="myextensions")
    print(extension.extensions)

    for ext in extension.extensions:
        print(ext.plugin().get_name())

In this case for the driver I’m using stevedore’s DriverManager class to load the simple_driver1. From there I just print out the name. Moving onto the extensions I used the ExtensionManager class to load both extensions that were defined. I loop through those extensions and print their names.

Running the code you should see a result similar to this:

<class 'drivers.my_driver.SimpleDriver1'>
This is SimpleDriver1
[ <stevedore.extension.Extension object at 0x106e5574>, <stevedore.extension.Extension object at 0x106e55cf8>]
This is SimpleExtension1
This is SimpleExtension2

Note: Keep in mind stevedore does a lot more, this application is just meant to be a basic introduction. I suggest digging around in projects like pifpaf to see how to utilize stevedore.

Let’s now move onto Pike.

To quote the readme:

“Pike is a dynamic plugin management library for Python. Unlike most Python plugin managers, Pike allows for you to load Python packages from anywhere on a filesystem without complicated configuration. This enables applications to easily add the ability to expand their functionality through plugin modules. In addition to plugin management, Pike also includes a set of discovery functions for custom module and class discovery.”

Pike is very easy to use and the docs pretty much show you a full working example. Create a project and add a main.py and a my_driver.py file. Place the my_driver.py file inside a folder named drivers (include an __init__.py file in there as well). Then install Pike.

$ pip install -U pike

Add the following code to the my_driver.py file (will look very similar to the stevedore example above):

class SimpleDriver(object):

    def get_name(self):
        return "This is SimpleDriver"

The code for the main.py is as follows:

from pike.manager import PikeManager

if __name__ == "__main__":
    driver_path = '/Users/chadlung/PythonProjects/pike-example/src/drivers'

    with PikeManager([driver_path]) as mgr:
        classes = mgr.get_classes()
        print(classes)

        for item in classes:
            obj = item()
            print(obj.get_name())

Run the code above. The output should be:

[class <'my_driver.SimpleDriver'>]
This is SimpleDriver

I’m showing Pike’s filesystem discovery however it also supports an imported module discovery. Also, you will notice I’m loading using a context manager (you don’t need to load this way, but for this example it fits well).

Add another file in the drivers folder called my_other_driver.py with the following code in it:

class MyOtherDriver(object):

    def get_name(self):
        return "This is MyOtherDriver"

My project structure now looks like:

Pike project structure

Run the app again and you should see:

[class <'my_driver.SimpleDriver'>, class <'my_other_driver.MyOtherDriver'>]
This is SimpleDriver
This is MyOtherDriver

You can see the list now has two drivers initially loaded. Keep in mind we could also specify additional file paths via the PikeManager class in the search_paths param, for simplicity I just added the one.

Hopefully this gives you an idea on how to start using a couple popular plugin libraries for Python. Again, keep in mind both examples are very basic and each library can do much more.

Post to Twitter

This entry was posted in Open Source, OpenStack, Python. Bookmark the permalink.