Deploy a Python 3 Application to an Apache Mesos Cluster using the Marathon API

Post to Twitter

Where I work we are actively using Apache Mesos to deploy our production applications written in Go, Python, Lua, etc. We also use Chronos and Marathon. In this article I’ll show you how to setup a local Mesos Cluster complete with Marathon (using Docker) and how to build a simple Python 3.x application to deploy into it.

I shopped around Github a bit and couldn’t quite get exactly what I wanted so I borrowed some code and put together this repo. The Instructions in the Readme should get you up and running pretty easily especially if your using Docker on OSX. Simply clone the repo and follow the instructions to get a local Mesos Cluster (for dev purposes its just one agent running but you can add more).

Once the local Mesos Cluster is running and assuming you are in the mesos-compose folder (although it doesn’t matter) create a new folder called: mypythonapp and add the following Python code in a new file called app.py:

from flask import Flask

app = Flask(__name__)

@app.route('/')
def hello():
    return 'Hello World'

if __name__ == "__main__":
    app.run(host="0.0.0.0", debug=True)

In the mypythonapp folder add a file called requirements.txt with the following inside it:

flask

In the mypythonapp folder add a file called Dockerfile with the following contents:

FROM python:3-onbuild
CMD [ "python", "./app.py" ]

You can see I’m utilizing the official Python repo on Docker hub. It makes deploying Python apps very easy with minimal work on your part especially for learning purposes.

Save all the files and build the Docker image (from the mypythonapp folder):

$ docker build -t mypythonapp .

Note: This will probably take a little time to finish all the downloads. Also, the Dockerfile will be copied into your image, this is most likely not a desired result for production usage but for learning and simplicity in folder/file setup its fine.

Once the image has built you should be able to see it with the following command:

$ docker images

Output:

Output from docker images command

Let’s test the code to make sure it returns the Hello World message before we push it into Mesos via the Marathon API.

$ docker run -p 5000:5000 -t -i mypythonapp

Check the output in your web browser: http://localdocker:5000/

Output:

Hello World

Note: To shut down the container simply: CTRL-C

Now we can deploy this new image to our (make sure its still running) local Mesos (single node) cluster using the Marathon API:

$ curl -X POST -H "Accept: application/json" -H "Content-Type: application/json" localdocker:8080/v2/apps -d ' \
{
    "id": "mypythonapp",
    "container": {
        "docker": {
            "image": "mypythonapp"
        },
        "type": "DOCKER",
        "volumes": []
    },
    "cpus": 0.2,
    "mem": 32.0,
    "instances": 1
}'

Or if you want the command all in one line:

curl -X POST -H "Accept: application/json" -H "Content-Type: application/json" localdocker:8080/v2/apps -d '{"id": "mypythonapp", "container": {"docker": {"image": "mypythonapp"}, "type": "DOCKER", "volumes": []}, "cpus": 0.2, "mem": 32.0, "instances": 1}'

Take a peek at the Marathon and Mesos UIs.

Marathon API UI

Marathon API UI

Mesos UI

Validate its working from a web browser: http://localdocker:5000/

Congratulations you now have a Mesos + Marathon development environment to deploy your Python apps for testing and learning.

Post to Twitter

This entry was posted in Apache Mesos, Open Source, Python. Bookmark the permalink.