Trying out the Docker Compose support for PyWebhooks (a Python 3 webhooks server project)

Post to Twitter

When I introduced PyWebhooks I wrote an article on how to get started trying out the service easily using Vagrant. It was always my plan to add Docker Compose support to PyWebhooks to make it even easier for someone to try out PyWebhooks. In this article I’ll walk you through the steps to use the recently added Docker Compose support.

Note: You can find the PyWebhooks repository on Github here. PyWebhooks is open sourced under the Apache 2 license.

The easiest way to go through these steps is to create a virtualenv for the project as we will be running the example endpoint server (for testing) locally and everything else in Docker containers.

Note: Make sure you are running Docker version 1.10.0 and Docker Compose 1.6.0 or newer.

Let’s setup the project locally first. I keep my Python projects in a specific folder but feel free to change to your needs.

$ cd PythonProjects
$ python3 -m venv pywebhooks-github
$ cd pywebhooks-github
$ source bin/activate
$ git clone https://github.com/chadlung/pywebhooks.git
$ cd pywebhooks
$ pip install -r requirements.txt

By default the configuration settings are setup for Vagrant testing. You can easily change that by going into the pywebhooks\__init.py__ to edit.

Comment out the default Vagrant configuration:

# RETHINK_HOST = 'localhost'
# CELERY_BROKER_URL = 'redis://localhost:6379/0'

Un-Comment the Docker Compose configuration:

RETHINK_HOST = 'rethinkdb'
CELERY_BROKER_URL = 'redis://redis:6379/0'

Save the file changes.

With that out of the way move into the docker folder and run:

$ cd docker
$ docker-compose up

If you can don’t run that in daemon mode as you can more easily capture the admin secret_key and api_key from the console output. It will look similar to this:

pywebhooks-server | Adding admin account
pywebhooks-server | {'secret_key': 'd620fb92a70b7e5c127de74fcd717aa803f7e300', 'api_key': '7e8d21dda1c5738a30882e4520fbbfac55eebe3f'}

Make sure to record those keys.

On your local machine start the endpoint server.

$ python3 pywebhooks/examples/endpoint_development_server.py

That should start listening on: http://0.0.0.0:9090/

Now, let’s create an initial (non-admin) account. Since this will be the PyWebhooks server within Docker communicating with the endpoint server you need to use the following cURL call using your machine’s IP. In my case it is: 10.0.1.4 Adjust this to your machine’s IP. Same goes with the POST, you need to adjust this to the location of where you Docker VM is hosted (if not using Linux). In my case I’m using Virtualbox via Docker Machine to host the Docker VM and the IP is: 192.168.99.100 In a future version of Docker for Mac (in closed beta when this article was written) you can work much more closely to how Linux deals with Docker.

$ curl -v -X POST "http://192.168.99.100:8081/v1/account" -d '{"endpoint": "http://10.0.1.4:9090/account/endpoint", "username": "johndoe"}' -H "content-type: application/json"

Result:

{
  "api_key": "13d11bbf64b9db385c2357cbe21a732bed6695b7",
  "endpoint": "http://10.0.1.4:9090/account/endpoint",
  "epoch": 1459115939.933271,
  "failed_count": 0,
  "id": "a871db84-10ab-4217-97f9-dd670252e89f",
  "is_admin": false,
  "secret_key": "a8fe0d6b6b0d029388ef8503028029d660a86f99",
  "username": "johndoe"
}

Stop the endpoint_development_server.py from running and modify the code at the top:

# Adjust this as needed
SECRET_KEY = 'a8fe0d6b6b0d029388ef8503028029d660a86f99'

Using the output above put in the correct SECRET_KEY value, in the case of the example it would be: a8fe0d6b6b0d029388ef8503028029d660a86f99

Go ahead start the endpoint_development_server.py server again.

Create a new Webhook (adjust IPs as needed):

$ curl -v -X POST "http://192.168.99.100:8081/v1/webhook/registration" -H "content-type: application/json" -H "username: johndoe" -H "api-key: 13d11bbf64b9db385c2357cbe21a732bed6695b7" -d '{"description": "This is my registered webhook", "event_data": {"items": [{"item1": 1}, {"item2": 2}], "message": "hello world"}, "event": "mywebhook.event"}'

Result:

{
  "account_id": "a871db84-10ab-4217-97f9-dd670252e89f",
  "description": "This is my registered webhook",
  "epoch": 1459116313.4610372,
  "event": "mywebhook.event",
  "event_data": {
    "items": [
      {
        "item1": 1
      },
      {
        "item2": 2
      }
    ],
    "message": "hello world"
  },
  "id": "1100ae72-3db6-421d-b050-11cd1576d4e1"
}

Now you (or another account) can subscribe to that WebHook using it’s ID and receive notifications when it gets triggered (PyWebhooks will notify your endpoint server you setup).

$ curl -v -X POST "http://192.168.99.100:8081/v1/webhook/subscription/1100ae72-3db6-421d-b050-11cd1576d4e1" -H "content-type: application/json" -H "api-key: 13d11bbf64b9db385c2357cbe21a732bed6695b7" -H "username: johndoe"

Result:

{
  "account_id": "a871db84-10ab-4217-97f9-dd670252e89f",
  "epoch": 1459116429.3455124,
  "id": "53eac92d-a75b-4d9f-9a73-d69e9e020563",
  "registration_id": "1100ae72-3db6-421d-b050-11cd1576d4e1"
}

Now that someone is subscribed let’s go ahead and trigger the WebHook:

$ curl -v -X POST "http://192.168.99.100:8081/v1/webhook/triggered/1100ae72-3db6-421d-b050-11cd1576d4e1" -H "content-type: application/json" -H "api-key: 13d11bbf64b9db385c2357cbe21a732bed6695b7" -H "username: johndoe"

Result:

{
  "epoch": 1459116678.6967037,
  "id": "e6b8dbd2-e50e-4964-8dd9-32c55d061654",
  "registration_id": "1100ae72-3db6-421d-b050-11cd1576d4e1"
}

If you look in the output from the endpoint server you will see the following:

Event: mywebhook.event
Content-Length: 65
Connection: keep-alive
User-Agent: python-requests/2.9.1
Accept-Encoding: gzip, deflate
Accept: application/json
Pywebhooks-Signature: b1291274a5dd4c287a0e1223abbbd6bbe136c2c3
Host: 10.0.1.4:9090
Content-Type: application/json


b'{"items": [{"item1": 1}, {"item2": 2}], "message": "hello world"}'
{"items": [{"item1": 1}, {"item2": 2}], "message": "hello world"}
Is Signature Valid?: True
10.0.1.4 - - [27/Mar/2016 17:11:18] "POST /account/endpoint HTTP/1.1" 200 -

These are just the most basic API commands for PyWebhooks. Check out the fully detailed Readme for more API calls as well as admin API calls.

One last thing, if you look at the Docker Compose output in your terminal you can also verify everything that we’ve done so far:

pywebhooks-server | Creating database...
pywebhooks-server | Adding admin account
pywebhooks-server | {'secret_key': '6a250e31346d5680758d9c51bc858bb9b0ba66fd', 'api_key': 'edf24009cc30d7c061039503f76866c9c8df2771'}
pywebhooks-server | Complete
pywebhooks-server |  * Running on http://0.0.0.0:8081/ (Press CTRL+C to quit)
pywebhooks-server |  * Restarting with stat
pywebhooks-server |  * Debugger is active!
pywebhooks-server |  * Debugger pin code: 160-326-415
pywebhooks-server | 192.168.99.1 - - [27/Mar/2016 21:57:59] "POST /v1/account HTTP/1.1" 400 -
pywebhooks-server | 192.168.99.1 - - [27/Mar/2016 21:58:59] "POST /v1/account HTTP/1.1" 201 -
pywebhooks-server | 192.168.99.1 - - [27/Mar/2016 22:05:13] "POST /v1/webhook/registration HTTP/1.1" 201 -
pywebhooks-server | 192.168.99.1 - - [27/Mar/2016 22:07:09] "POST /v1/webhook/subscription/1100ae72-3db6-421d-b050-11cd1576d4e1 HTTP/1.1" 201 -
pywebhooks-worker | [2016-03-27 22:11:18,691: INFO/MainProcess] Received task: pywebhooks.tasks.webhook_notification.notify_subscribed_accounts[06e09781-5d48-4d0a-98ca-cd92b93f30bf]
pywebhooks-server | 192.168.99.1 - - [27/Mar/2016 22:11:18] "POST /v1/webhook/triggered/1100ae72-3db6-421d-b050-11cd1576d4e1 HTTP/1.1" 201 -
pywebhooks-worker | [2016-03-27 22:11:18,714: INFO/Worker-1] Starting new HTTP connection (1): 10.0.1.4
pywebhooks-worker | [2016-03-27 22:11:18,753: INFO/MainProcess] Task pywebhooks.tasks.webhook_notification.notify_subscribed_accounts[06e09781-5d48-4d0a-98ca-cd92b93f30bf] succeeded in 0.05999482100014575s: None

Post to Twitter

This entry was posted in Open Source, Python. Bookmark the permalink.