Introducing PyWebhooks: An Easy to Use Webhooks Service (Written in Python 3)

Post to Twitter

A while back (and even recently) I was looking for any sort of project that allowed you to create and define your own webhooks. Features like allowing others to subscribe to those events, and then trigger your custom webhooks when needed which then would go and notify each subscription’s endpoint. I couldn’t find anything like that. Pieces that did some of those features, but not the whole package. So PyWebhooks was born (Github repo here).

Note: PyWebhooks utilizes the excellent RethinkDB and Redis projects.

Although PyWebhooks is written in Python 3.4 (Python 2 support is not planned) it can be used by any service that “Speaks HTTP” so to speak. Your endpoints can be written in any language that can provide an “endpoint” service (a simple REST service). You can use simple cURL calls to interact with the PyWebhooks service or write your own client (you’ll find examples that will help you get started here and here). The readme contains all the information to create accounts, subscriptions, etc.

Most developers are already familiar with the concept of a webhook. In fact, most probably have had exposure of webhooks through services like Github or Dropbox (and many other services). The idea is you can POST some event data to subscribed service’s endpoints. This could be on your internal network where a particular event happens at any given time or other services that are interested in knowing the results of that particular event or even just that it happened.

How do you test out PyWebhooks easily? Glad you asked.

With Vagrant installed clone this repo: https://github.com/chadlung/vagrant-pywebhooks

$ git clone https://github.com/chadlung/vagrant-pywebhooks
$ cd vagrant-pywebhooks
$ vagrant up

This will take a bit as RethinkDB and Redis are installed along with the master branch of PyWebhooks.

Once the virtual machine is up and provisioned run:

$ vagrant ssh

When your logged into the VM:

vagrant@pywebhooks-dev:~$ cd pywebhooks/
vagrant@pywebhooks-dev:~/pywebhooks$ export PYTHONPATH="/home/vagrant/pywebhooks:$PYTHONPATH"
vagrant@pywebhooks-dev:~$ python3 pywebhooks/app.py --initdb

Store the admin API and Secret keys that get printed out. Here’s an example output:

{'secret_key': '45e4cfddab848ed39ec818e6da2a83a1fae2ad79', 'api_key': '9ddcbde5c5f493877f39d7861c0f3ba975f92edf'}

Start the PyWebhooks API Server (using screen):

vagrant@pywebhooks-dev:~$ screen -S pywebhooks-server
vagrant@pywebhooks-dev:~/pywebhooks$ python3 pywebhooks/app.py

Now: Ctrl+a d (detaches from the screen session)

Start the Celery worker:

vagrant@pywebhooks-dev:~/pywebhooks$ screen -S celery-worker
vagrant@pywebhooks-dev:~/pywebhooks$ celery -A pywebhooks.tasks.webhook_notification worker --loglevel=info

Now: Ctrl+a d (detaches from the screen session)

Start the endpoint server:

vagrant@pywebhooks-dev:~/pywebhooks$ screen -S endpoint-server
vagrant@pywebhooks-dev:~/pywebhooks$ python3 pywebhooks/examples/endpoint_development_server.py

Now: Ctrl+a d (detaches from the screen session)

Verify the running screens:

vagrant@pywebhooks-dev:~/pywebhooks$ screen -ls

Expect output (should be somewhat similar):

vagrant@pywebhooks-dev:~/pywebhooks$ screen -ls
There are screens on:
	1382.endpoint-server	(09/09/2015 01:33:31 AM)	(Detached)
	1360.celery-worker	(09/09/2015 01:31:56 AM)	(Detached)
	1336.pywebhooks-server	(09/09/2015 01:30:05 AM)	(Detached)
3 Sockets in /var/run/screen/S-vagrant.

Create a new account:

vagrant@pywebhooks-dev:~/pywebhooks$ curl -v -X POST "http://127.0.0.1:8081/v1/account" -d '{"endpoint": "http://127.0.0.1:9090/account/endpoint", "username": "sarahfranks"}' -H "content-type: application/json"

Output:

{
  "api_key": "3f9d97014910817f348737546fe415b38ef8ed48",
  "endpoint": "http://127.0.0.1:9090/account/endpoint",
  "epoch": 1441755386.3142877,
  "failed_count": 0,
  "id": "4d075cfe-a238-426d-867f-3f261971f14b",
  "is_admin": false,
  "secret_key": "eddf3bf145a30eec9b49c1bd12ef165ac55fed75",
  "username": "sarahfranks"
}

List the new account (optional):

vagrant@pywebhooks-dev:~/pywebhooks$ curl -v -X GET "http://127.0.0.1:8081/v1/account/4d075cfe-a238-426d-867f-3f261971f14b" -H "content-type: application/json" -H "api-key: 3f9d97014910817f348737546fe415b38ef8ed48" -H "username: sarahfranks"

Output:

{
  "api_key": "pbkdf2:sha1:1000$NefOvB4f$6f76f2366c5af870ba592731df9ece577556fbf1",
  "endpoint": "http://127.0.0.1:9090/account/endpoint",
  "epoch": 1441755386.3142877,
  "failed_count": 0,
  "id": "4d075cfe-a238-426d-867f-3f261971f14b",
  "is_admin": false,
  "secret_key": "eddf3bf145a30eec9b49c1bd12ef165ac55fed75",
  "username": "sarahfranks"
}

Register (create) a webhook:

vagrant@pywebhooks-dev:~/pywebhooks$ curl -v -X POST "http://127.0.0.1:8081/v1/webhook/registration" -H "content-type: application/json" -H "username: sarahfranks" -H "api-key: 3f9d97014910817f348737546fe415b38ef8ed48" -d '{"description": "This is my registered webhook", "event_data": {"message": "hello world"}, "event": "mywebhook.event"}'

Output:

{
  "account_id": "4d075cfe-a238-426d-867f-3f261971f14b",
  "description": "This is my registered webhook",
  "epoch": 1441755520.0332994,
  "event": "mywebhook.event",
  "event_data": {
    "message": "hello world"
  },
  "id": "59948829-7526-4fcb-b6b9-56c1023124a6"
}

Make note of the id, in this case it was: 59948829-7526-4fcb-b6b9-56c1023124a6

Subscribe (in this case to our own) to the webhook:

curl -v -X POST "http://127.0.0.1:8081/v1/webhook/subscription/59948829-7526-4fcb-b6b9-56c1023124a6" -H "content-type: application/json" -H "api-key: 3f9d97014910817f348737546fe415b38ef8ed48" -H "username: sarahfranks"

Output:

{
  "account_id": "4d075cfe-a238-426d-867f-3f261971f14b",
  "epoch": 1441755658.4106872,
  "id": "84ff52d0-bcdd-4650-8b90-2ae9210a302f",
  "registration_id": "59948829-7526-4fcb-b6b9-56c1023124a6"
}

Trigger the webhook:

curl -v -X POST "http://127.0.0.1:8081/v1/webhook/triggered/59948829-7526-4fcb-b6b9-56c1023124a6" -H "content-type: application/json" -H "api-key: 3f9d97014910817f348737546fe415b38ef8ed48" -H "username: sarahfranks"

Switch to the endpoint server screen and your should see the triggered webhook came in (the secret key for the user is hardcoded in the endpoint server example file so make sure to update that if you want the hmac signature to work — return true)

vagrant@pywebhooks-dev:~/pywebhooks$ screen -r endpoint-server

Output:

127.0.0.1 - - [09/Sep/2015 01:36:26] "GET /account/endpoint?echo=54ff260ba5a3b4573cbc909a70945ff5ffaeb342 HTTP/1.1" 200 -
User-Agent: python-requests/2.7.0 CPython/3.4.0 Linux/3.16.0-30-generic
Accept-Encoding: gzip, deflate
Content-Type: application/json
Pywebhooks-Signature: Ada14C0onZem65rQJwC5vdjTSPg=
Host: 127.0.0.1:9090
Event: mywebhook.event
Connection: keep-alive
Accept: application/json
Content-Length: 26


b'{"message": "hello world"}'
Is Signature Valid?: False
127.0.0.1 - - [09/Sep/2015 01:43:40] "POST /account/endpoint HTTP/1.1" 200

Now: Ctrl+a d (detaches from the screen session)

You can visit the other screens easily:

screen -r pywebhooks-server
screen -r endpoint-server
screen -r celery-worker

This just showed a minimal example of creating an account, registering a webhook, subscribing to it, and then triggering it. There are many more API calls supported so make sure to look over the readme for more details.

You can stop the screen sessions above by attaching to each one and running: Ctrl+c and then Ctrl+a k

PyWebhooks is open source published under the Apache 2 license.

Post to Twitter

This entry was posted in Open Source, Python. Bookmark the permalink.

One Response to Introducing PyWebhooks: An Easy to Use Webhooks Service (Written in Python 3)

  1. Pingback: Trying out the Docker Compose support for PyWebhooks (a Python 3 webhooks server project) | Giant Flying Saucer

Comments are closed.