Installing Hashicorp’s Consul and using Python 3 to communicate with it

Hashicorp builds several great tools and I’m going to focus on one in particular today called Consul. In particular one of the many ways to utilize Consul is it’s ability to support service discovery, failure detection and to be multi-datacenter aware for things like microservices (or other types of projects). Consul also has a sophisticated key/value storage feature in addition to the other features. Today I want to cover how to easily get a small two node Consul cluster up and running and use Python to do some simple communication with it.

Continue reading

Posted in Open Source, Python | Leave a comment

Exploring Python 3’s Asyncio by Example

In spring 2014 Python 3.4 shipped a provisional package (asyncio) which according to the docsprovides infrastructure for writing single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, running network clients and servers, and other related primitives“. I can’t possibly cover everything in this article but I can introduce some of the things you can do with it. As per my New’s Years resolution I’ll be building these examples using Python 3.4.2 (Asyncio has been ported back to Python 3.3 now as well).

Continue reading

Posted in Open Source, Python | 2 Comments

Installing Apache Kafka and using Python 3 to communicate with it

Apache Kafka is being used a fair amount these days and where I work is no exception. Kafka is getting a lot done in the highly scaled ‘Cloud’ and elsewhere. Getting a developer instance of Kafka up and running can be a little daunting for those new to it so I’ve come up with an easy way to get your development environment up and running in very little time. We will test out the single node Kafka server using Python 3 and kafka-python.

Continue reading

Posted in Open Source, Python | Leave a comment

Tutorial: Building a Minecraft Microservice with Python 3

Probably many of you have heard of Microservices. They are essentially a small single-purpose, API accessible (typically REST) based application. You can build your microservices in a variety of programming languages or just one depending on your needs. Today I want to guide you through a Python 3 microservice that focuses on one thing and is self-contained in the sense of not requiring any external dependencies that need to be installed via PIP.

Continue reading

Posted in Open Source, Python | 1 Comment

Easily setting up a Salt Master and Minion using Vagrant

I wrote a couple articles previously on getting started with Salt (see here and here). Those articles are good for learning however once you know the general idea of what it takes to install Salt (well one way to install it) sometimes you just want to quickly spin up and Salt master and a minion or two to experiment with. If that sounds like something you would use then go ahead and checkout my repo vagrant-salt-starter. Simply follow the instructions in the readme and you’ll have a Salt master and minion running in no time.

Posted in Open Source, Salt | Leave a comment

Introducing ViperPy – A Python library for interacting with the EMC ViPR API

ViperPy is a Python library for interacting with the EMC ViPR API. ViperPy began originally as an easy way to hook up numerous projects that needed to interact with the ViPR product in order to perform certain metering and billing extraction and handle authentication tokens behind the scenes. Very quickly I realized that it could be useful to others in my group if I added support for additional APIs and certain other features. ViperPy gives a great alternative to the existing Java SDK for ViPR if you would prefer to write your application in Python.

Continue reading

Posted in Open Source, Python | Leave a comment

Building a Python 3 REST application that scales “good enough”

Often developers get caught up in discussions about scaling their applications – usually before one line of code has even been written. Of course its helpful to keep the scalability of an application in mind, it however shouldn’t be a primary focus – at least initially on (most) new applications. I say this simply because the vast majority of applications out there will run just fine under most scenarios with minimal horsepower behind them. Most of us are not building Reddit, Twitter, or Amazon. In fact, you’d be surprised at how even some large projects like Twitter started out and eventually scaled. Suffice to say no at Twitter originally probably could’ve prefigured out how to have scaled Twitter to the levels its at now. Many times you’ll find that handling a few hundred requests per second will handle many corporate (or otherwise) workloads and you’ll never need to go beyond that.

Continue reading

Posted in Open Source, Python | Leave a comment

Python Multiprocessing: A Second Look

A while back I blogged about Python’s multiprocessing. I was talking with a friend the other day and was looking over his team’s Github repos and came across an interesting use of the multiprocessing package. I broke the original code down into a very small and easy to use example that you can use to power your own creations. Its not really anything somebody couldn’t figure out on their own, but my blog has always been popular with people new to programming so I think this could help new Python developers out.

Continue reading

Posted in Open Source, Python | Leave a comment

Clustering Graphite – The Easy Way

Graphite is used extensively by many companies. Right now, I’m working on a large Cloud based project that collects metering/billing data from several sources like EMC ViPR, OpenStack, etc. In order to quickly generate reports we cache some of the metering data inside Graphite. Today, I’ll go over how to setup a simple Graphite cluster and test it using Vagrant.

Continue reading

Posted in Open Source, OpenStack, Python, Ubuntu | Leave a comment

The two minute guide to using Viper – configuration management with Go (golang)

A per the website “Viper is a complete configuration solution. Designed to work within an application to handle file based configuration and seamlessly marry that with command line flags which can also be used to control application behavior”. You can load configuration files in: YAML, TOML or JSON. Viper allows you to easily add configuration management to your Go (golang) based projects.

Continue reading

Posted in Go, Open Source | Leave a comment