Installing Apache Kafka and using Python 3 to communicate with it

Apache Kafka is being used a fair amount these days and where I work is no exception. Kafka is getting a lot done in the highly scaled ‘Cloud’ and elsewhere. Getting a developer instance of Kafka up and running can be a little daunting for those new to it so I’ve come up with an easy way to get your development environment up and running in very little time. We will test out the single node Kafka server using Python 3 and kafka-python.

Continue reading

Posted in Open Source, Python | Leave a comment

Tutorial: Building a Minecraft Microservice with Python 3

Probably many of you have heard of Microservices. They are essentially a small single-purpose, API accessible (typically REST) based application. You can build your microservices in a variety of programming languages or just one depending on your needs. Today I want to guide you through a Python 3 microservice that focuses on one thing and is self-contained in the sense of not requiring any external dependencies that need to be installed via PIP.

Continue reading

Posted in Open Source, Python | Leave a comment

Easily setting up a Salt Master and Minion using Vagrant

I wrote a couple articles previously on getting started with Salt (see here and here). Those articles are good for learning however once you know the general idea of what it takes to install Salt (well one way to install it) sometimes you just want to quickly spin up and Salt master and a minion or two to experiment with. If that sounds like something you would use then go ahead and checkout my repo vagrant-salt-starter. Simply follow the instructions in the readme and you’ll have a Salt master and minion running in no time.

Posted in Open Source, Salt | Leave a comment

Introducing ViperPy – A Python library for interacting with the EMC ViPR API

ViperPy is a Python library for interacting with the EMC ViPR API. ViperPy began originally as an easy way to hook up numerous projects that needed to interact with the ViPR product in order to perform certain metering and billing extraction and handle authentication tokens behind the scenes. Very quickly I realized that it could be useful to others in my group if I added support for additional APIs and certain other features. ViperPy gives a great alternative to the existing Java SDK for ViPR if you would prefer to write your application in Python.

Continue reading

Posted in Open Source, Python | Leave a comment

Building a Python 3 REST application that scales “good enough”

Often developers get caught up in discussions about scaling their applications – usually before one line of code has even been written. Of course its helpful to keep the scalability of an application in mind, it however shouldn’t be a primary focus – at least initially on (most) new applications. I say this simply because the vast majority of applications out there will run just fine under most scenarios with minimal horsepower behind them. Most of us are not building Reddit, Twitter, or Amazon. In fact, you’d be surprised at how even some large projects like Twitter started out and eventually scaled. Suffice to say no at Twitter originally probably could’ve prefigured out how to have scaled Twitter to the levels its at now. Many times you’ll find that handling a few hundred requests per second will handle many corporate (or otherwise) workloads and you’ll never need to go beyond that.

Continue reading

Posted in Open Source, Python | Leave a comment

Python Multiprocessing: A Second Look

A while back I blogged about Python’s multiprocessing. I was talking with a friend the other day and was looking over his team’s Github repos and came across an interesting use of the multiprocessing package. I broke the original code down into a very small and easy to use example that you can use to power your own creations. Its not really anything somebody couldn’t figure out on their own, but my blog has always been popular with people new to programming so I think this could help new Python developers out.

Continue reading

Posted in Open Source, Python | Leave a comment

Clustering Graphite – The Easy Way

Graphite is used extensively by many companies. Right now, I’m working on a large Cloud based project that collects metering/billing data from several sources like EMC ViPR, OpenStack, etc. In order to quickly generate reports we cache some of the metering data inside Graphite. Today, I’ll go over how to setup a simple Graphite cluster and test it using Vagrant.

Continue reading

Posted in Open Source, OpenStack, Python, Ubuntu | Leave a comment

The two minute guide to using Viper – configuration management with Go (golang)

A per the website “Viper is a complete configuration solution. Designed to work within an application to handle file based configuration and seamlessly marry that with command line flags which can also be used to control application behavior”. You can load configuration files in: YAML, TOML or JSON. Viper allows you to easily add configuration management to your Go (golang) based projects.

Continue reading

Posted in Go, Open Source | Leave a comment

Getting started with etcd and Python

In a nutshell etcd is an open-source distributed key value store and is written in Go. You can run a single instance but it really shines when you set it up in a cluster and it will gracefully handle master election during network partitions and the loss of the current master. Today we are going to install etcd, use the CLI tool it ships with, and then do some simple Python scripts to interact with etcd.

Continue reading

Posted in Docker, Open Source, Python, Ubuntu | 1 Comment

Isolating your Ruby gems to a local project folder with a sample Sinatra app

In my previous blog post I wrote about installing rbenv on OS X 10.9 (Mavericks) to manage multiple Ruby installations easily. Today, I’m going to go over how I setup an initial Sinatra application while keeping the installed gems in the Ruby project and not going into the default system’s gem repository. I work a lot with Python so I tend to like how the separation of installed dependencies are handled using virtualenv and kept in the local project’s isolated instance. For more information on that see my blog post here.

Continue reading

Posted in Open Source, Ruby | Leave a comment