Setting up Repose: The REstful PrOxy Service Engine

From the Repose website: Repose is an open-source RESTful HTTP proxy service that scales to the cloud. Repose provides the solution to common API processing tasks such as rate limiting, client authentication, versioning, and logging so that web service developers can focus on the unique features of their services.

The Repose source code is available on GitHub. All files contained in Repose are licensed either under the Apache License v2.0 or the GNU General Public License v2.0.

If you were at the recent Openstack conference you may have heard of Repose already since the Repose client authentication component supports an OpenStack Identity Service authentication scheme.

Today I’ll walk through how to use a few of the Repose filters: Rate Limiting, HTTP Logging, Default Router, and IP Identity. Repose has several filters, so feel free to explore whats available. I’ll be setting up the standalone version of Repose but keep in mind you can also deploy it under an application server like Apache Tomcat.

For more detailed information you can check out the Repose documentation located here.

I’ll be setting this up on OS X Lion using the JDK 1.7 from Oracle. I have a previous article on how to get Java JDK 1.7 setup on OS X Lion. I’ll place Repose in front of Atom Hopper (Atom Hopper is a an AtomPub server). You can get more information about Atom Hopper here.

First, we will setup Atom Hopper. Download Apache Tomcat 7 and extract it in your home directory. Grab a copy of the latest Atom Hopper WAR file and place it into the Apache-Tomcat/webapps folder. Rename the Atom Hopper WAR file to ROOT.war. If ROOT already exists you can delete it, or you can simply modify the path later on in this article. If you have Netbeans 7 with Maven feel free to build your own copy of Repose and Atom Hopper if you wish.

You will need to create three folders for Atom Hopper:
/etc/atomhopper/
/opt/atomhopper/
/var/log/atomhopper/

Note: Make sure those folders allow permission for user starting Atom Hopper to have read and write access.

Grab a copy of the following files and copy them to the /etc/atomhopper/ folder:

• https://github.com/rackerlabs/atom-hopper/blob/master/atomhopper/src/main/webapp/META-INF/application-context.xml
• https://github.com/rackerlabs/atom-hopper/blob/master/atomhopper/src/main/webapp/META-INF/template-log4j.properties
• https://github.com/rackerlabs/atom-hopper/blob/master/atomhopper/src/main/resources/META-INF/atom-server.cfg.xml

We will just leave those config files as the default settings which will use H2 as the database and a single feed as: namespace/feed For more information on using Atom Hopper see the wiki or my previous articles.

At this point you can start Apache Tomcat by going into the Tomcat bin folder and running:

$ ./startup.sh

With Tomcat running Atom Hopper now we can test to ensure it’s working with the following URL: http://localhost:8080/namespace/feed/ You should just get some minimal default XML back since there is nothing in the database. Follow this article to learn how to put an ATOM entry into Atom Hopper.

Now we will focus on Repose.

Create the following folders and make sure the permissions are set that the user starting Repose has read and write access to those folders.
Configuration files: /etc/repose/
EAR file drop location: /usr/share/repose/filters/
Valve location (this is the standalone Repose location): /usr/share/lib/repose/
Log file location: /var/log/repose/
Deployment location (where the EAR is extracted): /var/repose/

In the configuration folder ( /etc/repose/ ) add all the following files at this location:
https://github.com/rackerlabs/repose/tree/master/project-set/core/core-lib/src/main/resources/META-INF/schema/examples

Add the IP identity config file ( /etc/repose/ ):
https://github.com/rackerlabs/repose/blob/master/project-set/components/client-ip-identity/src/main/resources/META-INF/schema/examples/ip-identity.cfg.xml

Add the rate limiting config file ( /etc/repose/ ):
https://github.com/rackerlabs/repose/blob/master/project-set/components/rate-limiting/src/main/resources/META-INF/schema/examples/rate-limiting.cfg.xml

Add the HTTP logging config file ( /etc/repose/ ):
https://github.com/rackerlabs/repose/blob/master/project-set/components/http-logging/src/main/resources/META-INF/schema/examples/http-logging.cfg.xml

In the EAR file drop folder ( /usr/share/repose/filters/ ) add the latest release from this location:
http://maven.research.rackspacecloud.com/content/repositories/releases/com/rackspace/papi/components/filter-bundle/

In the Valve location ( /usr/share/lib/repose/ ) copy in the latest Valve JAR release:
http://maven.research.rackspacecloud.com/content/repositories/releases/com/rackspace/papi/core/valve/

Since I have Tomcat with Atom Hopper running on port 8080 I’m going to set Repose to listen on port 8888. Keep in mind in a production environment you would most likely have Repose listening on port 80 and then forwarding to port 8080 to Tomcat.

I’ve modified the following config files in /etc/repose/

container.cfg.xml

<?xml version="1.0" encoding="UTF-8"?>

<repose-container xmlns='http://docs.rackspacecloud.com/repose/container/v2.0'>
    <deployment-config http-port="8888" connection-timeout="30000" read-timeout="30000">
        <deployment-directory auto-clean="false">/var/repose</deployment-directory>
        
        <artifact-directory check-interval="60000">/usr/share/repose/filters</artifact-directory>
        
        <logging-configuration href="log4j.properties"/>

    </deployment-config>
</repose-container>

http-logging.cfg.xml

<?xml version="1.0" encoding="UTF-8"?>

<http-logging xmlns="http://docs.rackspacecloud.com/repose/http-logging/v1.0">
	<!-- The id attribute is to help the user easily identify the log -->
	<!-- The format includes what will be logged.  The arguments with % are a subset of the apache mod_log_config
		 found at http://httpd.apache.org/docs/2.2/mod/mod_log_config.html#formats -->
	<http-log id="my-special-log" format="Response Code Modifiers=%200,201U\tModifier Negation=%!401a\tRemote IP=%a\tLocal IP=%A\tResponse Size(bytes)=%b\tRemote Host=%h\tRequest Method=%m\tServer Port=%p\tQuery String=%q\tTime Request Received=%t\tStatus=%s\tRemote User=%u\tURL Path Requested=%U\n">
		<targets>
			<!-- The actual log file -->
			<file location="/var/log/repose/repose.log"/>
		</targets>
	</http-log>
	
	<http-log id="my-other-log" format="Remote IP=%a Local IP=%A">
		<targets>
			<file location="/var/log/repose/repose_other.log"/>
		</targets>
	</http-log>
</http-logging>

ip-identity.cfg.xml

<?xml version="1.0" encoding="UTF-8"?>

<ip-identity  xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
   xmlns='http://docs.api.rackspacecloud.com/repose/ip-identity/v1.0'
   xsi:schemaLocation='http://docs.api.rackspacecloud.com/repose/ip-identity/v1.0'>
   
   <quality>0.2</quality>

    <white-list quality="1.0">
        <ip-address>127.0.0.1</ip-address>
    </white-list>
   
</ip-identity>

rate-limiting.cfg.xml

<?xml version="1.0" encoding="UTF-8"?>

<rate-limiting delegation="false" xmlns="http://docs.rackspacecloud.com/repose/rate-limiting/v1.0">
    <!--
        Defines an endpoint with a matching regex to bind GET requests for
        returning live rate limiting information.
    -->
    <request-endpoint uri-regex="/limits/" limits-format="OPENSTACK" />
    
    <!--
        Defining a limit group.

        The following headers can be found in the class
        com.rackspace.cloud.powerapi.http.PowerApiHeader in the Power API
        Filterlet library, maven group id com.rackspace.cloud.powerapi, artifact
        id filterlet.

        Groups are matched on the HTTP header: X-PP-Groups
        User information is matched on the HTTP header: X-PP-User
    -->
    <limit-group id="standard-ip-limits" groups="IP_Standard">
        <limit uri="/*" uri-regex="/(.*)" http-methods="GET" unit="SECOND" value="3" />
    </limit-group>

    <limit-group id="standard-ip-limits-superuser" groups="IP_Super">
        <limit uri="/*" uri-regex="/(.*)" http-methods="GET" unit="SECOND" value="5" />
    </limit-group>
</rate-limiting>

Finally, edit the system-model.cfg.xml (this is where you enable each filter you want to use):

<?xml version="1.0" encoding="UTF-8"?>

<system-model xmlns="http://docs.rackspacecloud.com/repose/system-model/v2.0">
  <repose-cluster id="repose">
    <nodes>
      <node id="node1" hostname="localhost" http-port="8888"/>
    </nodes>
    <filters>
      <!--
      <filter name="header-id-mapping" />
      -->
      <filter name="ip-identity" />
      <filter name="rate-limiting" />
      <filter name="http-logging" />
      <filter name="default-router"/>
    </filters>
    <destinations>
      <endpoint id="openrepose" protocol="http" hostname="localhost" root-path="/" port="8080" default="true"/>
    </destinations>
  </repose-cluster>
</system-model>

I won’t go through all the possibilities of how you can use HTTP Logging or Rate Limiting, etc. Refer to the docs for that information but suffice to say that Repose is extremely configurable.

Assuming everything is set correctly you can now run Repose:

$ java -jar valve-2.1.1.jar start -p 8888 -s 8188 -c /etc/repose/

The command above makes Repose listen on port 8888 (which is configured to proxy to Tomcat on port 8080). The port 8188 is the port Repose listens on for a shutdown command. This can done with a simple HTTP GET to this address: http://localhost:8188/ The final argument tells Repose to look in the /etc/repose/ folder for the configuration files.

Have a peek into the repose.log file once you’ve done an HTTP GET to Atom Hopper: http://localhost:8888/namespace/feed/

Response Code Modifiers=-	Modifier Negation=0:0:0:0:0:0:0:1%0	Remote IP=0:0:0:0:0:0:0:1%0	Local IP=0:0:0:0:0:0:0:1%0	Response Size(bytes)=2048	Remote Host=0:0:0:0:0:0:0:1%0	Request Method=GET	Server Port=8888	Query String=null	Time Request Received=04-06-2012-02:04:44.826	Status=304	Remote User=0:0:0:0:0:0:0:1%0;q=0.2	URL Path Requested=http://localhost:8080/namespace/feed

Response Code Modifiers=-	Modifier Negation=0:0:0:0:0:0:0:1%0	Remote IP=0:0:0:0:0:0:0:1%0	Local IP=0:0:0:0:0:0:0:1%0	Response Size(bytes)=2048	Remote Host=0:0:0:0:0:0:0:1%0	Request Method=GET	Server Port=8888	Query String=null	Time Request Received=04-06-2012-02:04:44.989	Status=404	Remote User=0:0:0:0:0:0:0:1%0;q=0.2	URL Path Requested=http://localhost:8080/favicon.ico

Play around with the rate limiting and rate limit yourself to see it in action.