Managed services in Stackato
Daniel Watrous This post is an extension of Managed services in CloudFoundry and follows the discussion of external services integration. The echo service and python service broker implementation are deployed using the documented procedure for Stackato. If necessary, you can get a Stackato instance up and running quickly. The result should be two apps deployed as shown below.
Optional validation
It is possible to validate that the deployed apps work as expected. The curl commands below validate the echo service is working properly.
watrous@watrous-helion:~$ curl -X PUT http://echo-service.stackato.danielwatrous.com/echo/51897770-560b-11e4-b75a-9ad2017223a9 -w "\n" {"instance_id": "51897770-560b-11e4-b75a-9ad2017223a9", "dashboard_url": "http://localhost:8090/dashboard/51897770-560b-11e4-b75a-9ad2017223a9", "state": "provision_success"} watrous@watrous-helion:~$ curl -X PUT http://echo-service.stackato.danielwatrous.com/echo/51897770-560b-11e4-b75a-9ad2017223a9/myapp -w "\n" {"state": "bind_success", "app": "myapp", "id": "51897770-560b-11e4-b75a-9ad2017223a9"} watrous@watrous-helion:~$ curl -X POST http://echo-service.stackato.danielwatrous.com/echo/51897770-560b-11e4-b75a-9ad2017223a9/myapp -H "Content-Type: application/json" -d '{"message": "Hello World"}' -w "\n" {"response": "Hello World"} watrous@watrous-helion:~$ curl -X POST http://echo-service.stackato.danielwatrous.com/echo/51897770-560b-11e4-b75a-9ad2017223a9/myapp -H "Content-Type: application/json" -d '{"message": "Hello World 2"}' -w "\n" {"response": "Hello World 2"} watrous@watrous-helion:~$ curl -X POST http://echo-service.stackato.danielwatrous.com/echo/51897770-560b-11e4-b75a-9ad2017223a9/myapp -H "Content-Type: application/json" -d '{"message": "Hello Worldz!"}' -w "\n" {"response": "Hello Worldz!"} watrous@watrous-helion:~$ curl -X GET http://echo-service.stackato.danielwatrous.com/echo/dashboard/51897770-560b-11e4-b75a-9ad2017223a9 <table class="pure-table"> <thead> <tr> <th>Instance</th> <th>Bindings</th> <th>Messages</th> </tr> </thead> <tbody> <tr> <td>51897770-560b-11e4-b75a-9ad2017223a9</td> <td>1</td> <td>3</td> </tr> </tbody> </table> watrous@watrous-helion:~$ curl -X DELETE http://echo-service.stackato.danielwatrous.com/echo/51897770-560b-11e4-b75a-9ad2017223a9/myapp -w "\n" {"state": "unbind_success", "app": "myapp", "id": "51897770-560b-11e4-b75a-9ad2017223a9"} watrous@watrous-helion:~$ curl -X DELETE http://echo-service.stackato.danielwatrous.com/echo/51897770-560b-11e4-b75a-9ad2017223a9 -w "\n" {"state": "deprovision_success", "id": "51897770-560b-11e4-b75a-9ad2017223a9", "messages": 3, "bindings": 0} |
The service broker API implementation can also be verified using curl as shown below.
watrous@watrous-helion:~$ curl -X GET http://service-broker.stackato.danielwatrous.com/v2/catalog -H "X-Broker-Api-Version: 2.3" -H "Authorization: Basic dXNlcjpwYXNz" -w "\n" {"services": [{"plans": [{"name": "large", "description": "A large dedicated service with a big storage quota, lots of RAM, and many connections", "id": "big_0001", "free": false}], "description": "Echo back the value received", "id": "echo_service", "bindable": true, "name": "Echo Service", "dashboard_client": {"redirect_uri": "http://echo-service.stackato.danielwatrous.com/echo/dashboard", "id": "client-id-1", "secret": "secret-1"}}, {"plans": [{"name": "small", "description": "A small shared service with a small storage quota and few connections", "id": "small_0001"}], "description": "Invert the value received", "id": "invert_service", "bindable": true, "name": "Invert Service", "dashboard_client": null}]} watrous@watrous-helion:~$ curl -X PUT http://service-broker.stackato.danielwatrous.com/v2/service_instances/mynewinstance -H "Content-type: application/json" -H "Authorization: Basic dXNlcjpwYXNz" -d '{"service_id": "echo_service", "plan_id": "small_0001", "organization_guid": "HP", "space_guid": "IT"}' -w "\n" {"dashboard_url": "http://echo-service.stackato.danielwatrous.com/echo/dashboard/mynewinstance"} watrous@watrous-helion:~$ curl -X PUT http://service-broker.stackato.danielwatrous.com/v2/service_instances/mynewinstance/service_bindings/myappid -H "Content-type: application/json" -H "Authorization: Basic dXNlcjpwYXNz" -d '{"service_id": "echo_service", "plan_id": "small_0001", "app_guid": "otherappid"}' -w "\n" {"credentials": {"uri": "http://echo-service.stackato.danielwatrous.com/echo/mynewinstance/myappid"}} watrous@watrous-helion:~$ curl -X DELETE http://service-broker.stackato.danielwatrous.com/v2/service_instances/mynewinstance/service_bindings/myappid -H "Authorization: Basic dXNlcjpwYXNz" -w "\n" {} watrous@watrous-helion:~$ curl -X DELETE http://service-broker.stackato.danielwatrous.com/v2/service_instances/mynewinstance -H "Authorization: Basic dXNlcjpwYXNz" -w "\n" {} |
Manage the Service Broker
Service brokers can be managed in Stackato using the command line client. After issuing the add-service-broker command, several prompts as for details about the service broker. The username and password values should be credentials for the service broker, not your stackato credentials. The python service broker used in this example will accept any combination of username and password. The brokers can then be listed out using the service-brokers command.
C:\Users\watrous\Documents\GitHub\cf-service-broker-python>stackato add-service-broker Enter name: echo-broker Enter url: http://service-broker.stackato.danielwatrous.com Enter username: user Enter password: pass Creating new service broker [echo-broker] ... OK C:\Users\watrous\Documents\GitHub\cf-service-broker-python>stackato service-brokers +-------------+--------------------------------------------------+ | Name | Url | +-------------+--------------------------------------------------+ | echo-broker | http://service-broker.stackato.danielwatrous.com | +-------------+--------------------------------------------------+ |
Make the service plan public
Before the service can be provisioned by apps, it must be made public. This can be done using the update-service-plan command with the stackato client.
C:\Users\watrous\Documents\echogo>stackato update-service-plan large --vendor "Echo Service" --public --free Updating service plan [large.Echo Service] ... Name (large): Description (A large dedicated service with a big storage quota, lots of RAM, and many connections): Changed public to "1" Changed free to "1" OK |
From the command line, it is possible to list available services with the marketplace command. This shows the Echo Service that was just made public.
C:\Users\watrous\Documents\GitHub\cf-service-broker-python>stackato marketplace https://api.stackato.danielwatrous.com -> Test -> somespace +--------------+-------+-------------------------------+---------------------------------------------------------------------------------------+------+--------+----------+---------+------+ | Vendor | Plan | Description | Details | Free | Public | Provider | Version | Orgs | +--------------+-------+-------------------------------+---------------------------------------------------------------------------------------+------+--------+----------+---------+------+ | filesystem | free | Persistent filesystem service | free | yes | yes | core | 1.0 | | | mysql | free | MySQL database service | free | yes | yes | core | 5.5 | | | postgresql | free | PostgreSQL database service | free | yes | yes | core | 9.1 | | | Echo Service | large | Echo back the value received | A large dedicated service with a big storage quota, lots of RAM, and many connections | yes | yes | | | | +--------------+-------+-------------------------------+---------------------------------------------------------------------------------------+------+--------+----------+---------+------+ |
The Stackato web interface also shows the services provided by the broker under Admin -> Services -> Available Services. Notice that this shows both public and non-public services.
Manage Services
With the new service in the marketplace, it’s now possible to provision, bind and use instances of the new service by way of the stackato client. The most simple way to accomplish this is to call create-service and Provide the name of the service. Calling stackato services shows all service plans (including non-public) and provisioned services.
C:\Users\watrous\Documents\GitHub\cf-service-broker-python>stackato create-service "Echo Service" 1. large: A large dedicated service with a big storage quota, lots of RAM, and many connections Please select the service plan to enact: 1 Creating new service [Echo Service-399dc] ... OK C:\Users\watrous\Documents\GitHub\cf-service-broker-python>stackato services ============== Service Plans ================ +----------------+-------+-------------------------------+---------------------------------------------------------------------------------------+------+--------+----------+---------+------+ | Vendor | Plan | Description | Details | Free | Public | Provider | Version | Orgs | +----------------+-------+-------------------------------+---------------------------------------------------------------------------------------+------+--------+----------+---------+------+ | filesystem | free | Persistent filesystem service | free | yes | yes | core | 1.0 | | | mysql | free | MySQL database service | free | yes | yes | core | 5.5 | | | postgresql | free | PostgreSQL database service | free | yes | yes | core | 9.1 | | | Echo Service | large | Echo back the value received | A large dedicated service with a big storage quota, lots of RAM, and many connections | yes | yes | | | | | Invert Service | small | Invert the value received | A small shared service with a small storage quota and few connections | yes | no | | | | +----------------+-------+-------------------------------+---------------------------------------------------------------------------------------+------+--------+----------+---------+------+ =========== Provisioned Services ============ +-----------------+-----------------------+--------------+----------+---------+-------+----------------+ | Space | Name | Service | Provider | Version | Plan | Applications | +-----------------+-----------------------+--------------+----------+---------+-------+----------------+ | Test::somespace | Echo Service-399dc | Echo Service | | | large | | +-----------------+-----------------------+--------------+----------+---------+-------+----------------+ |
Push an app
It is now possible to bind apps to services. There are two ways to do this. The first is to provision a new instance of the service when the app is pushed. The other is to bind an existing service to an app when it is pushed. Both options come as prompts during the push process.
The service details will be injected into the app environment in a JSON document under the key VCAP_SERVICES. The structure of the service details is shown below.
{
"Echo Service": [
{
"name": "Echo Service-399dc",
"label": "Echo Service",
"tags": [
],
"plan": "large",
"credentials": {
"uri": "http://echo-service.stackato.danielwatrous.com/echo/db5e119a-823b-4368-a8d6-ff5f0cc90cb9/85629bfa-736b-42fd-9554-3f5df0bea097"
}
}
]
} |
For this example, a simple Go script illustrates how to interact with the environment to extract the service connection details, then to make a call to the service.
package main import ( "bytes" "fmt" "io/ioutil" "log" "net/http" "os" "strings" "encoding/json" "io" ) func get_echo_uri() string { vcap_services := os.Getenv("VCAP_SERVICES") log.Print(vcap_services) decoder := json.NewDecoder(strings.NewReader(vcap_services)) type Service struct { Echo_Service []struct { Credentials struct { Uri string `json:"uri"` } `json:"credentials"` Label string `json:"label"` Name string `json:"name"` Plan string `json:"plan"` Tags []interface{} `json:"tags"` } `json:"Echo Service"` } var service Service for { if err := decoder.Decode(&service); err == io.EOF { break } else if err != nil { log.Fatal(err) } } return service.Echo_Service[0].Credentials.Uri } func vcap_json_decode(w http.ResponseWriter, r *http.Request) { uri := get_echo_uri() fmt.Fprintf(w, "URI: %s\n", uri) } func environment(w http.ResponseWriter, r *http.Request) { fmt.Fprintf(w, "<a href=\"/decode\">decode</a>\n") fmt.Fprintf(w, "<a href=\"/echo\">echo</a>\n") for _, e := range os.Environ() { pair := strings.Split(e, "=") fmt.Fprintf(w, "%s = %s\n", pair[0], pair[1]) } } func echo(w http.ResponseWriter, r *http.Request) { uri := get_echo_uri() var message = []byte(`{"message": "Hello Go!"}`) req, err := http.NewRequest("POST", uri, bytes.NewBuffer(message)) req.Header.Set("Content-Type", "application/json") client := &http.Client{} resp, err := client.Do(req) if err != nil { panic(err) } defer resp.Body.Close() fmt.Fprintf(w, "response Status: %s\n", resp.Status) fmt.Fprintf(w, "response Headers: %s\n", resp.Header) body, _ := ioutil.ReadAll(resp.Body) fmt.Fprintf(w, "response Body: %s\n", string(body)) } func main() { http.HandleFunc("/", environment) http.HandleFunc("/decode", vcap_json_decode) http.HandleFunc("/echo", echo) addr := ":" + os.Getenv("PORT") fmt.Printf("Listening on %v\n", addr) log.Fatal(http.ListenAndServe(addr, nil)) } |
Additional requirements
In order to push this script to Stackato, a few additional files are required: Procfile, manifest.yml and .godir. The Procfile tells Stackato which command to use to start the application. The manifest indicates the Go version and application name. The .godir file contains a path (a name really) that will be used to create the binary version of the application during the staging process.
Download all echo application files as a zip
When the app is pushed, the service is created, bound and injected directly.
C:\Users\watrous\Documents\echogo>stackato push Would you like to deploy from the current directory ? [Yn]: Using manifest file "manifest.yml" Application Deployed URL [go-environment.stackato.danielwatrous.com]: Application Url: https://go-environment.stackato.danielwatrous.com Enter Disk Reservation [2048]: Enter GOVERSION [1.2]: Adding Environment Variable [GOVERSION=1.2] Creating Application [go-environment] as [https://api.stackato.danielwatrous.com -> Test -> somespace -> go-environment] ... OK Map https://go-environment.stackato.danielwatrous.com ... OK Bind existing services to 'go-environment' ? [yN]: Create services to bind to 'go-environment' ? [yN]: y What kind of service ? 1. free.filesystem 2. free.mysql 3. free.postgresql 4. large.Echo Service 5. small.Invert Service Choose: 4 Specify the name of the service [large.Echo Service-b8309]: Creating new service [large.Echo Service-b8309] ... OK Binding large.Echo Service-b8309 to go-environment ... OK Create another ? [yN]: Uploading Application [go-environment] ... Checking for bad links ... OK Copying to temp space ... OK Checking for available resources ... OK Processing resources ... OK Packing application ... OK Uploading (10K) ... OK Push Status: OK Starting Application [go-environment] ... OK http://go-environment.stackato.danielwatrous.com/ deployed |
The bound service instance is shown in the Services tab on the application page in the Stackato web interface.
The new application can be loaded to confirm that the service is working and properly injected into the application environment.
References
ASIDE: I found it unsatisfying to work with JSON in Go. Many languages, like PHP and Python, find clean ways of mapping JSON on to core data types, such as associative arrays or dictionaries. The need to define structs in Go to accommodate the anticipated JSON document felt like a throwback to XML and DTDs. Here are some references and it’s possible there’s a more elegant or sophisticated way to accomplish what I’ve done. If so, leave it in the comments below.
http://mervine.net/json2struct
http://golang.org/pkg/encoding/json/
http://blog.golang.org/json-and-go
https://github.com/ChimeraCoder/gojson