Architektur

Developing FaaS with Oracle fn Project

fn-project

In my last post I wrote about general concepts behind Function as a Service (FaaS) and my view on this application development style. As an example I used the basic pipes and filters concept implemented by Unix Shells and many shell commands. Within this post I will use the same example and implement it using a Function as a Service platform. To recap the scenario, we had a simple JSON file containing booking records. From this file we identified certain records based on purchased products. Afterwards we extracted customer emails which could be used as a list for targeted marketing.

The FaaS platform I will be using is Oracle fn Project. It is an Open Source project which aims to develop an open and standard based FaaS Platform as alternative to mostly proprietary solutions existing in the market today. According to Oracle a managed service will be released in near future. Let us start and develop FaaS with Oracle fn Project!

Developing FaaS with Oracle fn Project

To begin with, the beauty of fn Project is that one is able to develop FaaS applications on any machine. This includes local development machines. From the decision to use fn to the first deployed function one needs less than nine shell commands. The mandatory „Hello World“ example can be found on fn Project’s GitHub repository. Just follow the instructions and come back to this post with a running fn server.

Now let us start to implement our FaaS example using fn Project. The first function should extract all booking records containing a specific product. Let us call this function searchProduct.

fn init --runtime go searchProduct

fn Project supports multiple languages. For our example we will be using Go. Why? Somehow everyone does and I wanted to test it myself!

So what did we get?

$ ls -lh searchProduct/
total 32
-rw-r--r--  1 dude  dudes   127B Jun 28 16:21 Gopkg.toml
-rw-r--r--  1 dude  dudes   1.5K Aug 16 17:51 func.go
-rw-r--r--  1 dude  dudes    79B Aug 16 17:51 func.yaml
-rw-r--r--  1 dude  dudes   505B Jun 28 16:21 test.json

As one can see fn init generated some files in the searchProduct folder, namely:

  • Gopkg.toml – Go package management configuration
  • func.go – Go file containing function code
  • func.yaml – Metadata required by fn Project
  • test.json – Test definition file

In general, only func.go content is relevant for our example. Replace it with the following code:

package main

import (
        "context"
        "encoding/json"
        "io"
        "strings"

        fdk "github.com/fnproject/fdk-go"
)

func main() {
        // Register handler function
        fdk.Handle(fdk.HandlerFunc(findBookingRecordsByProduct))
}

// Define booking record struct
type BookingRecordsStruct struct {
        ID        int    `json:"id"`
        FirstName string `json:"first_name"`
        LastName  string `json:"last_name"`
        Gender    string `json:"gender"`
        JobTitle  string `json:"job_title"`
        Contact   struct {
                Email   string `json:"email"`
                Address struct {
                        Street       string `json:"street"`
                        StreetNumber string `json:"street_number"`
                        PostalCode   string `json:"postal_code"`
                        City         string `json:"city"`
                } `json:"address"`
        } `json:"contact"`
        Booking struct {
                ID        int     `json:"id"`
                ItemID    int     `json:"item_id"`
                ItemTitle string  `json:"item_title"`
                Quantity  int     `json:"quantity"`
                Price     float64 `json:"price"`
                Currency  string  `json:"currency"`
        } `json:"booking"`
}

func findBookingRecordsByProduct(ctx context.Context, in io.Reader, out io.Writer) {
        // Create fn specific context object
        var fnctx = fdk.Context(ctx)

        // Get ProductName HTTP Header value
        var productName = fnctx.Header.Get("ProductName")
        var bookingRecords []BookingRecordsStruct
        var resultBookingRecords []BookingRecordsStruct = make([]BookingRecordsStruct, 0)

        // Decode request payload into BookingRecordsStruct
        json.NewDecoder(in).Decode(&bookingRecords)

        // Loop through booking records
        for _, bookingRecord := range bookingRecords {
                // Check if record contains searched product
                if strings.Contains(bookingRecord.Booking.ItemTitle, productName) {
                        // Add record to result struct
                        resultBookingRecords = append(resultBookingRecords, bookingRecord)
                }
        }

        // Encode result and write to output stream
        json.NewEncoder(out).Encode(&resultBookingRecords)
}

Now that we got our searchProduct function implementation in place, we need to build and deploy it.

$ fn build
$ fn deploy --app targetedMarketing --local
Deploying searchproduct to app: targetedMarketing at path: /searchProduct
Bumped to version 0.0.8
Building image searchproduct:0.0.8 .......
Updating route /searchProduct using image searchproduct:0.0.8...
$ fn ls routes targetedMarketing
PATH		IMAGE			ENDPOINT
/searchProduct	searchproduct:0.0.8	localhost:8080/r/targetedMarketing/searchProduct

The fn deploy command created a new application named targetedMarketing and deployed the function on our local fn Project server. Furthermore, the function version got increased automatically within our development environment. As the last command shows it is now reachable under http://localhost:8080/r/targetedMarketing/searchProduct. So let us call the function using curl.

$ curl -X POST -H "ProductName: Bacardi Raspberry" -d @./test_data/MOCK_DATA.json http://localhost:8080/r/targetedMarketing/searchProduct
[{"id":555,"first_name":"Read","last_name":"Whorlow","gender":"Male","job_title":"Structural Engineer","contact":{"email":"rwhorlowfe@un.org","address":{"street":"Gulseth","street_number":"162","postal_code":"4821","city":"Flagstaff"}},"booking":{"id":5,"item_id":20,"item_title":"Bacardi Raspberry","quantity":4,"price":118.93,"currency":"ZAR"}},{"id":900,"first_name":"Angelia","last_name":"MacKnocker","gender":"Female","job_title":"Food Chemist","contact":{"email":"amacknockeroz@diigo.com","address":{"street":"Fuller","street_number":"71","postal_code":"756 06","city":"Velké Karlovice"}},"booking":{"id":2,"item_id":22,"item_title":"Bacardi Raspberry","quantity":3,"price":118.99,"currency":"CZK"}}]

Congratulations, you just created your first function using Oracle fn Project and it does what we intended to do. Compared to the grep solution from my previous post the result actually improved as it produced valid JSON Output. Its structure is identical to the input JSON structure. So lets tackle our second function, extractEmail. Create a new project just like before and replace the generated code:

package main
  
import (
        "context"
        "encoding/json"
        "io"

        fdk "github.com/fnproject/fdk-go"
)

func main() {
        fdk.Handle(fdk.HandlerFunc(extractEmail))
}

type BookingRecords []struct {
        ID        int    `json:"id"`
        FirstName string `json:"first_name"`
        LastName  string `json:"last_name"`
        Gender    string `json:"gender"`
        JobTitle  string `json:"job_title"`
        Contact   struct {
                Email   string `json:"email"`
                Address struct {
                        Street       string `json:"street"`
                        StreetNumber string `json:"street_number"`
                        PostalCode   string `json:"postal_code"`
                        City         string `json:"city"`
                } `json:"address"`
        } `json:"contact"`
        Booking struct {
                ID        int     `json:"id"`
                ItemID    int     `json:"item_id"`
                ItemTitle string  `json:"item_title"`
                Quantity  int     `json:"quantity"`
                Price     float64 `json:"price"`
                Currency  string  `json:"currency"`
        } `json:"booking"`
}

func extractEmail(ctx context.Context, in io.Reader, out io.Writer) {
        var bookingRecords BookingRecords

        json.NewDecoder(in).Decode(&bookingRecords)

        var emailAddresses []string = make([]string, 0)

        for _, bookingRecord := range bookingRecords {
                emailAddresses = append(emailAddresses, bookingRecord.Contact.Email)
        }

        json.NewEncoder(out).Encode(&emailAddresses)
}

As one can see in the code, yet again the same BookingRecords structure is used for input. Now let us build and deploy this function as well.

$ fn deploy --app targetedMarketing --local
Deploying extractemail to app: targetedMarketing at path: /extractEmail
Bumped to version 0.0.3
Building image extractemail:0.0.3 ......
Updating route /extractEmail using image extractemail:0.0.3...
$ fn ls routes targetedMarketing
PATH		IMAGE			ENDPOINT
/extractEmail	extractemail:0.0.3	localhost:8080/r/targetedMarketing/extractEmail
/searchProduct	searchproduct:0.0.8	localhost:8080/r/targetedMarketing/searchProduct

We deployed the second function to the same application. As result we now have two routes within this application. If we now call our extractEmail function with the same data as searchProduct it will return a JSON Array containing all 1000 email addresses within this file. However, our requirement was to identify certain customers. In order to fulfil this requirement we need to chain our functions. As searchProduct uses the same input and output structure, this is no problem and we can reuse shell pipes in order to achieve our desired result.

$ curl -X POST -H "ProductName: Bacardi Raspberry" -d @./test_data/MOCK_DATA.json http://localhost:8080/r/targetedMarketing/searchProduct | curl -X POST -d @- http://localhost:8080/r/targetedMarketing/extractEmail
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  338k  100   708  100  338k   1223   584k --:--:-- --:--:-- --:--:--  584k
["rwhorlowfe@un.org","amacknockeroz@diigo.com"]

The command line above uses curl to call our searchProduct function and get a list of identified customers. Afterwards the result is piped into another curl command and extractEmail is called, which returns two email addresses. We now implemented the same functionality using the same principles as in the previous post. However this time we used a FaaS platform to fulfil our requirements. As you can see pipes and filters can be easily applied to FaaS development and just feels right.

However, in our current solution we still do the piping using shell functionality. Of course this does not seem right when aiming for enterprise application development. So how is the piping requirement addressed within fn Project? This topic will be elaborated in my next post.

Further Reading

Blog Post 1: Function as a Service (FaaS)
fn project – Oracle’s open FaaS Platform
Source Code Examples on GitHub.com