Software Engineering

Is it insane to use BPMN to create serverless workflows?

Last week my colleague Carsten Wiesbaum introduced Oracle’s fn Flow as a way to orchestrate Function-as-as-Service (FaaS) functions in his last article. Fn Flow claims to be code-driven. Carsten states that he is not convinced by the design decision that flows should be defined in code and nothing else. That made me think about the different ways of defining serverless workflows.

Let’s have a quick look at the example code-driven workflow definition in the official fn GitHub repository of fn Flow.

Can you understand what is going on at a glance? I personally cannot.

The Natural Way

Here is a feature description from the official fn Flow tutorial “Flow 101”:

“Flow does not rely on massive yaml or json descriptors or visual graph-designers. Workflows are defined in code, expressed as a function, naturally.”

I don’t think defining workflows in code is natural at all. Is it only me? I conducted a little anonymous survey to find out. I handed out the following assignment to people with different software development backgrounds:

“This is a small experiment. I can’t tell you what it is about.

You are given the contact information about a customer including the customer number, postal address, e-mail address, cell phone number. Please describe the following flow of actions as precisely and self-explanatory as possible:

„First, send a letter to the customer. Then, only if the customer number is an even number, send an e-mail and an SMS to the customer at the same time. You may use programming languages, pseudo languages, DSL (with JSON, YAML, etc), graphs, charts or tables.“

What do you think happened? Here are the anonymous results from four colleagues who have different focuses: data science, software architecture and backend. As a bonus, my wife, who doesn’t have any software development experience, also participated. Can you guess which result is hers?

A visual workflow

A code-driven workflow

A visual workflow

A visual workflow

A visual workflow

Only one person went with pseudocode. His comment „not same time“ points out that his code would not fulfill the requirement of executing both functions at the same time. Not a problem. Fn Flow has an API to define concurrent function calls and a lot of more features. But the versatility of code-driven workflows comes at the expense of readability and learnability. Even in this small scope, I think my little experiment shows that people prefer to design their workflows visually.

Reinventing BPMN for serverless workflows

Fn Flow comes with an experimental UI to visualize currently running and already executed flows. Netflix‘ Conductor, which introduces a JSON-based DSL to define flows, also has a similar UI. Clearly workflow engine producers also think that the visualization of workflows is important. So why do we make the round trip by using another language to define the workflows and then visualizing them afterwards, instead of defining them in a visual notation in the first place?

AWS does that. AWS Step functions provide a visual editor to create state machines based on the JSON-based Amazon States Language. The language has even concepts for timers and exceptions. There may be other visual workflow designers that work in a similar way. Doesn’t that look like reinventing Business Process Model and Notation (BPMN)?

Wouldn’t it be nice if we could use the well established ISO standard BPMN to create serverless workflows? BPMN has a great and growing tooling support. A lot of software developers have already worked with BPMN and can start using it out of the box instead of learning a new notation. Most importantly, BPMN has already standardized concepts for timers, compensation as in the SAGA pattern, hierarchical composition and versioning.

What about performance and scalability?

One concern about BPMN in the serverless world may be limited performance and scalability. If you are planning to have a serverless architecture, you naturally wouldn’t want an orchestration engine to limit your scalability and performance. AWS Step functions live serverless in the cloud whereas BPM engines must sit in an application server with a relational database, right? No. That might be the case for some BPM engines that are out there.  However, it is important to point out that BPMN itself is just the notation and does not limit performance and scalability per se.

As the BPMN community grows, so does the tooling. Zeebe is a good example. Zeebe is a new, highly scalable BPMN-based workflow engine for microservice orchestration by Camunda. If you are interested about how BPMN can blend into the serverless world, stay tuned for my next post. I will demonstrate how Zeebe and BPMN can be used to compose FaaS functions within the Oracle Fn platform.