1.1 Functions as a Service (FaaS)

FaaS is a revolutionary technology. It is a great boon for developers and businesses. FaaS allows businesses to adapt to rapidly changing business needs by enabling their development teams to develop products and features at a “high” velocity, thereby improving their Mean Time To Market (MTTM). Developers can develop functions without worrying about setting up, configuring, or maintaining the underlying infrastructure. FaaS models are also designed to use just the right quantity of infrastructure and compute time. They also can be scaled to fit exact demand, by focusing on billing for the number of invocations as compared to billing for uptime. FaaS has two parts, as shown in Figure 1-1.

• The function code encapsulates the business logic in any language, such as Java, C#, Python, Node, and so on.

• The underlying container hosts an application server and an operating system.

An illustration describes a container containing function code, app server, and operating system. An events cloud flows a consumer to a container, and a container flows a supplier to an events cloud.

1.1.4 Applying the Serverless Functions Concept to an Enterprise Application

Let’s dig deeper into the concept of functions and how AWS realizes the magic of infrastructure savings. Functions are tiny code pieces with a single input and a single output, and a processing layer (a predicate) acting as the glue. Compare this to enterprise apps, which are designed to do many things. Take a simple payroll system, for example. A payroll system has multiple input interfaces and multiple output interfaces. Here are some of those interfaces:

• Timecard system to get the hours employees worked in a month

• Performance evaluation system

• Peer feedback system

• Inflation adjustment calculator system

• The outgoing interface to the IRS

• The outgoing interface to the medical insurance provider

• An outgoing interface to the internal web portal where employees can download their paystubs

Running this payroll application is not trivial. I have seen such a payroll system use the following:

• Fourteen dedicated middleware application servers

• Two RDBMS database stores

• Two integration tools such as message queues and FTP

• Four dedicated bare-metal servers, with each server configured with 128GB RAM, 32 CPUs, 4TB of HDD, 10TB of vSAN, and the like

The key factor in determining whether this application can be hosted on a serverless functions infrastructure like Lambda is the time it takes for the application to boot up (the startup time or cold start) and the time it takes for the application to shut down (the shutdown time). The faster the startup and shutdown times, the larger the cost savings.

It is also important that these times be quick so that they don’t cause disruptions. If you were to research the startup times for large enterprise applications like the payroll application, you would find that it is not pretty. An average startup time is around 15 minutes for all components to come up and another 15 minutes for the application to come down. This would not fly. Imagine if you deployed this application to an AWS Lambda serverless function. Thirty minutes of downtime for each invocation? This will not work. Your users would abandon the application entirely. As you can see, large applications cannot benefit from resource release and resource reassignment because they take a long time to start up and shut down, which would impact the general operation of the application.

Can you make this large payroll application behave in an expected way for serverless functions? The answer is yes. A lot of refactoring is required, but it can be done.

Serverless Function in the Cloud

All cloud providers have now incorporated the serverless functions into their infrastructure offerings. AWS has Lambda Functions, Google has Cloud Functions, and Azure has Azure Functions. These providers, in the quest for making their customers captive, made sure to introduce proprietary elements into their environments. The two components that are essential to run the functions are:

• Serverless function code that serves the functions

• Serverless infrastructure (containers) that supports the code

Why Is It Important for Serverless Functions to be Non-Proprietary?

Enterprises are gravitating toward a multi-cloud, hybrid-cloud approach to their cloud strategy. As you can see in Figure 1-2, the survey of 3,000 global respondents indicated that 76 percent already work in a multi-cloud environment. This means they are not bound to one single cloud provider. The adoption of a multi-cloud strategy alleviates the risk of vendor lock-in.

A circle diagram depicts the cloud strategy for all organizations. 2% single private, and 9% single public, 89% multi-cloud. Multi-cloud is separated into 80% hybrid cloud, 7% multiple public, and 2% multiple private.

In a multi-cloud world, it is essential that enterprises subscribe to services that can be easily ported between clouds. This is especially important for commodity services.

FaaS, or serverless functions, have of late become a commodity with all the providers having some services around FaaS. It is therefore imperative that FaaS containers not have proprietary code.

Serverless functions become portable when they do not use proprietary code. Portable serverless functions allow for workload mobility across clouds. If, for instance, AWS Lambda functions are costly and Azure Functions are cheap, enterprises can avail the cost savings and move that Lambda workload to Azure Functions with very little effort.

The subsequent sections discuss in detail these portability issues and explain how you can solve them.

1.2 Code Portability Issues

Listing 1-1 shows the sample AWS Lambda code written in Java. This code was generated using the AWS SAM (Serverless Application Model) template. When observing the code, you can see that the AWS-specific library references and method calls bind the code to AWS. It is not much, but it is potent. In an enterprise, you typically have hundreds if not thousands of pieces of code that you must refactor if you want to move this type of code to another cloud provider. This is a costly affair.

Listing 1-1.Sample Code Using the AWS SAM Framework

A framework of amazon web services serverless application model. Package HelloWorld. It describes amazon web service-specific code and method calls that bind the code to amazon web services.

The following section explores the portability of the underlying serverless container, which impacts how multi-cloud migrations are conducted.

1.2.1 Serverless Container Portability Issue

What about Lambda’s underlying serverless framework? Is it portable?

If you deep dive into AWS Lambda, the virtualization technology used is Firecracker. Firecracker uses KVM (a kernel-based virtual machine) to create and manage microVMs. You can find more information on Firecracker at https://​aws.​amazon.​com/​blogs/​aws/​firecracker-lightweight-virtualization-for-serverless-computing/​.

The minimalist design principle that Firecracker is built on allows for fast startup and shutdown times. Google Cloud Functions, on the other hand, use gVisor and are not compatible with Firecracker. gVisor is an application kernel for containers. More information can be found at https://​github.​com/​google/​gvisor.

Azure Functions take a totally different approach of using the PaaS offering app service as their base. So, you can see that the major cloud providers use their own frameworks for the managing functions’ containers. This makes it difficult for functions to move between clouds in a multi-cloud environment. This portability issue becomes more pronounced due to the lack of portability at the container level.

A comparison table. Function codes for Lambda of them are Lambda run time, sandbox, guest O S. For Google, gVisor, Host Kernel, and Google computes engine. For Azure, some of them are language runtime, and web jobs script runtime.

You can see that the code and containers both differ from the provider and are not easily portable.

In the discussions so far, you have seen the following issues related to FaaS:

• Portability of code

• Portability of the serverless container

• Cold start of the serverless environment

How do you solve these issues?

Enter Spring Cloud Function and Knative. Spring Cloud Function addresses function code portability, and Knative addresses container portability.

Information on Spring Cloud Function is available at https://​spring.​io/​projects/​spring-cloud-function, and information about Knative is available at https://​knative.​dev/​docs/​.

The following sections deep dive into each of these topics.

1.3 Spring Cloud Function: Writing Once and Deploying to Any Cloud

Source: https://​spring.​io/​projects/​spring-cloud-function

The key goals are decoupling from a specific runtime and supporting a uniform programming model across serverless providers.

Figures 1-4 and 1-5 show how you can deploy Spring Cloud Function. When Spring Cloud Function is bundled with specific libraries, it can be deployed to AWS Lambda, Google Cloud Functions, or Azure Functions.

A model diagram describes the deployment of the spring cloud function. Spring cloud function deployed to Amazon cloud, Google cloud, and Azure using implementation-specific libraries.

A model diagram describes the deployment of the spring cloud function plus K native. It deployed to hyper-scale clouds and on-prem or hybrid.

When Spring Cloud Function is containerized on Knative, it can be deployed to any Kubernetes offering, whether on the cloud or on-premises. This is the preferred way to deploy it on hybrid and multi-cloud environments.

1.4 Project Knative and Portable Serverless Containers

Having a portable serverless container is also important. This minimizes the complexity and time required to move between cloud providers. Moving between cloud providers to take advantage of discounted pricing goes a long way toward saving costs. One methodology used is called cloud bursting (Figure 1-6). Cloud bursting compensates for the lack of infrastructure on-premises by adding resources to the cloud. This is usually a feature of a hybrid cloud.

A diagram describes normal traffic flows to hosted applications, and traffic spike flows to burst applications. A corporate data center and public cloud synchronize with hosted and burst applications.

Figure 1-6 shows that, to compensate for the lack of resources in a private cloud during a traffic spike, resources are allocated to the public cloud where the traffic is routed. When the traffic spike goes down, the public cloud resources are removed. This allows for targeted use of costs and resources—that is, it uses additional resources only during the traffic spike period. The burst of activity during an eCommerce event like Cyber Monday is a great example of a traffic spike.

This cannot be easily achieved with just a portable code. You need containers that are also portable. This way, containers can be moved across cloud boundaries to accommodate traffic spikes. In Figure 1-6, you can see that VMs from VMware are used as containers. Since the VMs hosted in the datacenter and hosted in the cloud are similar in construct, cloud bursting is possible.

Applying this to Functions as a Service, you need a new way to make the underlying serverless containers portable.

One such revolutionary approach in the cloud function world is Knative. The next section dives deep into Knative.

1.4.1 Containers, Serverless Platforms, and Knative

What was the need for containers /serverless platforms?

Over the course of the evolution of IT, there has been a need for secure isolation of running processes. In the early 90’s, chroot jail-based isolation allowed developers to create and host a virtualized copy of the software system. In 2008 Linux Containers (LXC) was introduced which gave the developers a virtualized environment. In 2011 Cloud Foundry introduced the concept of a container, and with Warden in 2019 container orchestration became a reality. Docker, introduced in 2013, provided containers that can host any operating system. Kubernetes, introduced in 2014, provided the capability to orchestrate containers based on Docker. Finally, Knative, introduced in 2018, extended Kubernetes to enable serverless workloads to run on Kubernetes clusters.

Serverless workloads (Knative) grew out of the need to help developers rapidly create and deploy applications without worrying about the underlying infrastructure. The serverless computing model takes care of provisioning, management, scheduling, and patching and allows cloud providers to develop the “pay only for resources used” model.

With Knative, you can create portable serverless containers that run on any Kubernetes environment. This allows for FaaS to be portable in a multi-cloud or hybrid-cloud environment.

Besides making developers more productive, the serverless environment offers faster deploys (see Figure 1-7). Developers can use the “fail fast and fail often” model and spin up or spin down code and infrastructure faster, which helps drive rapid innovation.

A diagram describes the servers deployed in months, virtual machines deployed in minutes, containers deployed in seconds, and serverless deployed in milliseconds.

1.5 Sample Use Case: Payroll Application

Let’s look at how you can apply serverless functions to a real-life example.

We introduced the payroll application in the beginning of the chapter, we’ll now build on it. Consider a payroll application with the configuration shown in Figure 1-8.

A table describes business functions databases and integrations. Some of them are in business functions of reading employee records, and payment information in a database of employee records, and in Integrations of I R S.

Figure 1-9 shows the configuration.

A flow diagram describes the employee records and monthly pay data flows to validate employee data to compute salary. Monthly pay rates and tax tables flow to compute salary and some of them are taxes and savings information.

Translating this flow diagram into an actual implementation that can be deployed in a corporate datacenter results in Figure 1-10. You see that the functions are developed as REST APIs that are executed in batch mode. The REST APIs expose SAP ECC payroll modules. These REST APIs are run as batch jobs every 15 days. The databases are hosted on Oracle, and integrations are exposed using IBM API Connect (APIC). Note that this is not an on-demand process and can consume a lot of resources when idle. These REST APIs cannot be easily shut down and started up because a typical time for booting up a SAP NetWeaver component can be anywhere from 2 to 15 minutes, depending on the JVM configuration. These application components must be running constantly to keep the payroll application from breaking down.

An architecture diagram describes an Oracle database of employee records and monthly pay data flows to validate employee data to compute salary. Monthly pay rates and tax tables flow to compute salary.

Using this use case, the following sections explore how you can leverage Spring Cloud Function to modernize and transform this application into a highly efficient, scalable, and portable system.

1.6 Spring Cloud Function Support

1.6.1 Spring Cloud Function on AWS Lambda on AWS

Transforming an application deployed on-premises to leverage an AWS Lambda environment and be portable requires function code that abstracts away the hard dependencies of AWS Lambda from the implementation and the serverless container. This example uses Spring Cloud Function for the function code framework and Lambda for the container. By writing once using Spring Cloud Function, you can use the DevOps pipeline, discussed in subsequent chapters, to deploy to other serverless environments. Figure 1-11 shows how AWS and its components help realize the payroll application in the cloud.

An architecture diagram describes a relational database of employee records and monthly pay data flows to validate employee data to compute salary. Monthly pay rates and tax tables flow to compute salary.

Now you need to deploy the payroll system on AWS Lambda. The deployment sequence is important, as you need to deploy SAP ECC and Oracle before integrating and then configure API and messaging for the Spring Cloud Function to integrate with SAP. Spring Cloud Function can be created and tested with dummy information, but it needs to be deployed after integration testing with SAP ECC:

1. 1.

   Deploy SAP ECC on the AWS EC2 instances.

    
2. 2.

   Deploy Oracle DB as an RDS instance.

    
3. 3.

   Configure the SAP to Oracle integration.

    
4. 4.

   Deploy Spring Cloud Function to AWS.

    
5. 5.

   Set up the Amazon API Gateway.

    
6. 6.

   Set up Amazon SQS for messaging.

    

1.6.2 Spring Cloud Function on Knative and EKS on AWS

If you want to implement a truly portable environment in AWS, you can leverage the AWS EKS, which is a Kubernetes platform that AWS offers. You can install Knative in EKS and this will give you a truly portable, serverless container that allows for faster deployment and improved cold starts. It uses appropriate technologies, such as Spring Cloud Function on GraalVMs. GraalVMs uses the AOT (Ahead of Time) compilation technique, which significantly improves execution times. Subsequent sections address GraalVMs and Spring Cloud Function. See Figure 1-12.

An architecture diagram describes a relational database of employee details flows to validation to compute salary. Pay rates and tax flow to compute salary. It flows through Amazon S Q S to the spring cloud function.

Follow this process to deploy the payroll system on Knative hosted on an Azure AKS and ensure that the SAP ECC and Oracle DB are up and integrated. You do this before developing and deploying Spring Cloud Function on Knative:

1. 1.

   Deploy SAP ECC on AWS EC2 instances.

    
2. 2.

   Deploy Oracle DB as an RDS instance.

    
3. 3.

   Configure the SAP to Oracle integration.

    
4. 4.

   Set up Knative on an AWS EKS cluster.

    
5. 5.

   Deploy Spring Cloud Function on Knative.

    
6. 6.

   Set up the Amazon API Gateway.

    
7. 7.

   Set up Amazon SQS for messaging.

    

1.6.3 Spring Cloud Function on Cloud Functions on GCP

GCP Cloud Functions provide a cloud alternative to AWS Lambda Functions. The GCP offering is newer than Lambda, but with the Anthos strategy, it is gaining a good amount of the function space. Spring.io works closely with Google to make the Spring.io components work seamlessly with the GCP components.

To deploy the payroll system on Cloud Functions on GCP, follow the process outlined here (see Figure 1-13). Ensure that the SAP ECC and Oracle DB are up and integrated before developing and deploying Spring Cloud Function on Cloud Functions:

1. 1.

   Deploy SAP ECC onto GCE.

    
2. 2.

   Deploy Oracle DB on GCE VMs, as there is no AWS RDS-like service on GCP.

    
3. 3.

   Configure the SAP to Oracle integration.

    
4. 4.

   Set up a GCP Cloud Function project.

    
5. 5.

   Deploy Spring Cloud Function onto GCP Cloud Functions.

    
6. 6.

   Deploy Apigee on GCP to host function APIs.

    
7. 7.

   Set up the Google Cloud pub/sub messaging platform.

    

An architecture diagram describes an Oracle on G C E V Ms of employee details flows to validation to compute salary. It flows through the cloud pub to the spring cloud function.

1.6.4 Spring Cloud Function on Knative and GKE on GCP

Knative, as discussed, is a way to make the functions portable. Knative, incidentally, was created by Google. With GCP, you can set up Knative on GKE, which is the Kubernetes engine provided by Google.

To deploy the payroll system on Knative hosted on a GCP GKE, follow the process outlined here (Figure 1-14). Ensure that the SAP ECC and Oracle DB are up and integrated before developing and deploying Spring Cloud Function on Knative:

1. 1.

   Deploy SAP ECC as a Docker image onto the GKE cluster.

    
2. 2.

   Deploy Oracle DB as a Docker image onto the GKE cluster.

    
3. 3.

   Configure the SAP to Oracle integration.

    
4. 4.

   Configure a GKE cluster with Knative.

    
5. 5.

   Deploy Spring Cloud Function onto Knative.

    
6. 6.

   Set up the Apigee API Gateway.

    
7. 7.

   Set up RabbitMQ on GKE for messaging.

    
8. 8.

   Set up Google cloud pub/sub.

    

An architecture diagram describes an Oracle on G C E V Ms of employee details flows to validation to compute salary. Pay rates and tax flow to compute salary. It flows through the cloud pub to the spring cloud function.

1.6.5 Spring Cloud Function on Azure Functions on Azure

Spring Cloud Function deployed on Azure Functions is not portable due to the explicit use of an Azure Function Invoker class. While Lambda and Google Cloud Functions require a change to Pom.xml (if you are using Maven), Azure needs an additional class. This makes it less portable. If you have a portfolio of one thousand Spring Cloud Functions in AWS that you need to move to Azure, you have to do a lot of development activity. This is disruptive.

To deploy the payroll system on Azure Functions, follow the process outlined here (see Figure 1-15). Ensure that the SAP ECC and Oracle DB are up and integrated before developing and deploying Spring Cloud Function on Azure Functions:

1. 1.

   Deploy SAP ECC on Azure VMs.

    
2. 2.

   Deploy Oracle DB on Azure VMs.

    
3. 3.

   Configure the SAP to Oracle integration.

    
4. 4.

   Configure Azure Functions.

    
5. 5.

   Deploy Spring Cloud Function on Azure Functions.

    
6. 6.

   Set up the Azure API Gateway on Azure.

    
7. 7.

   Set up Azure Queue storage on Azure for messaging.

    

An architecture diagram describes an Oracle on Azure V Ms of employee details flows to validation to compute salary. It flows through the Azure queue to the Azure plus spring cloud function.

1.6.6 Spring Cloud Function on Knative and AKS on Azure

Knative on Azure AKS is the only option for deploying Spring Cloud Function on Azure that makes it portable. As in any Kubernetes implementation, it requires an implementation of Knative to run the functions. Transforming the payroll application to AKS requires an AKS cluster.

To deploy the payroll system on Knative hosted on an Azure AKS environment, follow the process outlined here (Figure 1-16). Ensure that the SAP ECC and Oracle DB are up and integrated before developing and deploying Spring Cloud Function on Knative:

1. 1.

   Deploy SAP ECC on Azure VMs.

    
2. 2.

   Deploy Oracle DB on Azure VMs.

    
3. 3.

   Configure the SAP to Oracle integration.

    
4. 4.

   Configure an AKS cluster with Knative.

    
5. 5.

   Deploy Spring Cloud Function onto Knative.

    
6. 6.

   Set up an Azure API Gateway on AKS.

    
7. 7.

   Set up Azure Queue on Azure for messaging.

    

An architecture diagram describes an Oracle on Azure V Ms of employee details flows to validation to compute salary. It flows through the Azure queue storage to the A K S plus K native spring cloud function.

1.6.7 Spring Cloud Function on VMware Tanzu (TKG, PKS)

VMware Tanzu is an evolution of Pivotal Cloud Foundry (PCF). Those who are familiar with PCF should be aware of the “cf push” experience. It was a one-click provisioning approach and was very popular in the developer community. This is the same experience that Knative provides through its Knative build feature. To transform the payroll application to run on VMware Tanzu, you need the Tanzu Kubernetes grid, also known as TKG. TKG is built using the main branch of Kubernetes code. This can be deployed on-premises and in the cloud and can facilitate a multi-cloud or hybrid-cloud strategy. You can start up an instance of TKG on AWS, Azure, or Google by subscribing to the service in the provider’s marketplace.

In a datacenter, you need a collection of servers or a hyper-converged infrastructure like VxRail with PRA (Pivotal Ready Architecture). You also need to upgrade your vSphere to Version 7.

Going back to the payroll application, you need to follow the process outlined here (Figure 1-17). Ensure that the SAP ECC and Oracle DB have been up and integrated before developing and deploying Spring Cloud Function on Knative:

1. 1.

   Deploy SAP ECC as a Docker image onto TKG.

    
2. 2.

   Deploy Oracle DB as a Docker image onto TKG.

    
3. 3.

   Configure the SAP to Oracle integration.

    
4. 4.

   Configure a TKG cluster with Knative.

    
5. 5.

   Deploy Spring Cloud Function onto Knative.

    
6. 6.

   Set up a Spring Cloud Gateway on TKG as an API Gateway.

    
7. 7.

   Set up RabbitMQ on TKG for messaging.

    

An architecture diagram describes an Oracle on docker plus T K G of employee details flows to validation to compute salary. It flows through the rabbit M Q to the T K G plus K native spring cloud function.

A diagram describes the cluster. It contains Tanzu Kubernetes grid, network, and services. It connected with the K 8s control plane through the v Sphere distributed switch of the management and workload port group.

1.6.8 Spring Cloud Function on Red Hat OpenShift (OCP)

Red Hat OpenShift can be an on-premises option for deploying Spring Cloud Function. As in any Kubernetes implementation, it requires an implementation of Knative to run the functions. OpenShift has its own serverless implementation, called OpenShift serverless. Transforming the payroll application to OpenShift requires an OpenShift cluster.

To deploy the payroll system to OpenShift hosted on a VMware vSphere environment, follow the outlined process. First ensure that the SAP ECC and Oracle DB are up and integrated before developing and deploying Spring Cloud Function on Knative:

1. 1.

   Deploy SAP ECC as a Docker image onto OpenShift cluster.

    
2. 2.

   Deploy an Oracle DB as a Docker image onto the OpenShift cluster.

    
3. 3.

   Configure the SAP to Oracle integration.

    
4. 4.

   Configure an OpenShift cluster with Knative.

    
5. 5.

   Deploy Spring Cloud Function onto Knative.

    
6. 6.

   Set up a Red Hat 3scale API Gateway on OpenShift.

    
7. 7.

   Set up RabbitMQ on TKG for messaging.

    

An architecture diagram describes an Oracle on docker plus open shift of employee details flows to validation to compute salary. It flows through the rabbit M Q to the open shift serverless plus spring cloud function.

1.7 Summary

This chapter discussed FaaS environments, Spring Cloud Function, and Knative. You saw that FaaS containers/environments provided by AWS, Google, or Microsoft Azure are not portable, as the underlying components that host the FaaS environment do not have the same architecture, which makes it difficult to move or migrate FaaS containers between cloud providers. You also saw that Spring Cloud Function can abstract the dependent AWS and Google libraries and provide a portable alternative. Spring Cloud Function on Knative can improve developer productivity by “writing once and deploying anywhere.” You saw how to apply Spring Cloud Function and Knative to an enterprise payroll application and learned about the various implementation approaches. The next chapter walks through the deployments step-by-step. You will also see how to develop and deploy code to various targets, such as AWS, GCP, Azure, OpenShift, and VMware Tanzu. This will help you understand the power of Spring Cloud Function.

2.1 Setting Up the Spring Boot and Spring Cloud Function Locally

This is a preliminary step that you undertake to determine if the Spring Cloud Function works. This will set the stage for deployment.

I am using IntelliJ, as it makes me more productive with all the capabilities such as plugins for the cloud, Docker, Maven, and so on.

2.1.1 Step 1: Create the Spring Boot Scaffolding

In IntelliJ, create a Spring Boot project using Spring Initializer. Make sure you provide the name and choose JDK 11. Then provide a name and a group name. Click Next to reach the New Project screen, as shown in Figure 2-1.

An application screenshot. The left side contains project types and the right side contains new project configurations. Some of them are severed U R L, name, location, language, type, group, and artifact.

On the next screen, it is important to select Spring Web and Function. If you just choose Function, you will not be able to test the function locally, as the functions will not persist. See Figure 2-2.

An application screenshot of a new project. It contains a set of dependencies on the left side. From dependencies, a function is selected, Function description and add dependencies are on the right side.

A screenshot of the spring boot deployment logs. It allows for the function to be tested by tools.

As you can see from the Added Dependencies section in Figure 2-2, I also included an H2 Database. You can use the database of your choice.

A screenshot of testing in postman. The page contains authorization, headers, body, pre-request script, tests, and settings. The body contains the name, employee identifier, email, and salary.

Upon clicking the Finish button, the IDE will create the code bits and take you to the project screen. Alternatively, you can bring the code down from GitHub by cloning the project.

Listing 2-1 shows the pom.xml file with Maven dependencies.

<dependency>

    <groupId>org.springframework.boot</groupId>

    <artifactId>spring-boot-starter-data-jpa</artifactId>

</dependency>

<dependency>

    <groupId>org.springframework.boot</groupId>

    <artifactId>spring-boot-starter-data-rest</artifactId>

</dependency>

<dependency>

    <groupId>org.springframework.boot</groupId>

    <artifactId>spring-boot-starter-web</artifactId>

</dependency>

<dependency>

    <groupId>org.springframework.cloud</groupId>

    <artifactId>spring-cloud-function-web</artifactId>

</dependency>

<dependency>

    <groupId>com.h2database</groupId>

    <artifactId>h2</artifactId>

    <scope>runtime</scope>

</dependency>

Listing 2-1

pom.xml File with Maven Dependencies

Listing 2-2 shows the application.properties file. I included spring.h2.console.enabled=true to be able to verify if the database has been populated.

spring.cloud.function.definition=employeeConsumer

spring.datasource.url=jdbc:h2:mem:employee

spring.datasource.driverClassName=org.h2.Driver

spring.datasource.username=sa

spring.datasource.password=

spring.jpa.hibernate.ddl-auto=create

spring.h2.console.enabled=true

Listing 2-2

application.properties

PayrollApplication.java, as shown Listing 2-3, is the main entry point for the application. It is important to define three beans that represent the consumer, supplier, and function, so that these functions can be accessed after deployment.

If you do not do this, you will not be able to invoke the function.

package com.kubeforce.payroll;

import org.springframework.boot.SpringApplication;

import org.springframework.boot.autoconfigure.SpringBootApplication;

import org.springframework.cloud.function.context.FunctionalSpringApplication;

import org.springframework.context.annotation.Bean;

@SpringBootApplication

public class PayrollAwsApplication {

    public static void main(String[] args) {SpringApplication.run(PayrollApplication.class,args);

        //SpringApplication.run can be used to run the function application locally.

        //Change SpringApplication.run to FunctionalSpringApplication.run when deploying to a serverless platforms

        // such as lambda, knative, etc.

    }

    @Bean

    public EmployeeFunction employeeFunction() {

        return new EmployeeFunction();

    }

    @Bean

    public EmployeeConsumer employeeConsumer() {

        return new EmployeeConsumer();

    }

    @Bean

    public EmployeeSupplier employeeSupplier() {

        return new EmployeeSupplier();

    }

}

Listing 2-3

PayrollApplication.java

Once the scaffolding and code bits are completed, you are ready for the next step, which is to create the model for your employee.

2.1.7 Step 7: Test the Function

You can use curl or Postman to test your function. Here I use Postman. You can download Postman at https://www.postman.com/downloads/.

In Postman, choose POST, the http://localhost:8080/employeeConsumer URL, and the JSON format input.

{

  "name": "xxx",

    "employeeIdentifier":"2",

    "email": xxx@yahoo.com,

      "salary":"1000"

}

You can test all the three interfaces here, but when you go to Lambda or other environments, you will be restricted to just one interface.

You can also verify if the data has been added by going to the H2 console, as shown in Figure 2-5.

A screenshot of the H 2 console. S Q L statement is Select star from an employee. The employee details are I D, email, employee identifier, name, and salary.

This completes the steps for creating, running, and testing the code of a local function.

In the following sections, you’ll see how to use the same code with little to no changes and deploy it to a serverless function container of your choice.

2.2 Setting Up Spring Cloud Function and AWS Lambda

This section discusses deploying the code that you created into AWS Lambda.

You have to do a little bit of tweaking to make the code compatible with Lambda. The tweaking is not too disruptive. Again, going back to real situations where you have over a thousand functions you want to move to AWS, this tweak will not be that significant. Of course, it depends on what cloud provider you are starting to create functions for. Ideally, I have seen enterprises gravitate toward Lambda, but there are enterprises that start with Azure or Google. In any case, making these functions portable across clouds requires a few code changes and some pom.xml changes.

Step 1: First follow Steps 1-6 outlined in Section 2.1. The code remains the same for any serverless offerings from the cloud providers.

The next step is to change the pom.xml file to add the AWS Lambda adapter.

The POM needs to be changed to accommodate the Spring Cloud Function adapter for AWS, Lambda events, and the Maven shade plugin (optional).

These changes will create a package that can be deployed to AWS Lamdba.

Step 3: Package the application into a JAR file to deploy to AWS Lambda. Figure 2-6 shows how to trigger a Maven package run in IntelliJ. Note that payroll-aws-0.0.1-SNAPSHOT-aws.jar is created in the target folder.

A screenshot of Run Maven. The left side contains project details. The right side contains Maven. Some of them are packages run Maven build, run payroll-a w s, debug payroll-a w s, and profile payroll-a w s.

Step 4: Create a function definition in AWS Lambda.

Log in to AWS and subscribe to AWS Lambda. Information on how to subscribe to Lambda is available at https://docs.aws.amazon.com/lambda/latest/dg/getting-started.html.

Create a function defined on the Lambda portal/dashboard, as shown in Figure 2-7.

A screenshot of Create function. It contains the author from scratch and uses a blueprint, and container image. Author from scratch contains basic information, permissions, and advanced settings.

A screenshot of an employee consumer page. It contains a function overview and employee consumer layers.

Upload the JAR file created in Step 3 and provide the settings, as illustrated in Figure 2-7.

Use the Upload From button, as shown in Figure 2-9, to access the choices for uploading a file.

A screenshot of the edit runtime setting page. The page contains runtime settings, runtime as Java 11, handler, and architecture.

Set the handler as shown. The Lambda function handler is the method in your function code that processes events. This will indicate that the Spring Cloud Function adapter will be invoked when the function is called. I used Java 11. Refer to the pom.xml.

Once you click Save, you can start your testing process.

Step 5: Test the function. The AWS Portal allows developers to run the tests against the function. Clicking the Test tab will take you to the testing dashboard, as shown in Figure 2-10.

A screenshot of the function overview page from the test function. The page contains employee consumer layers and descriptions. Last modified details, function A R N, and function U R L.

The testing dashboard allows you to send a JSON file as an event to trigger the function.

In the Test Event section, you can provide the event name or choose other settings. You can run the event with just the defaults for test purposes; the only parameter that needs to be provided is the input JSON payload. Click Test when you’re ready, as shown in Figure 2-11.

A screenshot of the test event page. It contains test event action, event name, event sharing settings as private, and template-optional. Event J S O N contains name, employee identifier, email, and salary.

Figure 2-12 shows you the details of the test. Note that the test is successful.

A screenshot of an execution result of success. The page contains the details of the test, summary, and log output. Summaries of code, request I D, duration.

2.3 Setting Up Spring Cloud Function and Google Cloud Functions

You will use the same example from Section 2.1 and modify it a bit to make it run on Google Cloud Functions. The approach is similar to AWS Lambda, but you will be leveraging Google’s libraries.

Step 2: Configure POM. The pom.xml file needs to be modified to include the Spring Cloud Function adapter and any plugins that are specific to GCP. See Listing 2-11.

Step 3: Build, package, and deploy to Google Cloud Functions. The Google cloud website for this activity is a bit clumsy. You have to zip up your entire directory and upload it. This may not be allowed in some enterprises. I therefore reverted to using the CLI to accomplish this task. See Figure 2-13.

A screenshot of G cloud C L I execution results. It uses the C L I to accomplish the task. Some of the details on the page contain available memory M b, build I D, build name, docket registry, and entry point.

Once the command has been successfully executed, you can go to your Google Cloud Functions console to see the deployed function. See Figure 2-14.

A screenshot of the Google cloud function dashboard. It contains cloud functions and functions. Some of the details it contains the environment, name, last deployed, region, trigger, runtime, and memory allocated.

Now you can test it to verify that it works.

Step 4: Test the function in GCP. Google provides a way to test from the website itself. You can provide the input in the form of the JSON file, as you did before, and then execute the tests.

Figure 2-15 shows the successful execution of the test.

A set of two screenshots of the cloud functions testing dashboard. a. Google cloud platform described triggering event. b. Cloud functions payroll g c p.

2.4 Setting Up Spring Cloud Function Azure Functions

You will use the same example from Section 2.1 and modify it a bit to make it run on Azure Functions. You will see that the approach for Azure Functions is different from AWS or Google. You have to add another class called EmployeeConsumerHandler, which extends a FunctionInvoker<I,O>. This FunctionInvoker class is merely a pass through for the EmployeeConsumer function.

Step 1: Follow Steps 1-6 outlined in Section 2.1.

You have to introduce a FunctionInvoker <I,O> class, which makes the code non-portable to other cloud providers. While AWS and Google externalize the invocation of the function, as you have seen in previous sections, Azure forces you to provide a handler class that extends a FunctionInvoker.

Step 2: Build and package the function. IntelliJ offers you the ability to execute Maven commands through a Maven window, as shown in Figure 2-16.

A screenshot of the Maven package. The left side contains coding. The right side contains commands. Run payroll Azure package command is selected.

You have to run the maven clean command before you run the maven package command. These commands are under the Lifecycle menu in the Maven window. Run the Maven package to get the deployable JAR file. These JAR files are stored in the target folder in your project directory

Figure 2-17 shows a successful run of the Maven package.

A screenshot of a successful package. It contains payroll Azure package logs.

Step 3: Deploy the function in Azure. You will see some artifacts created in the target folder. Payroll-azure-0.0.1-SNAPSHOT.jar is the file you are interested in. This file needs to be deployed to the Azure Cloud.

IntelliJ has a feature/plugin for Azure that you can enable. Once you enable that feature, you can perform Azure Function-based activities from the IDE.

In this case, to start the deploy process, choose Run Azure Functions ➤ Deploy from the Azure Functions list, as shown in Figure 2-18.

A screenshot of payroll-azure. The left side contains projects. Target is selected. In coding payroll, a function is selected. The right side contains the commands. Azure-function deploy is selected.

After the function is successfully deployed to the Azure Cloud, you can use the URL provided to runs tests in Postman or proceed to the Azure Functions console to test.

The Azure Functions console is shown in Figure 2-19.

A screenshot of the function application. A payroll-azure function is selected. In the payroll-azure function, the function tag is selected, and it contains the name, trigger, and status.

You will see that the function is deployed in Azure and is running.

Step 4: Test. Clicking the employeeConsumer function will take you to the detail console, as shown in Figure 2-20. Here, you can conduct tests by clicking the Code + Test link. The dashboard has an Input section, where you can specify the JSON in the body, choose the HTTP method as POST (in this example), and click the Run button. You can see the results of the execution in the command console, as shown in Figure 2-21.

A screenshot of employee consumer code plus test page. The test tag is selected. In payroll-azure, employee consumer input is the H T T P method, key, query, headers, and body. It contains a run page.

A screenshot of employee consumer code plus test page. Code plus test tag is selected. The output is the H T T P response code, successful H T T P response content. It contains the run logs.

A screenshot of the execution log. It contains creating cluster k native. Some of the checks are to ensure node image, prepare nodes, and writing configuration. The output is you can now use your cluster.

Figure 2-21 shows successful execution of the test.

2.5 Setting Up Locally with Kubernetes and Knative and Spring Cloud Function

This section explores another way to deploy Spring Cloud Function. Kubernetes, as you might be aware, is a portable approach to containerization and running containers. Kubernetes allows you to host containers that are packaged as images and deployed on Kubernetes pods.

Knative is an environment that sits on Kubernetes and can host a function. Chapter 1 explained how the cloud providers created environments such as AWS Lambda that could start up on demand and shut down when a resource is not in use. Similarly, you can create such an on-demand environment with Knative for functions. This section explores how to set up the Knative environment locally, which is difficult with AWS and Google Functions, and deploy and work with your functions locally and on the cloud. This section starts with local deployment and then moves on to the cloud.

You will notice that I used EmployeeSupplier as the function. To do this, you need to prepopulate the H2 database with some data so that you can query the database using the EmployeeSupplier function.

Here is the process of setting up Kubernetes and Knative and deploying Spring Cloud Function.

Step 1: Set up a Kubernetes cluster locally. In this step, you install a local Kubernetes cluster. You deploy a KIND (Kubernetes IN Docker). Cluster. Of course, you can use Minikube or Minishift to deploy locally.

Now you have the KIND bits that will allow you to install the cluster and deploy Knative.

Step 2: Configure Knative. In order to configure Knative, you need a KIND cluster. You will create a cluster with custom configuration.

Install Kourier by running the following command (see Figure 2-23):

$ kubectl apply --filename kourier.yaml

A screenshot of the execution log. It includes configuration, service account, cluster role, deployment, and service of kourier. The output is service kourier-internal created.

You can see in Figure 2-24 that all the components are up and running. You can go to the next step of publishing your Spring Cloud Function app on Knative.

A screenshot of the run log. It includes Kubert l get pods k native serving name, an activator, autoscaler, default domain, and domain mapping.

Step 3: Containerize the app and push it to a repository. Create a Dockerfile with the parameters set shown in Listing 2-16. I used jdk8.

FROM openjdk:8-jdk-alpine

ARG JAR_FILE=target/*.jar

COPY ${JAR_FILE} app.jar

ENTRYPOINT ["java","-jar","/app.jar"]

Listing 2-16

The Dockerfile

1. 1.

   Push to Dockerhub. You can use the IDEs Docker features to push it to the Docker registry, as shown in Figure 2-25.

    

A screenshot of the docker file. The left side contains the project. A docker file is selected. Push image page created. It contains a registry, namespace, repository, and tag.

A YAML execution gives you more control over the target environment. Note this URL, as it is required for the testing step. The URL shown here for example is https://payroll.default.127.0.0.1.sslip.io.

You can run the following command to get the URL and check if the endpoint is ready for testing.

Step 5: Test . Since employeeSupplier queries the database and gets the records, you need to use a GET operation. See Figure 2-26.

A screenshot of the test with the postman. The authorization tag is selected and the body contains I D, name, email, salary, and employee identifier.

In this section, you created a KIND-based Kubernetes cluster, configured Knative, and deployed the application. You created a portable image that can be deployed to any Kubernetes cluster that has been configured with Knative.

2.6 Setting Up AWS with EKS and Knative with Spring Cloud Function

This section looks at deploying to an AWS EKS cluster that has Knative configured. You will see similarities to Section 2.5, because you create the cluster, enable Knative, and deploy a Docker image.

The process takes 10 to 15 minutes. At the end of the process, you will have created a cluster in EKS. You can verify it in the AWS EKS console in the cloud. See Figure 2-27.

A screenshot of amazon elastic Kubernetes service. The payroll cluster contains an overview and its node's details. They are node name, instance type, node group, created, and status.

You will see that the dashboard shows the name of the cluster you provided in the Listing 2-18 and the creation of three nodes as set for desiredCapacity in the listing.

Verify that the clusters are up and running (see Figure 2-28). You can do that by running the following command:

$kubectl get pods --all-namespaces

A screenshot of elastic Kubernetes service running logs. It contains Kubect l get pods all namespaces, name, ready, status, restarts, and age. All namespaces were running successfully.

A screenshot of K native components running logs. It contains name, ready, status, restarts, and age. All namespaces were running status.

Step 3: Containerize the app with Docker and push it to a repository (optional).

Step 4: Push to the Docker hub. The next step is to push the Docker image to the Dockerhub repository, as shown in Figure 2-30. Be sure to log in to Dockerhub at https://hub.docker.com/ and create a repository and namespace. You will need it for the Docker push.

A screenshot of payroll. The left side contains the project. In that, a docker file is selected. The push image page is created. It includes a registry, namespace, repository, and tag.

Once the push is successful, you can navigate to Dockerhub to check for the deployment of the image, as shown in Figure 2-31.

A screenshot of the deploy image of the docker hub. The general page contains spring cloud functions, docker commands, tags and scans, and automated builds.

Step 5: Deploy the app to Knative on EKS. In this step, you create a YAML file and set the location of the image that you will deploy to Knative.

Deploy the app to Knative on EKS.

Figure 2-32 shows the result of running the kn cli. A YAML execution gives you more control over the target environment. Note the URL, as it is required for the testing step. The URL shown here for example is https://payroll.default.13.58.221.247.sslip.io.

A screenshot of payroll deployment logs. The run commands are route is still working to reflect, configuration payroll is waiting, waiting for a load balancer to be ready, and ready to serve.

The URL that you need for testing or posting is payroll.default.13.58.221.247.sslip.io

Step 6: Testing. Use the URL you got in Step 5 and run the test. The URL in this case is https://payroll.default.13.58.221.247.sslip.io/employeeConsumer

You can use Postman or curl to run the tests. Figure 2-33 shows the Postman test results.

A screenshot of the postman page. The header tags are home, workspace, A P I network, reports, and explore. The body contains the name, employee identifier, email, and salary.

A screenshot of cluster running logs. It contains the name, location, master version, master I P, machine type, node version, number of nodes, and status.

2.7 Setting Up GCP with Cloud Run/GKE and Knative with Spring Cloud Function

This section looks at deploying to GCP a GKE cluster that has Knative configured. You will see similarities to Section 2.5, because you create the cluster, enable Knative, and deploy a Docker image.

Step 1: Set up a Kubernetes cluster with GKE. In this step, you create a GKE cluster. Make sure you have the sufficient permissions to create the cluster. Additional information can be found at https://cloud.google.com/kubernetes-engine/docs/deploy-app-cluster.

Before running the command ensure, that you have logged in to GCP using the Gcloud CLI. Run the command in Listing 2-22.

$gcloud container clusters create payroll-clstr \

   --zone=us-central1-a \

   --cluster-version=latest \

   --machine-type=n1-standard-4 \

   --enable-autoscaling --min-nodes=1 --max-nodes=10 \

   --enable-autorepair \

   --scopes=service-control,service-management,compute-rw,storage-ro,cloud-platform,logging-write,monitoring-write,pubsub,datastore \

   --num-nodes=3

Listing 2-22

Create a Cluster in GCP

A screenshot of the status of the G K E cluster. It contains the name, status, roles, age, and version. All the statuses were ready.

Navigate to Google Clouds GKE dashboard to verify the cluster. See Figure 2-36.

A screenshot of the Google Kubernetes engine. Cluster tag selected. It contains Kubernetes clusters. Its overview contains status, name, location, number of nodes, total v C P Us, and total memory.

A screenshot of Kourier running logs. It contains namespace, type, and patch. The network was successfully dispatched.

Step 2: Configure Knative on GKE. This step is similar to what you did earlier with EKS or local Kubernetes:

Now that you have Knative configured and running, you can proceed to the next step of pushing the app image to Knative.

Step 3: Containerize the app with Docker and push it to a repository (optional).

This is an optional step, as you already deployed an image in Section 2.5. You can skip this step and go to Step 4. I used JDK 8 here, but you can use the latest JDK by changing the FROM statement to FROM adoptopenjdk/openjdk11:latest, as shown in Listing 2-23.

FROM openjdk:8-jdk-alpine

ARG JAR_FILE=target/*.jar

COPY ${JAR_FILE} app.jar

ENTRYPOINT ["java","-jar","/app.jar"]

Listing 2-23

The Dockerfile that Helps Create the Container and Image

A screenshot of K native serving running logs. It contains name, ready, status, restarts, and age. All namespaces were running successfully.

Step 4: Push to the Docker hub. The next step is to push the Docker image to the Dockerhub repository, as shown in Figure 2-40. Make sure to log in to Dockerhub at https://hub.docker.com/ and create a repository and namespace. You will need it for the Docker push.

A screenshot of Kourier service status run logs. It contains name, type, cluster I P, external I P, ports, and age.

A screenshot of payroll. The left side contains the project. In that, a docker file is selected. The push image page is created. It includes a registry, namespace, repository, and tag.

A screenshot of the deployment image of the docker hub. It contains spring cloud functions, docker commands, tags and scans, and automated builds.

A screenshot of payroll application run logs. The commands are route is still working, configuration payroll is waiting, and ready to serve. Output is service payroll created to the latest revision.

A screenshot of the successful testing with the postman. The header tags are home, workspaces, A P I network, reports, and explore. The body contains the name, employee identifier, email, and salary.

A screenshot of A K S cluster running logs. It contains names, statuses, roles, and ages. All the clusters were in ready status.

A screenshot of Kourier ingress patched running logs. It contains namespace, type merge, and patch. The output-configured network is patched.

A screenshot of the application deployed on A K S with K native logs. The commands are route is still working, and ready to serve. Output is service payroll created to the latest revision.

A screenshot of testing of the payroll on A K S with K native. The collections tag was selected and displayed create collection tag. The body contains the name, employee identifier, email, and salary.

A screenshot of the installation of T K G bits on a local machine running logs. Installing builder, cluster, Kubernetes release, login, and management cluster. The output is installation completed.

A screenshot of install of the T K G cluster. It contains the core package repo status. the output is cluster created.

A screenshot of application deployment on T K G with K native logs. The commands are route still working, and ready to serve. The output is the service payroll created to the latest revision payroll.

Once the push is successful, you can navigate to Dockerhub to check for the deployment of the image, as shown in Figure 2-41.

Figure 2-42 shows the results of running the kn cli. A YAML execution gives you more control over the target environment. Note the URL, as it is required for the testing step. The URL shown here for example is https://payroll.default.34.69.156.24.sslip.io.

Step 6: Test. Use the DNS name provided in Step 5 to test against the following URL:

https://payroll.default.34.69.156.24.sslip.io/employeeConsumer

Now you have successfully deployed the payroll app on GKE with Knative.

2.8 Setting Up Azure with AKS and Knative with Spring Cloud Function

This section looks at deploying to an Azure AKS cluster that has Knative configured. You will see similarities to Section 2.5, because you create the cluster, enable Knative, and deploy a Docker image.

Step 1: Set up a Kubernetes cluster with AKS. In this step, you create an AKS cluster. Make sure you have the sufficient permissions to create the cluster. Additional information can be found at https://docs.microsoft.com/en-us/azure/aks/learn/quick-kubernetes-deploy-cli.

Step 2: Configure Knative on AKS. This step is similar to what you did earlier with EKS or local Kubernetes.

Now that you have the Knative configured and running, you can proceed to the next step of pushing the app image to Knative.

Step 3: Containerize the app with Docker and push it to a repository. Follow Step 3 in Sections 2.5, 2.6, or 2.7 if you choose to create and deploy an image to the Dockerhub repository.

Step 4: Deploy the app to Knative on AKS. Here again, I use the Knative CLI kn to deploy the app, as it convenient for this simple exercise.

Note the URL that is generated. You will use this URL for testing.

Step 5: Test. Use the DNS name provided in Step 4 to test against the following URL:

https://payroll.default.20.121.248.sslip.io/employeeConsumer

2.9 Setting Up VMware Tanzu TKG and Knative

As with other Kubernetes offerings, setting up and installing Knative on TKG on your local laptop is pretty straightforward.

VMware released a community edition that can be locally installed. Additional information can be found at https://tanzucommunityedition.io/download/.

Step 1: Set up TKG locally. Follow the instructions provided on the website to set up TKG locally. This will give you an unmanaged cluster.

Figure 2-48 shows a successful install.

You will get a message that the payroll cluster has been completed, as shown in Figure 2-49.

Step 4: Deploy the app on Knative, as shown in Figure 2-50.

Note the URL provided. In this case, it is http://payroll.default.127.0.0.241.nip.io.

Step 5: Test Follow Section 2.5, Step 5 and use Postman or curl to test against the http://payroll.default.127.0.0.241.nip.io/employeeConsumer URL.

This completes the successful deployment of Tanzu Kubernetes Grid or TKG.

2.10 Summary

This chapter explained how Spring Cloud Function can be developed and deployed to various target platforms, including AWS Lambda, Google Functions, Azure Functions, and Kubernetes environments such as AWS EKS, Google’s GKE, Azure AKS, VMware Tanzu, and the like. With Kubernetes, you were able to use Knative to deploy the same image to all the other Kubernetes flavors without changing the code. Knative ensures that the code is portable across any Kubernetes platform. This is critical, as it ensures movement to any cloud with minimal disruptions. To summarize, you should now have a good understanding of how to build serverless functions with Spring Cloud Function. The next chapter looks at how to automate the deployment process for Spring Cloud Function.

3.1 GitHub Actions

This is a CI/CD platform tightly integrated with GitHub; it allows you to create and trigger workflows from GitHub. So, if you are a fan of GitHub, you will really like this new feature. When you sign up for GitHub, GitHub Actions will automatically be integrated into your project, so you do not have to use a separate tool like Jenkins or Circle CI. Of course, this means that you are restricted to GitHub as your code repository. Creating a workflow is quite straightforward. You can create a workflow directly on the GitHub website by navigating to your project and clicking the New Workflow button in the Actions tab, as shown in the Figure 3-1.

A screenshot of the banup-kubeforce window. Tab action is selected. It describes all workflows. It contains runs from all workflows, events, statuses, brands, and actors.

Upon clicking New Workflow, as shown in Figure 3-2, you will be taken to the workflow “marketplace,” where you can choose from the suggested flows or set up a workflow yourself. Click the Set Up a Workflow Yourself link to start creating a custom workflow.

A screenshot of the banup-kubeforce window. Tab actions is selected which displays a message to choose a workflow.

The Set Up a Workflow Yourself window, as shown in Figure 3-3, will take you to the page where you can write the script to create your workflow.

A screenshot of a banup-kubeforce window. Tab code is selected. Code is present on the left side of the window. The right side of the window contains feature actions under the tab marketplace. The featured actions are upload a build artifact and set up Java J D K.

Let’s dive deep into the components of the YAML file.

A screenshot of workflow code elaboration. It contains name, on-push, jobs, runs-on ubuntu-latest, steps, uses, and run along with its details.

This sets up the GitHub Actions workflow and the triggers. Now, every time you commit or push code to this project repository in GitHub, this workflow will execute. Figure 3-5 shows a sample execution of this workflow.

A screenshot of chapter 3 S C F plus Lambda. Build-deploy is selected under jobs. Build-deploy window contains set up job, run action, run a w s-actions, build with Maven, run Sam package and deploy, post run a w s, etc.

3.2 ArgoCD

Declarative GitOps allows the programmer or the ones who created the application to control the configuration of the environment in which the environment will run. This means the programmer does not have to rely on different teams, such as infrastructure or DevOps teams, to manage the pieces of the application. The programmers are in control, and this is a good thing.

ArgoCD set up is mostly programmatic and relies on the underlying Kubernetes configmaps. This is, as you can see, is different from other tools like Jenkins.

Here is how I set up the ArgoCD environment.

Step 1: Create a namespace for ArgoCD.

A screenshot of creating an Argo C D namespace. The screenshot reads, dollar sign Kubectl create namespace Argo c d namespace forward slash Argo c d created.

A screenshot describes the status of an Argo C D installation run logs. It includes name, status, restarts, etc.

Now you can see that the ArgoCD services are up and running. Notice that an external IP has not been associated with service/argocd-server.

A screenshot of the external I P of Argo c d-server run logs. It contains name, type, cluster-I P, external-I P, port, and age.

You will see an external IP associated with argocd-server. This will allow you to connect to the argocd-server.

Before you start to use ArgoCD, you need to change the “admin” user password. You can use Kubectl to read the secret associated with the “admin” user.

A screenshot of getting the password of admin user. The screenshot depicts a code.

The output of this command is the password for the “admin” user.

You can now log in using the web browser. Then navigate to the 20.119.112.240 URL to change the password.

A screenshot of logging in to Argo C D run logs. If it gets a warning error, the command is a certificate signed by an unknown authority. Run logs contain details and the admin logged in successfully.

You have successfully installed and connected to ArgoCD. Figure 3-11 shows a sample of the ArgoCD UI.

A screenshot of deploying an application in Argo C D U I. It depicts Payroll-h 2 which contains project, labels, status, repository, path, destination, etc. The status is healthy and synced.

Now that you have learned about GitHub Actions and ArgoCD, you can move on to deploying your application and automating the CI/CD process.

3.3 Building a Simple Example with Spring Cloud Function

You will use the same example from Chapter 2, but instead of using the EmployeeConsumer interface, this example uses EmployeeSupplier. In order to do that, you need a prepopulated database. You’ll then query the database using a supplier function. You can find the code at https://github.com/banup-kubeforce/payroll-h2.

Here are the required changes.

Add these two files to the resources folder of the main project, as shown in Figure 3-12.

A screenshot of the spring boot project structure. In a project structure, the s r c folder is selected. It contains a resources folder. Resources folder includes data. s q l, and schema. s q l folders along with other folders.

Also add spring.jpa.defer-datasource-initialization=true to ensure that the data gets populated on startup.

No other code changes are required. It is important to note that the changes you made only affect the configuration file.

3.4 Setting Up a CI/CD Pipeline to Deploy to a Target Platform

As discussed in the introduction of this chapter, you’ll use two tools for the CI/CD process. GitHub Actions can be used as a tool for both CI (Continuous Integration) and CD (Continuous Deployment), while ArgoCD is a CD tool.

ArgoCD was designed for Kubernetes, so you can leverage this tool exclusively for Knative/Kubernetes deployment. You’ll use GitHub Actions for serverless environments such as Lambda.

Figure 3-13 shows the flow when deploying to serverless environments like AWS Lambda, Google Cloud Functions, and Azure Functions.

A flow diagram of deploying to serverless functions. The code created is pushed to GitHub through Git flows repo name and branch name. Event-triggered from GitHub results in GitHub actions and deploys to spring cloud function.

Figure 3-13 shows the flow when deploying Spring Cloud Function to a Kubernetes environment with Knative configured.

A flow diagram describes deploying to a Knative-kubernetes environment. Commit code through Git is pushed to GitHub. Event-triggered GitHub actions are pushed to docker and synced and pulled to Argo. It deploys the spring cloud function on kubernetes.

3.5 Deploying to the Target Platform

This section looks at the process of deploying the Spring Cloud Function to the target environments such as AWS Lambda, Google Cloud Functions, Azure Functions, and Knative on Kubernetes.

3.5.1 Deploying to AWS Lambda

Deploying to AWS Lambda requires using a SAM (Serverless Application Model) based GitHub Actions script. This section explains how to use SAM and GitHub Actions. There is no additional coding required.

Prerequisites:

• AWS account

• AWS Lambda Function subscription

• S3 bucket to store the code build

• AWS CLI (optional) to verify deployments through the CLI

• Code from GitHub at https://github.com/banup-kubeforce/payroll-aws-h2

A flow diagram describes deploying spring cloud function with GitHub actions. Commit code through Git is pushed to GitHub. Event-triggered GitHub actions deploy spring cloud function on lambda.

Step 1: Spring Cloud Function code in GitHub.

Push the code to GitHub. You can bring down code from GitHub at https://github.com/banup-kubeforce/payroll-aws-h2. This code can be modified to your specs and deployed to the repository of your choice.

Step 2: Implement GitHub Actions with AWS SAM. AWS SAM (Serverless Application Model) is a framework for building serverless applications. More information can be found at https://aws.amazon.com/serverless/sam/.

AWS has a sample SAM-based actions script that is available in the GitHub marketplace that you can leverage. This script will execute the SAM commands.

The action code in Listing 3-5 can be created on the GitHub Actions dashboard or in your IDE

name: CI

on:

  push:

    branches: [ "master" ]

jobs:

  build-deploy:

    runs-on: ubuntu-latest

    steps:

      - uses: actions/checkout@v2

      - uses: actions/setup-java@v2

        with:

          java-version: '11'

          distribution: 'temurin'

          cache: maven

      - uses: aws-actions/setup-sam@v1

      - uses: aws-actions/configure-aws-credentials@v1

        with:

          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}

          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

          aws-region: us-east-2

      - name: Build with Maven

        run: mvn -B package --file pom.xml

      # sam package

      - run: sam package --template-file template.yaml --output-template-file packaged.yaml --s3-bucket payrollbucket

# Run Unit tests- Specify unit tests here

# sam deploy

      - run: sam deploy --no-confirm-changeset --no-fail-on-empty-changeset --stack-name payroll-aws --s3-bucket payrollbucket --capabilities CAPABILITY_IAM --region us-east-2

Listing 3-5

Workflow for Payroll Function to be Deployed on AWS Lambda

The secrets for these two elements can be stored in GitHub secrets:

          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}

          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

Figure 3-16 shows the place to store configuration secrets for GitHub Actions. It is under the Settings tab.

A screenshot of the ban up-Kubeforce window. The settings tab is selected. It displays action secrets and environmental secrets with its details.

Step 3: Execute GitHub Actions. Once the workflow is configured, the actions can be triggered from the GitHub console or through a code commit, as defined by the following code in the sam-pipeline.yaml file.

on:

  push:

    branches: [ "master" ]

GitHub Actions execute the steps outlined in the YAML file and deploy the function to AWS Lambda. Figure 3-17 shows a successful run.

A screenshot of the banup-kubeforce window. The build deploy button is selected. It displays a new window on build deploy which includes set up job, build with Maven, run Sam package, etc.

Step 4: Verify that the function is up and running in AWS Lambda.

A screenshot of A W S Lambda window. The function is selected. It displayed the function name.

Since the payroll-aws-h2 application exposes EmployeeSupplier, you will do a simple GET against the function to see if you get the result of data that has been inserted into the database on Spring Cloud Function startup.

A screenshot of the Lambda function window. It depicts function overview with its description.

In Figure 3-19, you can see that the test was successful and see a JSON result of what is in the database.

3.6 Deploying to GCP Cloud Functions

Deploying to GCP Cloud Functions using GitHub Actions is a bit intrusive, as you have to add a MANIFEST.MF file to the resources folder. See the code in GitHub.

A flow diagram describes G C P and GitHub actions flow. Commit code through Git is pushed to GitHub. Event-triggered GitHub actions deploy spring cloud function in google cloud function.

Step 1: Spring Cloud Function code in GitHub. Push your code to GitHub. If you have cloned the https://github.com/banup-kubeforce/payroll-gcp-h2.git then you have everything that you need to push the code to your repository.

Step 2: Set up Cloud Functions actions.

Note that you will have to store your GCP_CREDENTIALS in the GitHub Secrets dashboard.

As in the previous example with AWS Lambda, note that the steps to check out, set up, and build Maven are the same. For the authentication and deployment, you use the Google Cloud CLI. The Set up Cloud SDK task will download and set up the Google CLI. You can use the same command line script that you used when you deployed from a laptop in Chapter 2.

This can be done on the GitHub Actions website or in the IDE. You can go to GitHub and commit a change by doing a simple modification at the “master” branch. This will start the GitHub Action flow.

A screenshot of the banup-kubeforce window. The build deploy button is selected. A new window on build deployment is displayed, it includes set up job, build with Maven, run Sam package, etc.

Once the actions successfully complete the job, you can go to the Google Cloud Functions dashboard and test the function. Again, you execute a simple GET against the EmployeeSupplier function.

Step 4: Test the function.

Before you test the function, ensure that you pick the function to be invoked from an unauthenticated device such as your laptop. Once you’re done testing, remove the privilege to avoid unnecessary invocations.

A screenshot of the unauthenticated set for payroll-g c p- h 2 which is allowed. It includes a link to 1st gen payroll along with other details.

You can go to the console of your function in the Google Cloud Functions dashboard and execute the test. You do not have to provide any input; simply click the Test the Function button to execute the test. You will see the output of EmployeeSupplier in the Output section, as shown in Figure 3-23.

A screenshot of the google cloud window. It displays cloud functions. In payroll-g c p-h 2, a function is selected. It configures triggering events and displays the test command.

3.7 Deploying to Azure Functions

Spring Cloud Function on Azure Functions require a bit of tweaking, as you learned in Chapter 2. This is because the configuration is not externalized, as with AWS Lambda or GCP Cloud Functions. This does not mean that you cannot deploy easily. You have to understand how Azure Function code interprets Spring Cloud Function code and execute. See Chapter 2 for discussions around this issue; make sure that you execute and test locally before pushing to the Azure cloud.

Step 1: Spring Cloud Function code in GitHub. Push your code to GitHub. If you cloned the https://github.com/banup-kubeforce/payroll-azure-h2.git, you have everything that you need to push the code to your repository.

A flow diagram describes flow of spring cloud function deployment on azure. Commit code through Git is pushed to GitHub. Event-triggered GitHub actions and deploy spring cloud function in Azure functions.

Any commit or push to the “master” branch will trigger an execution of GitHub Actions; see Figure 3-25.

A screenshot of the summary window. The build and deploy button is selected which displays build deployment window which includes set up job, build with Maven, run Sam package, etc.

After successfully deploying using GitHub actions, you need to verify the deployment in the Azure Functions dashboard; see Figure 3-26.

A screenshot of the azure functions dashboard. Functions selected under payroll-kubeforce includes name, trigger and status with its details.

Click the employeeSupplier link to get to Figure 3-27.

A screenshot of the employee supplier dashboard. The tab overview is selected. The get function under overview is selected which displays essentials along with other details such as function app, status, etc.

A screenshot of the employee supplier dashboard. The tab overview is selected. It displays the Get function U r l along with the default function key and ok tab.

The URL of the function is https://payroll-kubeforce.azurewebsites.net/api/employeeSupplier. Use this URL for testing.

Step 4: Testing. You will use an external testing tool to see if the deployed functions work. The tool you use here is Postman.

You can simply use a GET operation to test, as shown in Figure 3-29.

A screenshot of the window of testing of the payroll-Kubeforce azure website. The tab body is selected. It displays options raw, binary, and Graph Q L along with other details.

This completes the deployment of Spring Cloud Function on Azure Functions using GitHub Actions.

3.8 Deploying to Knative on Kubernetes

The CI/CD for deploying Spring Cloud Function on Knative are similar for every Kubernetes. The only change is the cluster name. This section uses ArgoCD (http://argoproj.github.io) for CD even though you can achieve the same result with GitHub Actions. I found GitHub Actions a bit code-intensive. I wanted to separate the CD process and have a good visual tool that shows the deployment. ArgoCD provides a good visual interface.

To have a common repository for all the cloud environments, you’ll use Docker hub in this example. Docker hub provides a good interface for managing images and it is popular with developers. If you use ECR, GCR, or ACR, you’ll experience vendor lock-in.

A flow diagram describes deployment flow for spring cloud function with Github actions and Argo C D. Commit code through Git pushes it to GitHub. Event-triggered Github actions push to docker and pull and sync to Argo. It deploys spring cloud function in kubernetes.

Once you have the prerequisites set up, you can begin configuring an automated CI/CD pipeline. For this example implementation, you’ll use the code from GitHub at https://github.com/banup-kubeforce/payroll-h2.git.

Step 1: Spring Cloud Function code in GitHub. Push your code to GitHub. You can use the code for payroll-h2 in GitHub.

This code creates a Docker image and pushes it to the Docker hub. You can store the username and password as secrets in the GitHub site (see Figure 3-31).

A screenshot of the payroll-Kubeforce window. In settings, action is selected. It displays environment and repository secrets and their details.

                  username: ${{ secrets.DOCKERHUB_USERNAME }}

                  password: ${{ secrets.DOCKERHUB_TOKEN }}

Step 3: Execute GitHub Actions to build and push the Docker image.

A screenshot of the banup-kubeforce window. The docker button is selected. under docker the following is displayed setup job, set up Q E M U, build with Maven, run the Sam package, etc.

Step 4: Configure ArgoCD.

This will create app payroll-h2 in ArgoCD, as shown in Figure 3-33.

A screenshot of the application window. It depicts the Payroll-h 2 which includes project, labels, status, repository, path, destination, etc. The status is healthy and synced.

Step 5: Sync the project in Argo CD.

Now that you have created the app and it is pointing to the GitHub repository, make sure you have a connection to the repo, as shown in Figure 3-34. I connected to the repo using HTTPS. This will allow the app to poll for changes and trigger the flow to push the Docker image to the specified Kubernetes environment.

A screenshot of the repository window. It depicts type, name, repository, and connection status.

You can also run a deployment manually by clicking SYNC, as shown in Figure 3-35.

A screenshot of payroll-h 2 window. It depicts the project, labels, status, repository, target, path, destination, and name. The status is healthy and synced.

Figure 3-36 shows a successful sync process in ArgoCD.

A screenshot of deploying the application window. Select synced button. Payroll-h 2 is displayed. It includes app health, current synchronize status, and last synchronize result.

Step 6: Check if the function has been deployed. Navigate to the Kubernetes dashboard on the Azure Portal and verify that the service has been deployed. See Figure 3-37.

A screenshot of Kubernetes services dashboard. It displays payroll c l s t r of services and ingresses. Services and ingresses are selected and the status all are ok.

Step 7: Testing. The best way to get the URL to test is to connect to the cluster via the command line and get the URL, as explained in Chapter 2.

Run $kn service list to get the URL for testing, as shown in Figure 3-38.

A screenshot of service list run logs. It contains name, U R L, latest, age, conditions, ready, and reason.

You can use Postman for testing. See Figure 3-39.

A screenshot of the Get window. params selected. It contains query parameters of key, value, description, and bulk edit. The body includes a code with other tabs such as pretty, raw, preview, etc.

This completes the successful deployment using GitHub Actions and ArgoCD.

3.9 Summary

In this chapter, you learned how to set up some CI/CD tools to create an automated deployment for your Spring Cloud Function.

You learned how to trigger the deployment of functions on Lambda, Google Cloud Functions, and Azure Functions.

You also learned that you can combine the build of Docker images stored in Docker hub and ArgoCD to deploy the image to any Kubernetes cluster that is running Knative.

If you want to achieve “write-once deploy-anywhere,” you have to look at using Kubernetes and Knative. Spring Cloud Function is really a portable function.

4.1 Data Event Pipelines

Let’s look at an example. Say you are driving a car and suddenly you collide with a vehicle. After a few minutes, you get a call from OnStar asking you if you are alright as you were in a collision. How did OnStar know that you were in a collision? All this happens in a flash. Sensors in your vehicle are triggered by the collision event and begin sending data as streams into the OnStar system. The OnStar system processes the data from numerous other systems and triggers a response in the form of an OnStar operator calling you. The event that triggered the data processing along with the output is the event data pipeline. Figure 4-1 illustrates this process.

An illustration depicts the OnStar event-driven data pipeline implementation for vehicles. The vehicle sends data to nearby Cell tower, Cell tower relays data to OnStar datacenter. OnStar datacenter routes information to the call center. Through this emergency services such as police, medical response team and towing company are activated.

All data regarding the incident is collated, processed, analyzed, and sent to various target systems, both internal and external to OnStar. This is an example of event data pipeline.

A data pipeline is a set of processes that takes raw data from disparate sources, and then filters, transforms, and moves the data to target stores, where it can be analyzed and presented in a visually compelling way.

The data can be in the form of streams or batches. Streaming is usually associated with real-time to near real-time data movement and batches are usually scheduled and non-real-time. Figure 4-2 illustrates this.

An illustration depicts the data pipeline as a set of processes that ingests the structured or unstructured data and retrieves and transforms the data through data lakes or data warehouses which is further analyzed through A I or M L processes.

The data can also be unstructured or structured. Unstructured data can be from a multitude of resources, including Internet-based platforms such as Twitter, to an automotive device emanating data.

Structured data usually originates from within the company’s applications. This data needs to be combined at a landing zone of object stores. These object stores can be on-premises or in the cloud. Once the data is ingested into the object stores, it is then retrieved and transformed, usually through an ETL process in other datastores such as data lakes or data warehouses. This data is then further analyzed using BI tools such as Power BI and Tableau or further processed with the AI/ML processes.

This whole process—from data ingestion to consumption—is called the data pipeline.

Let’s dive a bit deeper into the various sub-processes outlined in the data pipeline process.

4.2 Spring Cloud Function and Spring Cloud Data Flow and Spring Cloud Streams

Spring Cloud Data Flow (SCDF) is a Spring.io-based product that supports the creation and deployment of a data pipeline. It supports batch and event-driven data, which makes it versatile. SCDF pipelines can be built programmatically or wired up through a GUI. It is heavily based on the Spring Framework and Java, which makes it very popular among Spring developers.

Figure 4-3 shows a sample dashboard of SCDF with a simple data pipeline that has a source, a processor, and a sink. You can drag-and-drop from the available components to build the pipeline. You can also build your custom source, processors, and sinks and deploy them for use within your enterprise.

A screenshot of a S C D F. It shows the stream vehicle data pipelines definition, status and applications.

As you can see from the dashboard, you can build stream-based or batch-based (task) data pipelines and manage these through a single dashboard.

SCDF, unlike other the data pipeline tools available in the cloud, can be deployed in a Kubernetes, Docker, or Cloud Foundry environment, making it a portable tool for data pipeline development and deployment.

4.2.1 Spring Cloud Function and SCDF

Spring Cloud Function and SCDF are perfectly matched, as they are built out of the same framework, Spring. You can deploy Spring Cloud Function as a source, processor, sink, or as a trigger for the pipeline. Since the data pipelines are usually invoked sporadically for processing data, you can optimize utilization of resources and costs with a Spring Cloud Function.

Let’s look at a sample implementation of Spring Cloud Function with SCDF.

In this example, you will build a simple data pipeline using RabbitMQ as a source, do a simple transformation, and store the messages in a log. You will publish sample vehicle information into a RabbitMQ topic called VehicleInfo and do a simple transformation, then store it in a log file.

RabbitMQ ➤ Transform ➤ Log

Prerequisites:

• SCDF deployed on Kubernetes or locally in Docker

• Kubernetes or Docker

• A RabbitMQ cluster/instance

• A queue to publish messages

• Code from GitHub at https://github.com/banup-kubeforce/SCDF-Rabbit-Function.git

Additional prerequisites for each environment can be found at https://dataflow.spring.io/docs/installation

Step 1: Installing Spring Cloud Data Flow.

• Local machine: Docker Compose is an easy way to get an instance of Spring Cloud Data Flow up and running. The details of the installation are available at https://dataflow.spring.io/docs/installation/local/docker/

• Kubernetes: Instructions for installing this on Kubernetes are available athttps://dataflow.spring.io/docs/installation/kubernetes/.

For this example, I used an existing cluster that I created in Chapter 2 and ran the following helm command to install the SCDF, which results in the output shown in Figure 4-4.

$helm install scdf bitnami/spring-cloud-dataflow -n scdf-system --set server.service.type=LoadBalancer --set server.service.loadBalancerIP=${ingress} --set server.ingress.enabled=true --set server.ingress.server.host=scdf.${ingress}.xip.io

A screenshot depicts the helm command to install the S C D F. It consists of name, last deployed, namespace, status, test suite, notes, chart name, chart version and app version.

Once the command runs successfully, as shown in Figure 4-4, you can check the status by running kubectl get pods. You can also get the external IP with the kubectl get svc commands, as shown in Figures 4-5 and 4-6.

$kubectl get pods -n scdf-system

A screenshot depicts the command to get external I P of the S C D F. The command used is dollar kubect 1 get pods-n s c d f-system.

A screenshot depicts the command to get the status of the S C D F. The command used is dollar kubect 1 get s v c-n s c d f-system.

$kubectl get svc -n scdf-system

You can now access SCDF using the external IP. For example, http://20.241.228.184:8080/dashboard

The next step is to add applications. Spring provides a standard set of templates that you can use to build your pipeline.

Step 2: Add Applications to your SCDF Instance

Use the Add Application(s) button to create some starter apps that you can use to create a data pipeline; see Figures 4-7 and 4-8.

A screenshot depicts the S C D F add application view to create a data pipeline.

A screenshot depicts the options to add applications and docker-based or rabbit M Q kafka-based.

Figure 4-8 shows you the options to add applications that are custom built through the Registering One or More Applications option. You import the application coordinates from an HTTP URI or use a properties file. There is also an option to import some prebuilt starters from Spring. Furthermore, you can choose starters that are Maven- or Docker-based or RabbitMQ- or Kafka-based. RabbitMQ and Kafka are used as backbone messaging systems for internal SCDF components and not for external use. When deploying to a Kubernetes cluster, you have to choose a Docker-based starter. When deploying locally in a Docker environment, you can choose between Maven and Docker-based starters.

Figure 4-9 shows prebuilt templates with the Docker images URI. SCDF is running on a Kubernetes environment, so the prebuilt images have a Docker URI.

A screenshot depicts the S C D F running on a Kubernetes environment and prebuilt templates installed in S C D F.

These prebuilt templates come in three categories—source, processor, and sink. They allow you to wire up a data pipeline without the need for coding. If you want a custom component, you can follow the examples in https://dataflow.spring.io/docs/stream-developer-guides/.

The next step is to create a stream using the starter templates you loaded.

Step 3: Create a Stream

The Streams dashboard allows you to wire up a data pipeline. Click the Create Streams button to start the process, as shown in Figure 4-10.

A screenshot depicts the streams button to create a stream in the streams dashboard.

Then perform the following steps:

1. 1.

   Pick the source.

    

This example uses a RabbitMQ as the source, so pick Rabbit from the available options, as shown in Figure 4-11.

A screenshot depicts the create stream and picking the available option from the source.

1. 2.

   Pick a processor.

    

Pick the Transform component from the list, as shown in Figure 4-12.

A screenshot depicts a selection and creation of a processor for the transform component.

1. 3.

   Pick a sink.

    

Pick the Log component from the list, as shown in Figure 4-13.

A screenshot depicts a selection and creation of a sink for the log component.

Now you wire them by dragging from the output of the first component to the input of the second component, as depicted in Figure 4-14.

A screenshot depicts wiring the data pipeline by dragging the output of the first component to the input of the second component.

The next step is to configure the pipeline.

1. 4.

   Configure RabbitMQ.

    

You have to point the Rabbit source to your RabbitMQ instance. There are three categories in the form to populate. Open and fill in the appropriate fields. I used username, password, port, host, and queue, as shown in Figure 4-15.

Two screenshot depicts the configured Rabbit M Q and the properties of the Rabbit. It consists of a username, password, port and host.

Click Update once you’re done with the configuration.

1. 5.

   Configure the transform.

    

In this example, you simply transform the incoming string to uppercase. Note that you have to use SpEL (Spring Expression Language). More information on SpEL Is available at https://docs.spring.io/spring-framework/docs/3.0.x/reference/expressions.html. See Figure 4-16.

A screenshot depicts the properties for transform. It consists of general and spel. function and update and cancel buttons.

1. 6.

   Configure the sink.

    

In this case, you just use a simple log. No configuration is required. See Figure 4-17.

A screenshot depicts setting properties for log. It represents general, log along with cancel and update buttons.

1. 7.

   Wire up the data pipeline.

    

Once you wire up, you will get a data pipeline. This will also show a Stream DSL (Domain Specific Language) expression that you can save and reuse using a SCDF shell or dashboard, as shown in Figure 4-18.

rabbit --password=pa55word --port=5672 --host=20.40.208.246 --username=banup --queues=VehicleInfo | transform --expression='new String(payload).toUpperCase()' |log

A screenshot depicts the wiring up of the data pipeline. It represents stream D S L expression on create stream dashboard.

The pipeline can be triggered by an external Spring Cloud Function on Lambda or using Google Cloud Functions, and so on. See Figure 4-19.

1. 8.

   Deploy the stream

    

A screenshot depicts the successful deployment of a data pipeline by an external Spring Cloud Function.

Step 4: Create a function to publish data to RabbitMQ

Here, you create a Spring Cloud Function to publish the data to start the data-pipeline process.

The components that you will create are as follows:

• SenderConfig

• QueueSender

• SenderFunction

Prerequisites:

• A POM with the necessary dependencies for RabbitMQ and Spring Cloud Functions, as shown in Listing 4-1.

    <dependencies>

        <dependency>

            <groupId>org.springframework.boot</groupId>

            <artifactId>spring-boot-starter-data-jpa</artifactId>

        </dependency>

        <dependency>

            <groupId>org.springframework.boot</groupId>

            <artifactId>spring-boot-starter-web</artifactId>

        </dependency>

        <dependency>

            <groupId>org.springframework.boot</groupId>

            <artifactId>spring-boot-starter-amqp</artifactId>

        </dependency>

        <dependency>

            <groupId>org.springframework.cloud</groupId>

            <artifactId>spring-cloud-function-web</artifactId>

        </dependency>

        <dependency>

            <groupId>org.springframework.boot</groupId>

            <artifactId>spring-boot-starter-test</artifactId>

            <scope>test</scope>

        </dependency>

    </dependencies>

    <dependencyManagement>

        <dependencies>

            <dependency>

                <groupId>org.springframework.cloud</groupId>

                <artifactId>spring-cloud-dependencies</artifactId>

                <version>${spring-cloud.version}</version>

                <type>pom</type>

                <scope>import</scope>

            </dependency>

    </dependencyManagement>

Listing 4-1

pom.xml with RabbitMQ Dependencies

Add the RabbitMQ server and configuration references to the application.properties file, as in Listing 4-2.

spring.cloud.function.definition=senderFunction

spring.rabbitmq.host=20.40.208.246

spring.rabbitmq.port=5672

spring.rabbitmq.username=banup

spring.rabbitmq.password=pa55word

queue.name=VehicleInfo

Listing 4-2

application.properties

     1.     Create the SenderConfig Component

This is a simple setter for the queue name. You can expand this to include other RabbitMQ configurations. This is the entity definition (see Listing 4-3).

package com.kubeforce.scdffunctiontigger;

import org.springframework.amqp.core.Queue;

import org.springframework.beans.factory.annotation.Value;

import org.springframework.context.annotation.Bean;

import org.springframework.context.annotation.Configuration;

@Configuration

public class SenderConfig {

    @Value("${queue.name}")

    private String message;

    @Bean

    public Queue queue() {

        return new Queue(message, true);

    }

}

Listing 4-3

application.properties

     2.     Create the QueueSender Component

You use a RabbitTemplate from the springframework amqp library to define the connection to the RabbitMQ instance (see Listing 4-4).

package com.kubeforce.scdftrigger;

import org.springframework.amqp.core.Queue;

import org.springframework.amqp.rabbit.core.RabbitTemplate;

import org.springframework.beans.factory.annotation.Autowired;

import org.springframework.stereotype.Component;

@Component

public class QueueSender {

    @Autowired

    private RabbitTemplate rabbitTemplate;

    @Autowired

    private Queue queue;

    public void send(String order) {

        rabbitTemplate.convertAndSend(this.queue.getName(), order);

    }

}

Listing 4-4

QueueSender.java

     3.     Wrap the Sender in Spring Cloud Function framework

This example uses the queueSender to send the data (see Listing 4-5).

package com.kubeforce.scdftrigger;

import org.springframework.beans.factory.annotation.Autowired;

import java.util.function.Function;

public class SenderFunction implements Function<String,String> {

    @Autowired

    private QueueSender queueSender;

    @Override

    public String apply(String s) {

        queueSender.send("Vehicle:SUV,Make:Ford,Model:Edge,Year:2021");

        return "ok. done";

    }

}

Listing 4-5

SenderFunction.java

Step 5: Test the function using Postman

Use the GET function on Postman and provide the URL to the senderFunction.

A screenshot depicts the G E T function on Postman with the U R L.

You should get the result shown in this image.

Check the RabbitMQ queue for any messages.

A screenshot depicts the Rabbit M Q Queue for any messages. It represents Exchange, Routing key, Redelivered, Properties, and Payload under the Queries tab with a message the server reported 0 messages remaining.

Now you have a function that can post messages to a RabbitMQ topic. The SCDF data pipeline will be listening to the queue and will start processing.

You can also look at the logs associated with each of the components to monitor the status of the data pipeline, as shown in Figure 4-20.

A screenshot depicts the details of the vehicle data pipeline. It shows the logs associated with each of the components to monitor.

You have seen how to create a data pipeline in SCDF that monitors a topic in RabbitMQ. You created a Spring Cloud Function that posts messages into the RabbitMQ topic. Spring Cloud Function can also be deployed as a source in SCDF; more information on how to develop code for SCDF is available on the SCDF site.

4.3 Spring Cloud Function and AWS Glue

AWS Glue works very similarly to Spring Cloud Data Flow in that you can wire up a data pipeline that has a source, processor, and sink. More information can be found at https://us-east-2.console.aws.amazon.com/gluestudio/home?region=us-east-2#/.

Spring Cloud Function can participate in the data pipeline process as a trigger, or simply by integrating with one of the components in the data pipeline.

For example, if you have AWS Kinesis as a source and you need to get data from a vehicle, you can have Spring Cloud Function stream the data that it gets into AWS Kinesis.

In the example in this section, you will be publishing data into AWS Kinesis and then kick off an AWS Glue job manually.

The flow will be:

Spring Cloud Function ➤ Kinesis ➤ AWS Glue Job ➤ S3

It is assumed that you have some knowledge of AWS Glue, as we do not delve into the details of this product. The focus is on creating the Spring Cloud Function.

4.3.1 Step 1: Set Up Kinesis

You can get to Kinesis at https://us-east-1.console.aws.amazon.com/kinesis/home?region=us-east-1#/home.

Once you subscribe to Kinesis, you can begin creating the data stream. Make sure you select on-demand (see Figure 4-21), as this will save you some money.

A screenshot depicts the setup for Kinesis to create a data stream for a vehicle. It shows the data stream configuration, data stream capacity and data stream settings.

A screenshot depicts the active vehicle data stream under the monitoring tab.

You can now connect and publish to the stream using your Spring Cloud Function.

4.3.2 Step 2: Set Up AWS Glue

You can access the AWS Glue Studio at https://us-east-1.console.aws.amazon.com/gluestudio/home?region=us-east-1#/.

Once you have the subscription, you can begin creating a glue job.

Go to the AWS Glue Studio to start the creation of the glue job, as shown in Figure 4-23.

A screenshot depicts the creation of a job on A W S glue studio.

Create a job called vehicledatapipeline. Use Amazon Kinesis as the source and Amazon S3 as the target. Ensure that you set the proper configurations for each of these components.

The job shown in Figure 4-24 will read Amazon Kinesis shards and post the data into Amazon S3.

A screenshot depicts the Configuration of amazon kinesis as the source and amazon S 3 integration for each component.

Now you have a job in AWS Glue that you can trigger manually or via a function.

4.3.3 Step 3: Create a Function to Load Data into Kinesis

1: Set the application properties. Make sure that you provide the stream name. AWS information has to be set, as shown in Listing 4-6.

spring.cloud.function.definition=producerFunction

#use your aws credentials here

aws.access_key =

aws.secret_key =

aws.region = us-east-2

#use your stream name that you have created

aws.stream_name = vehicledatastream

Listing 4-6

application.properties

2: Add the kinesis dependencies. Make sure to add the necessary libraries. The latest version of producer has some bugs, so I used the working version, as shown in Listing 4-7.

<dependency>

    <groupId>com.amazonaws</groupId>

    <artifactId>amazon-kinesis-client</artifactId>

    <version>1.14.1</version>

</dependency>

<dependency>

    <groupId>com.amazonaws</groupId>

    <artifactId>amazon-kinesis-producer</artifactId>

    <version>0.13.1</version>

</dependency>

<dependency>

    <groupId>com.amazonaws</groupId>

    <artifactId>amazon-kinesis-client</artifactId>

    <version>1.14.1</version>

</dependency>

Listing 4-7

pom.xml dependecies

3: Create the model. This is a simple model for tracking a vehicle’s detail. See Listing 4-8.

public class TrackDetail {

    private String vehicleId;

    private String driverId;

    private String driverName;

    public String getVehicleId() {

        return vehicleId;

    }

    public void setVehicleId(String vehicleId) {

        this.vehicleId = vehicleId;

    }

    public String getDriverId() {

        return driverId;

    }

    public void setDriverId(String driverId) {

        this.driverId = driverId;

    }

    public String getDriverName() {

        return driverName;

    }

    public void setDriverName(String driverName) {

        this.driverName = driverName;

    }

}

Listing 4-8

TrackDetail.java

4: Create the Kinesis producer. This interface is nice to have; see Listing 4-9.

public interface ProducerService {

    public void putDataIntoKinesis(String payload) throws Exception;

    public void stop();

}

Listing 4-9

ProducerService.java

5: Create the producer implementation. The implementation provided in Listing 4-10 sets the method for posting to Kinesis.

import com.amazonaws.auth.AWSStaticCredentialsProvider;

import com.amazonaws.auth.BasicAWSCredentials;

import com.amazonaws.services.kinesis.producer.*;

import com.google.common.util.concurrent.FutureCallback;

import com.google.common.util.concurrent.Futures;

import com.google.common.util.concurrent.ListenableFuture;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

import org.springframework.beans.factory.annotation.Value;

import org.springframework.stereotype.Service;

import java.io.UnsupportedEncodingException;

import java.nio.ByteBuffer;

import java.util.concurrent.ExecutorService;

import java.util.concurrent.Executors;

import java.util.concurrent.atomic.AtomicLong;

@Service

public class ProducerServiceImpl implements ProducerService {

    private static final Logger LOG = LoggerFactory.getLogger(ProducerServiceImpl.class);

    @Value(value = "${aws.stream_name}")

    private String streamName;

    @Value(value = "${aws.region}")

    private String awsRegion;

    @Value(value = "${aws.access_key}")

    private String awsAccessKey;

    @Value(value = "${aws.secret_key}")

    private String awsSecretKey;

    private KinesisProducer kinesisProducer = null;

    // The number of records that have finished (either successfully put, or failed)

    final AtomicLong completed = new AtomicLong(0);

    private static final String TIMESTAMP_AS_PARTITION_KEY =

            Long.toString(System.currentTimeMillis());

    public ProducerServiceImpl() {

        this.kinesisProducer = getKinesisProducer();

    }

    private KinesisProducer getKinesisProducer() {

        if (kinesisProducer == null) {

            BasicAWSCredentials awsCreds = new BasicAWSCredentials("AKIAZKRBTXXLPA3V2RW5", "kUqeO3bJyHEGziM09ru83/yU5vulbYagqHXmM4zG");

            KinesisProducerConfiguration config = new KinesisProducerConfiguration();

            config.setRegion("us-east-2");

            config.setCredentialsProvider(new AWSStaticCredentialsProvider(awsCreds));

            config.setMaxConnections(1);

            config.setRequestTimeout(6000); // 6 seconds

            config.setRecordMaxBufferedTime(5000); // 5 seconds

            kinesisProducer = new KinesisProducer(config);

        }

        return kinesisProducer;

    }

    @Override

    public void putDataIntoKinesis(String payload) throws Exception {

        FutureCallback<UserRecordResult> myCallback = new FutureCallback<UserRecordResult>() {

            @Override

            public void onFailure(Throwable t) {

                // If we see any failures, we will log them.

                int attempts = ((UserRecordFailedException) t).getResult().getAttempts().size() - 1;

                if (t instanceof UserRecordFailedException) {

                    Attempt last =

                            ((UserRecordFailedException) t).getResult().getAttempts().get(attempts);

                    if (attempts > 1) {

                        Attempt previous = ((UserRecordFailedException) t).getResult().getAttempts()

                                .get(attempts - 1);

                       LOG.error(String.format(

                                "Failed to put record - %s : %s. Previous failure - %s : %s",

                                last.getErrorCode(), last.getErrorMessage(),

                                previous.getErrorCode(), previous.getErrorMessage()));

                    } else {

                        LOG.error(String.format("Failed to put record - %s : %s.",

                               last.getErrorCode(), last.getErrorMessage()));

                    }

                }

                LOG.error("Exception during put", t);

            }

            @Override

            public void onSuccess(UserRecordResult result) {

                long totalTime = result.getAttempts().stream()

                        .mapToLong(a -> a.getDelay() + a.getDuration()).sum();

                LOG.info("Data writing success. Total time taken to write data = {}", totalTime);

                completed.getAndIncrement();

            }

        };

        final ExecutorService callbackThreadPool = Executors.newCachedThreadPool();

        ByteBuffer data = null;

        try {

            data = ByteBuffer.wrap(payload.getBytes("UTF-8"));

        } catch (UnsupportedEncodingException e) {

            e.printStackTrace();

        }

        // wait until unfinished records are processed

        while (kinesisProducer.getOutstandingRecordsCount() > 1e4) {

            Thread.sleep(1);

        }

        // write data to Kinesis stream

        ListenableFuture<UserRecordResult> f =

                kinesisProducer.addUserRecord(streamName, TIMESTAMP_AS_PARTITION_KEY, data);

        Futures.addCallback(f, myCallback, callbackThreadPool);

    }

    @Override

    public void stop() {

        if (kinesisProducer != null) {

            kinesisProducer.flushSync();

            kinesisProducer.destroy();

        }

    }

}

Listing 4-10

ProducerServiceImpl.java

6: Create the producer function. This is the critical function that will be exposed to post data into Kinesis; see Listing 4-11.

import com.fasterxml.jackson.core.JsonProcessingException;

import com.fasterxml.jackson.databind.ObjectMapper;

import org.springframework.beans.factory.annotation.Autowired;

import java.util.function.Function;

public class ProducerFunction implements Function<TrackDetail,String> {

    @Autowired

    private ProducerService producerService;

    @Override

    public String apply(TrackDetail trackDetail) {

        ObjectMapper mapper = new ObjectMapper();

        String data = "";

        try {

            data = mapper.writeValueAsString(trackDetail);

            producerService.putDataIntoKinesis(data);

        } catch (JsonProcessingException e) {

            e.printStackTrace();

        } catch (Exception e) {

            e.printStackTrace();

        }

        return "Saved data into Kinesis successfully!";

    }

}

Listing 4-11

ProducerFunction.java

7: Run the function; see Figure 4-25.

A screenshot depicts the successful run of the function.

8: Test with Postman. Run a POST-based test against ProducerFunction to publish data into Kinesis.

Add text that introduces and gives context to Figure 4-26.

A screenshot depicts the post-based test against the producer function. It includes the tabs params, authorization, headers, body, pre-request script, tests, and settings with a button to send.

You will get a message that the data is saved.

9: Check Kinesis for data. This information can be found on the Kinesis dashboard under Monitoring, as shown in Figure 4-27.

A screenshot depicts the data metrics found on the kinesis dashboard.

10: Run Glue manually. From the Glue Studio, start the process by clicking Run, as shown in Figure 4-24.

The job starts to run, as shown in Figure 4-28. Check the s3 bucket for any data.

A screenshot depicts the glue job run from the glue studio.

In this section, you learned how to create a Spring Cloud Function that can post data into AWS Kinesis that is part of the data pipeline. You learned that you can publish data into Kinesis and trigger the AWS Glue pipeline manually, but I also encourage you to explore other ways you can implement Spring Cloud Function for AWS Glue, such as creating and deploying triggers. More information on how to create AWS Glue triggers in Spring is available at https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/examples-glue.html.

4.4 Spring Cloud Function and Google Cloud Dataflow

Google Cloud Dataflow is very similar to Spring Cloud Data Flow, in that it allows you to wire up a data pipeline with a source, processor, and sink. The Dataflow product is easier to develop with. You can read about Dataflow and its capabilities at https://cloud.google.com/dataflow.

For the example in this section, you will create a dataflow that includes cloud pub/sub:

Spring Cloud Function ➤Dataflow {Cloud Pub/Sub ➤ Cloud Storage}

Step 1: Create and configure a cloud pub/sub instance.

Before coming to this step, ensure that you are subscribed to cloud pub/sub. Also ensure that you have proper subscriptions to the APIs.

Navigate to Cloud Pub/Sub console in your subscription to create a topic. See Figure 4-29.

A screenshot depicts the creation and configuration of cloud pub or sub-instance in google cloud.

Step 2: Create and configure a bucket in cloud storage. Create a cloud storage instance and bucket. I created a bucket called vehiclebucket1 to store the file coming from cloud pub/sub; see Figure 4-30.

A screenshot depicts the google cloud storage with vehicle bucket 1. It includes the tabs create and refresh with a list of names under buckets.

Now you are ready to build the Dataflow data pipeline.

Step 3: Create a data pipeline. Navigate to the Dataflow dashboard and create a pipeline. I created a pipeline using a prebuilt template.

1: Pick the template

This example uses the Pub/Sub to Text Files on Cloud Storage template, as shown in Figure 4-31.

A screenshot of a data flow option in google cloud. It shows the template used to create the data pipeline in the drop-down and the filter option shows many data flow template formats.

2: Set the parameters for pub/sub and cloud storage.

You will be able to pick the topic that you created in Step 1 and the bucket in Step 2. See Figure 4-32.

A screenshot depicts the data flow parameter setup. It shows the option to create a pipeline template, data flow template, regional endpoint, input pub per subtopic, output file name prefix and temporary file location.

3: Verify the creation of the data pipeline.

You can see from Figure 4-33 that a data pipeline has been created.

A screenshot depicts the successful creation of vehicle data pipeline and its pipeline summary.

An associated job will also be created, as shown in Figure 4-34.

A screenshot depicts the created job from the template in google cloud.

You can find a graphical representation of the job by drilling further into the URL of the job, as shown in Figure 4-35.

A screenshot depicts the graphical representation of the job. It consists of read pub-sub event, a 5-meter window and write files.

A screenshot depicts the vehicle bucket details. It consists of vehicle bucket 1, location, storage class, public access and protection, under the objects tab, it shows the vehicle info.

Step 4: Create the Spring Cloud Function.

1: Create the MessageEntity class.

2: Create the TrackDetail class.

3: Create the PubSubPublisher.

4: Create the Spring Cloud Function.

5: Run the application and test if a message is published to Cloud Pub/Sub.

Go to the Pub/Sub console in your Google Cloud console and verify that the message has been posted; see Figure 4-37.

A screenshot depicts the message posted in the pub per sub console in google cloud.

6: Verify is the message has been loaded into Cloud Storage.

Navigate to the Cloud Storage console in your Google Cloud to verify that the message has been loaded; see Figure 4-38.

A screenshot depicts the cloud storage console to verify that the message has been loaded into the storage. It shows the bucket details of vehicle bucket one.

In this section, you learned how to use Spring Cloud Function to trigger a Google Cloud Dataflow-based data pipeline.

4.5 Summary

This chapter explained how to create dataflow and data pipelines, whether on-premises using SCDF or in the cloud. For the cloud, you can use SCDF or cloud-native tools.

Spring Cloud Function is versatile and can be used in the context of data pipelines as a trigger or as part of the flow.

With AWS Glue and Google Data Flow, you saw that you can use Spring Cloud Function as a trigger for the flows. This requires some additional coding by adding some relevant libraries and invoking the flow.

Upcoming chapters discuss other use cases of Spring Cloud Function.

5.1 AI/ML in a Nutshell

AI/ML is gaining popularity, as it is being offered by almost all cloud providers. For AI/ML to work properly, it is important to understand the process behind it. See Figure 5-1.

A process of the M L lifecycle. The 4 processing steps are, gathering requirements, setting A I slash M L pipeline, performing A I slash M L, and monitoring.

Let’s evaluate these activities from a compute perspective. This will allow us to determine what kind of compute elements we can assign to these process

Some of the activities in this process are short lived and some are long running process. For example, deploying models and accessing the deployed models is a short lived process. While Training models and model evaluation require both a manual and programmatic intervention and will take a lot of processing time.

AI/ML processes require varying compute and storage requirements. Depending on the model size, the time taken to train, the complexity of the model, and so on, the process may require different compute and storage at different times. So, the environment should be scalable. In earlier days, AI/ML activities were conducted with a fixed infrastructure, through over-allocated VMs, dedicated bare metal servers, or parallel or concurrent processing units. This made the whole process costly and it was left to companies with deep pockets to be able to conduct proper AI/ML activities.

Today, with all the cloud providers providing some level of AI/ML activities through an API or SaaS approach, and with the ability to pay per use or pay as you go, companies small and big have begun to utilize AI/ML in their compute activities.

5.1.1 Deciding Between Java and Python or Other Languages for AI/ML

Most of the popular frameworks such as TensorFlow are written in Python, so the models’ outputs are also Python based. Therefore, it’s easy for anyone working on AI/ML to code in Python. See Figure 5-2.

A pictorial horizontal bar graph of the most popular programming languages. It mentions the logo of worldwide languages with percentages and a laptop image. Python has the highest at 25.95 and Ruby, has the lowest at 1.53.

It is very important to understand that the popularity of a language does not equate to it being a good, robust, secure language for use in AI/ML.

There are several reasons to choose Java over Python or R:

• Enterprises have standardized on Java, so they prefer to have their AI/ML platform written in Java to ease the integration into existing systems.

• Apache.org , the open source community for Java, is very robust and has many libraries and tools that have been tuned toward speed of compute, data processing, and so on. Tools such as Hadoop, Hive, and Spark are integral to the AI/ML process. Developers can easily use these tools and libraries in their java code.

• Java can be used at all touchpoints in the AI/ML process, including data collection, cleansing, labeling, model training, and so on. This way you can standardize on one language for AI/ML needs.

• JVMs allow for applications to be portable across different machine types.

• Due to Java’s object-oriented mechanisms and JVMs, it is easier to scale.

• Java-based computation for AI/ML can be made to perform faster with some tuning at the algorithm and JVM level. Therefore, it is a preferred language for sites like Twitter, Facebook, and so on.

• Java is a strong typing programming language, meaning developers must be explicit and specific about variables and types of data.

• Finally, production codebases are often written in Java. If you want to build an enterprise-grade application, Java is the preferred language.

Since Java is a preferred enterprise language for AI/ML, we can safely say that Spring Cloud Function is a better framework to use when developing enterprise-grade functions for AI/ML.

This chapter explores the different offerings from the different cloud providers and explains how you can use Spring Cloud Function with these offerings.

5.2 Spring Framework and AI/ML

A lot of frameworks have been developed in Java that can be leveraged using the Spring Framework. The latest of these frameworks was developed by AWS and is called DJL (Deep Java Library). This library can integrate with PyTorch, TensorFlow, Apache MXNet, ONNX, Python, and TFLite based models.

One of the important capabilities that you need is model serving, where you can leverage Spring Cloud Function to serve trained models, and DJL provides this capability out of the box. It’s called djl-serving.

Spring Cloud Function is unique in its ability to transcend the on-premises and cloud, especially in the realm of AI/ML. Even though cloud has become popular, not all companies have fully transitioned to the cloud. Most of them in fact have adopted a hybrid approach. Some of the applications and data still reside in the company-owned datacenters or are co-hosted in datacenters operated by third-party service providers. AI/ML activities revolving the data that is residing in the datacenters will need to have models that are trained and stored and be served using cloud functions that can be hosted in the datacenter. Cloud functions hosted in the datacenter are nearer to their data and therefore have better performance than cloud functions that are hosted in the cloud and access models that are trained and stored in the on-premises datacenters. This is where Spring Cloud Function can help serve models on-premises. See Figure 5-3.

An illustrated model of the on-prem data center. The flow of data includes trained model data, the model serving, an enterprise application, a firewall, and an external application.

5.3 Model Serving with Spring Cloud Function with DJL

Before you explore the cloud provider’s option, it’s a good idea try this out locally. To do that, you need a framework installed and access to a good tensor model and an image. The framework that you use in this example is called djl-serving.

5.3.1 What Is DJL?

Deep Java Library (DJL) https://docs.djl.ai/ is a high-level, engine-agnostic Java framework for deep learning. It allows you to connect to any framework like TensorFlow or PyTorch and conduct AI/ML activities from Java.

DJL has also great hooks to Spring Boot and can easily be invoked through the Spring Framework. DJL acts as an abstraction layer across frameworks and makes it easy to interact with those frameworks, as shown in Figure 5-4.

A model of layers of the deep java library. It includes a java framework with a logo of spring boot, and spring cloud function. An M L framework of m x net, Tensor flow, PyTorch. Deep Lava Library is mentioned between the frameworks.

There are many components in DJL that are useful to look at, but the DJL serving is interesting.

Run the following commands to get the djl-serving bits. Then unzip the file into your directory of choice and set the path to the serving.bat located at ~\serving-0.19.0\bin\serving.bat. This will allow you to execute serving from anywhere on your machine.

curl -O https://publish.djl.ai/djl-serving/serving-0.19.0.zip

unzip serving-0.19.0.zip

Listing 5-1 shows a sample run of djl-serving with a TensorFlow model.

# Load a TensorFlow model from TFHub

C:\Users\banua>serving -m "resnet=https://tfhub.dev/tensorflow/resnet_50/classification/1"

[INFO ] - Starting djl-serving: 0.19.0 ...

[INFO ] -

Model server home: C:\Users\banua

Current directory: C:\Users\banua

Temp directory: C:\Users\banua\AppData\Local\Temp\

Command line:

Number of CPUs: 16

Max heap size: 8114

Config file: N/A

Inference address: http://127.0.0.1:8080

Management address: http://127.0.0.1:8080

Default job_queue_size: 1000

Default batch_size: 1

Default max_batch_delay: 300

Default max_idle_time: 60

Model Store: N/A

Initial Models: resnet=https://tfhub.dev/tensorflow/resnet_50/classification/1

Initial Workflows: N/A

Netty threads: 0

Maximum Request Size: 67108864

[INFO ] - Initializing model: resnet=https://tfhub.dev/tensorflow/resnet_50/classification/1

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-synch-l1-2-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-file-l1-2-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/THIRD_PARTY_TF_JNI_LICENSES.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-file-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-crt-environment-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-synch-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-string-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-memory-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/msvcp140.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-util-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-datetime-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/vcruntime140.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/concrt140.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-sysinfo-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/ucrtbase.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-interlocked-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-processenvironment-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-file-l2-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/tensorflow_cc.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/libiomp5md.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/vcomp140.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-timezone-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/jnitensorflow.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-crt-convert-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-errorhandling-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-namedpipe-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-crt-math-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-crt-locale-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-crt-heap-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-profile-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/LICENSE.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-crt-utility-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-heap-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-localization-l1-2-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-debug-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-processthreads-l1-1-1.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-libraryloader-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-crt-time-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-rtlsupport-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-crt-runtime-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-crt-stdio-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-console-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/vcruntime140_1.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-processthreads-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-core-handle-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-crt-filesystem-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-crt-multibyte-l1-1-0.dll.gz ...

[INFO ] - Downloading https://publish.djl.ai/tensorflow-2.7.0/win/cpu/api-ms-win-crt-string-l1-1-0.dll.gz ...

2022-09-21 12:09:38.465035: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2

To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

[INFO ] - initWorkers for resnet (cpu()): -1, -1

[INFO ] - Loading model resnet on cpu()

2022-09-21 12:09:38.595923: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:43] Reading SavedModel from: C:\Users\banua\.djl.ai\cache\repo\model\undefined\ai\djl\localmodelzoo\ffdb59c80e9d66dc0ce00e409e06e710

2022-09-21 12:09:38.641647: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:107] Reading meta graph with tags { serve }

2022-09-21 12:09:38.641933: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:148] Reading SavedModel debug info (if present) from: C:\Users\banua\.djl.ai\cache\repo\model\undefined\ai\djl\localmodelzoo\ffdb59c80e9d66dc0ce00e409e06e710

2022-09-21 12:09:38.837590: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:210] Restoring SavedModel bundle.

2022-09-21 12:09:39.330251: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:194] Running initialization op on SavedModel bundle at path: C:\Users\banua\.djl.ai\cache\repo\model\undefined\ai\djl\localmodelzoo\ffdb59c80e9d66dc0ce00e409e06e710

2022-09-21 12:09:39.746608: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:283] SavedModel load for tags { serve }; Status: success: OK. Took 1150043 microseconds.

[INFO ] - scanning for plugins...

[INFO ] - plug-in folder not exists:C:\Users\banua\plugins

[INFO ] - 0 plug-ins found and loaded.

[INFO ] - Initialize BOTH server with: NioServerSocketChannel.

[INFO ] - BOTH API bind to: http://127.0.0.1:8080

[INFO ] - Model server started.

Listing 5-1

djl-serving Run with a Tensorflow Model

On the initial run, the model you specified will be loaded:

"resnet=https://tfhub.dev/tensorflow/resnet_50/classification/1"

On subsequent runs, the model server will be available at port 8080 at http://localhost:8080.

This example provides an image of kitten and it will try to recognize the kitten by providing output with probabilities:

$curl -O https://resources.djl.ai/images/kitten.jpg

This will show the image in Figure 5-5.

A photograph of a kitten with closed eyes and legs laid out.

Next, run the following and you will see the output with probabilities.

You provide the djl-serving instance that is running at http://localhost:8080/predictions with the kitten image that is located in the current directory, and you get a response shown in Figure 5-6, which shows that the image is probably a tabby cat. The probability is 0.4107377231121063. This is close.

A screenshot of the window. It has the class name and probability input and outputs. It represents the D J L results.

An X-ray image. It diagnosis the lungs using a saved model.

Next, you see how you can use DJL to create a Spring Cloud Function to serve models.

5.3.2 Spring Cloud Function with DJL

For this example, we borrow an example from DJL called pneumonia detection. This sample is available at https://github.com/deepjavalibrary/djl-demo/tree/master/pneumonia-detection.

This example uses an Xray image from https://djlai.s3.amazonaws.com/resources/images/chest_xray.jpg.

It predicts using a model from https://djl-ai.s3.amazonaws.com/resources/demo/pneumonia-detection-model/saved_model.zip.

The Spring Cloud Function you create will take an image, load the model, and provide a prediction, as in the cat example.

Prerequisites:

• DJL libraries

• A model: https://djl-ai.s3.amazonaws.com/resources/demo/pneumonia-detection-model/saved_model.zip

• The URL of the image to analyze: https://djl-ai.s3.amazonaws.com/resources/images/chest_xray.jpg

Step 1: Create the Spring Cloud Function with DJL framework. Add dependencies to the Hadoop file.

Add the DJL highlighted dependencies along with spring-cloud-function-web and GCP dependencies, as shown in Listing 5-2.

       <dependencies>

        <dependency>

            <groupId>org.springframework.boot</groupId>

            <artifactId>spring-boot-starter-web</artifactId>

        </dependency>

        <dependency>

            <groupId>org.springframework.cloud</groupId>

            <artifactId>spring-cloud-function-web</artifactId>

        </dependency>

        <!-- https://mvnrepository.com/artifact/ai.djl/bom -->

        <dependency>

            <groupId>ai.djl</groupId>

            <artifactId>bom</artifactId>

            <version>0.12.0</version>

            <type>pom</type>

        </dependency>

        <dependency>

            <groupId>ai.djl</groupId>

            <artifactId>api</artifactId>

            <version>0.12.0</version>

        </dependency>

        <dependency>

            <groupId>ai.djl.tensorflow</groupId>

            <artifactId>tensorflow-api</artifactId>

            <version>0.12.0</version>

        </dependency>

        <dependency>

            <groupId>ai.djl.tensorflow</groupId>

            <artifactId>tensorflow-engine</artifactId>

            <version>0.12.0</version>

        </dependency>

        <dependency>

            <groupId>ai.djl.tensorflow</groupId>

            <artifactId>tensorflow-native-auto</artifactId>

            <version>2.4.1</version>

            <scope>runtime</scope>

        </dependency>

        <dependency>

            <groupId>org.projectlombok</groupId>

            <artifactId>lombok</artifactId>

            <optional>true</optional>

        </dependency>

Listing 5-2

Dependencies for DJL

Step 2: Create the Spring Cloud Function.

Now create an XRAYFunction that stores a model from the URL provided: https://djl-ai.s3.amazonaws.com/resources/demo/pneumonia-detection-model/saved_model.zip. See Listing 5-3.

package com.kubeforce.djlxray;

import ai.djl.inference.Predictor;

import ai.djl.modality.Classifications;

import ai.djl.modality.cv.Image;

import ai.djl.modality.cv.ImageFactory;

import ai.djl.modality.cv.translator.ImageClassificationTranslator;

import ai.djl.modality.cv.util.NDImageUtils;

import ai.djl.repository.zoo.Criteria;

import ai.djl.repository.zoo.ZooModel;

import ai.djl.translate.Translator;

import lombok.SneakyThrows;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

import java.io.IOException;

import java.util.Arrays;

import java.util.List;

import java.util.Map;

import java.util.function.Function;

public class XRAYFunction implements Function<Map<String,String>, String> {

    private static final Logger logger = LoggerFactory.getLogger(XRAYFunction.class);

    private static final List<String> CLASSES = Arrays.asList("Normal", "Pneumonia");

    String imagePath;

    String savedModelPath;

    @SneakyThrows

    @Override

    public String apply(Map<String, String> imageinput) {

            imagePath= imageinput.get("url");

            savedModelPath = imageinput.get("savedmodelpath");

            Image image;

            try {

                image = ImageFactory.getInstance().fromUrl(imagePath);

            } catch (IOException e) {

                throw new RuntimeException(e);

            }

            Translator<Image, Classifications> translator =

                    ImageClassificationTranslator.builder()

                            .addTransform(a -> NDImageUtils.resize(a, 224).div(255.0f))

                            .optSynset(CLASSES)

                            .build();

            Criteria<Image, Classifications> criteria =

                    Criteria.builder()

                            .setTypes(Image.class, Classifications.class)

//                           .optModelUrls("https://djl-ai.s3.amazonaws.com/resources/demo/pneumonia-detection-model/saved_model.zip")

                            .optModelUrls(savedModelPath)

                            .optTranslator(translator)

                            .build();

            try (ZooModel<Image, Classifications> model = criteria.loadModel();

                 Predictor<Image, Classifications> predictor = model.newPredictor()) {

                Classifications result = predictor.predict(image);

                logger.info("Diagnose: {}", result);

                return result.toJson();

            }

        }

    }

Listing 5-3

XRAYFunction.java

Step 3: Test locally. Run the Spring Cloud Function and invoke the endpoint http://localhost:8080/xrayFunction

Provide input:

 {

"url":"https://djl-ai.s3.amazonaws.com/resources/images/chest_xray.jpg",

"savedmodelpath":https://djl-ai.s3.amazonaws.com/resources/demo/pneumonia-detection-model/saved_model.zip

}

This is executed in Postman, as shown in Figure 5-8.

A screenshot of the postman testing. It highlights the local host, body, and row options with a testing program.

Upon invoking the function, the model is downloaded and then loaded into memory. This takes about a minute to load, after which it comes back with a successful message. The model took 802066 microseconds (80 seconds) to load, and this is critical for your function calls, as you will have to accommodate for this model-loading time. See Figure 5-9.

A screenshot of programming functions with number, date, and time. It represents the results from image evaluation.

A flow diagram of the saved model. It classifies as training and deployment. It represents the tensor flow components. The output from the training is sent to the saved model and further to the deployment.

This section successfully demonstrated that Spring Cloud Function can act as a model server in AI/ML. This is a critical function, as you can move the loading and serving of models from traditional servers to a function-based, “pay-per-use” model.

You also learned how to use deep learning Java libraries in your functions. You can deploy this Spring Cloud Function to any cloud, as shown in Chapter 2.

5.4 Model Serving with Spring Cloud Function with Google Cloud Functions and TensorFlow

This section explores the model serving on Google. It uses TensorFlow, which is a Google product from AI/ML and explains how to build and save an AI model with datasets such as MNIST (https://en.wikipedia.org/wiki/MNIST_database).

5.4.1 TensorFlow

TensorFlow was developed by Google and is an open source platform for machine learning. It is an interface for expressing and executing machine learning algorithms. The beauty of TensorFlow is that a model expressed in TensorFlow can be executed with minimal changes on mobile devices, laptops, or large-scale systems with multiple GPUs and CPUs. TensorFlow is flexible and can express a lot of algorithms, including training and inference algorithms for deep neural networks, speech recognition, robotics, drug discovery, and so on.

In Figure 5-9, you can see that TensorFlow can be deployed to multiple platforms and has many language interfaces. Unfortunately, TensorFlow is written in Python, so most of the models are written and deployed in Python. This poses a unique challenge for enterprises who have standardized on Java.

Even though TensorFlow is written in Python, there are lots of frameworks written in Java that work on a saved model.

Let’s look at how you can work with TensorFlow on the Google Cloud platform.

Google Cloud provides different approaches to working on models. You can use a container- or Kubernetes-based approach, an SaaS-based approach, or a Cloud Functions-based approach. Each has advantages and disadvantages. Google has published a good guide for you to pick the right platform for your needs, as shown in Table 5-2.

Table 5-2

Google and AI/ML Environment²

A table with 4 columns and 6 rows. The column headers are the features, compute engine, A L platform prediction, and cloud functions. It categorizes features and descriptions.

As you can see from Table 5-2, cloud functions are recommended for experimentation. Google recommends Compute Engine with TF Serving, or its SaaS platform (AI Platform) for predictions for production deployments.

The issue with this approach is that a function-based approach is more than just an experimentation environment. Functions are a way of saving on cost while exposing the serving capabilities for predictions through APIs. It is a serverless approach, so enterprises do not have to worry about scaling.

5.4.2 Example Model Training and Serving

In this section you see how to train an AI model locally and upload it to Google Cloud Storage. You will then download the model and test an image through a Cloud Function API. You will use a model that is based on MNIST. More about MNIST can be found at https://en.wikipedia.org/wiki/MNIST_database.

In this example, you develop the model in Python and then expose the model through Spring Cloud Function using DJL; see Figure 5-11.

A model of spring cloud function. It indicates the tensor flow of H T T P J SON request, cloud storage, and cloud function with two logos of spring cloud function, and deep java library.

You use the same example outlined at https://cloud.google.com/blog/products/ai-machine-learning/how-to-serve-deep-learning-models-using-tensorflow-2-0-with-cloud-functions. This will allow you to concentrate more on the Spring Cloud Function code that you will be creating rather than the actual implementation in Python.

You will then serve the model using Spring Cloud Function.

Step 1: Install TensorFlow.

python3 -m pip install tensorflow

Step 2: Create a project. Create a project called MNIST and then create a main.py file with the code in Listing 5-4. I used PyCharm to run this code.

On your Mac, make sure to run this command before running the code; otherwise you will get a certificate error. The code tries to download packages from googleapis:

open /Applications/Python\ 3.7/Install\ Certificates.command

I ran main.py and the whole process took me 13 minutes. At the end of it, I was able to get two model files as output.

from __future__ import absolute_import

from __future__ import division

from __future__ import print_function

from __future__ import unicode_literals

import tensorflow as tf

from tensorflow.keras.layers import Dense, Flatten, Conv2D

from tensorflow.keras import Model

EPOCHS = 10

mnist = tf.keras.datasets.mnist

fashion_mnist = tf.keras.datasets.fashion_mnist

(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()

x_train, x_test = x_train / 255.0, x_test / 255.0

# Add a channels dimension e.g. (60000, 28, 28) => (60000, 28, 28, 1)

x_train = x_train[..., tf.newaxis]

x_test = x_test[..., tf.newaxis]

train_ds = tf.data.Dataset.from_tensor_slices(

    (x_train, y_train)).shuffle(10000).batch(32)

test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(32)

class CustomModel(Model):

    def __init__(self):

        super(CustomModel, self).__init__()

        self.conv1 = Conv2D(32, 3, activation='relu')

        self.flatten = Flatten()

        self.d1 = Dense(128, activation='relu')

        self.d2 = Dense(10, activation='softmax')

    def call(self, x):

        x = self.conv1(x)

        x = self.flatten(x)

        x = self.d1(x)

        return self.d2(x)

model = CustomModel()

loss_object = tf.keras.losses.SparseCategoricalCrossentropy()

optimizer = tf.keras.optimizers.Adam()

train_loss = tf.keras.metrics.Mean(name='train_loss')

train_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='train_accuracy')

test_loss = tf.keras.metrics.Mean(name='test_loss')

test_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='test_accuracy')

@tf.function

def train_step(images, labels):

    with tf.GradientTape() as tape:

        predictions = model(images)

        loss = loss_object(labels, predictions)

    gradients = tape.gradient(loss, model.trainable_variables)

    optimizer.apply_gradients(zip(gradients, model.trainable_variables))

    train_loss(loss)

    train_accuracy(labels, predictions)

@tf.function

def test_step(images, labels):

    predictions = model(images)

    t_loss = loss_object(labels, predictions)

    test_loss(t_loss)

    test_accuracy(labels, predictions)

for epoch in range(EPOCHS):

    for images, labels in train_ds:

        train_step(images, labels)

    for test_images, test_labels in test_ds:

        test_step(test_images, test_labels)

    template = 'Epoch {}, Loss: {}, Accuracy: {}, Test Loss: {}, Test Accuracy: {}'

    print(template.format(epoch + 1,

                          train_loss.result(),

                          train_accuracy.result() * 100,

                          test_loss.result(),

                          test_accuracy.result() * 100))

# Save the weights

model.save_weights('fashion_mnist_weights')

tf.saved_model.save(model, export_dir="c://Users//banua//Downloads/MNIST/models")

Listing 5-4

main.py

The key is tf.saved_model.save(model, export_dir="c://Users//banua//Downloads/MNIST/models").

This will save the model so that any model server can use it.

Step 3: Run the project.

Execute main.py from the IDE. See Figure 5-12.

A screenshot of the window. It represents the successful run of M N I S T. The pop-up text reads as generated model files located in the root of the project.

Zip the assets, variables, and the saved_model.pb. file as Savedmodel3.zip and upload it to the Google Cloud Storage.

Step 4: Upload the models into Cloud Storage. Navigate to your Google Cloud Console and subscribe to Cloud Storage. It is available at cloud.google.com.

Create a storage bucket in Google Cloud Storage and upload the two files into the storage bucket. Use the defaults for this example. If you are using free credits from Google, this storage should be covered.

I created a bucket called mnist-soc. You will use the bucket name in the Cloud Functions call. See Figure 5-13.

A screenshot of the Google cloud storage bucket. It includes the name of the bucket, choosing store data, choosing default storage, choosing control access, and protecting access with highlighted lines.

Name your bucket mnist-soc and leave the others set to the defaults; then click Create.

Upload the savedmodel3.zip file to this folder by clicking Upload Files. See Figure 5-14.

A screenshot of a mnist soc application. It highlights the object option and mentions the model's name, size, and type. It represents the models located in cloud storage.

Click the file to get the details of the URL you need to connect to, as shown in Figure 5-15.

A screenshot represents the U R L for a model. It indicates a mnist soc application with a live object option. It highlights the public and authentication U R L with a link.

The URL you use for testing this example is https://storage.googleapis.com/mnist-soc/savedmodel3.zip.

The test image you use for this example is https://storage.googleapis.com/mnist-soc/test.png.

Note that the function that you created in Section 5.2 will be deployed in Step 5. If you use savedmodel3.zip and test.png, it will fail. But you will know that the function is working because you will get an error message that the model could not be loaded. This is an acceptable outcome for the model you created.

Step 5: Deploy the Spring Cloud Function to Google Functions. In this step, you take the function you created in Section 5.2 and deploy it into the Google Cloud Functions environment. The prerequisites and steps are the same as discussed in Chapter 2.

Prerequisites:

• Google account

• Subscription to Google Cloud Functions

• Google CLI (This is critical, as it is a more efficient way than going through the Google Portal)

• Code from GitHub at https://github.com/banup-kubeforce/DJLXRay-GCP.git

Modify the Spring Cloud Function to fit the Google Cloud Functions environment. See Listing 5-5.

<dependency>

    <groupId>org.springframework.cloud</groupId>

    <artifactId>spring-cloud-function-adapter-gcp</artifactId>

</dependency>

<dependencyManagement>

        <dependencies>

            <dependency>

                <groupId>org.springframework.cloud</groupId>

                <artifactId>spring-cloud-dependencies</artifactId>

                <version>${spring-cloud.version}</version>

                <type>pom</type>

                <scope>import</scope>

            </dependency>

            <dependency>

                <groupId>com.google.cloud</groupId>

                <artifactId>spring-cloud-gcp-dependencies</artifactId>

                <version>3.3.0</version>

                <type>pom</type>

                <scope>import</scope>

            </dependency>

        </dependencies>

    </dependencyManagement>

<build>

        <plugins>

            <plugin>

                <groupId>org.springframework.boot</groupId>

                <artifactId>spring-boot-maven-plugin</artifactId>

                <configuration>

                    <outputDirectory>target/deploy</outputDirectory>

                </configuration>

                <dependencies>

                    <dependency>

                        <groupId>org.springframework.cloud</groupId>

                        <artifactId>spring-cloud-function-adapter-gcp</artifactId>

                        <version>3.2.7</version>

                    </dependency>

                </dependencies>

            </plugin>

            <plugin>

                <groupId>com.google.cloud.functions</groupId>

                <artifactId>function-maven-plugin</artifactId>

                <version>0.9.1</version>

                <configuration>

                    <functionTarget>org.springframework.cloud.function.adapter.gcp.GcfJarLauncher</functionTarget>

                    <port>8080</port>

                </configuration>

            </plugin>

        </plugins>

    </build>

Listing 5-5

Dependencies for GCP Added

Deploy the Spring Cloud Function to Google Cloud Functions. Make sure that you build and package before you run the following command. A JAR file must be present in the target/deploy directory in the root of your project.

The saved model that you are going to test with is 400MB, so you have to accommodate this by increasing the memory to 4096 and setting the timeout to 540 seconds:

gcloud functions deploy DJLXRay-GCP --entry-point org.springframework.cloud.function.adapter.gcp.GcfJarLauncher --runtime java11 --trigger-http --source target/deploy --memory 4096MB --timeout 540

Once this runs successfully, you will get the output shown in Figure 5-16.

A screenshot of multiple programming functions. It denotes the deployed function, memory, and time.

Navigate to your Google Cloud Functions console to verify and to get the URL to test. See Figure 5-17.

A screenshot includes functions and cloud functions in the console. It highlights the create function and refresh. It indicates a filter option with details of the first gen.

You now test in the Cloud Function console by providing input (see Figure 5-18). Note that you have to increase the memory to 4096MB with a timeout set to 540s just to be safe:

{"url":"https://djl-ai.s3.amazonaws.com/resources/images/chest_xray.jpg",

"savedmodelpath":"https://storage.googleapis.com/mnist-soc/saved_model.zip"}

A screenshot highlights the cloud function, D J L X ray -G C P. The testing option with Test the function is highlighted. The test was successfully executed.

If you scroll down the test console, you get the execution times. This shows that the function execution took 16392ms, as shown in Figure 5-19. This is 16s for execution, which is phenomenal. This is faster because you stored the saved model in Google Cloud Storage, which is closer to the function.

A screenshot of the window denotes the multiple output logs. It highlights one log code no-16392, code- 200 with box .It displays the execution time.

This section explored the capabilities of TensorFlow and explained how you can use DJL and Spring Cloud Function together to access a saved TensorFlow model. DJL makes it easy for Java programmers to access any of the saved models generated using Python frameworks, such as PyTorch (pytorch.org) and TensorFlow.

You also found that you have to set the memory and timeout based on the saved model size and store the model closer to the function, such as in Google’s storage offerings.

5.5 Model Serving with Spring Cloud Function with AWS Lambda and TensorFlow

This section emulates what you did in Chapter 2 for Lambda. It is best to finish that exercise before trying this one.