Undertanding load balancing in Spring Cloud with Ribbon and example

What is Load Balancer?
Netflix Ribbon
Objective of lesson
Create Spring Boot project
Configure Ribbon
Controller
Run application

1. What is Load Balancer?

No ADS

Let's imagine that you have a distributed system that consists of many services (applications) running on different computers. However, when the number of users is large, a service (application) is usually created multiple replicas. Each replica runs on a different computer. At this time, "Load Balancer" appears. It helps to distribute incoming traffic equally among servers.

Server side Load Balancer

Traditionally, Load Balancers are components placed at the Server Side. When the requests come from the Client, they will go to the load balancer, and the load balancer will designate a Server for the request. The simplest algorithm used by the load balancer is random designation. Most load balancers in this case are hardware, integrated software to control load balancing.

Client-Side Load Balancer:

When the load balancer is located at Client side, it will actively decide which server it will send requests to based on some criteria
.

Actually, servers are different. They differ from each other in the following criteria:

Availability: No server also works atall times.
Performance: The speed of servers is different.
Geography: Servers are located in different locations, for instance, they are located in different countries. They may be close to this Client but far from other Client.

Client-side load balancers usually send requests to servers in the same zone, or with a fast response.

2. Netflix Ribbon

No ADS

Ribbon is part of Netflix Open Source Software (Netflix OSS) family. It is a library that provides a Client-side load balancer. Because it is a member of the Netflix family, it can automatically interact with Netflix Service Discovery (Eureka).

Spring Cloud creates APIs to help you to easily use Ribbon libraries.

OK, We will discuss the main concepts related to the Ribbon:

List of servers.
Filtered list of servers.
Load Balancer
Ping

List Of Servers:

A list of servers that can respond to a particular service for 1 Client. For example, 1 Client needs information about the weather. There will be a list of servers being able to provide this information. This list consists of servers configured directly in the Client application, and servers discovered by the Client.

Filtered List of Servers:

We continue the above example. One Client needs weather information, and a list of servers can provide such information. However, not all of those servers operate, or those servers are too far from the Client; therefore, they responds very slowly. The Client will remove these servers from the list, and eventually it will have a more suitable list of servers (a filtered list).

Load Balancer (Ribbon):

Ribbon is a load balancer. It is a Client -side component. It decides which server will be called (of the filtered list of servers).

There are some strategies to make a decision. But they usually rely on a "Rule Component" to make a true decision. By default, the Spring Cloud Ribbon uses the ZoneAwareLoadBalancer strategy (Servers in the same zone as Client).

Rule Component is a smart module. It creates a "Call or uncall" decision. By default, Spring Cloud uses the ZoneAvoidanceRule rule.

Ping:

Ping is the way that the Client uses to rapidly check whether a server is active at that time or not? The default behavior of Spring Cloud is to delegate Eureka to automatically check this information. However, the Spring Cloud allows you to customize to check in your way.

3. Objective of lesson

No ADS

It is noted that this lesson is directly related to 2 previous lessons.

Lesson (1): We have created one "Service Registration" (Eureka Server).
Lesson (2): We have created one "ABC Service" application which is a Discovery Client (Eureka Client).

Understanding Spring Cloud Eureka Server with Example
Understanding Spring Cloud Discovery Eureka Client with Example

In this lesson, we are going to create a "XYZ Service" application. It is also a Discovery Client (Eureka Client) and it will call "ABC Service". On the "XYZ Service" application, we will use a load balancer.

To be able to test this application, we need to run all three applications, of which, "ABC Service" application will be created 5 replicas (simulating that it is running on 5 different computers). When the Client (XYZ Service) calls the "ABC Service", the load balancer will decide which replica of "ABC Service" is called.

OK!, It is an interesting and also quite easy job.

4. Create Spring Boot project

On the Eclipse, create aSpring Boot project.

Enter:

Name: SpringCloudLoadBalancerRibbon
Group: org.o7planning
Artifact: SpringCloudLoadBalancerRibbon
Description: Spring Cloud Load Balancer (Ribbon)
Package: org.o7planning.ribbon

The application being used by us will call "ABC Service". It needs a Load Balancer ; therefore, we will declare the use of Ribbon library.
This application should also discover other services (applications) running in the distributed system; therefore, it needs to use Discovery library. We will use Eureka Discovery (Eureka Client).

OK, the project has been created.

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
    http://maven.apache.org/xsd/maven-4.0.0.xsd">
    
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.o7planning</groupId>
    <artifactId>SpringCloudLoadBalancerRibbon</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>SpringCloudLoadBalancerRibbon</name>
    <description>Spring Cloud Load Balancer (Ribbon)</description>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.5.9.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
        <spring-cloud.version>Edgware.RELEASE</spring-cloud.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-eureka</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-ribbon</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>


</project>

5. Configure Ribbon

No ADS

application.yml

spring:
  application:
    name: XYZ-SERVICE
 
server:
  port: 5555

# -- Configure for Ribbon:
 
ping-server:
  ribbon:
    eureka:
      enabled: false # Disable Default Ping
    listOfServers: localhost:8000,localhost:8001,localhost:8002,,localhost:8003
    ServerListRefreshInterval: 15000
    
# -- Configure Discovery Client (Eureka Client).    
# Configure this application to known "Service Registration".

eureka:
  instance:
    appname: XYZ-SERVICE  # ==> This is an instance of XYZ-SERVICE
  client:    
    fetchRegistry: true
    serviceUrl:
#      defaultZone: http://my-eureka-server.com:9000/eureka
      defaultZone: http://my-eureka-server-us.com:9001/eureka
#      defaultZone: http://my-eureka-server-fr.com:9002/eureka
#      defaultZone: http://my-eureka-server-vn.com:9003/eureka

SpringCloudLoadBalancerRibbonApplication.java

package org.o7planning.ribbon;

import org.o7planning.ribbon.config.RibbonConfiguration;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
import org.springframework.cloud.netflix.ribbon.RibbonClient;

@RibbonClient(name = "ping-a-server", configuration = RibbonConfiguration.class)
@EnableEurekaClient
@SpringBootApplication
public class SpringCloudLoadBalancerRibbonApplication {

    public static void main(String[] args) {
        SpringApplication.run(SpringCloudLoadBalancerRibbonApplication.class, args);
    }

}

RibbonConfiguration.java

package org.o7planning.ribbon.config;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;

import com.netflix.client.config.IClientConfig;
import com.netflix.loadbalancer.IPing;
import com.netflix.loadbalancer.IRule;
import com.netflix.loadbalancer.PingUrl;
import com.netflix.loadbalancer.WeightedResponseTimeRule;

 
public class RibbonConfiguration {
    
    @Autowired
    private IClientConfig ribbonClientConfig;
 
    @Bean
    public IPing ribbonPing(IClientConfig config) {
        return new PingUrl();
    }
 
    @Bean
    public IRule ribbonRule(IClientConfig config) {
        return new WeightedResponseTimeRule();
    }
    
}

6. Controller

Example1Controller.java

package org.o7planning.ribbon.controller;

import java.util.List;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.client.discovery.DiscoveryClient;
import org.springframework.cloud.client.loadbalancer.LoadBalancerClient;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.client.RestTemplate;

@RestController
public class Example1Controller {

    @Autowired
    private DiscoveryClient discoveryClient;

    @Autowired
    private LoadBalancerClient loadBalancer;

    @ResponseBody
    @RequestMapping(value = "/", method = RequestMethod.GET)
    public String home() {

        return "<a href='testCallAbcService'>/testCallAbcService</a>";
    }

    @ResponseBody
    @RequestMapping(value = "/testCallAbcService", method = RequestMethod.GET)
    public String showFirstService() {

        String serviceId = "ABC-SERVICE".toLowerCase();

        // (Need!!) eureka.client.fetchRegistry=true
        List<ServiceInstance> instances = this.discoveryClient.getInstances(serviceId);

        if (instances == null || instances.isEmpty()) {
            return "No instances for service: " + serviceId;
        }
        String html = "<h2>Instances for Service Id: " + serviceId + "</h2>";

        for (ServiceInstance serviceInstance : instances) {
            html += "<h3>Instance :" + serviceInstance.getUri() + "</h3>";
        }

        // Create a RestTemplate.
        RestTemplate restTemplate = new RestTemplate();

        html += "<br><h4>Call /hello of service: " + serviceId + "</h4>";

        try {
            // May be throw IllegalStateException (No instances available)
            ServiceInstance serviceInstance = this.loadBalancer.choose(serviceId);

            html += "<br>===> Load Balancer choose: " + serviceInstance.getUri();

            String url = "http://" + serviceInstance.getHost() + ":" + serviceInstance.getPort() + "/hello";

            html += "<br>Make a Call: " + url;
            html += "<br>";

            String result = restTemplate.getForObject(url, String.class);

            html += "<br>Result: " + result;
        } catch (IllegalStateException e) {
            html += "<br>loadBalancer.choose ERROR: " + e.getMessage();
            e.printStackTrace();
        } catch (Exception e) {
            html += "<br>Other ERROR: " + e.getMessage();
            e.printStackTrace();
        }
        return html;
    }

}

7. Run application

No ADS

To be able to test this application fully, you need to run 2 applications of the 2 previous lessons ("Service Registration" and "ABC Service")