Tag Archives: system architecture

How to Solve Remote Desktop Protocol 0xd06 Error

My environment:

Target computer win11 virtual machine vm16.2.0 (win10 in the virtual machine is copied from the original computer)

The original computer is win10 virtual machine vm15.5.0

Error reporting process:

Connect the win10 system in the virtual machine with the remote desktop connection tool mstsc provided by win11. The error is as follows:

The session will be disconnected due to a protocol error. Please reconnect to the remote computer
error code: 0xd06
extended error code: 0x0

Solution:

After a series of searches, no solution was found on the Internet. After careful consideration, considering that the win10 system in the virtual machine is copied from another original computer and the original computer uses the vm15.5.0 version of the virtual machine, it may be that the driver has a problem.

Solution:

Uninstall the display driver of win10 system in the virtual machine.

Java uses single thread pool to realize multi thread sequential execution (non alternating, non synchronous)

1. Principle and mechanism

Use one-way pool to ensure orderly submission and execution.

explain:

1) Alternate execution between threads is not supported.

2) Multithreading is not synchronous, only to ensure the order of execution, multithreading is concurrent execution.

2. Code examples

package com.thread;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;

public class TasksWithoutLockTest extends Thread {

    private static int num = 0;
    private int id;

    public TasksWithoutLockTest(int id) {
        this.id = id;
    }


    @Override
    public void run() {
        System.out.println("Thread" + id + "output num:" + num++);
        try {
            Thread.sleep(3);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) {
        //ExecutorService exec = Executors.newFixedThreadPool(1);
        ExecutorService exec = Executors.newSingleThreadScheduledExecutor();
        for (int i = 0; i < 12; i++) {
            exec.submit(new TasksWithoutLockTest(i));
        }

        exec.shutdown();

    }
}

3. Output results

Asynchronous processing of HTTP request by Java_ Method 2: through deferredresult

1.DeferredResult

Spring supports HTTP asynchronous return

2. Asynchronous processing

Start a new thread, process the data and return the value.

3. Code examples

1)controller

package com.liuxd.controller;

import com.liuxd.entity.Responses;
import com.liuxd.service.TaskService2;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.context.request.async.DeferredResult;

@Slf4j
@RestController
public class AsyncCtr2 {

    @Autowired
    private TaskService2 taskService2;

    @GetMapping(value = "/getResult")
    public DeferredResult<Responses<String>> getResult() {
        log.info("HTTP request received...");
        long startTime = System.currentTimeMillis();

        DeferredResult<Responses<String>> deferredResult = new DeferredResult<Responses<String>>();

        new Thread(new Runnable() {
            @Override
            public void run() {
                taskService2.getData(deferredResult);
            }
        }).start();

        log.info("The task of the receiving HTTP request thread has been completed, exit!");
        long endTime = System.currentTimeMillis();
        log.info("http Total request time. " + (endTime - startTime) + "ms");

        return deferredResult;

    }
}

2)service

package com.liuxd.service;

import com.liuxd.entity.Responses;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;
import org.springframework.web.context.request.async.DeferredResult;

@Slf4j
@Service
public class TaskService2 {

    public void getData(DeferredResult<Responses<String>> deferredResult) {

        log.info("Call service asynchronous with return method, start execution...");
        long startTime = System.currentTimeMillis();

        try {
            Thread.sleep(2500L);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        log.info("Call service asynchronous has return method, the end of execution!!!") ;

        long endTime = System.currentTimeMillis();
        log.info("Total time taken to call service asynchronous return method. " + (endTime - startTime) + "ms");

        deferredResult.setResult(new Responses<>(0, "Done", "SUCCESS"));

    }


}

3)Responses

package com.liuxd.entity;

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class Responses<T> {

    private Integer code;

    private String msg;

    private T data;

}

4) Print results

5) Result analysis

1) The HTTP main thread accepts the request and ends after processing the request

2) After the asynchronous thread ends, returns the

Three ways of thread sequence alternate execution in Java lock free programming

1. Principle analysis

Atomicinteger is thread safe, and autoincrement is thread safe.

2. Code examples

package com.thread;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;

public class TasksTest extends Thread {
    private static AtomicInteger atomic = new AtomicInteger(1);
    private int id;

    public TasksTest(int id) {
        this.id = id;
    }

    @Override
    public void run() {
        while (atomic.get() <= 12) {
            while (atomic.get() % 3 == id) {
                System.out.println("thread_" + id + " output:" + atomic.get());
                atomic.incrementAndGet();
            }
        }
    }

    public static void main(String[] args) {
        Thread thread0 = new TasksTest(0);
        Thread thread1 = new TasksTest(1);
        Thread thread2 = new TasksTest(2);

        ExecutorService exec = Executors.newFixedThreadPool(3);

        exec.submit(thread0);
        exec.submit(thread1);
        exec.submit(thread2);

        exec.shutdown();

    }
}

Implementation of retrial mechanism in Java

Keywords: thread, thread pool, queue, task scheduling

1. Retrying tool class

package com.iretry;

import com.alibaba.fastjson.JSON;
import lombok.extern.slf4j.Slf4j;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler;

import java.util.Queue;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.Executor;
import java.util.concurrent.ThreadPoolExecutor;

/**
 * Retry Tools Class
 */
@Slf4j
public class RetryUtils {

    private final static int DEFAULT_RETRY_TIMES = 6;
    private final static int[] DEFAULT_DELAY_SECONDS = {3, 30, 180, 600, 1800, 3600};

    private static Queue<RetryRunnable> TASK_QUEUE = new ConcurrentLinkedQueue<>();

    public static ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    public static ThreadPoolTaskScheduler scheduler = new ThreadPoolTaskScheduler();

    static {
        //Configure the number of core threads
        executor.setCorePoolSize(10);
        // Configure the maximum number of threads
        executor.setMaxPoolSize(20);
        //configure queue size
        executor.setQueueCapacity(1000);
        //configure the name prefix of the threads in the thread pool
        executor.setThreadNamePrefix("async_");
        // rejection-policy: how to handle new tasks when the pool has reached max size
        // CALLER_RUNS: instead of executing the task in a new thread, the caller's thread will execute it
        executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
        //Perform initialization
        executor.initialize();

        scheduler.setThreadNamePrefix("scheduler_");
        scheduler.setPoolSize(5);
        scheduler.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
        scheduler.initialize();
    }

    public RetryUtils() {
        //Check every second: iterate through the task queue, if needed, the thread pool schedules execution
        scheduler.scheduleAtFixedRate(() -> {
            for (RetryRunnable task : TASK_QUEUE) {
                long nextRetryMillis = task.nextRetryMillis;
                if (nextRetryMillis != -1 && nextRetryMillis <= System.currentTimeMillis()) {
                    task.nextRetryMillis = -1;
                    executor.execute(task);
                }
            }
        }, 1000);
    }

    public void doRetry(Task task) {
        doRetry(DEFAULT_RETRY_TIMES, DEFAULT_DELAY_SECONDS, task);
    }

    public void doRetry(int maxRetryTime, Task task) {
        doRetry(maxRetryTime, DEFAULT_DELAY_SECONDS, task);
    }

    public void doRetry(int[] retryDelaySeconds, Task task) {
        doRetry(retryDelaySeconds.length, retryDelaySeconds, task);
    }

    /**
     * @param maxRetryTime      Maximum number of retries
     * @param retryDelaySeconds array of retry interval times
     * @param task task
     */
    public void doRetry(final int maxRetryTime, final int[] retryDelaySeconds, final Task task) {
        Runnable runnable = new RetryRunnable(maxRetryTime, retryDelaySeconds, task);
        executor.execute(runnable);
    }

    /**
     * Custom Threads Class
     */
    private static class RetryRunnable implements Runnable {

        private final Task task;
        private final int maxRetryTimes;
        private final int[] retryDelaySeconds;

        private int retryTimes;
        private volatile long nextRetryMillis;

        //Constructors
        public RetryRunnable(final int maxRetryTimes, final int[] retryDelaySeconds, final Task task) {
            this.task = task;
            if (maxRetryTimes <= 0) {
                this.maxRetryTimes = DEFAULT_RETRY_TIMES;
            } else {
                this.maxRetryTimes = maxRetryTimes;
            }
            if (retryDelaySeconds == null || retryDelaySeconds.length == 0) {
                this.retryDelaySeconds = DEFAULT_DELAY_SECONDS;
            } else {
                this.retryDelaySeconds = retryDelaySeconds;
            }
        }

        //Implementation of business methods
        @Override
        public void run() {
            try {

                task.run();

            } catch (Throwable e) {
                int sleepSeconds = retryTimes < retryDelaySeconds.length ?retryDelaySeconds[retryTimes] : retryDelaySeconds[retryDelaySeconds.length - 1];

                if (retryTimes < maxRetryTimes) {
                    if (retryTimes == 0) {
                        TASK_QUEUE.add(this);
                        log.error("task executed error, " + sleepSeconds + " seconds do next... ", e);
                    } else {
                        log.error("retry " + retryTimes + " times error, " + sleepSeconds + " seconds do next... ", e);
                    }
                    nextRetryMillis = System.currentTimeMillis() + sleepSeconds * 1000;

                } else {
                    log.error("retry " + retryTimes + " times error", e);
                    log.error("retry snapshot: {}", JSON.toJSONStringWithDateFormat(task.snapshot(), "yyyy-MM-dd HH:mm:ss"));

                    TASK_QUEUE.remove(this);
                    task.retryFailed(e);

                }

                retryTimes++;

            }
        }
    }

}

2. Business class interface

package com.iretry;

public interface Task {

    void run() throws Exception;

    default void retryFailed(Throwable e) {
    }

    default Object snapshot() {
        return null;
    }
}

3. Test class

package com.iretry;

import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler;

import java.util.concurrent.ThreadPoolExecutor;

@Slf4j
public class IRetryTest {

    public static void main(String[] args) {
        RetryUtils retryUtils = new RetryUtils();

        IRetryTest iRetryTest = new IRetryTest();
        iRetryTest.executeBusiness(retryUtils, new IRetryTest().new UserInfo("Jack"));

    }

    //Business methods that require retries
    private void executeBusiness(RetryUtils retryUtils, UserInfo user) {

        retryUtils.doRetry(new int[]{6, 12}, new Task() {
            @Override
            public void run() throws Exception {
                user.setName("Henry");
                log.info("Execute the operation, change the name of the employee to: " + user.getName());
                Integer a = 1/0;

            }

            // Support rollback operations after final failure
            @Override
            public void retryFailed(Throwable e) {
                user.setName("Jack");
                log.info("Retry failed, business data rolled back...") ;
                log.info("Username is called." + user.getName());
            }

            @Override
            public Object snapshot() {
                return user;
            }
        });
    }


    @Data
    @NoArgsConstructor
    @AllArgsConstructor
    private class UserInfo {
        private String name;
    }


}

4. Implementation results

Split log by date in log4j2 of spring boot

1、application.yml

logging:
  config: classpath:log4j2.xml

2、log4j2.xml

<?xml version="1.0" encoding="UTF-8"?>
<Configuration monitorInterval="30">
    <Properties>
        <!--System variable ${sys:logPath:-/var/logs} Retrieve the value of the system variable logPath, or set the default value if it is empty./var/logs-->
       <Property name="LOG_HOME">${sys:catalina.home:-.}/logs</Property>
        <Property name="LOG_BACK_HOME">${sys:catalina.home:-.}/logs/backup</Property>
        <Property name="LOG_PATTERN_LAYOUT">%date %highlight{[%-5.5level]}{STYLE=Logback} [%-10.10thread] [%X{employeeCode}] [%X{X-B3-TraceId}/%X{X-B3-SpanId}] %cyan{[%-50.50class:%-4.4line]} - %msg%xEx%n </Property>
        <Property name="DEFAULT_CHARSET">UTF-8</Property>
        <Property name="ERROR_FILE_NAME">error</Property>
        <Property name="INFO_FILE_NAME">info</Property>
    </Properties>

    <Appenders>
        <!-- Configure daily logs -->
        <RollingFile name="${INFO_FILE_NAME}" fileName="${LOG_HOME}/${INFO_FILE_NAME}.log" filePattern="${LOG_BACK_HOME}/$${date:yyyy-MM}/${INFO_FILE_NAME}-%d{yyyy-MM-dd}.log.gz" append="true">
            <PatternLayout charset="${DEFAULT_CHARSET}" pattern="${LOG_PATTERN_LAYOUT}"/>
            <!-- Setting Policy -->
            <Policies>
                <TimeBasedTriggeringPolicy interval="1" modulate="true" />
            </Policies>
            <DefaultRolloverStrategy >
                <!--Delete log from 30 days ago-->
                <Delete basePath="${LOG_BACK_HOME}" maxDepth="2">
                    <IfFileName glob="*/*.log.gz" />
                    <IfLastModified age="30d" />
                </Delete>
            </DefaultRolloverStrategy>
        </RollingFile>
        <!-- Configuration error log -->
        <RollingFile name="${ERROR_FILE_NAME}" fileName="${LOG_HOME}/${ERROR_FILE_NAME}.log" filePattern="${LOG_BACK_HOME}/$${date:yyyy-MM}/${ERROR_FILE_NAME}-%d{yyyy-MM-dd}.log.gz" append="true">
            <PatternLayout charset="${DEFAULT_CHARSET}" pattern="${LOG_PATTERN_LAYOUT}"/>
            <!-- Setting Policy -->
            <Policies>
                <TimeBasedTriggeringPolicy interval="1" modulate="true" />
            </Policies>
            <Filters>
                <!--Logs with levels greater than or equal to WARN can be written-->
                <ThresholdFilter level="WARN" onMatch="ACCEPT" onMismatch="DENY"/>
            </Filters>
        </RollingFile>
    </Appenders>

    <Loggers>
        <Logger name="org.springframework" level="INFO" additivity = "false">
            <AppenderRef ref="${INFO_FILE_NAME}"/>
            <AppenderRef ref="${ERROR_FILE_NAME}"/>
        </Logger>
        <Logger name="com.picc" level="DEBUG" additivity = "false">
            <AppenderRef ref="${INFO_FILE_NAME}"/>
            <AppenderRef ref="${ERROR_FILE_NAME}"/>
        </Logger>
        <Root level="INFO">
            <AppenderRef ref="${INFO_FILE_NAME}"/>
            <AppenderRef ref="${ERROR_FILE_NAME}"/>
        </Root>
    </Loggers>
</Configuration>

3. Use result display

cd backup/

You’ll find monthly backups

 cd 2020-06/

Log is compressed, self decompression view

Code case of XXL job executor

1. POM is introduced. Note: the version number should be consistent with XXL job admin.

       <dependency>
            <groupId>com.xuxueli</groupId>
            <artifactId>xxl-job-core</artifactId>
            <version>1.9.2-SNAPSHOT</version>
        </dependency>

2、application.yml

xxl:
  job:
    admin:
      addresses: http://47.102.168.36:9500
    executor:
      log-path: ${catalina.home:/usr}/logs/job
      log-retention-days: 30
      ip: 127.0.0.1
      port: 10001

3. Configure executor bean

import com.xxl.job.core.executor.XxlJobExecutor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;


/**
 * xxl-job config
 */
@Configuration
public class XxlJobConfig {
    private Logger logger = LoggerFactory.getLogger(XxlJobConfig.class);

    @Value("${xxl.job.admin.addresses}")
    private String adminAddresses;

    @Value("${spring.application.name}")
    private String appName;

    @Value("${xxl.job.executor.ip}")
    private String ip;

    @Value("${xxl.job.executor.port}")
    private Integer port;

    @Value("${xxl.job.executor.log-path}")
    private String logPath;

    @Value("${xxl.job.executor.log-retention-days}")
    private int logRetentionDays;


    @Bean(initMethod = "start", destroyMethod = "destroy")
    public XxlJobExecutor xxlJobExecutor() {
        logger.info(">>>>>>>>>>> xxl-job config init.");
        XxlJobExecutor xxlJobExecutor = new XxlJobExecutor();
        xxlJobExecutor.setAdminAddresses(adminAddresses);
        xxlJobExecutor.setAppName(appName);
        xxlJobExecutor.setIp(ip);
        xxlJobExecutor.setPort(port);
        xxlJobExecutor.setPort(-1);
        xxlJobExecutor.setAccessToken(null);
        xxlJobExecutor.setLogPath(logPath);
        xxlJobExecutor.setLogRetentionDays(logRetentionDays);
        return xxlJobExecutor;
    }
}

4. Task code

import com.xxl.job.core.biz.model.ReturnT;
import com.xxl.job.core.handler.IJobHandler;
import com.xxl.job.core.handler.annotation.JobHandler;
import com.xxl.job.core.log.XxlJobLogger;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;

import javax.annotation.Resource;

/**
 * JOB
 */
@JobHandler(value = "iFlyTenantJob")
@Component
public class IFlyTenantJob extends IJobHandler {

    private static final Logger log = LoggerFactory.getLogger(IFlyTenantJob.class);

    @Override
    public ReturnT<String> execute(String s) {
        XxlJobLogger.log("The mission begins...") ;
        log.info("Task started...");

        iFlyAiService.queryTenants();
        try {
            Thread.sleep(6000);
            //int a = 1/0;
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        XxlJobLogger.log("End of mission...");
        log.info("End of mission...");
        return ReturnT.SUCCESS;

    }


}

Spring boot uses thread pool to realize asynchronous processing without return and asynchronous processing with return

1. Scene

HTTP requests are processed asynchronously;

1) No need to return, such as sending SMS, push messages, e-mail and other ancillary services, asynchronous processing, reduce the total time consumption of HTTP requests, improve customer experience.

2) If you need to return, the front-end will wait for the business processing result data, and the asynchronous processing will return to the main thread of HTTP request, and return to the front-end; if it is a single business that can be optimized here, please check the previous and next articles in this blog; if it is multiple business processing, you can use multiple asynchronous processing with return, and the total weight will summarize the result and return.

2. Knowledge points

1) Thread pool threadpooltaskexecutor

2) Note: @ enableasync, @ async (“asyncserviceexecutor”)

3) Return value encapsulates future & lt; T & gt;, new asyncresult & lt; & gt; (T)

3. Code examples

1)controller

package com.liuxd.controller;

import com.liuxd.entity.Responses;
import com.liuxd.service.TaskService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

import java.util.concurrent.Future;

@Slf4j
@RestController
public class AsyncCtr {

    @Autowired
    private TaskService taskService;

    @RequestMapping(value = "/get", method = RequestMethod.GET)
    public Responses<String> getResult() throws Exception {

        log.info("HTTP request received...") ;
        long startTime = System.currentTimeMillis();

        //1. Asynchronous execution_processing_data_without_waiting
        taskService.handleData();

        //2. Asynchronous execution_processing_data_waiting_for_processing_results
        Future<String> future = taskService.getData();
        String result = future.get();

        log.info("Receiving HTTP request thread task completed, exit!") ;
        long endTime = System.currentTimeMillis();
        log.info("Total time spent on http request. " + (endTime - startTime) + "ms");

        return new Responses<>(0, result, "SUCCESS");

    }

}

2)service

package com.liuxd.service;

import lombok.extern.slf4j.Slf4j;
import org.springframework.scheduling.annotation.Async;
import org.springframework.scheduling.annotation.AsyncResult;
import org.springframework.stereotype.Service;

import java.util.concurrent.Future;

@Slf4j
@Service
public class TaskService {

    @Async("asyncServiceExecutor")
    public void handleData() {

        log.info("Call service no return asynchronous method, start execution...");
        long startTime = System.currentTimeMillis();
        try {
            Thread.sleep(2500L);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        log.info("Call service no return asynchronous method, the end of execution!!!") ;
        long endTime = System.currentTimeMillis();
        log.info("Total time taken to call service without return asynchronous method. " + (endTime - startTime) + "ms");

    }

    @Async("asyncServiceExecutor")
    public Future<String> getData(){

        log.info("Call service has return asynchronous method, start execution...");
        long startTime = System.currentTimeMillis();

        try {
            Thread.sleep(2500L);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        log.info("Call service has returned asynchronous methods, the end of execution!!!") ;

        long endTime = System.currentTimeMillis();
        log.info("Total time to call service with return asynchronous method: " + (endTime - startTime) + "ms");

        return new AsyncResult<>("Asynchronous processing completed!");

    }


}

3) Thread pool

package com.liuxd.config;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

import java.util.concurrent.Executor;
import java.util.concurrent.ThreadPoolExecutor;

/**
 * @program: unif-insure-service
 * @description: Thread Pool
 **/
@Configuration
@EnableAsync
public class ExecutorConfig {
    @Bean
    public Executor asyncServiceExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        //Configure the number of core threads
        executor.setCorePoolSize(5);
        // Configure the maximum number of threads
        executor.setMaxPoolSize(5);
        //configure queue size
        executor.setQueueCapacity(1000);
        //configure the name prefix of the threads in the thread pool
        executor.setThreadNamePrefix("async-system-");

        // rejection-policy: How to handle new tasks when the pool has reached its max size
        // CALLER_RUNS: instead of executing the task in the new thread, the caller's thread will execute it
        executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
        //Execution initialization
        executor.initialize();
        return executor;
    }
}

4)Responses

package com.liuxd.entity;

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class Responses<T> {

    private Integer code;

    private String msg;

    private T data;

}

5) Print results

6) Result analysis

1) There are three threads, HTTP request main thread, asynchronous no return thread, asynchronous return thread

2) The main thread does not need to wait for no return thread, it will wait for the result of return thread, and finally return

Asynchronous processing of HTTP request by Java_ Method 1: through callable

1.Callable

1) Runnable, which performs an independent task but does not return a value. If you want a return value after the task is completed, you can use the callable interface;

2) Callable is a paradigm with type parameters. Its type parameter method is expressed as the value returned by the method call () instead of run (), and it must be used ExecutorService.submint () method.

difference:

1. Callable, which accepts a generic type and returns a value of this type in the call() method; however, runnable’s run() method has no return value;
2. Callable, whose call() method can throw an exception, while runnable’s run() method does not.

2. Business scenario:

In HTTP requests, the business processing process takes a long time, such as large queries, remote calls and other scenarios. The main thread that receives HTTP requests will be occupied all the time, and the number of threads in Tomcat thread pool is limited. Therefore, the number of HTTP requests accepted per unit time will decrease.

3. Code examples

1)controller

package com.liuxd.controller;

import com.liuxd.entity.Responses;
import com.liuxd.service.BusinessService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.concurrent.Callable;

@Slf4j
@RestController
public class AsyncCallableController {

    @Autowired
    private BusinessService businessService;

    @GetMapping(value = "/getData")
    public Callable<Responses<String>> getData() {

        log.info("HTTP request received...");

        Callable<Responses<String>> data = (() -> {
            return businessService.getData();
        });

        log.info("The task of the receiving HTTP request thread has been completed, exit!");

        return data;
    }

}

2) service

package com.liuxd.service;

import com.liuxd.entity.Responses;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;

@Slf4j
@Service
public class BusinessService {

    public Responses<String> getData(){

        log.info("Call the service method and start the execution...");

        try {
            Thread.sleep(2500L);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        log.info("Call service method, end of execution!");

        return new Responses<>(0,"Done","SUCCESS");

    }

}

3) Print results

Design of MQ asynchronous collection report data

1. MQ asynchronous data acquisition

1) Using MQ asynchronization to break away from the main service and shorten the response time of the main service.

2) Using MQ asynchronous data storage operation to reduce the pressure of the main business server.

3) Storage uses es to support big data query and improve query efficiency.

4) In process asynchronous MQ message sending operation can be realized by using Google thread bus or spring @ async

2. The design is as follows

 

Why must microservices have gateways?

1、 What is a service gateway

Service gateway = route forwarding + filter

1. Route forwarding: receive all external requests and forward them to the back-end micro service.

2. Filter: in the service gateway, a series of crosscutting functions can be completed, such as permission verification, current limiting and monitoring. All these can be completed through the filter (in fact, routing and forwarding are also realized through the filter).

2、 Why a service gateway is needed
the above crosscutting function (taking permission verification as an example) can be written in three places:

1) each service is implemented by itself [not recommended]

2) write to a public service, and then all other services depend on this service [not recommended]

3) write it to the prefilter of the service gateway, and all requests come to check the permission [recommended]

First, the disadvantages are too obvious to use;
Second, compared with the first point, the code development is not redundant, but there are two disadvantages:

(1) because each service introduces this public service, it is equivalent to introducing the same permission verification code in each service, which increases the jar package size of each service for no reason. Especially for the deployment scenario using docker image, the smaller the jar, the better;

       ② Since this public service is introduced into every service, it may be difficult for us to upgrade this service in the future. Moreover, the more functions the public service has, the more difficult it will be to upgrade. Suppose we change the way of permission verification in public service. If we want all services to use the new way of permission verification, we need to re package all previous services , compile the deployment.

The service gateway can solve this problem

    write the logic of permission verification in the filter of the gateway, the back-end service does not need to pay attention to the code of permission verification, so the logic of permission verification will not be introduced into the jar package of the service, and the size of the jar package will not be increased; if you want to modify the logic of permission verification, you only need to modify the filter of permission verification in the gateway, without upgrading all existing micro services.

So, need service gateway!!!

 

3、 Service gateway technology selection

 

After the introduction of service gateway, the microservice architecture is as above, including three parts: service gateway, open service and service.

1. Overall process:

The service gateway, open service and service are registered in the registry when they are started; the user requests the gateway directly, and the gateway performs intelligent routing and forwarding (including service discovery and load balancing) to the open service, which includes permission verification, monitoring, current limiting and other operations. The open service aggregates the internal service response, returns it to the gateway, and the gateway returns it to the user

2. Points for attention in introducing gateway

With the addition of gateway and one more layer of forwarding (originally, the user requested to directly access the open service), the performance will decline (but the decline is not big. Generally, the performance of gateway machine will be very good, and the access between gateway and open service is usually intranet access, which is very fast); single point problem of gateway: there must be a single point in the whole network call process, which may be Gateway, nginx, DNS server, etc. To prevent gateway single point, you can hang another nginx in front of the gateway layer. The performance of nginx is very high, and it will not hang basically. After that, the gateway service can continuously add machines. However, such a request is forwarded twice, so the best way is to deploy the gateway single point service on a powerful machine (estimate the configuration of the machine through pressure test). Moreover, the performance comparison between nginx and zuul is similar according to the experiment done by a foreign friend. Zuul is an open source framework for gateway of Netflix, and the gateway should be fully implemented Light weight.

3. Basic functions of service gateway

Intelligent routing: receive all external requests and forward them to the external service open service of the back end;

Note: we only forward external requests, and requests between services do not go through the gateway. This means that full link tracking, internal service API monitoring, fault tolerance of calls between internal services, and intelligent routing cannot be completed in the gateway. Of course, all service calls can go through the gateway, and almost all functions can be integrated into the gateway, but in this case, the gateway’s pressure can be reduced It’s going to be very heavy. Permission verification: only the user’s request to the open service is verified, and the internal request of the service is not verified. Is it necessary to verify the request inside the service?API monitoring: only monitor the requests passing through the gateway and some performance indicators of the gateway itself (for example, GC, etc.); current limiting: cooperate with the monitoring to carry out current limiting operation; API log unified collection: similar to an aspect aspect aspect, record the relevant log when the interface enters and goes out… Follow up supplement

The above functions are the basic functions of the gateway, and the gateway can also realize the following functions:

A | B test: a | B test is a relatively large thing, including background experiment configuration, data burial point (see conversion rate) and streaming engine. In the service gateway, the streaming engine can be realized, but in fact the streaming engine will call internal services, so if it is in accordance with the architecture in the figure above, the streaming engine should be in the open service rather than in the service gateway…. Follow up supplement

 

4. Technology selection

The author is going to build a lightweight service gateway

Development language: java + groovy, the advantage of groovy is that the gateway service can dynamically add filter to achieve some functions without restart; microservice basic framework: springboot; gateway basic component: Netflix zuul; service registry: consult; permission verification: JWT; API monitoring: Prometheus + grafana; API unified log collection: logback+ Elk; stress test: JMeter;… Follow up supplement

In the follow-up introduction, will gradually introduce each knowledge point, and complete a lightweight service gateway!!!