Tag Archives: java

Destructor abnormal stuck bug

The cause of the bug

When sending messages, this interface needs to be rewritten. If the rewritten method throws an exception and does not capture it, the program will be stuck and will not execute the
Destroy source code

package com.lmax.disruptor;

public interface EventTranslatorVararg<T> {
    void translateTo(T var1, long var2, Object... var4);
}

Problem code

public class Main {
    public static void main(String[] args) {
        Disruptor<Message> disruptor = new Disruptor<>(
                Message::new,
                1024,
                (ThreadFactory) Thread::new);
        disruptor.handleEventsWith((EventHandler<Message>) (message, l, b) -> {
            System.out.println("Handling messages " + message);
        });
        disruptor.start();
        RingBuffer<Message> ringBuffer = disruptor.getRingBuffer();

        for(int i = 0; i<10; i++){
            Message message = new Message(String.valueOf(i));
            ringBuffer.publishEvent((m, l) -> 
            	throw new RuntimeException();
            );// Throwing exceptions without catching and finding that the program cannot be terminated
        }

        System.out.println("hi");	//hi Will not output
        disruptor.shutdown();//shutdown the disruptor, the method will block until all events have been handled.
    }
    
}

result:

The main thread exits, but the program continues to run without stopping

solve:

Handle exceptions on call

public class Main {
    public static void main(String[] args) {
        Disruptor<Message> disruptor = new Disruptor<>(
                Message::new,
                1024,
                (ThreadFactory) Thread::new);
        disruptor.handleEventsWith((EventHandler<Message>) (message, l, b) -> {
            System.out.println("Handling messages " + message);
        });
        disruptor.start();
        RingBuffer<Message> ringBuffer = disruptor.getRingBuffer();

        for(int i = 0; i<10; i++){
            Message message = new Message(String.valueOf(i));
            ringBuffer.publishEvent((m, l, m2) -> {
                try {
                    throw new RuntimeException();
                }catch (Exception e){
                    e.printStackTrace();
                }
            });// Handle exceptions, find program print exceptions and can end
        }

        System.out.println("hi");
        disruptor.shutdown();//Close the disruptor and the method will block until all events have been processed.
    }
    
}

Message.java

public class Message {
    String id;

    public Message(String id) {
        this.id = id;
    }

    public Message() {
    }

    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

    @Override
    public String toString() {
        return "Message{" +
                "id='" + id + '\'' +
                '}';
    }

}

Error:Cannot build artifact xxx:war exploded’ because it is included into a circular dependency

Error:Cannot build artifact xxx:war exploded’ because it is included into a circular dependency solution
IDEA error: Error:Cannot build artifact xxx:war exploded’ because it is included into a circular dependency

How to Solve:
ctrl + alt + shift + s Open project structure (or ctrl alt + a to search for project structure)
Click on the left artifacts and delete the two extra ones, which are
xxx:warxxx:war exploded
Delete is OK

Spring boot real time HTML page

Add the following configuration in application. Properties

spring.thymeleaf.cache=false # close cache

Introduce debugging tools into POM. XML and set it to open

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-devtools</artifactId>
    <optional>true</optional>
</dependency>

alt + ctrl + S open settings：
Build,Execution,Deployment —— Complier：
Check Build project automatically
ctrl + shift + alt + /:
Check Registry
Check compiler.automake.allow.when.app.running

Java garbage collection

Most of the source network, I summarize, mainly in the interview garbage collection related to high frequency questions and answers

Common garbage collection algorithms

Mark clear algorithm

   mark clear algorithm scans from the root set (GC roots) to mark the surviving objects. After marking, it scans the unmarked objects in the whole space for recycling, as shown in the figure below. Mark and clear algorithm does not need to move objects, but only needs to deal with the non surviving objects. It is very efficient when there are many surviving objects. However, because mark and clear algorithm directly recycles the non surviving objects, it will cause memory fragmentation.

Replication algorithm

   replication algorithm is proposed to overcome the overhead of handle and solve the problem of memory fragmentation. At the beginning, the heap is divided into one object surface and several free surfaces. The program allocates space for the object from the object surface. When the object is full, the garbage collection based on copying algorithm scans the active objects from the GC roots and copies each active object to the free surface (so that there is no free hole between the memory occupied by the active objects), so that the free surface becomes the object surface, The original object face becomes a free face, and the program allocates memory in the new object face.

Mark tidy algorithm

   mark clean algorithm uses the same way as mark clean algorithm to mark objects, but it is different in cleaning. After reclaiming the space occupied by the non surviving objects, it will move all the surviving objects to the left free space, and update the corresponding pointer. Mark and clean algorithm is based on mark and clean algorithm, and moves objects, so the cost is higher, but it solves the problem of memory fragmentation. The specific process is shown in the figure below:

Generational collection algorithm

   generational collection algorithm is currently used by most JVM garbage collectors. Its core idea is to divide the memory into several different regions according to the life cycle of the object. In general, the reactor area is divided into the aged generation and the young generation, and there is another generation outside the reactor area that is the permanent generation. The characteristics of the old era is that only a small number of objects need to be recycled in each garbage collection, while the characteristics of the new generation is that a large number of objects need to be recycled in each garbage collection, so the most suitable collection algorithm can be adopted according to the characteristics of different generations.

Recycling algorithm of young generation

a) All newly generated objects are first placed in the younger generation. The goal of the young generation is to collect the objects with short life cycle as quickly as possible.

b) The Cenozoic memory is divided into one Eden area and two survivor (survivor0, survivor1) areas according to the ratio of 8:1:1. One Eden area and two survivor areas (generally speaking). Most of the objects are generated in the Eden area. When recycling, first copy the surviving objects in the Eden area to a survivor0 area, and then empty the Eden area. When the survivor0 area is also full, copy the surviving objects in the Eden area and survivor0 area to another survivor1 area, and then empty the Eden area and the survivor0 area. At this time, the survivor0 area is empty, and then exchange the survivor0 area with survivor1 area, that is, keep the survivor1 area empty, So back and forth.

c) When survivor1 is not enough to store the surviving objects of Eden and survivor0, the surviving objects are directly stored in the old age. If the old age is full, a full GC will be triggered, that is, the new generation and the old generation will recycle.

d) Cenozoic GC is also called minor GC. Minor GC occurs more frequently (not necessarily when Eden area is full).

Recycling algorithm of old generation

a) Objects that survive n garbage collections in the younger generation will be put into the older generation. Therefore, it can be considered that the objects stored in the old generation are all objects with long life cycle.

b) The memory is also much larger than that of the Cenozoic generation (the ratio is about 1:2). When the memory of the old generation is full, the major GC, or full GC, is triggered. The frequency of full GC is relatively low, and the survival time of objects in the old generation is relatively long, and the survival rate is high.

Recycling algorithm of permanent generation

   used to store static files, such as Java classes, methods, etc. Persistent generations have no significant impact on garbage collection, but some applications may dynamically generate or call some classes, such as hibernate. In this case, a large persistent generation space needs to be set to store these new classes.

 
 
Garbage collector

1、 Seven kinds of garbage collectors

(1) Serial (serial GC) – XX: + useserialgc

(Replication Algorithm)
the new generation of single threaded collector, marking and cleaning are single threaded, and its advantage is simple and efficient. It is the default GC mode of client level, which can be specified by - XX: + useserialgc .

(2) Parnew (parallel GC) – XX: + useparnewgc

(stop copy algorithm)
the new generation collector can be considered as the multithreaded version of serial collector, which has better performance than serial in multi-core CPU environment.

(3) Parallel scavenge (GC)

(stop copy algorithm)
parallel collector pursues high throughput and efficient utilization of CPU. Throughput is generally 99%, throughput = user thread time/(user thread time + GC thread time). It is suitable for background applications and other scenes with low interaction requirements. It is the default GC mode of server level, which can be specified by - XX: + useparallelgc , and the number of threads can be specified by - XX: parallelgcthreads = 4 .

(4) Serial old (MSc) (serial GC) – XX: + useserialgc

(tag collation algorithm)
the older generation of single threaded collector, the older version of serial collector.

(5) CMS (concurrent GC) – XX: + useconcmarksweepgc

(Mark clean algorithm)
high concurrency, low pause, pursuit of the shortest GC recovery pause time, high CPU occupation, fast response time, short pause time, multi-core CPU pursuit of high response time.

(6) Parallel old (parallel GC) – XX: + useparallelold GC

(stop copy algorithm)

The old version of collector, parallel collector, throughput first.

(7) G1 (jdk1.7update14 can be used for commercial use)

 

2、 1 ~ 3 is used for garbage collection of young generation: garbage collection of young generation is called minor GC

3、 4 ~ 6 is used for the garbage collection of the old generation (of course, it can also be used for the garbage collection of the method area): the garbage collection of the old generation is called full GC

G1 independently completes “generation by generation garbage collection”

Note: parallelism and concurrency

Parallel: multiple garbage collection threads operate simultaneously

Concurrency: the garbage collection thread operates with the user thread

4、 Five common combinations

Serial/Serial Old

Parnew/serial old: compared with the above, it’s just more multithreaded garbage collection than the younger generation

Parnew/CMS: a more efficient combination at present

Parallel scavenge/parallel old: a combination of automatic management

G1: the most advanced collector, but need jdk1.7update14 or above

5、 Serial/serial old

The young generation serial collector uses a single GC thread to implement the “copy” algorithm (including scan and copy)

The old generation of serial old collector uses a single GC thread to implement the “mark and tidy” algorithm

Both serial and serial old suspend all user threads (STW)

explain:

STW (stop the world): when compiling code, inject safepoint into each method (the point at which the loop ends and the method execution ends). When pausing the application, you need to wait for all user threads to enter safepoint, then pause all threads, and then garbage collection.

Applicable occasions:

CPU cores & lt; 2, physical memory & lt; 2G machine (in short, single CPU, the new generation of small space and STW time requirements are not high)

-20: Useserialgc: forces the use of this GC combination

-20: Printgcapplicationsstoppedtime: View STW time

6、 Parnew/serial old:

Parnew is the same as serial except that it uses multiple GC threads to implement the replication algorithm. However, the serial old in this combination is a single GC thread, so it is an awkward combination. It is not as fast as serial/serial old in the case of single CPU (because parnew needs to switch multiple threads), In the case of multi CPU, it is not as fast as the following three combinations (because serial old is a single GC thread), so it is not used much.

-20: Parallelgcthreads: Specifies the number of parallelgcthreads. By default, it is the same as the number of CPU cores. This parameter may also be used in CMS GC combination

7、 Parallel scavenge/parallel old:

characteristic:

The younger generation parallel sweep collector uses multiple GC threads to implement the “copy” algorithm (including scanning and copying). The older generation Parallel old collector uses multiple GC threads to implement the “mark collate” algorithm. Both parallel sweep and parallel old suspend all user threads (STW)

explain:

Throughput: CPU running code time/(CPU running code time + GC time) CMS mainly focuses on the reduction of STW (the shorter the time, the better the user experience, so it is mainly used to handle a lot of interactive tasks). Parallel scavenge/parallel old mainly focuses on throughput (the larger the throughput, the higher the CPU utilization, So it is mainly used to deal with a lot of CPU computing tasks and less user interaction tasks.)

Parameter setting:

-20: + useparalleloldgc: use this GC combination

-20: Gctimeratio: set the throughput directly. Suppose it is set to 19, then the maximum GC time allowed accounts for 1/(1 + 19) of the total time. The default value is 99, that is, 1/(1 + 99)

-20: Maxgcpause millis: maximum GC pause time. The smaller the parameter, the better

-20: + useadaptive sizepolicy: turn on this parameter, – XMN/- XX: survivorratio/- XX: preemptsizethhreshold these parameters will not work. The virtual opportunity will automatically collect monitoring information and dynamically adjust these parameters to provide the most appropriate pause time or maximum throughput (GC adaptive adjustment strategy). What we need to set is – Xmx, -20: + useparalleloldgc or – XX: gctimeratio (of course, – XMS is also specified to be the same as – Xmx)

be careful:

-20: Gctimeratio and – XX: maxgcpausemillis just set one

If – XX: + useadaptive sizepolicy, – XMN/- XX: survivorratio/- XX: preteuresizethreshold is not enabled, these parameters can still be configured. Take the rest server as an example

-Xms2048m -Xmx2048m -Xmn512m -Xss1m -XX:PermSize=256M -XX:MaxPermSize=256M -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=15 -XX:+UseParallelOldGC -XX:GCTimeRatio=19 -XX:+PrintGCDetails -XX:+PrintGCTimeStamps View Code

Applicable occasions:

In the case of many CPU computing tasks and few user interaction tasks, I don’t want to pay more attention to GC parameters, and I want to let the virtual machine do its own tuning work

 
When did GC trigger (one of the most common interview questions)

   because the objects are processed by generations, the garbage collection area and time are also different. There are two types of GC: scavenge GC and full GC.

  Scavenge GC

   in general, when a new object is generated and Eden fails to apply for space (if it is larger than the maximum space, start the guarantee mechanism. You can learn about the guarantee mechanism), scavenge GC will be triggered to GC the Eden area, clear the non living objects, and move the surviving objects to the survivor area. And then sort out the two areas of survivor. This way GC is for the younger generation

Solving Chinese garbled code in Java compressed file

Solving Chinese garbled code in Java compressed file

Introducing Maven dependency

<dependency>
    <groupId>ant</groupId>
    <artifactId>ant</artifactId>
    <version>1.6.5</version>
</dependency>

Click here to view all versions of ant on Maven’s official website

Using ant.jar class in ant

import org.apache.tools.zip.ZipEntry;
import org.apache.tools.zip.ZipOutputStream;

Replace the class in JDK API

import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;

Java will convert Excel to list set or JSON, export excel file to local, excel import and export, easyexcel tool class

Introducing Maven coordinates

        <!--fastjson-->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.76</version>
        </dependency>        
        <!--easyexcel-->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>easyexcel</artifactId>
            <version>2.2.8</version>
        </dependency>

According to the file path, the excel file is parsed into a list

    /**
     * Parsing Excel files into Lists
     *
     * @param pathName file path (e.g. D:\test\xzh.xlsx)
     * @param head Table header
     * @param <T> generic
     * @return
     */
    public static <T> List<T> upload(String pathName, Class<T> head) {
        List<T> list = new ArrayList<>();
        AnalysisEventListener<T> analysisEventListener = new AnalysisEventListener<T>() {
            // This will be called for every data parsing
            @Override
            public void invoke(T data, AnalysisContext context) {
                log.info("Parsing a piece of data:{}", JSON.toJSONString(data));
                list.add(data);
                // TODO Here you can also manipulate the data
            }

            // All the data parsing is done and will be called
            @Override
            public void doAfterAllAnalysed(AnalysisContext context) {
                log.info("All data parsing is complete!") ;
                // TODO Here you can also manipulate the data
            }
        };
        EasyExcel.read(pathName, head, analysisEventListener).sheet().doRead();
        return list;
    }

According to the file stream, the excel file is parsed into a list

    /**
     * Parsing Excel files into Lists
     *
     * @param file File
     * @param head table header
     * @param <T> generic
     * @return
     */
    public static <T> List<T> upload(MultipartFile file, Class<T> head) {
        List<T> list = new ArrayList<>();
        AnalysisEventListener<T> analysisEventListener = new AnalysisEventListener<T>() {
            // This will be called every time the data is parsed
            @Override
            public void invoke(T data, AnalysisContext context) {
                log.info("Parsed a piece of data: {}", JSON.toJSONString(data));
                list.add(data);
                // TODO You can also manipulate the data here
            }

            // All the data parsing is done and it will call
            @Override
            public void doAfterAllAnalysed(AnalysisContext context) {
                log.info("All data parsing is complete!") ;
                // TODO Here you can also manipulate the data
            }
        };
        InputStream inputStream;
        try {
            inputStream = file.getInputStream();
        } catch (IOException e) {
            log.error("upload-InputStream-Exception：", e);
            return null;
        }
        EasyExcel.read(inputStream, head, analysisEventListener).sheet().doRead();
        return list;
    }

Download excel file to file path

    /**
     * Excel file download
     *
     * @param pathName File path
     * @param sheetName Worksheet name
     * @param data data
     * @param head Table header
     * @param <T> generic
     */
    public static <T> void download(String pathName, String sheetName, List<T> data, Class<T> head) {
        EasyExcel.write(pathName, head).sheet(sheetName).doWrite(data);
    }

Excel file download to file path response body response

    /**
     * Excel file download
     *
     * @param response response body
     * @param excelName File name
     * @param sheetName Worksheet name
     * @param data data
     * @param head table header
     * @param <T> generic
     */
    public static <T> void download(HttpServletResponse response, String excelName, String sheetName, List<T> data, Class<T> head) {
        response.setContentType("application/vnd.ms-excel");
        response.setCharacterEncoding("UTF-8");
        try {
            String fileName = URLEncoder.encode(excelName, "UTF-8");
            response.setHeader("Content-disposition", "attachment;filename=" + fileName + ".xlsx");
            EasyExcel.write(response.getOutputStream(), head).sheet(sheetName).doWrite(data);
        } catch (IOException e) {
            log.error("download-Exception：", e);
        }
    }

GitHub: https://github.com/xxiangzh/xzh-excel/tree/easyexcel

JAVA: Controller exception handling

With the @controlleradvice annotation, @exceptionhandler can specify the subdivision type of the exception on the specified controller

@ControllerAdvice
public class BaseController {

    private static final Logger logger = LoggerFactory.getLogger(BaseController.class);

    /**
     * Parameter type conversion error
     *
     * @param exception error
     * @return error message
     */
   @ExceptionHandler(HttpMessageConversionException.class)
   public String parameterTypeException(HttpMessageConversionException exception){       
        logger.error(exception.getCause().getLocalizedMessage());
        return ResultErr("Type conversion error");
   }

	/**
	 * Uniform exception handling
	 */
	@ExceptionHandler(value = Exception.class)
	@ResponseBody
	public String handlerException(Exception e) {
	    logger.error("Data check failure : errorMessage{"+e.getMessage()+"}");
		return ResultErr(e.getCode(), e.getMessage());
	}
	
}

Net Q & A: how to avoid the exception thrown by max() on emptyenumerable?

Consultation area

Naor：

I have the following query:


int maxShoeSize = Workers.Where(x => x.CompanyId == 8)
                          .Max(x => x.ShoeSize);

If workers. Where (x = & gt; x. Companyid = = 8) if no workers are found, the above code will throw an exception.

Now the idea is: query can return 0 if it can't be found, but don't throw an exception. How can I modify the query above?

Answer area

Ron K.：

You can use the extension method of IEnumerable defaultifempty() to avoid this embarrassment. Refer to the following code.


    class Program
    {
        static void Main(string[] args)
        {
            List<Worker> Workers = new List<Worker>()
            {
                new Worker(){ CompanyId=1, CompanyName="tweet", ShoeSize=10 },
                new Worker(){ CompanyId=2, CompanyName="google", ShoeSize=20 },
            };

            int maxShoeSize = Workers.Where(x => x.CompanyId == 8)
                          .Select(x => x.ShoeSize)
                          .DefaultIfEmpty(0)
                          .Max();

            Debug.WriteLine($"maxShoeSize={maxShoeSize}");
        }

    }

    class Worker
    {
        public int CompanyId { get; set; }

        public string CompanyName { get; set; }

        public int ShoeSize { get; set; }
    }

Output results:


maxShoeSize=0

Of course, the above 0 is not necessary. You can change it to any other number.

CptRobby：

Although the plan provided by the man upstairs can work normally, it doesn't look very eye-catching. It can be transformed into the following one.


    int maxShoeSize = Workers.Where(x => x.CompanyId == 8)
                             .Select(x => (int?)x.ShoeSize)
                              .Max() ?? 0;

Does the code look a little lengthy?The best way is to customize a extension method , as shown in the following code:


public static int MaxOrDefault<T>(this IQueryable<T> source, Expression<Func<T, int?>> selector, int nullValue = 0)
{
    return source.Max(selector) ?? nullValue;
}

For simplicity, this extension only deals with the int type. You can change it to any type, such as: (long, double,...), and then you can continue to reform the caller.


int maxShoeSize = Workers.Where(x => x.CompanyId == 8).MaxOrDefault(x => x.ShoeSize);

I hope my answer can help more people.

Comment area

Xiaobian never dares to do Max on the empty collection , after all, it's not once or twice, so every time we judge whether there is a value in the collection in advance, and then execute Max , we didn't expect that there are magic extension methods defaultifempty and empty type that can help us to do it, and we all forget about sensory science????????????.

package xxx does not exist cannot find symbol

When deploying services on Linux, a large number of packages were thrown and no exception was found

[ERROR] /home/jenkins/agent/workspace/pipeline_p-h6sbk-4/src/main/java/com/UserServiceImpl.java:[3,28] package com.alibaba.fastjson does not exist

[ERROR] /home/jenkins/agent/workspace/pipeline_p-h6sbk-4/src/main/java/com/User.java:[9,2] cannot find symbol

[ERROR]   symbol: class Data

[ERROR]   location: class com.xxx

[ERROR] /home/jenkins/agent/workspace/pipeline_p-h6sbk-4/src/main/java/com/User.java:[3,14] package lombok does not exist

[ERROR] /home/jenkins/agent/workspace/pipeline_p-h6sbk-4/src/main/java/com/User.java:[9,2] cannot find symbol

[ERROR]   symbol: class Data

[ERROR]   location: class com.xxx

[ERROR] -> [Help 1]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

This is Maven’s problem

    check whether the settings.xml file is configured. Check & lt; in the settings.xml file; localRepository> If the path is configured correctly, & lt; mirror> Whether the image address is correct and the network is unblocked. Whether the jar package and the corresponding version exist in Maven warehouse. Whether the Maven version configured in pom.xml corresponds to the Linux Installation version

Error installing redis: redis requires Ruby Version > = 2.3.0

Error installing redis: redis requires Ruby Version & gt= 2.3.0.

Error installing redis: redis requires Ruby Version & gt= 2.3.0.

Recently, the following error was reported when installing redis cluster:
the version of ruby in centos7 Yum library supports 2.0.0, but gem needs at least 2.3.0 to install redis, and uses RVM to update ruby

Follow the steps below

1. Install curl

yum -y install curl

2. Install RVM

curl - L get.rvm.io | bash - s stable
this is an installation failure error

this situation mostly occurs in self signed certificates. The error report means that the issuing certificate authority is not authenticated and cannot be identified.

3. Modify hosts

VI/etc/hosts
add the following
199.232.28.133 raw.githubusercontent.com

4. Execute curl command and download RVM

curl - L get.rvm.io | bash - s stable

Load shell configuration
source/usr/local/RVM/scripts/RVM

5. Check the known Ruby versions in RVM library

rvm list known

6. Install the ruby version

rvm install 2.6.3

7. Use Ruby Version

rvm use 2.6.3

8. Set the default version

ruby --version

9. Uninstall the old version

rvm remove 2.0.0

10. Install the interface version of redis and ruby

gem install redis

This is the perfect solution~

Using AspectJ to transfer the data written to FTP service to MySQL database

Recently, the company’s project has carried out the rectification of performance improvement. It was originally planned to use FTP to write the collected underlying data to the file, and the client will read the FTP file again, and then parse and display it according to the demand. In the actual application process, the display effect is not ideal due to the slow reading and parsing of the FTP file. Therefore, it is proposed that the data written to FTP should be parsed and stored in the database for the client Read the database directly, no need to read the file and parse, so as to improve the display effect and performance of this part.

After several options, in the light of minimal changes and smooth transition, the original part of FTP writing remains intact, and the client’s parsing display function is also retained. Using AspectJ, the method of writing FTP file (upload ()) is pointcut. Once FTP writes a file, the operation of writing database part is triggered. First, the parameters of upload () are obtained, and these parameters are parsed , stored in a global list, the SQL method will construct the insert statement through the global list, and finally addbatch will execute and submit. Considering that frequent connection to the database for insertion will seriously affect the performance, the insert operation is set as a scheduled task, and the corresponding insert will be constructed according to the global list every 60 seconds SQL, execute and submit transactions, so as to reduce the pressure of the database and clear the current list; in order to ensure data security, the insert and list clear operations are placed in a synchronized statement segment to prevent the list from being changed when constructing SQL. So far, the transfer of the original FTP data is completed.

Advantages: 1. Aspect programming, without affecting the original system architecture and business code, realizes the optimization of functions, and still maintains loose coupling;

2. Keep the original function and make a smooth transition;

Paste part of the code as follows (integrated development under the spring framework)

@Aspect

@Enable scheduling
@ enable aspect jautoproxy// enable aspect proxy
@ component()
public class dbsyncaop{

//Set upload (..) to pointcut and name it pointcustsignature ()

@Pointcut(“execution(* com.bjtct.oom . mms.service.FileManager .upload(..))”)
public void pointcustSignature(){}

//2. Set the loading priority of the uploadfileaspect method to high and load it first

@Around(“pointcustSignature()”)
@Order(value=1)
public Object uploadFileAspect(ProceedingJoinPoint pjp) throws Throwable{

//upload() execute before ，get upload() paramaters code here 

List<String> fileNames = (List<String>) pjp.proceed ();

//upload() execute after ，handle upload() paramaters  here,and store data in mysql code here 

//handle upload() paramaters

List<Map<String, String>> fieldMapList = handleMsg(type, mqMsg);  

//param is a global list. Here, the data originally written to FTP is saved to param, which is used to construct the inserted SQL statement

synchronized(params){
List<Object> param;
for(int i=0; i< fieldMapList.size (); i++){
param = new ArrayList<Object>();
msg = fieldMapList.get (i);
System.out.println (“msg: “+msg);
param.add (new Date());
param.add (msg);
param.add (type);

params.add (param);
}
}

//Schedule tasks regularly, execute the method regularly, and insert the data obtained from FTP

@Scheduled(cron = “${ DBSync.schedule.delay .cron}”)
private void proceedInsertion(){

if(!”Y”.equalsIgnoreCase(DBSyncEnabled)){
return;
}
log.info (“DBSyncAop scheduling start”);
Connection conn = null;
PreparedStatement ps = null;
log.info (“##params size: “+ params.size ());
synchronized(params){
if(! params.isEmpty ()){
try{
conn = DataSourceUtils.getConnection (scheduleDataSource);
conn.setAutoCommit (false);
List param;
ps = conn.prepareStatement (insertSql);
for(int i=0; i< params.size (); i++){
param = params.get (i);
ps.setDate (1, new java.sql.Date (((Date) param.get (0)).getTime()));//timestamp
ps.setString (2, (String) param.get (1));//fileName
ps.setString (3, (String) param.get (2)); //json msg
ps.setString (4, (String) param.get (3));//mq type
ps.addBatch ();
log.info (“DBSyncAop scheduling added batch”);
}
ps.executeBatch ();
conn.commit ();
log.info (“DBSyncAop scheduling committed”);
conn.setAutoCommit (true);
params.clear ();
} catch (Exception e) {
log.error (“insert MQ information data contact” + e.getmessage());
e.printstacktrace();
} finally {
try {
} DataSourceUtils.doCloseConnection (conn, scheduleDataSource);
} catch (SQLException e) {
log.error (e.getMessage());
e.printStackTrace();
}
}
}
}
log.info (“DBSyncAop scheduling finish”);

}

}