Recently, the company’s project has carried out the rectification of performance improvement. It was originally planned to use FTP to write the collected underlying data to the file, and the client will read the FTP file again, and then parse and display it according to the demand. In the actual application process, the display effect is not ideal due to the slow reading and parsing of the FTP file. Therefore, it is proposed that the data written to FTP should be parsed and stored in the database for the client Read the database directly, no need to read the file and parse, so as to improve the display effect and performance of this part.

After several options, in the light of minimal changes and smooth transition, the original part of FTP writing remains intact, and the client’s parsing display function is also retained. Using AspectJ, the method of writing FTP file (upload ()) is pointcut. Once FTP writes a file, the operation of writing database part is triggered. First, the parameters of upload () are obtained, and these parameters are parsed , stored in a global list, the SQL method will construct the insert statement through the global list, and finally addbatch will execute and submit. Considering that frequent connection to the database for insertion will seriously affect the performance, the insert operation is set as a scheduled task, and the corresponding insert will be constructed according to the global list every 60 seconds SQL, execute and submit transactions, so as to reduce the pressure of the database and clear the current list; in order to ensure data security, the insert and list clear operations are placed in a synchronized statement segment to prevent the list from being changed when constructing SQL. So far, the transfer of the original FTP data is completed.

Advantages: 1. Aspect programming, without affecting the original system architecture and business code, realizes the optimization of functions, and still maintains loose coupling;

2. Keep the original function and make a smooth transition;

Paste part of the code as follows (integrated development under the spring framework)

@Aspect

@Enable scheduling
@ enable aspect jautoproxy// enable aspect proxy
@ component()
public class dbsyncaop{

//Set upload (..) to pointcut and name it pointcustsignature ()

@Pointcut(“execution(* com.bjtct.oom . mms.service.FileManager .upload(..))”)
public void pointcustSignature(){}

//2. Set the loading priority of the uploadfileaspect method to high and load it first

@Around(“pointcustSignature()”)
@Order(value=1)
public Object uploadFileAspect(ProceedingJoinPoint pjp) throws Throwable{

//upload() execute before ，get upload() paramaters code here 

List<String> fileNames = (List<String>) pjp.proceed ();

//upload() execute after ，handle upload() paramaters  here,and store data in mysql code here 

//handle upload() paramaters

List<Map<String, String>> fieldMapList = handleMsg(type, mqMsg);  

//param is a global list. Here, the data originally written to FTP is saved to param, which is used to construct the inserted SQL statement

synchronized(params){
List<Object> param;
for(int i=0; i< fieldMapList.size (); i++){
param = new ArrayList<Object>();
msg = fieldMapList.get (i);
System.out.println (“msg: “+msg);
param.add (new Date());
param.add (msg);
param.add (type);

params.add (param);
}
}

//Schedule tasks regularly, execute the method regularly, and insert the data obtained from FTP

@Scheduled(cron = “${ DBSync.schedule.delay .cron}”)
private void proceedInsertion(){

if(!”Y”.equalsIgnoreCase(DBSyncEnabled)){
return;
}
log.info (“DBSyncAop scheduling start”);
Connection conn = null;
PreparedStatement ps = null;
log.info (“##params size: “+ params.size ());
synchronized(params){
if(! params.isEmpty ()){
try{
conn = DataSourceUtils.getConnection (scheduleDataSource);
conn.setAutoCommit (false);
List param;
ps = conn.prepareStatement (insertSql);
for(int i=0; i< params.size (); i++){
param = params.get (i);
ps.setDate (1, new java.sql.Date (((Date) param.get (0)).getTime()));//timestamp
ps.setString (2, (String) param.get (1));//fileName
ps.setString (3, (String) param.get (2)); //json msg
ps.setString (4, (String) param.get (3));//mq type
ps.addBatch ();
log.info (“DBSyncAop scheduling added batch”);
}
ps.executeBatch ();
conn.commit ();
log.info (“DBSyncAop scheduling committed”);
conn.setAutoCommit (true);
params.clear ();
} catch (Exception e) {
log.error (“insert MQ information data contact” + e.getmessage());
e.printstacktrace();
} finally {
try {
} DataSourceUtils.doCloseConnection (conn, scheduleDataSource);
} catch (SQLException e) {
log.error (e.getMessage());
e.printStackTrace();
}
}
}
}
log.info (“DBSyncAop scheduling finish”);

}

}