Phenomenon

The project uses the spring batch framework. Multiple slices use jsch to obtain SFTP connections to read files and report errors

In fact, it is multithreading, using jsch to obtain the session connection and report an error

com.jcraft.jsch.JSchException: Session.connect: java.net.SocketException: Connection reset

Jsch version

version=0.1.54
groupId=com.jcraft
artifactId=jsch

reason

Various reasons have been found on the Internet. Some say the number of SSH terminal connections is limited, and some say there is a TCP connection problem. The final reason has not been found yet. Please inform us in the comment area

Reappearance

public static Session getSshSession(String sftpHost, int sftpPort, String userName, String password) {
	JSch jsch = new JSch();
	// GET sshSession
	Session sshSession = null;
	try {
		sshSession = jsch.getSession(userName, sftpHost, sftpPort);
	} catch (JSchException e) {
		e.printStackTrace();
	}
	if (StringUtils.isNotBlank(password)) {
		sshSession.setPassword(password);
	}
	Properties sshConfig = new Properties();
	sshConfig.put("StrictHostKeyChecking", "no");
	sshSession.setConfig(sshConfig);
	return sshSession;
}


static void test() {
	for (int i = 1; i < 50; i++) {
		new Thread(() -> {
			Session sshSession = getSshSession("*.*.*.*", 22, "root", "***");
			try {
				Thread.sleep(100);
				sshSession.connect();
			} catch (Exception e) {
				e.printStackTrace();
			} finally {
				sshSession.disconnect();
			}

		}).start();
	}
}

Solution:

Create a channel pool using apache.commons.pool2

Since the SFTP configuration of the project is dynamic and not fixed, the following code is not encapsulated as a spring boot managed bean

Connection pool configuration:

public class ConnPoolConfig extends GenericObjectPoolConfig {
    public ConnPoolConfig() {
        // https://blog.csdn.net/weixin_42340670/article/details/108431381
        // The minimum number of free objects in the object pool should be
        setMinIdle(4);
        // The maximum capacity of the pool. The maximum number of objects to be stored in the pool
        setMaxTotal(10);
        // Check the validity of an object when it is borrowed from the pool.
        setTestOnBorrow(true);
        // How often the recycler thread performs idle object recovery (polling interval, in milliseconds)
        setTimeBetweenEvictionRunsMillis(60 * 60000);
        // Whether to verify the validity of the object when the recycler is scanning for idle objects.
        // If an object has not reached the specified threshold of idle time, and if testWhileIdle is configured to true
        // then it checks if the object is still valid, and if the object's resources have expired (e.g., the connection is disconnected), then he can be recycled.
        setTestWhileIdle(true);
    }
}

Connection pool factory:

public class ConnPoolFactory extends BasePooledObjectFactory<ChannelSftp> {

    private String host;
    private Integer port;
    private String userName;
    private String password;
    private final String strictHostKeyChecking = "no";

    public ConnPoolFactory(String host, Integer port, String userName, String password) {
        this.host = host;
        this.port = port;
        this.userName = userName;
        this.password = password;
    }

    @Override
    public ChannelSftp create() throws Exception {
        JSch jsch = new JSch();
        Session session = jsch.getSession(userName, host, port);
        session.setPassword(password);
        Properties config = new Properties();
        config.put("StrictHostKeyChecking", strictHostKeyChecking);
        session.setConfig(config);
        session.connect();
        ChannelSftp channel = (ChannelSftp) session.openChannel("sftp");
        channel.connect();
        return channel;
    }

    @Override
    public PooledObject<ChannelSftp> wrap(ChannelSftp obj) {
        return new DefaultPooledObject<>(obj);
    }

    // https://segmentfault.com/a/1190000003920723
    // Destroy the object, if the object pool detects that an "object" idle timeout,
    // or if the operator detects that the "object" is no longer valid when "returning the object" to the object pool, then this will result in "object destruction";
    // The design of the "destroy object" operation is far different, but it must be clear:
    // When this method is called, the life of the "object" must end. If object is a thread, then the thread must exit at this point;
    // If object is a socket operation, then the socket must be closed;
    // If object is a file stream operation, then "data flush" is done and closed normally.
    @Override
    public void destroyObject(PooledObject<ChannelSftp> pooledObject) throws Exception {
        Channel channel = pooledObject.getObject();
        Session session = channel.getSession();
        channel.disconnect();
        session.disconnect();
    }

    // Check if the object is "valid";
    // The Pool cannot hold invalid "objects", so the "background detection thread" will periodically check the validity of the "objects" in the Pool,
    // If the object is invalid, it will be removed from the Pool and destroyed;
    // In addition, when the caller gets an "object" from the Pool, it also checks the validity of the "object" to make sure that no "invalid" objects can be output to the caller;
    // When the caller returns the "object" to the Pool after use, the validity of the object is still checked. By validity,
    // The validity of the object is whether the object is in the expected state and can be used directly by the caller;
    // If the object is a socket, then its validity is whether the socket's channel is open/blocking timeout, etc.
    @Override
    public boolean validateObject(PooledObject<ChannelSftp> pooledObject) {
        return pooledObject.getObject().isConnected();
    }

    // "Activate" an object, an additional "activation" action when the Pool decides to remove an object for delivery to the caller,
    // For example, you can "reset" the list of parameters in the activateObject method to make it feel like a "newly created" object when the caller uses it;
    // If the object is a thread, you can reset the "thread break flag" in the "activate" operation, or wake up the thread from blocking, etc;
    // If the object is a socket, then you can refresh the channel in the "activate" operation,
    // or rebuild the link to the socket (if the socket is unexpectedly closed), etc.
    @Override
    public void activateObject(PooledObject<ChannelSftp> pooledObject) throws Exception {
        ChannelSftp channelSftp = pooledObject.getObject();
        Session session = channelSftp.getSession();
        if (!session.isConnected()) {
            session.connect();
            channelSftp.connect();
        }
    }

    // "Passivate" the object, when the caller "returns the object", the Pool will "passivate the object".
    // The implication of passivate is that the "object" needs a "rest" for a while.
    // If the object is a socket, then you can passivateObject to clear the buffer and block the socket;
    // If the object is a thread, you can sleep the thread or wait for an object in the thread during the "passivate" operation.
    // Note that the methods activateObject and passivateObject need to correspond to each other to avoid deadlocks or confusion about the state of the "object".
    @Override
    public void passivateObject(PooledObject<ChannelSftp> pooledObject) throws Exception {
    }
}

Connection pool:

public class ConnPool extends GenericObjectPool<ChannelSftp> {

    private static final Map<String, ConnPool> MAP = new ConcurrentHashMap<>();

    private ConnPool(String host, Integer port, String userName, String password) {
        super(new ConnPoolFactory(host, port, userName, password), new ConnPoolConfig());
    }

    public static ConnPool getConnPool(String host, Integer port, String userName, String password) {
        String key = host + ":" + port;
        ConnPool connPool = MAP.get(key);
        if (connPool == null) {
            synchronized (ConnPool.class) {
                connPool = MAP.get(key);
                if (connPool == null) {
                    connPool = new ConnPool(host, port, userName, password);
                    MAP.put(key, connPool);
                }
            }
        }
        return connPool;
    }
}

The connection pool supports the establishment of different pools for different remote IP

Tool class encapsulation:

public static ChannelSftp borrowChannel(ConnectionConfig connCfg) {
	ConnPool connPool = ConnPool.getConnPool(connCfg.getHost(), connCfg.getPort(), connCfg.getUserName(),
			connCfg.getPassword());
	try {
		return connPool.borrowObject();
	} catch (Exception e) {
		logger.error("Get channelSftp from pool fail", e);
	}
}

public static void returnChannel(ConnectionConfig connCfg, ChannelSftp channel) {
	ConnPool connPool = ConnPool.getConnPool(connCfg.getHost(), connCfg.getPort(), connCfg.getUserName(),
			connCfg.getPassword());
	try {
		connPool.returnObject(channel);
	} catch (Exception e) {
		logger.error("Return channelSftp to pool fail", e);
	}
}

No problem with the test:

static void test2() {
	AtomicInteger j = new AtomicInteger(0);
	for (int i = 0; i < 50; i++) {
		new Thread(() -> {
			ConnPool connPool = ConnPool.getConnPool("*", 22, "root", "*");
			System.out.println(connPool + "--" + j.getAndIncrement());
			ChannelSftp channelSftp = null;
			try {
				channelSftp = connPool.borrowObject();
			} catch (Exception e) {
				e.printStackTrace();
			} finally {
				connPool.returnObject(channelSftp);
			}
		}).start();
	}
}