For those suspected to be related to the NSS version of the above problem, the curl request will be verified in south China VPC and South China grayscale, respectively, below
One, in south China gray level environment test
1. Curl S3 internal domain name (error reporting)
[root@A06-R12-302F0714-I12-86 --PROD-- ~]# curl -I https://s3-internal.cn-south-1.jdcloud-oss.com/a/a -v
* About to connect() to s3-internal.cn-south-1.jdcloud-oss.com port 443 (#0)
* Trying 100.65.254.35...
* Connected to s3-internal.cn-south-1.jdcloud-oss.com (100.65.254.35) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* NSS error -12286 (SSL_ERROR_NO_CYPHER_OVERLAP)
* Cannot communicate securely with peer: no common encryption algorithm(s).
* Error in TLS handshake, trying SSLv3...
> HEAD /a/a HTTP/1.1
> User-Agent: curl/7.29.0
> Host: s3-internal.cn-south-1.jdcloud-oss.com
> Accept: */*
>
* Connection died, retrying a fresh connect
* Closing connection 0
* Issue another request to this URL: 'https://s3-internal.cn-south-1.jdcloud-oss.com/a/a'
* About to connect() to s3-internal.cn-south-1.jdcloud-oss.com port 443 (#1)
* Trying 100.65.254.35...
* Connected to s3-internal.cn-south-1.jdcloud-oss.com (100.65.254.35) port 443 (#1)
* TLS disabled due to previous handshake failure
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* NSS error -12286 (SSL_ERROR_NO_CYPHER_OVERLAP)
* Cannot communicate securely with peer: no common encryption algorithm(s).
* Closing connection 1
curl: (35) Cannot communicate securely with peer: no common encryption algorithm(s).
2. Test whether port 443 is open (all OK)
[root@A06-R12-302F0714-I12-86 --PROD-- ~]# telnet 127.0.0.1 443
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
^CConnection closed by foreign host.
[root@A06-R12-302F0714-I12-86 --PROD-- ~]# telnet 100.65.254.3 443
Trying 100.65.254.3...
Connected to 100.65.254.3.
Escape character is '^]'.
3. Curl S3 new domain name (results are OK)
[root@A06-R12-302F0714-I12-86 --PROD-- ~]# curl -I https://s3.cn-south-1.jdcloud-oss.com/a/a -v
* About to connect() to s3.cn-south-1.jdcloud-oss.com port 443 (#0)
* Trying 59.37.144.139...
* Connected to s3.cn-south-1.jdcloud-oss.com (59.37.144.139) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
* subject: CN=*.s3.cn-north-1.jdcloud-oss.com,O="BEIJING JINGDONG SHANGKE INFORMATION TECHNOLOGY CO., LTD.",L=beijing,ST=beijing,C=CN
* start date: Jan 29 09:31:09 2019 GMT
* expire date: Jan 30 09:31:09 2020 GMT
* common name: *.s3.cn-north-1.jdcloud-oss.com
* issuer: CN=GlobalSign Organization Validation CA - SHA256 - G2,O=GlobalSign nv-sa,C=BE
> HEAD /a/a HTTP/1.1
> User-Agent: curl/7.29.0
> Host: s3.cn-south-1.jdcloud-oss.com
> Accept: */*
>
< HTTP/1.1 404 Not Found
HTTP/1.1 404 Not Found
< Server: jfe
Server: jfe
< Date: Mon, 11 Mar 2019 04:53:31 GMT
Date: Mon, 11 Mar 2019 04:53:31 GMT
< Content-Length: 0
Content-Length: 0
< Connection: keep-alive
Connection: keep-alive
< x-req-id: B6E24BAF242EF989
x-req-id: B6E24BAF242EF989
<
* Connection #0 to host s3.cn-south-1.jdcloud-oss.com left intact
4. Curl S3 old domain name (OK)
[root@A06-R12-302F0714-I12-86 --PROD-- ~]# curl -I https://s3.cn-south-1.jcloudcs.com/a/a -v
* About to connect() to s3.cn-south-1.jcloudcs.com port 443 (#0)
* Trying 100.65.11.130...
* Connected to s3.cn-south-1.jcloudcs.com (100.65.11.130) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
* subject: CN=*.jdcloud.com,O="BEIJING JINGDONG SHANGKE INFORMATION TECHNOLOGY CO., LTD.",L=beijing,ST=beijing,C=CN
* start date: Nov 19 02:26:04 2018 GMT
* expire date: Feb 18 09:06:02 2020 GMT
* common name: *.jdcloud.com
* issuer: CN=GlobalSign Organization Validation CA - SHA256 - G2,O=GlobalSign nv-sa,C=BE
> HEAD /a/a HTTP/1.1
> User-Agent: curl/7.29.0
> Host: s3.cn-south-1.jcloudcs.com
> Accept: */*
>
< HTTP/1.1 404 Not Found
HTTP/1.1 404 Not Found
< Server: JDCloudOSS
Server: JDCloudOSS
< Date: Mon, 11 Mar 2019 04:53:46 GMT
Date: Mon, 11 Mar 2019 04:53:46 GMT
< Content-Length: 0
Content-Length: 0
< Connection: close
Connection: close
< x-req-id: 8228B04708806DDF
x-req-id: 8228B04708806DDF
<
* Closing connection 0
Ii. VPC test in South China
1, curl S3, internal network new domain name (OK), test other domain name is also OK here is not posted
[root@domain-cn-south-1 ~]# curl -I https://s3-internal.cn-south-1.jdcloud-oss.com/a/a -v
* About to connect() to s3-internal.cn-south-1.jdcloud-oss.com port 443 (#0)
* Trying 100.65.254.35...
* Connected to s3-internal.cn-south-1.jdcloud-oss.com (100.65.254.35) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
* subject: CN=*.s3.cn-north-1.jdcloud-oss.com,O="BEIJING JINGDONG SHANGKE INFORMATION TECHNOLOGY CO., LTD.",L=beijing,ST=beijing,C=CN
* start date: Jan 29 09:31:09 2019 GMT
* expire date: Jan 30 09:31:09 2020 GMT
* common name: *.s3.cn-north-1.jdcloud-oss.com
* issuer: CN=GlobalSign Organization Validation CA - SHA256 - G2,O=GlobalSign nv-sa,C=BE
> HEAD /a/a HTTP/1.1
> User-Agent: curl/7.29.0
> Host: s3-internal.cn-south-1.jdcloud-oss.com
> Accept: */*
>
< HTTP/1.1 404 Not Found
HTTP/1.1 404 Not Found
< Server: JDCloudOSS
Server: JDCloudOSS
< Date: Mon, 11 Mar 2019 05:13:33 GMT
Date: Mon, 11 Mar 2019 05:13:33 GMT
< Content-Length: 0
Content-Length: 0
< Connection: keep-alive
Connection: keep-alive
< x-req-id: 828EF8FED8952127
x-req-id: 828EF8FED8952127
<
* Connection #0 to host s3-internal.cn-south-1.jdcloud-oss.com left intact
Third, through the above tests, it is suspected that the NSS version problem caused the failure to load the new certificate
1. Check NSS version (South China Gray Scale machine NSS version is low)
South China gray scale machine
[root@A06-R12-302F0714-I12-86 --PROD-- ~]# curl -V
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.15.4 zlib/1.2.7 libidn/1.28 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz
[root@A06-R12-302F0714-I12-86 --PROD-- ~]# rpm -qa | grep nss
openssh-server-6.6.1p1-11.el7.x86_64
nss-softokn-freebl-3.16.2.3-9.el7.x86_64
nss-util-3.16.2.3-2.el7.x86_64
openssl-1.0.1e-42.el7.x86_64
openssh-clients-6.6.1p1-11.el7.x86_64
nss-tools-3.16.2.3-5.el7.x86_64
nss-3.16.2.3-5.el7.x86_64
nss-sysinit-3.16.2.3-5.el7.x86_64
jansson-2.4-6.el7.x86_64
openssh-6.6.1p1-11.el7.x86_64
nss-softokn-3.16.2.3-9.el7.x86_64
libsss_nss_idmap-1.12.2-58.el7.x86_64
openssl-devel-1.0.1e-42.el7.x86_64
openssl-libs-1.0.1e-42.el7.x86_64
South China VPC
[root@domain-cn-south-1 ~]# curl -V
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.34 zlib/1.2.7 libidn/1.28 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz unix-sockets
[root@domain-cn-south-1 ~]# rpm -qa | grep nss
nss-3.36.0-5.el7_5.x86_64
jansson-2.10-1.el7.x86_64
nss-util-3.36.0-1.el7_5.x86_64
nss-pem-1.0.3-4.el7.x86_64
openssl-1.0.2k-12.el7.x86_64
nss-sysinit-3.36.0-5.el7_5.x86_64
nss-tools-3.36.0-5.el7_5.x86_64
openssh-clients-7.4p1-16.el7.x86_64
nss-softokn-freebl-3.36.0-5.el7_5.x86_64
nss-softokn-3.36.0-5.el7_5.x86_64
openssl-libs-1.0.2k-12.el7.x86_64
openssh-7.4p1-16.el7.x86_64
libsss_nss_idmap-1.16.0-19.el7_5.5.x86_64
openssh-server-7.4p1-16.el7.x86_64
Four, update the South China gray scale machine NSS version
1.
[root@A06-R12-302F0714-I12-86 --PROD-- ~]# yum update nss
Loaded plugins: fastestmirror, langpacks
Repodata is over 2 weeks old. Install yum-cron?Or run: yum makecache fast
base | 3.6 kB 00:00:00
extras | 3.4 kB 00:00:00
updates | 3.4 kB 00:00:00
(1/2): extras/7/x86_64/primary_db | 180 kB 00:00:00
(2/2): updates/7/x86_64/primary_db | 2.5 MB 00:00:00
Determining fastest mirrors
Resolving Dependencies
--> Running transaction check
---> Package nss.x86_64 0:3.16.2.3-5.el7 will be updated
--> Processing Dependency: nss = 3.16.2.3-5.el7 for package: nss-sysinit-3.16.2.3-5.el7.x86_64
--> Processing Dependency: nss(x86-64) = 3.16.2.3-5.el7 for package: nss-tools-3.16.2.3-5.el7.x86_64
---> Package nss.x86_64 0:3.36.0-7.1.el7_6 will be an update
--> Processing Dependency: nss-util >= 3.36.0-1.1 for package: nss-3.36.0-7.1.el7_6.x86_64
--> Processing Dependency: nss-softokn(x86-64) >= 3.36.0-1 for package: nss-3.36.0-7.1.el7_6.x86_64
--> Processing Dependency: nspr >= 4.19.0 for package: nss-3.36.0-7.1.el7_6.x86_64
--> Processing Dependency: nss-pem(x86-64) for package: nss-3.36.0-7.1.el7_6.x86_64
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.31)(64bit) for package: nss-3.36.0-7.1.el7_6.x86_64
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.24)(64bit) for package: nss-3.36.0-7.1.el7_6.x86_64
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.21)(64bit) for package: nss-3.36.0-7.1.el7_6.x86_64
--> Running transaction check
---> Package nspr.x86_64 0:4.10.6-3.el7 will be updated
---> Package nspr.x86_64 0:4.19.0-1.el7_5 will be an update
---> Package nss-pem.x86_64 0:1.0.3-5.el7 will be installed
---> Package nss-softokn.x86_64 0:3.16.2.3-9.el7 will be updated
---> Package nss-softokn.x86_64 0:3.36.0-5.el7_5 will be an update
--> Processing Dependency: nss-softokn-freebl(x86-64) >= 3.36.0-5.el7_5 for package: nss-softokn-3.36.0-5.el7_5.x86_64
---> Package nss-sysinit.x86_64 0:3.16.2.3-5.el7 will be updated
---> Package nss-sysinit.x86_64 0:3.36.0-7.1.el7_6 will be an update
---> Package nss-tools.x86_64 0:3.16.2.3-5.el7 will be updated
---> Package nss-tools.x86_64 0:3.36.0-7.1.el7_6 will be an update
---> Package nss-util.x86_64 0:3.16.2.3-2.el7 will be updated
---> Package nss-util.x86_64 0:3.36.0-1.1.el7_6 will be an update
--> Running transaction check
---> Package nss-softokn-freebl.x86_64 0:3.16.2.3-9.el7 will be updated
---> Package nss-softokn-freebl.x86_64 0:3.36.0-5.el7_5 will be an update
--> Finished Dependency Resolution
Dependencies Resolved
=============================================================================================================================================================================================
Package Arch Version Repository Size
=============================================================================================================================================================================================
Updating:
nss x86_64 3.36.0-7.1.el7_6 updates 835 k
Installing for dependencies:
nss-pem x86_64 1.0.3-5.el7 base 74 k
Updating for dependencies:
nspr x86_64 4.19.0-1.el7_5 base 127 k
nss-softokn x86_64 3.36.0-5.el7_5 base 315 k
nss-softokn-freebl x86_64 3.36.0-5.el7_5 base 222 k
nss-sysinit x86_64 3.36.0-7.1.el7_6 updates 62 k
nss-tools x86_64 3.36.0-7.1.el7_6 updates 515 k
nss-util x86_64 3.36.0-1.1.el7_6 updates 78 k
Transaction Summary
=============================================================================================================================================================================================
Install ( 1 Dependent package)
Upgrade 1 Package (+6 Dependent packages)
Total download size: 2.2 M
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/8): nspr-4.19.0-1.el7_5.x86_64.rpm | 127 kB 00:00:00
(2/8): nss-pem-1.0.3-5.el7.x86_64.rpm | 74 kB 00:00:00
(3/8): nss-softokn-freebl-3.36.0-5.el7_5.x86_64.rpm | 222 kB 00:00:00
(4/8): nss-softokn-3.36.0-5.el7_5.x86_64.rpm | 315 kB 00:00:00
(5/8): nss-3.36.0-7.1.el7_6.x86_64.rpm | 835 kB 00:00:00
(6/8): nss-tools-3.36.0-7.1.el7_6.x86_64.rpm | 515 kB 00:00:00
(7/8): nss-sysinit-3.36.0-7.1.el7_6.x86_64.rpm | 62 kB 00:00:00
(8/8): nss-util-3.36.0-1.1.el7_6.x86_64.rpm | 78 kB 00:00:00
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 4.6 MB/s | 2.2 MB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : nspr-4.19.0-1.el7_5.x86_64 1/15
/sbin/ldconfig: /usr/local/lib/libpcreposix.so.0 is not a symbolic link
Updating : nss-util-3.36.0-1.1.el7_6.x86_64 2/15
/sbin/ldconfig: /usr/local/lib/libpcreposix.so.0 is not a symbolic link
Updating : nss-softokn-freebl-3.36.0-5.el7_5.x86_64 3/15
Updating : nss-softokn-3.36.0-5.el7_5.x86_64 4/15
/sbin/ldconfig: /usr/local/lib/libpcreposix.so.0 is not a symbolic link
Installing : nss-pem-1.0.3-5.el7.x86_64 5/15
Updating : nss-sysinit-3.36.0-7.1.el7_6.x86_64 6/15
Updating : nss-3.36.0-7.1.el7_6.x86_64 7/15
/sbin/ldconfig: /usr/local/lib/libpcreposix.so.0 is not a symbolic link
Updating : nss-tools-3.36.0-7.1.el7_6.x86_64 8/15
Cleanup : nss-tools-3.16.2.3-5.el7.x86_64 9/15
Cleanup : nss-sysinit-3.16.2.3-5.el7.x86_64 10/15
Cleanup : nss-3.16.2.3-5.el7.x86_64 11/15
/sbin/ldconfig: /usr/local/lib/libpcreposix.so.0 is not a symbolic link
Cleanup : nss-softokn-3.16.2.3-9.el7.x86_64 12/15
/sbin/ldconfig: /usr/local/lib/libpcreposix.so.0 is not a symbolic link
Cleanup : nss-util-3.16.2.3-2.el7.x86_64 13/15
/sbin/ldconfig: /usr/local/lib/libpcreposix.so.0 is not a symbolic link
Cleanup : nspr-4.10.6-3.el7.x86_64 14/15
/sbin/ldconfig: /usr/local/lib/libpcreposix.so.0 is not a symbolic link
Cleanup : nss-softokn-freebl-3.16.2.3-9.el7.x86_64 15/15
Verifying : nss-3.36.0-7.1.el7_6.x86_64 1/15
Verifying : nss-pem-1.0.3-5.el7.x86_64 2/15
Verifying : nss-tools-3.36.0-7.1.el7_6.x86_64 3/15
Verifying : nspr-4.19.0-1.el7_5.x86_64 4/15
Verifying : nss-sysinit-3.36.0-7.1.el7_6.x86_64 5/15
Verifying : nss-softokn-freebl-3.36.0-5.el7_5.x86_64 6/15
Verifying : nss-util-3.36.0-1.1.el7_6.x86_64 7/15
Verifying : nss-softokn-3.36.0-5.el7_5.x86_64 8/15
Verifying : nss-softokn-freebl-3.16.2.3-9.el7.x86_64 9/15
Verifying : nss-util-3.16.2.3-2.el7.x86_64 10/15
Verifying : nss-sysinit-3.16.2.3-5.el7.x86_64 11/15
Verifying : nss-tools-3.16.2.3-5.el7.x86_64 12/15
Verifying : nss-softokn-3.16.2.3-9.el7.x86_64 13/15
Verifying : nss-3.16.2.3-5.el7.x86_64 14/15
Verifying : nspr-4.10.6-3.el7.x86_64 15/15
Dependency Installed:
nss-pem.x86_64 0:1.0.3-5.el7
Updated:
nss.x86_64 0:3.36.0-7.1.el7_6
Dependency Updated:
nspr.x86_64 0:4.19.0-1.el7_5 nss-softokn.x86_64 0:3.36.0-5.el7_5 nss-softokn-freebl.x86_64 0:3.36.0-5.el7_5 nss-sysinit.x86_64 0:3.36.0-7.1.el7_6
nss-tools.x86_64 0:3.36.0-7.1.el7_6 nss-util.x86_64 0:3.36.0-1.1.el7_6
Complete!
2. Re-test the new DOMAIN name OK in South China Intranet
[root@A06-R12-302F0714-I12-86 --PROD-- ~]# curl -I https://s3-internal.cn-south-1.jdcloud-oss.com/a/a -v
* About to connect() to s3-internal.cn-south-1.jdcloud-oss.com port 443 (#0)
* Trying 100.65.254.3...
* Connected to s3-internal.cn-south-1.jdcloud-oss.com (100.65.254.3) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* subject: CN=*.s3.cn-north-1.jdcloud-oss.com,O="BEIJING JINGDONG SHANGKE INFORMATION TECHNOLOGY CO., LTD.",L=beijing,ST=beijing,C=CN
* start date: Jan 29 09:31:09 2019 GMT
* expire date: Jan 30 09:31:09 2020 GMT
* common name: *.s3.cn-north-1.jdcloud-oss.com
* issuer: CN=GlobalSign Organization Validation CA - SHA256 - G2,O=GlobalSign nv-sa,C=BE
> HEAD /a/a HTTP/1.1
> User-Agent: curl/7.29.0
> Host: s3-internal.cn-south-1.jdcloud-oss.com
> Accept: */*
>
< HTTP/1.1 404 Not Found
HTTP/1.1 404 Not Found
< Server: JDCloudOSS
Server: JDCloudOSS
< Date: Mon, 11 Mar 2019 04:54:29 GMT
Date: Mon, 11 Mar 2019 04:54:29 GMT
< Content-Length: 0
Content-Length: 0
< Connection: keep-alive
Connection: keep-alive
< x-req-id: ABE7F53066B4B389
x-req-id: ABE7F53066B4B389
<
* Connection #0 to host s3-internal.cn-south-1.jdcloud-oss.com left intact
Conclusion: NSS version problems cause curl error handling: YUM Update NSS