[Solved] Linux Service Start Error: *.service: Main process exited, code=exited, status=203/EXEC

Error when starting rocketmq service:

[[email protected] bin]# systemctl start rocketmq-nameserver
[[email protected] bin]# systemctl status rocketmq-nameserver
● rocketmq-nameserver.service - nameserver
   Loaded: loaded (/usr/lib/systemd/system/rocketmq-nameserver.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Fri 2021-12-10 18:47:05 CST; 3s ago
  Process: 2414 ExecStart=/home/rocketmq/bin/mqnamesrv (code=exited, status=203/EXEC)
 Main PID: 2414 (code=exited, status=203/EXEC)

Dec 10 18:47:05 rocketmq1-nameserver-test systemd[1]: Started nameserver.
Dec 10 18:47:05 rocketmq1-nameserver-test systemd[1]: rocketmq-nameserver.service: Main process exited, code=exited, status=203/EXEC
Dec 10 18:47:05 rocketmq1-nameserver-test systemd[1]: rocketmq-nameserver.service: Failed with result 'exit-code'.


First, use journalctl – Xe to view the detailed error reports:

[[email protected] bin]# journalctl -xe
Dec 10 18:47:05 rocketmq1-nameserver-test systemd[1]: Started nameserver.
-- Subject: rocketmq-nameserver.service Unit has ended start
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
-- 
-- rocketmq-nameserver.service Unit has ended start.
-- 
-- The start result is "done".
Dec 10 18:47:05 rocketmq1-nameserver-test systemd[2414]: rocketmq-nameserver.service: Failed to execute command: Permission denied
Dec 10 18:47:05 rocketmq1-nameserver-test systemd[2414]: rocketmq-nameserver.service: Failed at step EXEC spawning /home/rocketmq/bin/mqnamesrv: Permission denied
-- Subject: progression /home/rocketmq/bin/mqnamesrv could not be executed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
-- 
-- process /home/rocketmq/bin/mqnamesrv could not be executed and has failed.
-- 
-- The process returns an error code of 13.
Dec 10 18:47:05 rocketmq1-nameserver-test systemd[1]: rocketmq-nameserver.service: Main process exited, code=exited, status=203/EXEC
Dec 10 18:47:05 rocketmq1-nameserver-test systemd[1]: rocketmq-nameserver.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
-- 
-- The unit rocketmq-nameserver.service has entered the 'failed' state with result 'exit-code'.
Dec 10 18:47:05 rocketmq1-nameserver-test dbus-daemon[970]: [system] Activating service name='org.fedoraproject.Setroubleshootd' requested by ':1.4' (uid=0 pid=948 comm="/usr/sbin/sedispatch " label="system_u:system_r:auditd_t:s0") (using servicehelper)
Dec 10 18:47:05 rocketmq1-nameserver-test dbus-daemon[2417]: [system] Failed to reset fd limit before activating service: org.freedesktop.DBus.Error.AccessDenied: Failed to restore old fd limit: Operation not permitted
Dec 10 18:47:07 rocketmq1-nameserver-test dbus-daemon[970]: [system] Successfully activated service 'org.fedoraproject.Setroubleshootd'
Dec 10 18:47:07 rocketmq1-nameserver-test setroubleshoot[2417]: AnalyzeThread.run(): Cancel pending alarm
Dec 10 18:47:07 rocketmq1-nameserver-test setroubleshoot[2417]: failed to retrieve rpm info for /home/rocketmq/bin/mqnamesrv
Dec 10 18:47:07 rocketmq1-nameserver-test dbus-daemon[970]: [system] Activating service name='org.fedoraproject.SetroubleshootPrivileged' requested by ':1.94' (uid=995 pid=2417 comm="/usr/libexec/platform-python -Es /usr/sbin/setroub" label="system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023") (using servicehelper)
Dec 10 18:47:07 rocketmq1-nameserver-test dbus-daemon[2431]: [system] Failed to reset fd limit before activating service: org.freedesktop.DBus.Error.AccessDenied: Failed to restore old fd limit: Operation not permitted
Dec 10 18:47:09 rocketmq1-nameserver-test dbus-daemon[970]: [system] Successfully activated service 'org.fedoraproject.SetroubleshootPrivileged'
Dec 10 18:47:10 rocketmq1-nameserver-test setroubleshoot[2417]: SELinux is preventing /usr/lib/systemd/systemd from 'read, open' accesses on the file /home/rocketmq/bin/mqnamesrv. For complete SELinux messages run: sealert -l e1b1100f-c8cb-44d7-b3de-1559f1d87286
Dec 10 18:47:10 rocketmq1-nameserver-test setroubleshoot[2417]: SELinux is preventing /usr/lib/systemd/systemd from 'read, open' accesses on the file /home/rocketmq/bin/mqnamesrv.
                                                                  
                                                                  *****  Plugin restorecon (99.5 confidence) suggests   ************************
                                                                  
                                                                  If you want to fix the label. 
                                                                  /home/rocketmq/bin/mqnamesrv default label should be home_bin_t.
                                                                  Then you can run restorecon. The access attempt may have been stopped due to insufficient permissions to access a parent directory in which case try to change the following command accordingly.
                                                                  Do
                                                                  # /sbin/restorecon -v /home/rocketmq/bin/mqnamesrv
                                                                  
                                                                  *****  Plugin catchall (1.49 confidence) suggests   **************************
                                                                  
                                                                  If you believe that systemd should be allowed read open access on the mqnamesrv file by default.
                                                                  Then you should report this as a bug.
                                                                  You can generate a local policy module to allow this access.
                                                                  Do
                                                                  allow this access for now by executing:
                                                                  # ausearch -c '(qnamesrv)' --raw | audit2allow -M my-qnamesrv
                                                                  # semodule -X 300 -i my-qnamesrv.pp


There is a passage:

Dec 10 18:47:07 rocketmq1-nameserver-test dbus-daemon[2431]: [system] Failed to reset fd limit before activating service: org.freedesktop.DBus.Error.AccessDenied: Failed to restore old fd limit: Operation not permitted
Dec 10 18:47:09 rocketmq1-nameserver-test dbus-daemon[970]: [system] Successfully activated service 'org.fedoraproject.SetroubleshootPrivileged'
Dec 10 18:47:10 rocketmq1-nameserver-test setroubleshoot[2417]: SELinux is preventing /usr/lib/systemd/systemd from 'read, open' accesses on the file /home/rocketmq/bin/mqnamesrv. For complete SELinux messages run: sealert -l e1b1100f-c8cb-44d7-b3de-1559f1d87286

As mentioned above, we run: sealert – L e1b1100f-c8cb-44d7-b3de-1559f1d87286

[[email protected] bin]# sealert -l e1b1100f-c8cb-44d7-b3de-1559f1d87286
SELinux is preventing /usr/lib/systemd/systemd from 'read, open' accesses on the file /home/rocketmq/bin/mqnamesrv.

*****  Plugin restorecon (99.5 confidence level) Recommended ******************************************

If you want to fix the tags. /home/rocketmq/bin/mqnamesrv the default tag should be home_bin_t.
Then you can run restorecon. access attempts may have stopped due to insufficient permissions to access the parent directory, in which case try changing the following command accordingly.
Do
# /sbin/restorecon -v /home/rocketmq/bin/mqnamesrv

***** plug-in catchall (1.49 confidence level) Recommended ********************************************

If you believe (qnamesrv) should allow _BASE_PATH read open access to the mqnamesrv file by default.
Then this should be reported as a bug.
A local policy module can be generated to allow this access.
Do
Temporarily allow this access execute: #ausearch -c '(qnamesrv)'--raw | audit2allow -M my-qnamesrv #semodule -X 300 -i my-qnamesrv.pp
Omit the following

The last sentence above is to run the command: #ausearch – C ‘(qnamesrv)’ – raw | audit2alow – m my qnamesrv semodule – x 300 – I my qnamesrv pp

However, after running, it still reports an error
after verifying the data, it is the problem of SELinux:
SELinux believes that binary files can only be executed from some locations, and my user-defined directory is not clearly marked as allowed. It var_ T from/SRV /* (I think) inherited the type.

To get an extensive list of current rules for all directories, you can run semanage fcontext — list

I added an exception using the following ansible task:

name: set SELinux permissions on ts3server binaries
sefcontext:
target: “/srv/teamspeak/versions/[^/]+/ts3server”
setype: bin_ tname: reload SELinux policy to ensure that ts3server is executable
command: restorecon -irv /srv/teamspeak/
when: tarball. Changed
you can achieve the same goal by using the command followed by semanage fcontext restorecon – IRV/SRV/TeamSpeak/
therefore, we need to add a rocketmq startup flag:

restorecon -irv /home/rocketmq/bin/

Restart service succeeded:

[[email protected] bin]# semodule -i my-qnamesrv.pp
[[email protected] bin]# systemctl start rocketmq-nameserver
[[email protected] bin]# systemctl status rocketmq-nameserver
● rocketmq-nameserver.service - nameserver
   Loaded: loaded (/usr/lib/systemd/system/rocketmq-nameserver.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2021-12-10 18:47:48 CST; 1min 15s ago
 Main PID: 2459 (mqnamesrv)
    Tasks: 36 (limit: 10931)
   Memory: 172.9M
   CGroup: /system.slice/rocketmq-nameserver.service
           ├─2459 /bin/sh /home/rocketmq/bin/mqnamesrv
           ├─2463 sh /home/rocketmq/bin/runserver.sh org.apache.rocketmq.namesrv.NamesrvStartup
           └─2480 /usr/local/jdk1.8.0_151/bin/java -server -Xms256m -Xmx256m -Xmn128m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSP

Read More: