Tag Archives: linux

[Solved] yum command Error: “except keyboardinterrupt, e: syntax error: invalid syntax”“

Use the yum command prompt “except keyboardinterrupt, e: syntax error: invalid syntax”

Troubleshooting of error reporting using Yum command

Error reporting using Yum command

File "/usr/bin/yum", line 30
except KeyboardInterrupt, e:
^
SyntaxError: invalid syntax

Reason for the problem:
Yum package management uses python2 X, python2 X to Python 3.0 After X, problems occurred due to Python version syntax compatibility

Solution:

Modify the yum configuration file to point the python version to the previous version

vi /usr/bin/yum
#!/usr/bin/python2.7

Modify the urlgrabber ext down file and change the python version

vi /usr/libexec/urlgrabber-ext-down
#!/usr/bin/python2.7

redhat7. 5 system service is not started automatically and use systemctl start error: error getting authority: error initializing authority: CO, (the server will restart)

After the server is powered on, the default system service SSH network card will not start automatically
when using systemctl start sshd command, the following error will be reported. After the error is reported, the server will restart automatically
error getting authority: error initializing authority: could not connect: no such file or directory (g-io-error-wuark, 1)
view the system log, The following error is reported
#journalctl – XB
dependency failed for migrate local SELinux policy changes from the old store structure to the new structure

It may be that the attached system file does not exist.
#mount – a check whether the system attachment is normal, and the following error is reported.
mount special device XXX does not exist
#cat/etc/fstab check whether there is redundant attachment configuration for boot attachment, and delete the attachment configuration

[Solved] Linux executes SQL script Error: Syntax error

Demand scenario:

when deploying the production environment, you need to run the SQL script uploaded locally

Problem Description:

the local script runs normally, and liunux reports syntax error </ font>

syntax error

Cause analysis:

when running stored procedures or a long string of SQL scripts in Linux, you need to add fixed syntax at the beginning and end
beginning: \set sqlterm/
end:/\ set sqlterm

Solution:

add fixed statements \set sqlterm/and
/\ set sqlterm
as shown in the figure:
 

[Solved] Source Zsh error: zsh problem: compinit:503: no such file or directory

zsh problem: compinit:503: no such file or directory: /usr/local/share/zsh/site-functions/_brew
[oh-my-zsh]  Solution:

## zsh problem: compinit:503: no such file or directory: /usr/local/share/zsh/site-functions/_brew

Solution:

```zsh
sudo ln -fsv /opt/homebrew/completions/zsh/_brew /usr/local/share/zsh/site-functions/_brew

brew cleanup && source ~/.zshrc
```

Then it was solved.

[Solved] Kafka Start Log Error: WARN Found a corrupted index file due to requirement failed: Corrupt index found

Go straight to the theme, and the personal test is effective!!

Error content

WARN Found a corrupted index file due to requirement failed: Corrupt index found   A corrupt index file was found and the request failed

WARN Found a corrupted index file due to requirement failed: Corrupt index found, index file (/tmp/kafka-logs/__consumer_offsets-13/00000000000000000000.index) has non-zero size but the last offset is 0 which is no larger than the base offset 0.}. deleting /tmp/kafka-logs/__consumer_offsets-13/00000000000000000000.timeindex, /tmp/kafka-logs/__consumer_offsets-13/00000000000000000000.index, and /tmp/kafka-logs/__consumer_offsets-13/00000000000000000000.txnindex and rebuilding index... (kafka.log.Log)
[2021-11-27 10:02:24,168] INFO Recovering unflushed segment 0 in log __consumer_offsets-13. (kafka.log.Log)
[2021-11-27 10:02:24,168] INFO Loading producer state from offset 0 for partition __consumer_offsets-13 with message format version 2 (kafka.log.Log)
[2021-11-27 10:02:24,169] INFO Completed load of log __consumer_offsets-13 with 1 log segments, log start offset 0 and log end offset 0 in 2 ms (kafka.log.Log)
[2021-11-27 10:02:24,172] WARN Found a corrupted index file due to requirement failed: Corrupt index found, index file (/tmp/kafka-logs/test-0/00000000000000000000.index) has non-zero size but the last offset is 0 which is no larger than the base offset 0.}. deleting /tmp/kafka-logs/test-0/00000000000000000000.timeindex, /tmp/kafka-logs/test-0/00000000000000000000.index, and /tmp/kafka-logs/test-0/00000000000000000000.txnindex and rebuilding index... (kafka.log.Log)

Error reporting reason

The Kafka broker is abnormally closed and stopped, resulting in index corruption

Solutions

Step 1:   Enter the Kafka home directory of your installation and execute the following command to view the storage directory of Kafka data (Kafka log configuration)

vim config/server.properties

Step 2: directly delete the/TMP/Kafka LOHS folder

rm -rf /tmp/kafka-logs/

Result verification

Restart Kafka and check the log to verify that the problem has been solved

Check the Kafka log and confirm that the error has been resolved

To solve the problem, a problem is derived

deleting the Kafka data file is equivalent to starting Kafka for the first time. When you start up with the zookeeper provided by Kafka, you may report an error by viewing logs/zookeeper.out under the log information.

[Solved] C + + compile Error: error: default argument given for parameter 3

1. error: default argument given for parameter 3 of ‘void split(const string&, std::vector<std::__cxx11::basic_string<char> >&, char)’ [-fpermissive]
void split(const std::string &s, std::vector<std::string> &sv, const char flag = ‘ ‘)
^
In file included from /home/zhouying/lcl/test/FotaService/src/util.cpp:5:0:
/home/zhouying/lcl/test/FotaService/include/util.h:6:6: note: previous specification in ‘void split(const string&, std::vector<std::__cxx11::basic_string<char> >&, char)’ here
void split(const std::string &s, std::vector<std::string> &sv, const char flag = ‘ ‘);

 

Cause: The third formal parameter in the header file util.h is set to the default value, and the third formal parameter in the function definition in util.cpp is set to the default value, resulting in preservation, delete the third formal parameter default value in the function definition in util.cpp

mysqlclient in Ubuntu: How to Solve mysqlclient Install Error

After running pip3 install mysqlclient, the following error occurs:

ERROR: Command errored out with exit status 1:
     command: /usr/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-hmicg9lo/mysqlclient_17bc9786b14e4bff9a8bdcf2c8427558/setup.py'"'"'; __file__='"'"'/tmp/pip-install-hmicg9lo/mysqlclient_17bc9786b14e4bff9a8bdcf2c8427558/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-o3rs50h8/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.6/mysqlclient
         cwd: /tmp/pip-install-hmicg9lo/mysqlclient_17bc9786b14e4bff9a8bdcf2c8427558/
    Complete output (46 lines):
    mysql_config --version
    ['5.7.33']
    mysql_config --libs
    ['-L/usr/lib/x86_64-linux-gnu', '-lmysqlclient', '-lpthread', '-lz', '-lm', '-lrt', '-lssl', '-lcrypto', '-ldl']
    mysql_config --cflags
    ['-I/usr/include/mysql']
    ext_options:
      library_dirs: ['/usr/lib/x86_64-linux-gnu']
      libraries: ['mysqlclient', 'pthread', 'm', 'rt', 'dl']
      extra_compile_args: ['-std=c99']
      extra_link_args: []
      include_dirs: ['/usr/include/mysql']
      extra_objects: []
      define_macros: [('version_info', "(2,1,0,'final',0)"), ('__version__', '2.1.0')]
    /usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type'
      warnings.warn(msg)
    /usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'python_requires'
      warnings.warn(msg)
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.6
    creating build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/__init__.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/_exceptions.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/connections.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/converters.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/cursors.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/release.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/times.py -> build/lib.linux-x86_64-3.6/MySQLdb
    creating build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/__init__.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/CLIENT.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/CR.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/ER.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/FIELD_TYPE.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/FLAG.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    running build_ext
    building 'MySQLdb._mysql' extension
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/MySQLdb
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Dversion_info=(2,1,0,'final',0) -D__version__=2.1.0 -I/usr/include/mysql -I/usr/include/python3.6m -c MySQLdb/_mysql.c -o build/temp.linux-x86_64-3.6/MySQLdb/_mysql.o -std=c99
    MySQLdb/_mysql.c:46:20: fatal error: Python.h: No such file or directory
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-hmicg9lo/mysqlclient_17bc9786b14e4bff9a8bdcf2c8427558/setup.py'"'"'; __file__='"'"'/tmp/pip-install-hmicg9lo/mysqlclient_17bc9786b14e4bff9a8bdcf2c8427558/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-o3rs50h8/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.6/mysqlclient Check the logs for full command output.

 

Solution:

sudo apt-get install python3.6-dev

It may be due to the mismatch of dev version due to the installation of Python 3.6 in Ubuntu.

bandersnatch Error: No module Named [How to Solve]

ERROR: Unable to load entry point swift_plugin = bandersnatch_storage_plugins.swift:SwiftStorage: No module named ‘keystoneauth1’
This solution is missing at the time of installation due to python version, solution:
pip3 install keystoneauth1
Collecting keystoneauth1
Downloading https://files.pythonhosted.org/packages/7c/2e/dcfd2412941244e8a00a568654ce1687a1cab7be05d634ada0b5a078d0a3/keystoneauth1-4.4.0-py3-none-any.whl (314kB)
100% |████████████████████████████████| 317kB 240kB/s
Collecting pbr!=2.1.0,>=2.0.0 (from keystoneauth1)
Downloading https://files.pythonhosted.org/packages/73/c3/d45171501210b0305f4c93fafe50950f0c2228e87034ceb51744bd03ff08/pbr-5.8.0-py2.py3-none-any.whl (112kB)
100% |████████████████████████████████| 122kB 184kB/s
Requirement already satisfied: requests>=2.14.2 in /usr/lib/python3/dist-packages (from keystoneauth1)
Collecting stevedore>=1.20.0 (from keystoneauth1)
Downloading https://files.pythonhosted.org/packages/7a/bc/fcce9e50da73ea23af6d236e05e15db8a02da1099a5e0a479451bcea3833/stevedore-3.5.0-py3-none-any.whl (49kB)
100% |████████████████████████████████| 51kB 166kB/s
Requirement already satisfied: six>=1.10.0 in /usr/lib/python3/dist-packages (from keystoneauth1)
Collecting iso8601>=0.1.11 (from keystoneauth1)
Downloading https://files.pythonhosted.org/packages/df/e5/589bc81d410139ec4e4f37d9af5a50987566abf6d087b3c4fbed708109a9/iso8601-1.0.2-py3-none-any.whl
Collecting os-service-types>=1.2.0 (from keystoneauth1)
Downloading https://files.pythonhosted.org/packages/10/2d/318b2b631f68e0fc221ba8f45d163bf810cdb795cf242fe85ad3e5d45639/os_service_types-1.7.0-py2.py3-none-any.whl
Collecting importlib-metadata>=1.7.0; python_version < “3.8” (from stevedore>=1.20.0->keystoneauth1)
Downloading https://files.pythonhosted.org/packages/c4/1f/e2238896149df09953efcc53bdcc7d23597d6c53e428c30e572eda5ec6eb/importlib_metadata-4.8.2-py3-none-any.whl
Collecting zipp>=0.5 (from importlib-metadata>=1.7.0; python_version < “3.8”->stevedore>=1.20.0->keystoneauth1)
Using cached https://files.pythonhosted.org/packages/bd/df/d4a4974a3e3957fd1c1fa3082366d7fff6e428ddb55f074bf64876f8e8ad/zipp-3.6.0-py3-none-any.whl
Requirement already satisfied: typing-extensions>=3.6.4; python_version < “3.8” in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=1.7.0; python_version < “3.8”->stevedore>=1.20.0->keystoneauth1)
Installing collected packages: pbr, zipp, importlib-metadata, stevedore, iso8601, os-service-types, keystoneauth1
Successfully installed importlib-metadata-4.8.2 iso8601-1.0.2 keystoneauth1-4.4.0 os-service-types-1.7.0 pbr-5.8.0 stevedore-3.5.0 zipp-3.6.0

ERROR: Unable to load entry point swift_plugin = bandersnatch_storage_plugins.swift:SwiftStorage: No module named ‘swiftclient’
This one was installed with python-swiftclient missing.
Solution:
pip3 install python-swiftclient
Collecting python-swiftclient
Downloading https://files.pythonhosted.org/packages/f6/5f/6784a830e618a89272f8efee784930e614be285a6d3e82986916076fe69e/python_swiftclient-3.13.0-py2.py3-none-any.whl (86kB)
100% |████████████████████████████████| 92kB 281kB/s
Requirement already satisfied: six>=1.9.0 in /usr/lib/python3/dist-packages (from python-swiftclient)
Requirement already satisfied: requests>=1.1.0 in /usr/lib/python3/dist-packages (from python-swiftclient)
Installing collected packages: python-swiftclient
Successfully installed python-swiftclient-3.13.0

[Solved] Hive execute insert overwrite error: could not be cleared up

Problem description

    1. User Zhangsan executes insert overwrite:
INSERT OVERWRITE table temp.push_temp PARTITION(d_layer='app_video_uid_d_1')
SELECT ...

Could not be cleaned up:

Failed with exception Directory hdfs://Ucluster/user/hive/warehouse/temp.db/push_temp/d_layer=app_video_uid_d_1 could not be cleaned up.
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. Directory hdfs://Ucluster/user/hive/warehouse/temp.db/push_temp/d_layer=app_video_uid_d_1 could not be cleaned up.
      1. check the HDFS directory permissions and find that the directory is writable by everyone, and the directory owner is Lisi:
drwxrwxrwt   - lisi supergroup          0 2021-11-29 15:04 /user/hive/warehouse/temp.db/push_temp/d_layer=app_video_uid_d_1
        1. the user Lisi executes step 1. The SQL in step 1 can execute  successfully

Cause of problem

Three words – viscous bit
look carefully at the directory permissions above. The last bit is “t”, which means that the sticky bit is enabled for the directory, that is, only the owner of the directory can delete files under the directory

# Non-owner delete sticky bit files
$ hadoop fs -rm /user/hive/warehouse/temp.db/push_temp/d_layer=app_video_uid_d_1/000000_0
21/11/29 16:32:59 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 7320 minutes, Emptier interval = 0 minutes.
rm: Failed to move to trash: hdfs://Ucluster/user/hive/warehouse/temp.db/push_temp/d_layer=app_video_uid_d_1/000000_0: Permission denied by sticky bit setting: user=admin, inode=000000_0

Because insert overwrite needs to delete the original file in the directory, but it cannot be deleted due to sticky bits, resulting in HQL execution failure

Solution

Cancel the sticky bit of the directory

# Cancellation of sticking position
hadoop fs -chmod -R o-t /user/hive/warehouse/temp.db/push_temp/d_layer=app_video_uid_d_1

# Open sticking position
hadoop fs -chmod -R o+t /user/hive/warehouse/temp.db/push_temp/d_layer=app_video_uid_d_1

[Solved] Docker error: “unknown runtime specified NVIDIA” using GPU“

Question 1 recurrence

System: Ubuntu 18.04 docker version: 20.10.7
when I start a container, run the following command:

docker run -itd \
   --runtime=nvidia --gpus=all \
   -e NVIDIA_DRIVER_CAPABILITIES=compute,utility,video,graphics \
   image_name

report errors:

docker: Error response from daemon: Unknown runtime specified nvidia.
See 'docker run --help'.

Solution 1

This is because the user did not join the docker group and added his own user to the docker user group.

sudo usermod -a -G docker $USER

Question 2 recurrence

docker: Error response from daemon: Unknown runtime specified nvidia.
See 'docker run --help'.

Solution 2

NVIDIA docker2 needs to be installed

sudo apt-get install -y nvidia-docker2

Restart docker

sudo systemctl daemon-reload
sudo systemctl restart docker

[Solved] rsync Error: @ERROR: chdir failed rsync error

Question

The client uses the Rsync command to synchronize the following errors:

@ERROR: chdir failed
rsync error: error starting client-server protocol (code 5) at main.c(1649) [sender=3.1.2]

Solution:

After troubleshooting, it is caused by the server directory creation failure

[ew_dbbak]
path = /data/dbbak/ew_dbbak/