Problematic environment: centos7
Solution: turn off the firewall!
Problematic environment: centos7
Solution: turn off the firewall!
Problem Description:
got permission denied while trying to connect to the docker daemon socket at unix:///var/run/docker.sock: Get http://% 2fvar% 2frun% 2fdocker.sock/v1.40/version: Dial UNIX/var/run/docker.sock: Connect: permission denied
error reason:
docker process uses UNIX socket instead of TCP port. By default, UNIX socket belongs to the root user and needs root permission to access it
solution:
sudo groupadd docker #Add the docker user group
sudo gpasswd -a $USER docker #Add the logged-in user to the docker user group
newgrp docker #Update the user group
docker ps #Test that the docker command works properly with sudo
Environmental information
ansible
to call the docker
module of remote centos7
, you need to install the package docker = = 4.4.4
.
"msg": "Failed to import the required Python library (Docker SDK for Python: docker above 5.0.0 (Python >= 3.6) or docker before 5.0.0 (Python 2.7) or docker-py (Python 2.6)) on k8s-node-56-166.by.com's Python /usr/bin/python.
Please read the module documentation and install it in the appropriate location.
If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter, for example via `pip install docker` (Python >= 3.6) or `pip install docker==4.4.4` (Python 2.7) or `pip install docker-py` (Python 2.6). The error was: No module named requests.exceptions"
After using pip to install, the following error is still reported:
>>> import docker
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/site-packages/docker/__init__.py", line 2, in <module>
from .api import APIClient
File "/usr/lib/python2.7/site-packages/docker/api/__init__.py", line 2, in <module>
from .client import APIClient
File "/usr/lib/python2.7/site-packages/docker/api/client.py", line 8, in <module>
import websocket
File "/usr/lib/python2.7/site-packages/websocket/__init__.py", line 20, in <module>
from ._app import WebSocketApp
File "/usr/lib/python2.7/site-packages/websocket/_app.py", line 23, in <module>
import selectors
ImportError: No module named selectors
>>> qa
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
This line of error code comes from
File "/usr/lib/python2.7/site-packages/websocket/_app.py", line 23, in <module>
import selectors
This file is from the websocket client
package.
Observe the detailed installation information as follows:
[root@k8s-node-56-164 ~]# pip install docker==4.4.4
Collecting docker==4.4.4
Downloading https://files.pythonhosted.org/packages/c4/22/410313ad554477e87ec406d38d85f810e61ddb0d2fc44e64994857476de9/docker-4.4.4-py2.py3-none-any.whl (147kB)
100% |████████████████████████████████| 153kB 636kB/s
Requirement already satisfied (use --upgrade to upgrade): six>=1.4.0 in /usr/lib/python2.7/site-packages (from docker==4.4.4)
Requirement already satisfied (use --upgrade to upgrade): ipaddress>=1.0.16; python_version < "3.3" in /usr/lib/python2.7/site-packages (from docker==4.4.4)
Requirement already satisfied (use --upgrade to upgrade): backports.ssl-match-hostname>=3.5; python_version < "3.5" in /usr/lib/python2.7/site-packages (from docker==4.4.4)
Collecting requests!=2.18.0,>=2.14.2 (from docker==4.4.4)
Downloading https://files.pythonhosted.org/packages/92/96/144f70b972a9c0eabbd4391ef93ccd49d0f2747f4f6a2a2738e99e5adc65/requests-2.26.0-py2.py3-none-any.whl (62kB)
100% |████████████████████████████████| 71kB 827kB/s
Collecting websocket-client>=0.32.0 (from docker==4.4.4)
Downloading https://files.pythonhosted.org/packages/55/44/030ea47390896c8d6dc9995c8e9a4c5df3a161cd45416d88119036c73eda/websocket_client-1.2.1-py2.py3-none-any.whl (52kB)
100% |████████████████████████████████| 61kB 815kB/s
Collecting certifi>=2017.4.17 (from requests!=2.18.0,>=2.14.2->docker==4.4.4)
Downloading https://files.pythonhosted.org/packages/05/1b/0a0dece0e8aa492a6ec9e4ad2fe366b511558cdc73fd3abc82ba7348e875/certifi-2021.5.30-py2.py3-none-any.whl (145kB)
100% |████████████████████████████████| 153kB 740kB/s
Collecting urllib3<1.27,>=1.21.1 (from requests!=2.18.0,>=2.14.2->docker==4.4.4)
Downloading https://files.pythonhosted.org/packages/5f/64/43575537846896abac0b15c3e5ac678d787a4021e906703f1766bfb8ea11/urllib3-1.26.6-py2.py3-none-any.whl (138kB)
100% |████████████████████████████████| 143kB 4.1MB/s
Collecting idna<3,>=2.5; python_version < "3" (from requests!=2.18.0,>=2.14.2->docker==4.4.4)
Downloading https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl (58kB)
100% |████████████████████████████████| 61kB 732kB/s
Collecting chardet<5,>=3.0.2; python_version < "3" (from requests!=2.18.0,>=2.14.2->docker==4.4.4)
Downloading https://files.pythonhosted.org/packages/19/c7/fa589626997dd07bd87d9269342ccb74b1720384a4d739a1872bd84fbe68/chardet-4.0.0-py2.py3-none-any.whl (178kB)
100% |████████████████████████████████| 184kB 610kB/s
Installing collected packages: certifi, urllib3, idna, chardet, requests, websocket-client, docker
Found existing installation: chardet 2.2.1
Uninstalling chardet-2.2.1:
Successfully uninstalled chardet-2.2.1
Successfully installed certifi-2021.5.30 chardet-4.0.0 docker-4.4.4 idna-2.10 requests-2.26.0 urllib3-1.26.6 websocket-client-1.2.1
docker==4.4.4
withwebsocket-client>=0.32.0
It is OK
websocket-client
version is1.2.1
[root@k8s-node-56-164 ~]# pip show websocket-client
---
Metadata-Version: 2.1
Name: websocket-client
Version: 1.2.1
Summary: WebSocket client for Python with low level API options
Home-page: https://github.com/websocket-client/websocket-client.git
Author: liris
Author-email: [email protected]
Installer: pip
License: Apache-2.0
Location: /usr/lib/python2.7/site-packages
...
Solution:
Reduce the version of websocket client
to 0.32.0
.
[root@k8s-node-56-164 ~]# pip install websocket-client==0.32.0
The verification is successful, and the ansible
call is also successful.
[root@k8s-node-56-164 ~]# python
Python 2.7.5 (default, Nov 16 2020, 22:23:17)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import docker
>>> exit
docker run -itd –privileged –name=apache -v /var/www/html/:/var/www/html/ -p 8888:80 myapache:v1 /usr/sbin/init
Remember, remember
Error content:
The system has not been booted with systemd as init system (PID 1). Can’t operate.
Failed to connect to bus: Host is down
Solution:
docker run -itd — privileged –name myCentos centos /usr/sbin/init
After creation: use the following command to enter the container
docker exec -it myCentos /bin/bash
Pay special attention to the bold content and don’t forget it
The reason is that/bin/Bash is executed in the first step by default, and systemctl cannot be used because of a bug in docker
Therefore, we use/usr/SBIN/init and — privileged, so that we can use systemctl, but override the default/bin/bash
Therefore, if we want to enter the container, we can no longer use docker attach mycentos
Instead, you can only use docker exec -it myCentos /bin/bash Because exec allows us to execute the overridden default command/bin/bash
At the same time -it is also necessary.
Background
The docker
+ nginx
+ uwsgi
+ flask
deployment environment has always been able to run well before. This time, the Python
version of the basic image was upgraded from the original 3.6
to 3.8
, and the title was wrong.
docker
+nginx
+uwsgi
+flask
deployment can refer to this article
Problem analysis
Let’s take a look at the start log of docker:
Starting nginx: nginx.,
*** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 17 02:21:46 2021] ***,
[uWSGI] getting INI configuration from uwsgi.ini,
compiled with version: 8.2.0 on 10 February 2019 02:42:46,
os: Linux-3.10.0-957.el7.x86_64 #1 SMP Thu Nov 8 23:39:32 UTC 2018,
nodename: 9c8cc3ffd4ed,
machine: x86_64,
pcre jit disabled,
detected number of CPU cores: 2,
clock source: unix,
current working directory: /code,
detected binary path: /usr/bin/uwsgi-core,
uWSGI running as root, you can use --uid/--gid/--chroot options,
chdir() to /code,
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** ,
*** WARNING: you are running uWSGI without its master process manager ***,
your memory page size is 4096 bytes,
detected max file descriptor number: 1048576,
lock engine: pthread robust mutexes,
thunder lock: disabled (you can enable it with --thunder-lock),
uwsgi socket 0 bound to TCP address :5000 fd 3,
uWSGI running as root, you can use --uid/--gid/--chroot options,
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** ,
Python version: 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0],
Python main interpreter initialized at 0x55fa4f5a8990,
uWSGI running as root, you can use --uid/--gid/--chroot options,
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** ,
python threads support enabled,
your server socket listen backlog is limited to 100 connections,
your mercy for graceful operations on workers is 60 seconds,
mapped 825016 bytes (805 KB) for 8 cores,
*** Operational MODE: preforking+threaded ***,
Traceback (most recent call last):,
File "run.py", line 20, in <module>,
from server import create_app,
File "./server/__init__.py", line 14, in <module>,
from flask import Flask,
unable to load app 0 (mountpoint='') (callable not found or import error),
*** no app loaded. going in full dynamic mode ***,
uWSGI running as root, you can use --uid/--gid/--chroot options,
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** ,
*** uWSGI is running in multiple interpreter mode ***,
spawned uWSGI worker 1 (pid: 21, cores: 2),
spawned uWSGI worker 2 (pid: 22, cores: 2),
spawned uWSGI worker 3 (pid: 23, cores: 2),
spawned uWSGI worker 4 (pid: 24, cores: 2)
It can be seen from this that an error is reported when quoting flash
.
What causes this problem?
If the program does not find the python Library (flash) when running, it will report an error.
Here, we don’t care why he didn’t find it. Since he didn’t find it, we’ll take the initiative to tell him where to find it.
Solution
Modify uwsgi. Ini
, and set the value of Python path
to /usr/local/lib/Python 3.8/site packages/
.
The content of the modified uwsgi. Ini
file is:
[uwsgi]
chdir = /code
socket = :5000
pythonpath = /usr/local/lib/python3.8/site-packages/
wsgi-file = run.py
callable = app
chmod-socket = 666
plugins = python3
buffer-size = 65535
processes = 4
threads = 2
All right, problem solved.
After using CentOS 7 image to create a container, you may encounter such a problem that you use systemctl to start the service and report an error. For this error report, let’s analyze it next!
# docker run -itd –name centos7 centos:7
# docker attach centos7
# yum install vsftpd
# systemctl start vsftpd
Failed to get D-Bus connection: Operation not permitted
The reasons are as follows:
The design concept of docker is that there is no background service running in the container. The container itself is an independent main process on the host, which can also be indirectly understood as the application process running the service in the container. The life cycle of a container exists around the main process, so the correct way to use the container is to run the services in the foreground.
When it comes to SYSTEMd, this suite has become the default service management for mainstream Linux distributions (such as centos7 and Ubuntu 14 +), replacing the traditional Systemv style service management. SYSTEMd maintains system services, which require privileges to access the Linux kernel. The container is not a complete operating system, there is only one file system, and the default startup is only for ordinary users to access the Linux kernel, that is, there is no privilege, so it can’t be used naturally!
Therefore, please follow the container design principles and run a foreground service in one container!!!
Yes, run the container in privileged mode.
The solution is as follows:
Create container:
# docker run -d -name centos7 –privileged=true centos:7 /usr/sbin/init
Enter container:
# docker exec -it centos7 /bin/bash
This allows you to start the service using systemctl
Docker failed to start the container after installing mysql. Check the MySQL container log and find:
[ERROR] [FATAL] InnoDB: Table flags are 0 in the data dictionary but the flags in file ./ibdata1 are 0x4800!
Query all containers
docker ps -a
Delete container
docker rm CONTAINER ID; #CONTAINER ID is the actual container number
If so, be sure to delete the external mount directory.
Restart mysql. I installed version 5.7 of MySQL
docker run -p 3306:3306 --name mysql \
-v /mydata/mysql/log:/var/log/mysql \
-v /mydata/mysql/data:/var/lib/mysql \
-v /mydata/mysql/conf:/etc/mysql \
-e MYSQL_ROOT_PASSWORD=root \
-d mysql:5.7
Enter the container external mount profile
vim /mydata/mysql/conf/my.conf
Insert the following. Copy the following contents. After entering the file through the previous operation, press keyboard I to enter the insertion state, shift + insert to paste the contents, ESC to exit the insertion mode,: WQ save and exit
[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
init_connect='SET collation_connection = utf8_unicode_ci'
init_connect='SET NAMES utf8'
character-set-server=utf8
collation-server=utf8_unicode_ci
skip-character-set-client-handshake
skip-name-resolve
Restart MySQL
docker restart mysql
Question
The CentOS container is started based on docker. You need to use systemctl start XXX
to start the service in the CentOS container. An error occurred:
Failed to get D-Bus connection: Operation not permitted
reason
The permission of ptrace system call. By default, the container does not have permission to ptrace process.
resolvent
When starting the container, turn on the privileged mode through privileged = true
, and start it with /usr/SBIN/init
as the entry command, for example:
docker run -d -name centos7 --privileged=true centos:7 /usr/sbin/init
After entering the container with the command docker exec - it centos7/bin/bash
, you can start the service internally with systemctl start
.
(END)
1、 Background
In the previous article, the research group needs to encapsulate AFL as a web API with docker in order to make a visual display of AFL on the web side. When the script is redeployed, the AFL boot fails when calling AFL for fuzzing. Because before executing AFL fuzzy
, if the system is configured to send core dump file (core) notification to external programs, the delay between sending crash information to fuzzer will increase, and the crash may be falsely reported as timeout. So you have to modify the core temporarily_ Pattern
file, as follows:
echo core > /proc/sys/kernel/core_pattern
However, an error occurs when executing the above command in the docker container:
bash: /proc/sys/kernel/core_pattern: Read-only file system
Core_ Pattern is a system read-only file and cannot be modified
2、 Doubt
When docker is executed, the root user executes various commands internally. Why can’t you change the files with read-only permission?With this question, baidu found it again and again, but it didn’t find one. Therefore, it checked a machine turned post, which is also ambiguous.
Since Baidu can’t, then Google. After checking for two minutes, I found a blog of compatriots across the Strait and found a solution.
3、 Solution
Direct code:
# Add the --privileged parameter when building the container
docker run -idt -p xx:xx --privileged afl-api:0.0.3
As with the above command, the problem can be solved by adding an additional -- privileged
parameter when building the container.
4、 Cause
About version 0.6, privileged was introduced into docker
with this parameter, the root in the container has real root permissions
otherwise, the root in the container is only an external ordinary user permission
with the container started by privileged, you can see many devices on the host and execute mount
it even allows you to start the docker container in the docker container
That is, the container with the -- privileged
parameter can really execute root permission.
CUDA driver version is insufficient for CUDA runtime version
Question:
An error is reported when docker runs ONEFLOW code of insightface
Failed to get cuda runtime version: CUDA driver version is insufficient for CUDA runtime version
reason:
1. View CUDA runtime version
cat /usr/local/cuda/version.txt
The CUDA version in my docker is 10.0.130
CUDA Version 10.0.130
2. The CUDA version has requirements for the graphics card driver version, see the following link.
https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html
CUDA Toolkit | Linux x86 64 Driver Version | Windows x86 and 64 Driver Version |
---|---|---|
CUDA 11.0.3 Update 1 | ||
CUDA 11.0.2 GA | >= 450.51.05 | >= 451.48 |
CUDA 11.0.1 RC | >= 450.36.06 | >= 451.22 |
CUDA 10.2.89 | >= 440.33 | >= 441.22 |
CUDA 10.1 (10.1.105 general release, and updates) | >= 418.39 | >= 418.96 |
CUDA 10.0.130 | >= 410.48 | >= 411.31 |
CUDA 9.2 (9.2.148 Update 1) | >= 396.37 | >= 398.26 |
CUDA 9.2 (9.2.88) | >= 396.26 | >= 397.44 |
cat /proc/driver/nvidia/version took a look at the server’s graphics card driver is 418.67, CUDA 10.1 should be installed, and I installed 10.0.130 cuda.
NVRM version: NVIDIA UNIX x86_64 Kernel Module 418.67 Sat Apr 6 03:07:24 CDT 2019
GCC version: gcc version 7.3.0 (Ubuntu 7.3.0-27ubuntu1~18.04)
solve:
Installing CUDA 10.1
(1) First in https://developer.nvidia.com/cuda-toolkit-archive According to the machine environment, download the corresponding cuda10.1 installation file. For the installer type, I choose runfile (local). The installation steps will be simpler.
wget https://developer.download.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda_10.1.243_418.87.00_linux.runsudo sh
(2) Installation
sh cuda_10.1.243_418.87.00_linux.run
The same error occurred, unresolved
it will be updated when a solution is found later.
Today, apt failed to update when building the image with dockerfile on raspberry pie
...
---> Running in 05393fa6f242
Get:1 http://ports.ubuntu.com/ubuntu-ports focal InRelease [265 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports focal-updates InRelease [114 kB]
Err:1 http://ports.ubuntu.com/ubuntu-ports focal InRelease
At least one invalid signature was encountered.
Err:2 http://ports.ubuntu.com/ubuntu-ports focal-updates InRelease
At least one invalid signature was encountered.
Get:3 http://ports.ubuntu.com/ubuntu-ports focal-backports InRelease [101 kB]
Err:3 http://ports.ubuntu.com/ubuntu-ports focal-backports InRelease
At least one invalid signature was encountered.
Get:4 http://ports.ubuntu.com/ubuntu-ports focal-security InRelease [114 kB]
Err:4 http://ports.ubuntu.com/ubuntu-ports focal-security InRelease
At least one invalid signature was encountered.
Reading package lists...
W: GPG error: http://ports.ubuntu.com/ubuntu-ports focal InRelease: At least one invalid signature was encountered.
E: The repository 'http://ports.ubuntu.com/ubuntu-ports focal InRelease' is not signed.
W: GPG error: http://ports.ubuntu.com/ubuntu-ports focal-updates InRelease: At least one invalid signature was encountered.
E: The repository 'http://ports.ubuntu.com/ubuntu-ports focal-updates InRelease' is not signed.
W: GPG error: http://ports.ubuntu.com/ubuntu-ports focal-backports InRelease: At least one invalid signature was encountered.
E: The repository 'http://ports.ubuntu.com/ubuntu-ports focal-backports InRelease' is not signed.
W: GPG error: http://ports.ubuntu.com/ubuntu-ports focal-security InRelease: At least one invalid signature was encountered.
E: The repository 'http://ports.ubuntu.com/ubuntu-ports focal-security InRelease' is not signed.
...
The solution is:
in http://ftp.debian.org/debian/pool/main/libs/libseccomp/ Download the latest version of libseccomp2. The current version is libseccomp2_ 2.5.1-1_ Armhf. DEB
install on raspberry pie
>>> sudo dpkg -i libseccomp2_2.5.1-1_armhf.deb
“20999;” 21448;”
[br] https://askubuntu.com/a/1264921/685786
https://stackoverflow.com/a/64463211/7151777
run kubeadm init –config=kubeadm.yml –upload-certs | tee kubeadm-init.log command error: failed to pull image registry.aliyuncs.com/google_containers/coredns:v1.8.0
full error message as below:
kubeadm init --config=kubeadm.yml --upload-certs | tee kubeadm-init.log
[init] Using Kubernetes version: v1.21.2
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Hostname]: hostname "node" could not be reached
[WARNING Hostname]: hostname "node": lookup node on 127.0.0.53:53: server misbehaving
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR ImagePull]: failed to pull image registry.aliyuncs.com/google_containers/coredns:v1.8.0: output: Error response from daemon: manifest for registry.aliyuncs.com/google_containers/coredns:v1.8.0 not found: manifest unknown: manifest unknown
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
Prompt to pull registry.aliyuncs.com/google_ Containers/coredns: v1.8.0 this image failed
use kubedm config images list — config kubedm.yml to query the image to be downloaded
kubeadm config images list --config kubeadm.yml
registry.aliyuncs.com/google_containers/kube-apiserver:v1.21.2
registry.aliyuncs.com/google_containers/kube-controller-manager:v1.21.2
registry.aliyuncs.com/google_containers/kube-scheduler:v1.21.2
registry.aliyuncs.com/google_containers/kube-proxy:v1.21.2
registry.aliyuncs.com/google_containers/pause:3.4.1
registry.aliyuncs.com/google_containers/etcd:3.4.13-0
registry.aliyuncs.com/google_containers/coredns:v1.8.0
Use the docker images command to query images
docker images
registry.aliyuncs.com/google_containers/kube-apiserver v1.21.2 106ff58d4308 3 weeks ago 126MB
registry.aliyuncs.com/google_containers/kube-controller-manager v1.21.2 ae24db9aa2cc 3 weeks ago 120MB
registry.aliyuncs.com/google_containers/kube-scheduler v1.21.2 f917b8c8f55b 3 weeks ago 50.6MB
registry.aliyuncs.com/google_containers/kube-proxy v1.21.2 a6ebd1c1ad98 3 weeks ago 131MB
registry.aliyuncs.com/google_containers/pause 3.4.1 0f8457a4c2ec 6 months ago 683kB
registry.aliyuncs.com/google_containers/etcd 3.4.13-0 0369cf4303ff 10 months ago 253MB
It is found that there is no registry.aliyuncs.com/google in the downloaded image_ Containers/coredns: v1.8.0
use the docker command to pull the image
docker pull registry.aliyuncs.com/google_containers/coredns:1.8.0
Kubernetes needs registry.aliyuncs.com/google_ Containers/coredns: v1.8.0, rename the image with the docker tag command
# rename
docker tag registry.aliyuncs.com/google_containers/coredns:1.8.0 registry.aliyuncs.com/google_containers/coredns:v1.8.0
# detele all mirror
docker rmi registry.aliyuncs.com/google_containers/coredns:1.8.0
Run the initialization command again
kubeadm init --config=kubeadm.yml --upload-certs | tee kubeadm-init.log
Prompt success
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Follow the above prompts to configure kubectl
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
# NOT ROOT
chown $(id -u):$(id -g) $HOME/.kube/config
Verify success
kubectl get node
# The ability to print out node information indicates success
NAME STATUS ROLES AGE VERSION
node NotReady control-plane,master 31m v1.21.2