Tag Archives: course

[Solved] mmdet install error: ERROR: Could not build wheels for pycocotools 

x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/local/lib/python3.8/dist-packages/numpy/core/include -I./common -I/usr/include/python3.8 -c pycocotools/_mask.c -o build/temp.linux-x86_64-3.8/pycocotools/_mask.o -Wno-cpp -Wno-unused-function -std=c99
      pycocotools/_mask.c:4:10: fatal error: Python.h: No such file or directory
       #include "Python.h"
                ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pycocotools
Failed to build pycocotools
ERROR: Could not build wheels for pycocotools, which is required to install pyproject.toml-based projects

The python version I use is 3.8 (replaced according to your version).

Solution:

Execute sudo apt install python3.8-dev

Ubuntu Run Error: not syncing : VFS: Unable to mount root

Problem Description:

Previously, Ubuntu needed to be updated, but it was considered to be interrupted in the middle of the update, or it was easy to open and crash later, which was described in the title

When Ubuntu starts, not syncing: VFS: unable to mount root appears

The solution is as follows:

1. After boot, do not directly enter the boot option of Ubuntu, select advanced mode,

2. Then select recovery mode,

3. Select resume resume normal boot

4. After entering the system, open the terminal and input

sudo apt-get autoremove –purge

After running, update grub

sudo update-grub

Done!

To solve the problem of increasing video memory when training network (torch)

Method 1

torch.backends.cudnn.enabled = True
torch.backends.cudnn.benchmark = True

Principle:

Cundnn follows the following criteria:

    if the dimension or type of network input data changes little, set    torch.backends.cudnn.benchmark  =  true   It can increase the operation efficiency; If the input data of the network changes every iteration, cndnn will find the optimal configuration every time, which will improve the operation efficiency

     
    Method 2

    Tensor calculation should be written as follows:

    train_loss += loss.item()

Python parallel processing makes full use of CPU to realize acceleration

Recently, Python is used to process the public image database. Due to the large amount of data, it takes too long to process the images one by one in serial. Therefore, it is decided to adopt the parallel mode to make full use of the CPU on the host to speed up the processing process, which can greatly reduce the total processing time.

Here we use the concurrent.futures Module, which can use multiprocessing to achieve real parallel computing.

The core principle is: concurrent.futures It will run multiple Python interpreters in parallel in the form of sub processes, so that Python programs can use multi-core CPU to improve the execution speed. Since the subprocess is separated from the main interpreter, their global interpreter locks are also independent of each other. Each subprocess can use a CPU core completely.

The specific implementation is also very simple, the code is as follows. How many CPU cores the host has will start how many Python processes for parallel processing.

import concurrent.futures


def function(files):
    # To do what you want
    # files: file list that you want to process


if __name__ == '__main__':
    with concurrent.futures.ProcessPoolExecutor() as executor:
        executor.map(function, files)

After changing to parallel processing, my 12 CPUs run at full load, and the processing speed is significantly faster.