numpy.core._exceptions.MemoryError: Unable to allocate xxx GB

This error was reported during the training of 15W pieces of data recently, but I only used 60% of my local memory. After solving it, record it.

After checking, it should be the problem of Python bits. At first, my local is 32 bits.  

If your Python is 32-bit, your pandas and numpy can only be 32-bit. When your memory usage exceeds 2G, the memory will be automatically terminated

Later, it was changed to 64 bit, and there was no error

Read More: