Training network error reporting: RuntimeError: cuda runtime error (710) : Device – side assert triggered the at/pytorch aten/SRC/THC/generic/THCTensorScatterGather cu: 380
the terminate called after throwing an instance of ‘c10: : Error’
I () : CUDA Error: device-side assert triggered (insert_events at /pytorch/c10/cuda/CUDACachingAllocator.cpp:569)
Reason: The label is out of line
Method: Input
the terminate called after throwing an instance of ‘c10: : Error’
I () : CUDA Error: device-side assert triggered (insert_events at /pytorch/c10/cuda/CUDACachingAllocator.cpp:569)
Reason: The label is out of line
Method: Input
CUDA_LAUNCH_BLOCKING=1 python train.py
An error generates specific information
/pytorch/aten/src/THC/THCTensorScatterGather.cu:188: void THCudaTensor_scatterFillKernel(TensorInfo<Real, IndexType>, TensorInfo<long, IndexType>, Real, int, IndexType) [with IndexType = unsigned int, Real = float, Dims = -1]: block: [72,0,0], thread: [32,0,0] Assertion `indexValue >= 0 && indexValue < tensor.sizes[dim]` failed.
Can be seen to be Assertion ‘indexValue & GT; = 0 & & indexValue < The predicate error is tensor. Sizes [dim], which means the label is more than zero or more than the total number and crosses the line. After debugging, I found that there was a setting that was larger than the preset total number of categories when the category was labeled. I modified this label, and the problem was solved.
Read More:
- [DL Common Issue] RuntimeError: CUDA error 59: Device-side assert triggered
- [Solved] RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors
- [Solved] RuntimeError: cuda runtime error: device-side assert trigger
- [Solved] RuntimeError: cuda runtime error (100) : no CUDA-capable device is detected at
- [Solved] bushi RuntimeError: version_ <= kMaxSupportedFileFormatVersion INTERNAL ASSERT FAILED at /pytorch/caffe2/s
- [Solved] pytorch Error: KeyError: tensor(2, device=‘cuda:0‘)
- Pytorch torch.cuda.FloatTensor Error: RuntimeError: one of the variables needed for gradient computation has…
- RTX 3090 Run pytorch Error: CUDA error: no kernel image is available for execution on the device
- Pytorch CUDA Error: UserWarning: CUDA initialization: CUDA unknown error…
- How to Solve Error: RuntimeError CUDA out of memory
- RuntimeError: CUDA error: an illegal memory access was encountered
- [How to Solve] RuntimeError: CUDA out of memory.
- [Solved] RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasSgemm
- [Pytorch Error Solution] Pytorch distributed RuntimeError: Address already in use
- [Solved] Pytorch error: RuntimeError: one of the variables needed for gradient computation
- [Solved] RuntimeError: cuda runtime error (801) : operation not supported at
- [Solved] PyTorch Caught RuntimeError in DataLoader worker process 0和invalid argument 0: Sizes of tensors mus
- [Solved] Pytorch Error: RuntimeError: expected scalar type Double but found Float
- [Solved] pytorch loss.backward() Error: RuntimeError: Function AddBackward0 returned an invalid gradient at index 1…
- [Solved] RuntimeError : PyTorch was compiled without NumPy support