# Tests not working **URL:** \ **Category:** Applications\ **Created:** [November 6, 2020, 4:18pm UTC](https://gpusph.discourse.group/t/tests-not-working/125 "2020-11-06T16:18:49Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Manuel](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@Manuel](https://gpusph.discourse.group/u/Manuel)\ **Post date:** [November 6, 2020, 4:18pm UTC](https://gpusph.discourse.group/t/tests-not-working/125/1 "2020-11-06T16:18:49Z")
Hello, I have just installed GPUSPH and tried to run some tests of the src/problems folder. However I get warnings and the results at the end of the simulations are wrong. For example, regarding the DamBreak3D problem this is what I get: Simulation time t=4.523146e-02s, iteration=186, dt=2.659091e-04s, 84,444 parts (15, cum. 14 MIPPS), maxneibs 122+0 Simulation time t=5.006359e-02s, iteration=208, dt=2.017437e-04s, 84,444 parts (15, cum. 14 MIPPS), maxneibs 124+0 Simulation time t=5.512060e-02s, iteration=231, dt=2.659091e-04s, 84,444 parts (15, cum. 14 MIPPS), maxneibs 124+0 Simulation time t=6.015487e-02s, iteration=251, dt=2.424981e-04s, 84,444 parts (14, cum. 14 MIPPS), maxneibs 124+0 Simulation time t=6.505197e-02s, iteration=271, dt=2.659091e-04s, 84,444 parts (13, cum. 14 MIPPS), maxneibs 125+0 WARNING: at iteration 280 the number of particles changed from 84444 to 84443 for no known reason! WARNING: at iteration 280, time 0.0671853 particle ID 0 is at indices 0 and 1! WARNING: at iteration 280, time 0.0671853 particle ID 1 was not found! Recap of devices after roll call: - device at index 0 has 84,443 particles assigned and offset 0 Simulation time t=7.002765e-02s, iteration=293, dt=2.659091e-04s, 84,443 parts (15, cum. 14 MIPPS), maxneibs 125+0 WARNING: at iteration 310 the number of particles changed from 84443 to 84442 for no known reason! WARNING: at iteration 310, time 0.0737294 particle ID 67044 was not found! Recap of devices after roll call: - device at index 0 has 84,442 particles assigned and offset 0 Simulation time t=7.523180e-02s, iteration=317, dt=1.200993e-04s, 84,442 parts (16, cum. 14 MIPPS), maxneibs 125+0 WARNING: at iteration 330 the number of particles changed from 84442 to 84439 for no known reason! Simulation time t=8.005969e-02s, iteration=345, dt=1.771645e-04s, 84,439 parts (16, cum. 14 MIPPS), maxneibs 125+0 WARNING: at iteration 350 the number of particles changed from 84439 to 84434 for no known reason! WARNING: at iteration 1930 the number of particles changed from 71562 to 71512 for no known reason! WARNING: current max. neighbors numbers (132 | 0) greater than max possible neibs (127 | 0) at iteration 1930 possible culprit: 1216 (neibs: 132 + 0 | 0) WARNING: at iteration 1940 the number of particles changed from 71512 to 71487 for no known reason! WARNING: current max. neighbors numbers (131 | 0) greater than max possible neibs (127 | 0) at iteration 1940 possible culprit: 960 (neibs: 131 + 0 | 0) WARNING: particle 0 (id 0, type 0) has NAN position! (nan, nan, nan) @ (0, 0, 0) = (nan, nan, nan) at iteration 1941, time 0.330071 Simulation time t=3.300714e-01s, iteration=1,941, dt=2.085889e-04s, 71,487 parts (20, cum. 20 MIPPS), maxneibs 180+0 WARNING: at iteration 1950 the number of particles changed from 71487 to 71441 for no known reason! WARNING: particle 0 (id 0, type 0) has NAN position! (nan, nan, nan) @ (0, 0, 0) = (nan, nan, nan) at iteration 1950, time 0.331993 WARNING: at iteration 1960 the number of particles changed from 71441 to 71387 for no known reason! WARNING: particle 0 (id 0, type 0) has NAN position! (nan, nan, nan) @ (0, 0, 0) = (nan, nan, nan) at iteration 1967, time 0.335032 Simulation time t=3.350320e-01s, iteration=1,967, dt=2.200599e-04s, 71,387 parts (22, cum. 20 MIPPS), maxneibs 180+0 WARNING: at iteration 1970 the number of particles changed from 71387 to 71367 for no known reason! WARNING: particle 0 (id 0, type 0) has NAN position! (nan, nan, nan) @ (0, 0, 0) = (nan, nan, nan) at iteration 1970, time 0.335692 WARNING: at iteration 1980 the number of particles changed from 71367 to 71351 for no known reason! WARNING: particle 0 (id 0, type 0) has NAN position! (nan, nan, nan) @ (0, 0, 0) = (nan, nan, nan) at iteration 1990, time 0.340093 Simulation time t=3.400934e-01s, iteration=1,990, dt=2.200599e-04s, 71,351 parts (20, cum. 20 MIPPS), maxneibs 180+0 WARNING: at iteration 1990 the number of particles changed from 71351 to 71332 for no known reason! WARNING: particle 0 (id 0, type 0) has NAN position! (nan, nan, nan) @ (0, 0, 0) = (nan, nan, nan) at iteration 1990, time 0.340093 WARNING: at iteration 2000 the number of particles changed from 71332 to 71313 for no known reason! WARNING: at iteration 2010 the number of particles changed from 71313 to 71312 for no known reason! WARNING: particle 0 (id 0, type 0) has NAN position! (nan, nan, nan) @ (0, 0, 0) = (nan, nan, nan) at iteration 2014, time 0.345084 Simulation time t=3.450835e-01s, iteration=2,014, dt=2.200599e-04s, 71,312 parts (21, cum. 20 MIPPS), maxneibs 180+0 WARNING: at iteration 2020 the number of particles changed from 71312 to 71273 for no known reason! WARNING: particle 0 (id 0, type 0) has NAN position! (nan, nan, nan) @ (0, 0, 0) = (nan, nan, nan) at iteration 2020, time 0.346147 WARNING: at iteration 2030 the number of particles changed from 71273 to 71202 for no known reason! WARNING: particle 0 (id 0, type 0) has NAN position! (nan, nan, nan) @ (0, 0, 0) = (nan, nan, nan) at iteration 2038, time 0.350108 Simulation time t=3.501078e-01s, iteration=2,038, dt=2.200599e-04s, 71,202 parts (20, cum. 20 MIPPS), maxneibs 180+0 Basically, the particles slowly flow out of the domain. Do you have any idea why this happens? Could it be due to my GPU/CUDA which are not properly working? Here the GPU/CUDA details: CUDA Device Query (Driver API) statically linked version Detected 1 CUDA Capable device(s) Device 0: “Tesla V100-PCIE-32GB” CUDA Driver Version: 11.1 CUDA Capability Major/Minor version number: 7.0 Total amount of global memory: 32510 MBytes (34089730048 bytes) (80) Multiprocessors, ( 64) CUDA Cores/MP: 5120 CUDA Cores GPU Max Clock rate: 1380 MHz (1.38 GHz) Memory Clock rate: 877 Mhz Memory Bus Width: 4096-bit L2 Cache Size: 6291456 bytes Max Texture Dimension Sizes 1D=(131072) 2D=(131072, 65536) 3D=(16384, 16384, 16384) Maximum Layered 1D Texture Size, (num) layers 1D=(32768), 2048 layers Maximum Layered 2D Texture Size, (num) layers 2D=(32768, 32768), 2048 layers Total amount of constant memory: 65536 bytes Total amount of shared memory per block: 49152 bytes Total number of registers available per block: 65536 Warp size: 32 Maximum number of threads per multiprocessor: 2048 Maximum number of threads per block: 1024 Max dimension size of a thread block (x,y,z): (1024, 1024, 64) Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535) Texture alignment: 512 bytes Maximum memory pitch: 2147483647 bytes Concurrent copy and kernel execution: Yes with 7 copy engine(s) Run time limit on kernels: No Integrated GPU sharing Host Memory: No Support host page-locked memory mapping: Yes Concurrent kernel execution: Yes Alignment requirement for Surfaces: Yes Device has ECC support: Enabled Device supports Unified Addressing (UVA): Yes Device supports Managed Memory: Yes Device supports Compute Preemption: Yes Supports Cooperative Kernel Launch: Yes Supports MultiDevice Co-op Kernel Launch: Yes Device PCI Domain ID / Bus ID / location ID: 0 / 59 / 0 Compute Mode: \< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) \> Result = PASS Thanks for the help. Regards Manuel ---
**Author:** ![saikali](https://avatars.discourse-cdn.com/v4/letter/s/ecae2f/32.png) [@saikali](https://gpusph.discourse.group/u/saikali)\ **Post date:** [November 7, 2020, 12:22pm UTC](https://gpusph.discourse.group/t/tests-not-working/125/2 "2020-11-07T12:22:18Z")
Hello Manuel, Have you changed anything in the file DamBreakD.cu file ? Regards Elie ---
**Author:** ![Manuel](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@Manuel](https://gpusph.discourse.group/u/Manuel)\ **Post date:** [November 8, 2020, 11:36am UTC](https://gpusph.discourse.group/t/tests-not-working/125/3 "2020-11-08T11:36:02Z")
Hi Elie, thanks for your reply. No, I haven’t changed anything in the DamBreak3D.cu file. I installed first the CUDA Toolkit 11.1, then Project Chrono and finally GPUSPH. I executed “make test” and nothing else. I tried to run some other tests as well, but they all gave me similar errors. I am using Ubuntu 18.04.5 LTS. The CUDA 10.1 version was already installed on the machine. Now I have both (CUDA 11.1 and 10.1). Could it be that they are somehow conflicting with each other? Before compiling GPUSPH, in the Makefile I changed the cuda path to the 11.1 installation folder. Thank you for your help. Regards Manuel ---
**Author:** ![Manuel](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@Manuel](https://gpusph.discourse.group/u/Manuel)\ **Post date:** [November 9, 2020, 8:55am UTC](https://gpusph.discourse.group/t/tests-not-working/125/4 "2020-11-09T08:55:51Z")
I forgot to mention that I am trying to use GPUSPH on a remote server. Furthermore, going back to the CUDA 11.1 installation, I noticed that some of the samples do not run, e.g. nbody, Mandelbrot, etc… I get this type of error: CUDA error at bodysystemcuda\_impl.h:186 code=999(cudaErrorUnknown) “cudaGraphicsGLRegisterBuffer(&m\_pGRes[i], m\_pbo[i], cudaGraphicsMapFlagsNone)” I found this thread on the nvidia forum: > **[After cuda9.2 installed, got error when run nobody sample](https://forums.developer.nvidia.com/t/after-cuda9-2-installed-got-error-when-run-nobody-sample/61495/5)** > > @txbob thanks for the hint! You set me on the right track. This took the better part of an afternoon to solve and I’ll document here in case it helps the others. I’m not on a laptop nor do I have multiple GPUs. But this was the initial result from... and I am currently trying to solve this issue. However, could this be the reason why GPUSPH does not run properly? Thanks for the help. Regards Manuel ---
**Author:** ![Manuel](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@Manuel](https://gpusph.discourse.group/u/Manuel)\ **Post date:** [November 9, 2020, 9:10am UTC](https://gpusph.discourse.group/t/tests-not-working/125/5 "2020-11-09T09:10:43Z")
Another point: when I compile whatever problem, I get this warning messages: /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/detail/config/cpp\_dialect.h:104:13: warning: Thrust requires C++14. Please pass -std=c++14 to your compiler. Define THRUST\_IGNORE\_DEPRECATED\_CPP\_DIALECT to suppress this message. THRUST\_COMPILER\_DEPRECATION(C++14, pass -std=c++14 to your compiler); ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/cub/util\_arch.cuh:36:0, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/system/cuda/detail/util.h:32, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/system/cuda/detail/for\_each.h:34, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/system/detail/adl/for\_each.h:42, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/detail/for\_each.inl:27, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/for\_each.h:279, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/system/detail/generic/transform.inl:19, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/system/detail/generic/transform.h:105, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/detail/transform.inl:25, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/transform.h:724, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/system/detail/generic/reduce\_by\_key.inl:29, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/system/detail/generic/reduce\_by\_key.h:88, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/detail/reduce.inl:26, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/reduce.h:784, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/system/detail/generic/find.inl:19, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/system/detail/generic/find.h:62, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/detail/find.inl:25, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/find.h:384, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/system/detail/generic/sort.inl:26, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/system/detail/generic/sort.h:153, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/detail/sort.inl:26, from /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/thrust/sort.h:1361, from src/cuda/buildneibs.cu:36, from src/cuda/cudasimframework.cu:46, from src/problems/StillWater.cu:32: /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/cub/util\_cpp\_dialect.cuh:129:13: warning: CUB requires C++14. Please pass -std=c++14 to your compiler. Define CUB\_IGNORE\_DEPRECATED\_CPP\_DIALECT to suppress this message. CUB\_COMPILER\_DEPRECATION(C++14, pass -std=c++14 to your compiler); ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/cuda\_runtime.h: In instantiation of ‘cudaError\_t cudaBindTexture(size\_t\*, const texture\&, const void\*, size\_t) [with T = float4; int dim = 1; cudaTextureReadMode readMode = (cudaTextureReadMode)0; cudaError\_t = cudaError; size\_t = long unsigned int]’: src/cuda/forces.cu:479:31: required from ‘void CUDAForcesEngine\::bind\_textures(const BufferList&, uint, RunMode) [with KernelType kerneltype = (KernelType)3; SPHFormulation sph\_formulation = (SPHFormulation)1; DensityDiffusionType densitydiffusiontype = (DensityDiffusionType)3; ViscSpec = FullViscSpec\<(RheologyType)1, (TurbulenceModel)0, (ComputationalViscosityType)0, (ViscousModel)0, (AverageOperator)0, 1, true\>; BoundaryType boundarytype = (BoundaryType)3; long unsigned int simflags = 1; uint = unsigned int]’ /tmp/tmpxft\_0000534f\_00000000-6\_StillWater.cudafe1.stub.c:34:27: required from here /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/cuda\_runtime.h:1346:23: warning: ‘cudaError\_t cudaBindTexture(size\_t\*, const texture\&, const void\*, const cudaChannelFormatDesc&, size\_t) [with T = float4; int dim = 1; cudaTextureReadMode readMode = (cudaTextureReadMode)0; cudaError\_t = cudaError; size\_t = long unsigned int]’ is deprecated [-Wdeprecated-declarations] return cudaBindTexture(offset, tex, devPtr, tex.channelDesc, size); ~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /usr/local/cuda-11.1/bin/…/targets/x86\_64-linux/include/cuda\_runtime.h:1293:53: note: declared here static \_\_CUDA\_DEPRECATED **inline** **host** cudaError\_t cudaBindTexture( Thanks. Regards Manuel ---
**Author:** ![giuseppe.bilotta](https://yyz2.discourse-cdn.com/free1/user_avatar/gpusph.discourse.group/giuseppe.bilotta/32/72_2.png) [@giuseppe.bilotta](https://gpusph.discourse.group/u/giuseppe.bilotta)\ **Post date:** [November 10, 2020, 6:47pm UTC](https://gpusph.discourse.group/t/tests-not-working/125/6 "2020-11-10T18:47:44Z")
Hello @Manuel, thanks for the bug report. Your inability to run the CUDA samples is a strong indicator that there are issues with your CUDA installation, and that should take priority in being fixed on your side. Make sure in particular that your drivers are up to date and that you are using a single version of CUDA consistently (issues may arise if you compile and link against different versions of CUDA). That being said, I have received other reports of incompatibilities between GPU and CUDA 11.1, independently of the hardware. The issue however generally arise as unjustified illegal device memory accesses crashing most test-cases, and not the issues you are seeing. I’m currently in the process of investigating the problem, and adapting the code to the new requirements of CUDA 11. Some of the simplest fixes for this have been pushed to the `next` branch already (e.g. using C++14), but some others (such as the transition from traditional compile-time, static, bound textures to bindless textures) require more significant changes to the code and have not been released yet. For the time being, I would strongly recommend using the GPUSPH `next` branch (if you are not using that already), and if possible downgrade CUDA to CUDA 10.x (your GPU is a V100, which is supported by CUDA 10). Cheers, Giuseppe Bilotta ---
**Author:** ![Manuel](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@Manuel](https://gpusph.discourse.group/u/Manuel)\ **Post date:** [November 13, 2020, 9:33am UTC](https://gpusph.discourse.group/t/tests-not-working/125/7 "2020-11-13T09:33:43Z")
Hello Giuseppe, thank you for your reply and help. As you suggested, I downgraded CUDA to 10.2 and switched to the “next” branch and now everything works. Thank you very much. Regards Manuel ---
**Author:** ![giuseppe.bilotta](https://yyz2.discourse-cdn.com/free1/user_avatar/gpusph.discourse.group/giuseppe.bilotta/32/72_2.png) [@giuseppe.bilotta](https://gpusph.discourse.group/u/giuseppe.bilotta)\ **Post date:** [November 14, 2020, 12:32pm UTC](https://gpusph.discourse.group/t/tests-not-working/125/8 "2020-11-14T12:32:36Z")
Hello @Manuel, thanks for the update. I’m glad that CUDA 10.2 is working. I’m still in the process of investigating (and hopefully fixing) the issues with CUDA 11. I’ll make an announcement when I have some positive news. Cheers, Giuseppe Bilotta