Scene won't render in GPU even when memory used does not go over the amount of graphics memory

dakkuuan

Oddly I can't render this scene in GPU. There is no HDRI image. It seems I run out of video memory but the logs aren't indicating I'm even getting close to the 8GB I've got on my cards. (it's 8GB per card and I know they load the same stuff into memory). It's becoming clear if I ever want to do anything beyond simple one character renders I'm going to have to buy a 3090. Is there somenthing wrong or is iray just not reporting all the memory it's using?

2021-04-06 13:55:27.274 Iray [INFO] - IRAY:RENDER :: 1.13 IRAY rend progr: CUDA device 0 (NVIDIA GeForce GTX 1080): Processing scene...

2021-04-06 13:55:27.274 Iray [INFO] - IRAY:RENDER :: 1.4 IRAY rend progr: CUDA device 1 (NVIDIA GeForce GTX 1080): Processing scene...

2021-04-06 13:55:27.598 Iray [VERBOSE] - IRAY:RENDER :: 1.2 IRAY rend stat : Geometry memory consumption: 1.040 GiB (device 0), 0.000 B (host)

2021-04-06 13:55:27.602 Iray [VERBOSE] - IRAY:RENDER :: 1.19 IRAY rend stat : Geometry memory consumption: 1.040 GiB (device 1), 0.000 B (host)

2021-04-06 13:56:09.386 Iray [VERBOSE] - IRAY:RENDER :: 1.19 IRAY rend stat : Texture memory consumption: 3.741 GiB for 225 bitmaps (device 1)

2021-04-06 13:56:09.386 Iray [VERBOSE] - IRAY:RENDER :: 1.2 IRAY rend stat : Texture memory consumption: 3.741 GiB for 225 bitmaps (device 0)

2021-04-06 13:56:09.389 Iray [INFO] - IRAY:RENDER :: 1.19 IRAY rend info : Importing lights for motion time 0

2021-04-06 13:56:09.394 Iray [INFO] - IRAY:RENDER :: 1.19 IRAY rend info : Initializing light hierarchy.

2021-04-06 13:56:09.398 Iray [INFO] - IRAY:RENDER :: 1.19 IRAY rend info : Light hierarchy initialization took 0.003s

2021-04-06 13:56:09.399 Iray [VERBOSE] - IRAY:RENDER :: 1.19 IRAY rend stat : Lights memory consumption: 196.000 B (device 1)

2021-04-06 13:56:09.400 Iray [VERBOSE] - IRAY:RENDER :: 1.2 IRAY rend stat : Lights memory consumption: 196.000 B (device 0)

2021-04-06 13:56:09.407 Iray [VERBOSE] - IRAY:RENDER :: 1.19 IRAY rend stat : Material measurement memory consumption: 0.000 B (GPU)

2021-04-06 13:56:09.409 Iray [VERBOSE] - IRAY:RENDER :: 1.2 IRAY rend stat : Materials memory consumption: 875.660 KiB (GPU)

2021-04-06 13:56:09.558 Iray [INFO] - IRAY:RENDER :: 1.4 IRAY rend info : CUDA device 1 (NVIDIA GeForce GTX 1080): Scene processed in 42.282s

2021-04-06 13:56:09.563 Iray [INFO] - IRAY:RENDER :: 1.13 IRAY rend info : CUDA device 0 (NVIDIA GeForce GTX 1080): Scene processed in 42.289s

2021-04-06 13:56:09.569 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER :: 1.4 IRAY rend error: CUDA device 1 (NVIDIA GeForce GTX 1080): Not enough memory for kernel launches (372.828 MiB (380.953 MiB) required, 0.000 B available). Cannot allocate framebuffer.

2021-04-06 13:56:09.571 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER :: 1.13 IRAY rend error: CUDA device 0 (NVIDIA GeForce GTX 1080): Not enough memory for kernel launches (372.828 MiB (380.953 MiB) required, 0.000 B available). Cannot allocate framebuffer.

2021-04-06 13:56:09.572 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER :: 1.4 IRAY rend error: CUDA device 1 (NVIDIA GeForce GTX 1080): Failed to setup device frame buffer

2021-04-06 13:56:09.572 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER :: 1.13 IRAY rend error: CUDA device 0 (NVIDIA GeForce GTX 1080): Failed to setup device frame buffer

2021-04-06 13:56:09.572 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER :: 1.4 IRAY rend error: CUDA device 1 (NVIDIA GeForce GTX 1080): Device failed while rendering

2021-04-06 13:56:09.572 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER :: 1.13 IRAY rend error: CUDA device 0 (NVIDIA GeForce GTX 1080): Device failed while rendering

2021-04-06 13:56:09.573 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [WARNING] - IRAY:RENDER :: 1.4 IRAY rend warn : CUDA device 1 (NVIDIA GeForce GTX 1080) ran out of memory and is temporarily unavailable for rendering.

2021-04-06 13:56:09.573 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [WARNING] - IRAY:RENDER :: 1.13 IRAY rend warn : CUDA device 0 (NVIDIA GeForce GTX 1080) ran out of memory and is temporarily unavailable for rendering.

2021-04-06 13:56:09.574 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [WARNING] - IRAY:RENDER :: 1.13 IRAY rend warn : All available GPUs failed.

2021-04-06 13:56:09.575 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER :: 1.13 IRAY rend error: Fallback to CPU not allowed.

2021-04-06 13:56:09.576 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(359): Iray [ERROR] - IRAY:RENDER :: 1.13 IRAY rend error: All workers failed: aborting render

2021-04-06 13:56:09.579 Iray Render error: Internal rendering error.

2021-04-06 13:56:09.656 Saved image: C:\Users\dakku\AppData\Roaming\DAZ 3D\Studio4\temp\render\r.png

2021-04-06 13:56:09.667 Finished Rendering

2021-04-06 13:56:09.926 Total Rendering Time: 1 minutes 12.4 seconds

dakkuuan

Well I guess this is starting to make sense... it seems it's sitting with 1.5GB allocated without doing anything. 

margrave

For whatever reason, Iray just doesn't use all the VRAM that's available. There's some hidden overhead we're not privy to that can cut your max VRAM by 2/3rds.

PerttiA

margrave said:

For whatever reason, Iray just doesn't use all the VRAM that's available. There's some hidden overhead we're not privy to that can cut your max VRAM by 2/3rds.

It does use all the available VRAM, this test was made on W7/RTX 2070 Super - W10 uses 800MB's more base load, so an 8GB GPU on W10 should be able to render a scene that is reported to use 4.9GiB's of memory (total with the Frame Buffer and Work Space, which take 1-1.8GiB)

fubarthebuff

Without more context, the only thing I could tell you is your CUDA drivers might be buggered up. That's the issue I'm running into with the same gpu, I can't render on my 1080 at all, but for whatever reason can't get CUDA drivers to install.

dakkuuan

I found the real memory hog is the hair the goth girl has. But really, I can render an empty render and it eat up over 2GB of video ram (as per an external tool). I'm not sure it's always done this. 

PerttiA

dakkuuan said:

I found the real memory hog is the hair the goth girl has. But really, I can render an empty render and it eat up over 2GB of video ram (as per an external tool). I'm not sure it's always done this. 

The 2GB used in rendering empty scene sounds even low, as W10 reserves 1GB, DS base load would be around 250MB, workspace while rendering 1 - 1.7GB and since you are using non-RTX GPU, some more is needed for emulating the RTX functions in software.

duckdoom5

I have the exact same problem, but for me this only happens the second time I try to render.

The only way for me to render my scene is to sign out of my windows account and sign back in (or restart the pc), resetting the GPU in some way. It appears to be something CUDA kernel related rather that VRAM

onyxlee_9b3cce7f35

I can only recommend using some script to lower the texture resolution to 1/2 or even 1/4. Unfortunately, these things haven't been taken care of in an automatic way, such as only lowering the texture size on the objects that are far away.

If you don't manually manage these things, you will run out of vram sooner or later.

Btw, how do I get a notification when iray is not using GPU? So far my only solution is to look at the GPU cuda percentage, which is not fun.