Daz 4.12 Only Using GPU Once - Restart Required for Second Use
I've been racking my brain, making no progress on my redners, due to a strange (let's call it a bug for now) issue. I've read numerous threads, most ending in the solution of updating the graphics driver. I am presently using the Nvidia Studio driver 441.66, I'm on Windows 10, and have Daz Studio Pro 4.12. My advanced render settings are set to use the GPU only, CPU is unchecked, and I have Optix Prime Acceleration checked. According to everything I've read online, I should have no issue. But I do.
Through hair shedding agony, I've run a number of tests trying to figure out why my GPU only configuration is being ignored. I reduced the resolution for to 1080, to 720, to 480... still uses CPU. Things got interesting after I rebooted my computer and noticed Daz started using my GPU again. Yay! My celebration was short lived once I rendered a second time... it went back to my CPU. I restarted my computer again, same result... GPU first render, CPU second. Then I restarted just the application... GPU first render, CPU second. So, I seem to have isolated an issue with Daz Studio 4.12. I see a whole lot of people posting about this online and have to wonder if their madness was not entirely their fault, if this is in fact a bug.
Here are my machine spec:
OS: Windows 10 Pro
CPU: 7700k
RAM: 32GB DDR4
GPU: RTX 2070 (Studio driver 441.66)
Daz Studio Pro: 4.12.0.86
Frustration: 100%
If anyone has any ideas please let me know, because obviously restarting Daz after every render is insane, If I'm an idiot and missing something obvious, I don't mind. I just want to get back to work.
Thanks in advance
Comments
"GPU only" will be ignored if there's not enough VRAM to hold your scene. How much VRAM on your card, and what's the memory footprint of your scene?
OptiX Prime is on or off by card type, the check box in Render Settings is ignored (and has been removed in the current Public Build). It should be off for the 2070, since that is an RTX device. Are you using the Iray denoiser?
My card has 8GB of DDR6. Is there a native way to determine the scene's memory footprint or do I need a 3rd party app? I have to assume it's within limits, since it does use the GPU on first run and with no changes, switches to CPU when I render the same scene a second time. It's almost as if Daz Studio isn't releasing the VRAM after the render. Here is the first a second run output:
First render:
Rendering in NVIDIA Iray
Compiling Shaders - 0/1
Rendering image
Rendering...
Iray [VERBOSE] - IRAY:RENDER :: 1.7 IRAY rend progr: CUDA device 0 (GeForce RTX 2070): Processing scene...
Iray [VERBOSE] - IRAY:RENDER :: 1.8 IRAY rend stat : Geometry memory consumption: 1.1246 GiB (device 0), 0 B (host)
Iray [VERBOSE] - IRAY:RENDER :: 1.8 IRAY rend stat : Texture memory consumption: 735.387 MiB for 78 bitmaps (device 0)
Iray [VERBOSE] - IRAY:RENDER :: 1.8 IRAY rend stat : Lights memory consumption: 22.4537 MiB (device 0)
Iray [VERBOSE] - IRAY:RENDER :: 1.8 IRAY rend stat : Material measurement memory consumption: 0 B (GPU)
Iray [VERBOSE] - IRAY:RENDER :: 1.8 IRAY rend stat : PTX code (160 KiB) for SM 7.5 generated in 0.132s
Iray [VERBOSE] - IRAY:RENDER :: 1.8 IRAY rend stat : Materials memory consumption: 3.20839 MiB (GPU)
Iray Iteration: 1
Iray :Render target canvas was written.
Iray Iteration: 2
Then the second render:
Rendering in NVIDIA Iray
Compiling Shaders - 0/1
Rendering image
Rendering...
Iray [VERBOSE] - IRAY:RENDER :: 1.7 IRAY rend progr: CUDA device 0 (GeForce RTX 2070): Processing scene...
Iray [VERBOSE] - IRAY:RENDER :: 1.6 IRAY rend stat : Geometry memory consumption: 1.1246 GiB (device 0), 0 B (host)
Iray [VERBOSE] - IRAY:RENDER :: 1.7 IRAY rend progr: CPU: Processing scene...
Iray [VERBOSE] - IRAY:RENDER :: 1.8 IRAY rend stat : Native CPU code generated in 0.167s
Iray Iteration: 1
Iray :Render target canvas was written.
I found this in the log file:
2020-01-14 08:02:22.069 Iray [VERBOSE] - IRAY:RENDER :: 1.3 IRAY rend stat : Geometry memory consumption: 1.1246 GiB (device 0), 0 B (host)
2020-01-14 08:02:22.188 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(305): Iray [ERROR] - IRAY:RENDER :: 1.3 IRAY rend error: Unable to allocate 2294162816 bytes from 1281243545 bytes of available device memory
2020-01-14 08:02:22.192 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(305): Iray [ERROR] - IRAY:RENDER :: 1.2 IRAY rend error: CUDA device 0 (GeForce RTX 2070): Scene setup failed
2020-01-14 08:02:22.192 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(305): Iray [ERROR] - IRAY:RENDER :: 1.2 IRAY rend error: CUDA device 0 (GeForce RTX 2070): Device failed while rendering
2020-01-14 08:02:22.192 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(305): Iray [WARNING] - IRAY:RENDER :: 1.2 IRAY rend warn : All available GPUs failed.
2020-01-14 08:02:22.195 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(305): Iray [WARNING] - IRAY:RENDER :: 1.2 IRAY rend warn : No devices activated. Enabling CPU fallback.
2020-01-14 08:02:22.196 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(305): Iray [ERROR] - IRAY:RENDER :: 1.2 IRAY rend error: All workers failed: aborting render
I just unchecked OptiX (which was by default before). I only turned it on because another post said it was required. If the Iray denoiser is "Post Denoiser Available", then that is currently set to off. It had no effect on the render results, still using CPU when I render the same scene a second time.
@Mckelleon
Are you closing render frames between renders? The GPU will not release any VRAM if render frames are left open.
That just dawned on me this morning, because I hadn't been, so I closed all the frames and tried again positive this would be the fix. It wasn't. :/
You'd have to close all the frames and restart Daz (since your session had already dropped to CPU and once a Daz session does that once, it must be restarted). From then on, close each render frame before starting the next and it should hopefully remain in GPU each time.
This is probably what you did (and it still didn't work), I just wanted to make sure.
you need to do cntrl alt del to see the performance monitor that gives you your memory footprint both gpu and cpu...
and i have noticed that sometimes daz leaves a version running in ram even when you exit the programme so next time you use daz you will have even less memory until you kill the old process manually...
but i had a similar issue a few weeks ago when i had an item that had a high smoothing setting on the renderer (not viewport) by mistake....so i could preview it in iray no problem but when it came to acctually rendering it would blow out the gpu and drop down to cpu...it drove me nuts..lol.
Interesting. I closed all frames without restarting Daz Studio and it didn't work. I restart Daz Studio and close the frame after every render, it works 3 times before going to the CPU again. This seems like a memory leak to me now...
That's insane and a bit maddening! LOL how did you fix the smoothing issue? I'm thinking some of this issue might be the new environment I just bought. Seems heavy as hell when I go to render, and with other scenes I've created I don't seem to have this issue. Ever. This is the model I purchased that I believe is causing the issue: https://www.daz3d.com/modern-office-space
iRay and/or DS not completely releasing VRAM is a longstanding issue. Anything scene with more than just a single G3 or G8 character, I get one render out of my 6Gb GTX 1060 before I have to manually clear VRAM. For me, switching between GPU-only and GPU+CPU clears VRAM. That's in DS 4.10 and 4.11. Haven't upgraded so 4.12 so don't kow if it'll work there. Depending on the complexity of your scenes, you may be able to squeeze in more renders by selectively reducing the texture resolution (Scene Optimizer for the win) and sub-d level on background/peripheral props.
There may be third party scripts that display VRAM usage in DS, but I'm not aware of a native way to do that. Seems like most folks here use the free utility GPU-Z for that.
Interesting. I got Scene Optimizer, but haven't fully realized how to use it, LOL. I went hog wild and applied half size to everything and my render looked like ass, so have not gone back. I need to be more selective, so will likely give that a shot soon.
I just ran a test with preformance monitor up and here is what I've found:
First run the memory was 5.5/8GB while rendering then 4.4/8GB when stopped. I left that frame up for a second render, which would obviously go over my 8GB and it fell back on the CPU as expected. I close the frames and my VRAM drops to 1.3/8GB, so it should have zero issue doing a GPU render now. I run again and it jumps to 6.8/8GB then drops to 5.5/8GB and uses the GPU the whole time. Once done with the frame up it's at 4.4/8GB as before. I close the frame and back to 1.3/8GB as expected. Third run I get a run VRAM of 6.1/8GB... nothing has changed in the scene and it's still using the GPU but somehow using more VRAM than the previous 2 runs. When it completes the VRAM is at 4.4/8GB just like the other renders. Closing the frame brings me back down to 1.3/8GB. I run it up to the ninth attempt before it falls to the CPU this time, VRAM at 1.3/8GB before render. I uncheck the GPU and check CPU, VRAM looks good, but Daz still want to use the CPU. I switch to CPU only then render, cancel, then go to GPU only and it starts working again. If Daz Studio had a face I'd punch it. LOL
I'm now convinced there is a memory leak in the program.
Hmm, didn't know about the switch-to-cpu-then-back-to-GPU trick to clear memory. I'll still probably close/reopen Daz out of habit.
Many of my scenes also *eventually* drop to CPU after I'm working in them long enough (previewing, rendering, etc) so I can get behind the 'memory leak' idea.
Memory leaks, unfortunately, can be tough to fix. I'd be happy with a button that cleared GPU memory since it would be faster than closing/reopening (or switching to cpu, render, switch back to GPU).
I agree, memory leaks are a pain in the ass. I had one in a game I was developing and wound up rebuilding the engine rather than deal with the leak. LOL I do think they should address this ASAP with a button or quick method to clear the VRAM. I could only imagine how much of a hit someone must take when rendering an animation...
In the meantime, and since this has royalty annoyed me, I'm contemplating doing a Blender migration because I've heard there are new tools for exporting Daz models these days. Might give it a try, and if it sucks, maybe I'll appreciate Daz Studio again, bugs and all. Ha!
Good luck! Let me know if you find a tutorial on how to migrate. I've often wanted to use Blender for its fracture physics.
Unfortunately the latest tutorial I found still puts it in the 'too much work' category. https://www.youtube.com/watch?v=TI55AMr0fK4
All the manual fixing of bone alignment...no thanks.
I've had the same issue, it's not limited to 4.12. The issue is for whatever reason when the GPU runs out memory it detatches and won't work again with DAZ until it's restarted. Luckily, you don't need to restart your computer.
https://superuser.com/questions/178692/how-does-one-manually-restart-graphics-drivers
I've never used the utility or the keyboard shortcut they recommend, but I just go to the device manager and disable my video card and then enable it once the resolution adjusts.
In 4.10 and 4.11 you can leave GPU checked and check/uncheck CPU. IOW, just switch between GPU-only and GPU+CPU. Saves having to start and cancel a CPU render. Should be the case for 4.12 as well, but not running it so don't know for certain.
Why bother, when switching between GPU-only and GPU+CPU accomplishes the same thing without ever leaving DS?
It is nothing you can "fix" yourself. Restarting Daz is the only way to reduce the issue.
Switching back and forth with the check-boxes from CPU to GPU will do nothing, once it has gotten a rendering error. It will not release the memory and anything you try to render, after that point, will not fit, because it is still full anyways.
Some items, including Gen-8 model itself, causes the issue. (I can render the model from a distance, but if I zoom-in to the eyes, or toes or fingers, it instantly crashes at the point of render. Has nothing honestly to do with memory being full, but that is ALSO another issue.)
You can also make it crash the ability to render with GPU if you flip back and forth from IRAY-preview and texture-preview, on my machine. You would end-up with zero renders with GPU, at that point. (Since IRAY-preview mode is actually a render itself.)
Some items, for some reason, will consume 2x more in video memory, just at the beginning. If your card survives that spike, you can watch the video memory drop down to half the spike, as it actually renders. (I assume that is OptiX or something doing compression or removing all the unneeded components that Daz3D is sending to the card, for the render.)
You can watch your video-memory and processing power with the "Task manager". Switch to CUDA to see the processing. If processing suddenly stops, or if memory suddenly spikes to the top, then you have just failed a render. It may not even tell you in Daz, in the activity window. It will just start rendering a LOT slower. But, now you will know, can stop the render, save the scene, if needed, and reload to try rendering again.
Hopefully these issues will be fixed soon. They have gotten to the point where I can no loner render anything at all now. It has been a progressive downward spiral on these last updates. For me, I blame it on the fact that I have two types of cards, which are not the same. One set has Volta/Tensor cores, the other set does not. However, there is no way to "guess", which card is which, as they are all identified as the same cards in Daz. Even when I do figure it out, at that point, Daz stops working and when it restarts, they are not the same cards in the same checked slots.
Don't stress over it. There is nothing you can do, at the moment. It is a Daz and IRAY and nVidia-driver issue, for specific cards, with specific contents in the scene. (I can render Gen-3 models all day long, or any of my own creations made with other programs. But, eventually, at some point, Daz stops talking to the Video-cards and I have no other choice but to reboot Daz, to free-up the memory and reestablish a functional connection to the cards again.)
Have you actually tried it? YMMV if you're running 4.12, but it absolutely works in 4.10 and 4.11, because I do it ALL. THE. TIME. in both Win7 and Win10 after IRAY drops to CPU or throws up a black or blank render. GPU-Z shows it releases the VRAM and the scene renders just fine EVERY. STINKING. TIME.
Apparently Daz Forum removal of posts is like Fight Club; you can't talk about Fight Club. LOL. I won't speculate or accuse anyone of being like Nazi Germany (this is not speculative accusations), nor will I talk about any moderator’s actions (no specifics provided). This entire post is complaint with all rules because it states facts about the "ToS" without reference to anything specific and does not contain anything that would be speculative or accusatory towards any identifiable party. Man, I just looked at how much I've spent on Daz items between August and now... $678.37. Incredible how it all adds up.
Wow, I'm seeing hundreds of posts on this same topic. Sure seems like it should be addressed sometime soon. Would be neat.
@Richard Haseltine for what it's worth, the final 4.12.0.xxx release actually has TWO bugs going on with it as regards function of the OptiX Prime acceleration on/off toggle:
1. The underlying functionality is obsoleted/removed from the included version of Iray (a fact missed by Daz's developers due to Nvidia having failed to properly document the API change in the Iray changelogs), and
2. Daz Studio's GUI is erroneously interpreting that checkbox as being checked when it is unchecked, and vise-versa (a simple GUI programming error caused by someone at Daz)
For optimal function, the OptiX Prime acceleration toggle in all DS 4.12.0.xxx releases should be toggled to ON and left that way at all times regardless of GPU model/configuration. Unchecking it doesn't have any deleterious effects on rendering. Howver it does lead to a dirtier log file (since calls for no longer supported Iray features get logged as errors.)
The parenthetical bit is not self-evident.
The check box has been removed in the current Public Build.
I've read that the box has been removed in the Public Build a few places now, but I'm on 4.12.0.86 and still see it. No updates present in Download Manger, either. This is the current Public Build, right?
Public build is what they call the beta, pretty sure 1.12.1.40 is the latest
That's a silly name for a beta, but I suppose calling the non-beta release a "Stable Release" might be unfair to users. I'll give that a shot to see if anything improves.
Current public build is 4.12.1.xxx. My post is about 4.12.0.xxx.
Ah, you were instructing users. Sorry, I read it as an feature request for Daz.
The terms are pretty standard, and are explained here https://www.daz3d.com/forums/discussion/comment/990671/#Comment_990671
I was having this problem and using the scene optimizer helped a ton. Still need to figure out why my cpu is working instead of the gpu tho.
With all this talk about not releasing VRAM until quitting Daz and making sure render windows are closed. to release the memory. So my question is what does the purge memory script do?