DAZ / NVIDIA Card Crash... Ongoing Problem

I keep having a problem with NVIDIA cards and DAZ; I just got my third one, a Titan X, for use in rendering with Iray. But every time I render with my GPU it errors:
"Display driver stopped responding and recovered / Display driver NIVIDA Windows kernal mode driver version 258.91 stopped responding and has successfully recovered"
I've gotten this error for YEARS and yes my DAZ Studio is up to date and so is my graphics driver. I thought updating to more powerful cards would work; I even got a new more powerful power supply too. Also, I have 16 GB of RAM, and my operating System is Windows 7; my processor is an Intel Core i7-3770K CPU @ 3.50GHz, 2501 Mhz, 4 Core (s), 8 Logical Processor(s).
I had a GTX 960 before this Titan X but both are doing the same thing. I had a GTX 570 with the same problem before that.
This has happened with every version of DAZ Studio I have tried to render with; obviosuly with 3Delight first, and now with Iray. Now, this error does not always happen, but it often does. When it does "recover", DAZ will continue rendering with the CPU. But if I try to cancel the render, DAZ will completely crash with the following error: "Unable to recover from a kernal exception. The application must close." I have no clue what this means.
I also have an on-board Intel graphics card that cannot be disabled through the BIOS. This is what DAZ defaults to when the NVIDIA error occurs. It is an Intel HD Graphics 4000; and the only other one of course is the NVIDIA GeForce GTX TITAN X.
So what's the issue? It must be DAZ, right? Or something that is not DAZ or the graphics card? Is ANYONE else having this problem at all?
I'm trying to think of what else might be relevant... In the advanced render settings, I have both CPU and TITAN X chosen under both the Photoreal Devices and Interactive Devices categories. OptiX Prime Acceleration is currently checked, but this doesn't seem to matter whether or not it crashes.
Does anyone think in might be the motherboard itself?
This is from the NVIDIA help chat;
************
Arun: from the system information report the graphics card and its drivers are being detected correctly
Arun: the driver version seems to be older
Arun: however the onboard Intel graphics adapter is still enabled and could be the major cause of the issue
Arun: you would need to disable the onboard graphics adapter in the BIOS and the Device Manager with the assistance of your motherboard / computer manufacturer before trying to install the add on graphics card and its drivers
****
But I told him, both drivers are listed as up to date. Now I actually had the Intel card disabled in my Device Manager, but I don't know... I guess I still need to go into the BIOS; maybe I somehow missed it before. >.>
Then I mentioned this:
DEM: Actually according to my Device Manager, the Intel card was Disabled, not Enabled; do you think that would make a difference? Still go into the bios?
Arun: yes it will make a difference and unless the onboard graphics adapter is disabled in the BIOS and the Device Manager conflicts are expected
Arun: also if it is disabled in the Device Manager, then it would not be detected as enabled in the system information report Arun: it might have some how got enabled when you restart the computer
DEM: odd. Because i just enabled it in the device manager. Well I will try this method. Thank you.
*****
Now, as I was chatting, I opened DAZ again and unchecked the CPU option. Rendered a single charater, no crash. Rendered the scene which has been erroring, and it errored again (and will crash if I cancel the render).
The scene I am trying to do is two Genesis characters a simple rock outcrop. No lights except the IBL environnment.
When I went into the BIOS, we never found theb graphics driver listed (again). We did change the graphics or the monitor to PCI-E but it didn't change anything; it errored again, so here I am.
Comments
Its possible you have a lot of driver remnant trash and never getting a clean install maybe. I use this tool and it works great and reliably.
http://www.guru3d.com/files-details/display-driver-uninstaller-download.html
I know Nvidia driver version 359.00 is good with DAZ. Make sure you get that one and use the Geforce Experience App with it. Nvidia is really hard wiring that "thing" into their drivers now.
Use the above DDU tools "Installing a new graphics card" feature. Hook up the Nvidia card as the primary to your monitor before restarting so the bios won't see the Intel as the primary display device.
Hope this helps, I know your fustrated, this kind of issue sucks.
In short, your problem is NOT a Studio/Daz problem (this is way beyond their control) nor is it even an Nvidia problem. It IS a known Windows problem and is more system/install specific than most.
It was a new feature in Windows Vista, that is supposed to prevent bluescreens...but like many things Windows related, there is a price. In this case it is at the cost of the running application...terminated, with prejudice. If you Google Nvidia TDR disable it should give you a lot of hits...for gaming, the issue seems to be better, but for compute/rendering it can still be a problem, even with the latest drivers.
There are other causes than the timeout being set too low...heat and poor/insufficient power supply are the other 'main' ones.
Sone, the 258.91 driver is the only one that pops up for the NVIDIA TITAN X--doesn't this mean it is the only one compatible?
mjc, THANK YOU! Because you are the only one since I have been posting about this that has provided any hint of a real answer as to what the issue might be. For now I have edited the registry value for the TDR to 0... hopefully it works. As I stated above I bought a new power supply (1100 watts) before I got my TITAN X... and that video card should definitely be able to handle a two-figure-and-a-land-prop scene.
Well, I did as I said above, changing the TDR value to zero.... the error happened again at about 95% or so. And I forgot to mention, actually, it doesn't matter if I cancel the render or not. Once the render finishes, DAZ will say it has to close because of the kernal exception. >.>
you can in Nvidia control panel under manage 3D settings add DAZ studio to programs under CUDA only add your card, this may or may not help
No it's compatible, Go to Nvidia site (see attached)
Hi my name is Frederic and I'm new to this site.
I saw your problem and may have a solution. I use to have a Dell T7500 running a Quadro 6000 and continually had a similar driver issue whenever I used 3Ds Max. Sometimes using Internet or word would cause the same issue but more so when rendering. The driver would crash, the screen would black out then recover with the error message.
I shortly returned the PC and upgraded to the T7600 again with Quadro 6000 - again same issue! Turns out it wasn't a driver issue, Dell, 3Ds Max or compatibility issue! It was the power supply to the video card. I swapped out the 6000 card for a 5000 and 6 months in and I've never had the issue ever again.
So to summarise it has to be insufficient power going to your card. Couldn't promise this but seems like a good guess.
All the best - Frederic
@DrowElfMorwen "Sone, the 258.91 driver is the only one "
Sadly I think this is some system problem of yours. There are too many people running Windows 7, Daz Studio, and various Nvidia cards to believe it's a Win 7 or DS or Nvidia bug.
Did you make a typo in your driver number? The current one for the Geforce cards is 359.06. If not a typo then I would update the driver.
@fredericlavender "So to summarise it has to be insufficient power going to your card."
He siad he has an 1100 watt PS. That will be more than adequate. Unless of course it isn't properly connected.
It's definitely connected fine. I do not think this is a DS / NVIDIA bug... I think it must be something else.
For the 359.06 driver, I will look for it for sure. I don't know why the NVIDIA site has never showed me this driver, and even GeForce Experience won't find it.
Edit: Nevermind, I see it was released on Dec 1st. Last I checked was like November 30th >.>
Well, I tried the new driver--no luck. Same crash, and I did a clean install.
This can't be, say, caused by the motherboard can it?
I hope if ANYONE else is having this problem, they post >.>
Not entirely impossible to be the motherboard...but if the registry settings, drivers, presumably power supply and (and since this carried across to more than one card), card, checkout fine...system temps, memory and motherboard are all contenders.
I don't think it is the temp at all. I have monitored it before, plus the errors happen only a couple minutes into the rendering. I've done enough tests to know it is not that, and I have 16 GB of RAM so it shouldn't be the memory. All I can think of is the motherboard... but I have no other computer problems, no other errors. Nothing. It is JUST this error with DAZ and rendering on a GPU.
And I cannot for the life of me find any graphics driver in the BIOS. Me and my fiance have looked twice and nothing is there.
More like a problem with one of the sticks of RAM.
I currently have one stick that won't run at its rated speed and if I don't set the BIOS to run both sticks at this slower speed, not nice things happen.
What does the log file say at the crash?
not that it means much to me but may to someone
For example my card stops rendering in iray using animation sometimes with fiberhair and I get a weird repeated error in my log about my card, sadly nobody answered that one, I gave up and used Octane. The log did however show something
Hmm, I had no idea what to copy from it, so I put the entire log up here:
https://www.dropbox.com/s/xraltnqjmsainpj/log-daz-13DEC15.txt?dl=0
I hope it hints to an answer....
it seams to go pear shaped here
Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Received update to 00024 iterations after 59.934s.
WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER): 1.4 IRAY rend error: CUDA device 0 (GeForce GTX TITAN X): unknown error (while launching CUDA renderer)
WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER): 1.4 IRAY rend error: CUDA device 0 (GeForce GTX TITAN X): Failed to launch renderer
WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER): 1.2 IRAY rend error: CUDA device 0 (GeForce GTX TITAN X): unknown error (while de-allocating memory)
then this happens after a few tries
WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER): 1.2 IRAY rend error: CUDA device 0 (GeForce GTX TITAN X): Device failed while rendering
WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER): 1.2 IRAY rend error: CUDA device 0 (GeForce GTX TITAN X): unknown error (while initializing memory buffer)
WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER): 1.2 IRAY rend error: All workers failed: aborting render
and it switches to the CPU
WARNING: dzneuraymgr.cpp(261): Iray WARNING - module:category(IRAY:RENDER): 1.2 IRAY rend warn : All available GPUs failed.
Iray INFO - module:category(IRAY:RENDER): 1.2 IRAY rend info : Falling back to CPU rendering.
Iray INFO - module:category(IRAY:RENDER): 1.0 IRAY rend info : Retry rendering with other devices.
maybe run Speccy and GPU-Z see what your hardware devices are doing, chances are something is overheating or maybe a power issue
I am not a geek sadly but found my brothers fan was malfunctioning by doing that when they were playing Zynga games and their PC would shut down
It is not overheating and not a power issue. Already ruled those out, and yes I have CPU-Z.
Oddly, okay--I should explain this. So this one scene keeps crashing right? But if I open another scene, that has multiple still props and one Genesis figure, the scene doesn't crash. I have heard of certain items, like objects, making the render crash... but that just doesn't make sense does it? My fiance suggested to try to take pieces out at a time and see what happens; if it crashes each time. :/
So the crash doesn't always happen but the error seems picky about what's in the scene I guess.
Yes, there are certain shader settings that do cause specific items to crash a render. It is a bug in Iray that is fixed in the version in the 4.9 beta. And some hardware combinations are more suceptible than others.
ah yes could have told you that too
certain props will crash iray
finding which ones and why is a challenge
There's a couple of threads about some...one of the recently released space ships, is one. But it works flawlessly in 4.9.
I think it was missed that I have been having this problem for *years*... it started long before Iray came out. The same exact error. So, what would make 3Delight crash?
Edit: I just did the same render again, but instead of Iray chose 3Delight. Added a distant light with raytraced shadows; no crash. How very odd.
Edit 2: I told Iray to render the scene only. After six minutes, at 99%, it errored as usual and crashed after. Then I opened DAZ again and this time told Iray to render the "dome only". I've never done this before so I really was just expecting the dome... oddly it rendered my whole scene. And NO errors!! But, I am not considering this solved just yet. I rendered successfully a second time too. And a spot render with no problem.
Dome only means lighting from the environmental map only, not render the dome only.
By excluding the scene lights, using dome only, you have narrowed down the possible sources...
What are your lights in the scene? Turn all of them off and do some renders with one at a time...and if they all successfully render, individually, then start adding a second, third, etc back in until it crashes.
Actually this is the weird part! There are no other lights in the scene at all, not even as an emissive.
Adding to that, in the middle of all of this I did successfully render with Iray on my GPU on a scene using one distant light and the IBL/dome light. No error.
I was told the same thing by Dell multiple times, that it's not an power issue and that the power supply was more than capeable of powering a Quadro 6000.
Dell even replaced the motherboard, ALL the RAM sticks, power unit and even replaced the graphics card - guess what... the same issue persisted.
I ended up downgrading to the Quadro 5000 card and it works perfectly. It wasn't ideal but it fixed the issue.
Have you solved this yet?
Current Geforce Driver is 361.43, just released today.
By chance, have you had chrome open while rendering? In the recent past, there were some driver issues with geforce cards and chrome, I think somewhere around the 340-350 drivers.
DrowElfMoren - It probably doesn't make you feel any better but I have EXACTLY the same issue. I'm running Windows 7 Enterprise x64 on an HP-380t with two nvidia K2000 video adapters. For a while I thought not using the CPU during rendering - just using the two GPUs - fixed the issue, but it came back. Then I tried just one GPU, which fixed it sometimes and not others. I have updated the drivers whenever new ones came out, but it didn't solve the issue.
I'm using Daz Studio v4.8.0.59. My next thing to try is switching to the v4.9 beta.
I understand that some peoiple think it's somehow hardware or driver related, which is what I originally thought, but I'm starting to lean toward the idea that it's actually Iray related -- some object in the scene that Studio is passing to Iray, that Iray and the GPU instructions can't handle and die on -- but I don't really know.
This is a DAZ and iRay issue.
I run the latest version of 4.8 Pro on Win 7 Enterprise 64, 32 GB RAM, top of the line motherboard, and two NVIDIA GTX 970s. I can run 3DS Max, ZBrush, and a host of other VERY memory and video intensive applications without a problem. The ONLY application I've ever had crash on this system is DAZ. Canceling iRay doesn't crash DAZ every time I cancel a render, but it happens fairly consistently on one scene. Pretty sure I've isolated the problem down to one prop, but that prop is essential for what I'm doing. At this point, I am considering a return to Reality and LuxRender at least so far as this render is concerned. Maybe others, too. Never had any trouble with Reality and Lux. I considered upgrading to DAZ 4.9, but I don't like a myriad of things about it, mainly bugs a friend of mine is running into after upgrading.
So, yeah, if this problem continues, I suggest to the OP and anyone else having to try Reality and LuxRender for the scene that's causing them trouble.
Version 4.8 had a known bug that caused some products to crash the Iray render. The fix is to update to the latest version of studio.
No thank you. I'm definitely not a fan of 4.9. Yet another library reorganization, too problematic with ZBrush, cloud/drm/encrypted (have disliked cloud concept since going that route with Adobe) and the problems which can and do arise. Doubt I'll ever move on from 4.8 unless some later version of DAZ absolutely blows me away and is so superior I cannot achieve similar results (unlikely).