Speed up preparing render stage
lilweep
Posts: 2,402
It takes a long time to begin rendering a big scene (e.g, a scene of over 8 GB in size).
How do you speed up this process? More RAM? Multiple graphics cards? Often my computer will seem to freeze at this stage of loading the scene. I feel like i wait as long for the scene to load as i do for it to render.
Specs:
RTX 2080TI
16GB Ram
Ryzen 3 1300x (4 Cores - 3500 MHz)
On a side note, I was wondering can I use my laptop to help run my PC at all while rendering?
My laptop has:
GTX 1060 with 6GB VRAM and 16GB ram
Post edited by lilweep on
Daz 3D is part of 
Connect
DAZ Productions, Inc.
224 S 200 W, Suite #250
Salt Lake City, UT 84101
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
The preparation is (perhaps) baking out the final shapes and maps and then (certainly) passing everything over to the render engine - I suspect adding a second card would slow the process down, though I didn't do a comparison on my system. Smaller textures should help, by cutting the amount of data to be transferred, and to an extent the same would be true of mesh resolution.
im not sure i want to reduce the size of my scenes. Ideally was looking for a way to speed up process while keeping scene size the same.
I have Iray Memory Assistant, so maybe i can review what is causing the majority of memory usage and look at reducing scene size, but what's the point of having a 11GB card if i can never have a scene that size without waiting forever for it to load.
For example, currently I am at 56 minutes elapsed time and I-ray hasnt even rendered anything.
Last thing it said in the 'Rendering Image' dialogue box was "Native CPU code generates in 46.321s".
That CPU line means Iray is using your CPUs. That is why it is painfully slow.
... i dont think so... Unless? Maybe i should do some more tests.
CPU doesnt take 1 hour to do 1 iteration though?
Anyway, after 1 hour eventually it loaded and I went through a few iterations pretty quickly. Realised i set camera settings incorrect so will start over again. Maybe test my camera first.
The problem seems to be your render settings are for either both CPU and GPU or CPU only. That poor old 1300x is just not up to rendering and transferring all the data to the GPU. Turn off the CPU in render settings. It may helop some.
I will upgrade my CPU and RAM at some point as a general priority.
Does running the PC display etc take up VRAM?
This is probably the most asked topic on this forum, lots of threads you can search.
Always check your render window history/ log if the render is on GPU, your scene must fit on the card.
One figure can take about ~1gb , even more with detailed hair and clothing. A detailed scene easily takes as much. Windows 10 grabs about 1gb from your VRAM pool.
Best is try a very simple scene and see if it goes unto GPU, and increase the load.
If you have a scene over 8gb as you describe, you need to start optimizing your scene.
Yes. However on Win10 the OS reserves a video buffer on all consumer grade GPU's. So that shouldn't be having any impact that you could change.
But amount of VRAM is not your issue.
At idle, GPU-Z says I am using between 1GB and 2GB of VRAM? Does that idle usage of VRAM cut in to my render VRAM budget?
Another thing i notice monitoring GPU-Z: after I do a first test render, GPU-Z reports much higher VRAM use at idle (e.g., like 2GB higher than before render). Is that normal? It just decides to keep some scene memory for later?
DS/iRay has a memory leak. If you render multiple times in succession without shutting down DS you will run out of VRAM and drop to CPU.
Yes, it is normal for the video card to have some VRAM in use even at idle. Drawing the screen takes some VRAM.
OP is asking about "prepartion" time not rendering time. AFAIK, preparation is a CPU task where the scene is calculated BEFORE it's hand off for rendering.
Also, I think render "preparation" is a single threaded task which means you would want a CPU with higher clock. But I could be wrong in that scene preparation is single thread only though. Ryzen 1300x is a 3.5/3.7Ghz entry level CPU. Current mid-to-top tier consumer CPU can sustain 4Ghz easilly and some can do 5Ghz. Those will speed up your preparation time.
More DRAM (not VRAM) could also help. Scene prepartion is uncompressed so a 8GB compressed scene on GPU could use 2-3x more in DRAM for CPU to prepare. Your system currently has 2:1 16GB to 8GB DRAM/VRAM ratio. DRAM is also needed for multitasking so your ratio would always be lower, and could be much much lower. Extreme slowdown could be you running out of DRAM and CPU needs to constantly swapping data with HDD, which does sound like what you are experiencing with prepartion time taking as long as rendering time.
EDIT: Based on what you described, I think DRAM first (going up to 32GB is more than enough with a 8GB GPU) and CPU second will get the most for your buck in preparation time. If preparation is single threaded, you should aim for a mid-tier CPU with fewer cores but high clock. You get both more cores and high clock in top of the line CPU but those extra cores you paid for may not matter to Daz. The money saved could be used for a better GPU, for rendering.
No. There is no reason that the system should take hours to start a render. Adding $200 of RAM won't change that. 16Gb is more than enough for an 8Gb GPU in iRay.
If the op doesn't have CPU selected in his render settings his problems are fairly severe and might well be a failing mobo but he never replied about any of the suggestions made..
Huh, really. I will have to monitor this. I did a few renders and at idle it never went above 4GB VRAM in use. So maybe the memory leak is kind of capped? I guess constantly restarting DAZ to get more VRAM is a new joy I have yet to fully experience.
I ordered some more RAM anyway - whoops. I was meaning to get more RAM for the last few years - next i will upgrade CPU - which i need to do for marvelous designer anyway. I was surprised how cheap 16GB of DDR4 got. For the record, the GPU i have has 11GB VRAM, not 8GB but doubtful that makes a difference in your calculus as to whether i need more RAM or not.
I infer from your comment that I have the option to turn off using CPU for render? If so, wouldnt i just be recommended doing that and then i dont have to deal with Daz doing what i dont want it to do (which is use CPU which seems like it could be the reason the scene prep is so long)?
I think dawnblade was right before when they said my render was dropping to CPU and that's why it was taking so long to prepare scene. I say this because I was doing some tests last night, and I started to monitor VRAM with GPU-Z. I progressiely added stuff to scene, and rendered, and montiored the memory in GPU-Z. The scene prep was very quick (certainly under 10 minutes) every single time until i approached/went over limit. Unsure if i went over limit, because my pc kind of froze at this point when i -perhaps - went over limit and so i was unable to check GPU-Z to confirm it was using all VRAM. I didnt really have the patience to wait for it to respond so restarted.
That was dropping to CPU.
Render Settings->Advanced Unselect CPU and Allow CPU Fallback
@ilweep
I disagree w/ @kenshaw011267 More RAM is never going to hurt. IME 32GB of CPU ram will also minimize disk swapping and generally make all PC operations faster.
Ah okay. I will do this.
I assume if i go over VRAM it will just kindly notify me of this in some fashion?
Have you monitored system RAM when you've experienced the the long scene preparation times? I ask because I've had scenes take considerably more system RAM than double the amount required for the GPU. With only 16Gb of system RAM, you could easily be going over 16Gb for your scene, plus what Windows needs just to run. If this happens, Windows won't complain, it will just quietly start using swap-disk. This uses your hard drive as additional system RAM, and can be painfully slow, especially if you have a mechanical hard drive. Try rendering a scene that is causing problems while monitoring your system RAM. If usage goes over 16Gb, then this is almost certainly your problem, and the additional RAM you ordered should fix the issue.
My RAM was always at like 99% while rendering, so maybe that's true. I also only have like 22GB free on my SSD.
That sounds a lot like you were rendering on CPU.
As to exceeding VRAM if CPU fallback is disabled it should warn, or at least fail to render at all. But both are generally preferable to an unexpected CPU render.
The reason I think this may be an issue with exceeding system memory, more so than falling back to CPU, is that you are experiencing huge lag times of over an hour between when the render is initiated, and when it actually starts to render. I've never experienced any noticeable difference in lag times for a render starting when it drops to CPU. IMHO, it does stand to reason that exceeding system RAM and using swap disk could cause this sort of delay (though admittedly, I could be wrong). This is just a guess, as I have not had this happen with DS because I haven't used a system with less than 32Gb RAM in many years, nor a GPU with enough memory to challenge my system RAM.. But, in other work that I do, I have often gone over system RAM capacity (with an older computer), and once swap disk initiates everything slows to a crawl, even when the swap disk was an SSD. This seems to mimic your description of it taking forever to get Iray to start rendering, but once rendering begins it is fast (once everything is loaded onto the GPU, the performance hit from using swap disk would be comparatively minimal). Of course I could be way off base here, since I'm not sitting at the controls to see what is actually going on.
One sure way to determine if the problem is from using swap disk would for you to monitor that actual numbers for RAM usage, rather than just percentage. IIRC, with Win7 the percentage never went above 99%, even when I was using well over 32Gb on my old system. Windows would dynamically add more "RAM" to the swap disk as needed, which was counted as system RAM for the percentage value, thus never exceeding 99%, even though with swap disk I might be using about 38Gb of "RAM" (another way to get an idea if swap disk is being used is to watch your hard drive activity light if you have one, it will be nearly solid (on) and show near continuous use). I haven't run out of system ram with my current computer .... yet (128Gb - but I have come close), so I don't know how Win10 handles swap disk, if there is still a huge performance hit, and if it still shows as 99% usage when your well over actual RAM availability. I'm guessing it's still pretty much the same as it was with Win7.
Hope maybe this all helps in some way, and I haven't lead you astray.
With my RTX2080Ti that has 11GB Vram, had initially 32GB system ram. (Win7 Pro OS).
It was NOT enough. Went with next step up of 64GB and it's the sweet spot for my syetem, especially if muli-tasking at all.
Before that CPU load was 100% for too long as it was doing swapping. Have posted this before in these forums.
Have no idea why DAZ inflates so much the memory.
Can tell you my renders are much faster to start and I can go 10 hours without dropping to CPU in one Daz session.
i was monitoring by GPU-Z and it does seem like systemRAM could be bottlenecking things. Computer becomes really slow when RAM use is at peak.
My guess is that when you are seeing the peak usage, it's using swap disk. I've never seen Windows consume all available system ram, even when using swap disk, but it does get very close. Windows Task Manager used to show the total amount of memory used including swap disk, I don't know if it still does (not at PC to check). I'm pretty sure GPU-Z only shows real physical ram, not virtual swap disk 'ram`.
But, it looks like you have found your problem.
Yeah, with my old computer I had a GPU with 6Gb and for one render DS used over 24Gb. Part of the problem is DS keeps a copy of the data sent to the GPU in system ram to enable resuming the render. Also some things in DS just simply use a lot of RAM, like dForce simulations, so you can easily use up more than double the GPU memory with just scene data, and still easily have it fit on the GPU.
I am more convied by my original guess that it's the sytem memory. 16GB of DRAM vs. 11GB of VRAM is definitely not adequte (also 16GB stick is more like $80 than $200). On top of that you only have 22GB left on your SSD. It's the perfect receipe of system slowdown due to memory shortage (DRAM, not VRAM).
One easy way to monitor if Windows is running out of DRAM is watching the disk usage light on your PC case (not sure if that's common nowadays because I build my own desktop). The light flashes nonstop when Daz is preparing a scene and if it continues doing so during the long wait then it's 99% likelihood Windows has engaged in disk swap mode, and you need more DRAM.
There are a lot of odd beliefs about RAM, swap files and virtual memory.
Yes, Windows does swap RAM to disk to make the system have "unlimited" RAM. It's been doing so for more than 20 years and it goes on pretty much all the time, even when the system RAM is not full. Each program has a designated "pool" of RAM which is far smaller than the entire amount. You can easily enough check this yourself, Performance Monitor will let you monitor the paging file live.
Swapping to and from disk is mostly seamless. I have seen issues, very rarely in real time applications where swapping to disk seemed to cause a failure do to timing issues. I'm pretty sure iRay rendering on CPU has this sort of issue when it crashes, although a surprising number of people have the page file set to a fixed size which iswell known to cause crashes as well.
If swapping to disk routinely caused iRay loading times to run over an hour then we'd have all seen it, since it happens during every iRay prep. Yes having too little RAM could cause a delay, or even a crash, but to go from what should be a minute or less to an hour? That's simply impossible.
Is having only 22Gb of empty space on the disk an issue? Windows will either flat out cause programs to crash, if the page file size is fixed, or will pop up error messages about needing more room on the disk.
Finally just adding mismatched sticks of RAM to a computer is always a way to slow down not speed up a system. RAM has a whole host of timings related to accessing the data stored on the RAM. RAM kits sold together are the same timings and often, always?, come from the same original silicon wafer. Mismatched sticks could share the same primary timings, those can at least be checked because they are advertised, but not the secondary or tertiary ones. That can slow the RAM down a lot, all the way down to the base timing of the RAM, 1600 Mhz for DDR4. Further if you have 2 8 Gb sticks and drop in one 16 Gb stick your system will report having 32, assuming it will boot, but will run very poorly as that 16Gb stick will be single channel while the rest of the RAM is dual and that messes memory controllers up very badly. If you have a single stick getting a second matching stick is always a good idea. If you have 2, on a desktop system, adding more RAM should be done either by replacing the kit with a larger kit or if you don't care how slow the system runs as long as it runs by adding a second 2 stick kit. Adding just one stick to an existing 2 stick kit is just a bad time, there are motherboards where that won't even boot.
Going to 32GB has made a drastic difference in experience for me. For the particular big scene i was having problems with, the render preparation stage is noticeable faster. I was honestly not expecting much of a difference but it was at least half the time for the same scene (this scene took VRAM up to about 10GB). And i also noticed that when i cancel the render, I can immediately close the render and resume work, whereas before i would have to wait a few minutes or so for the render to finish cancelling.
Woohoo!! Sounds like the lack of enough RAM was the problem. Glad to know things are working much better now!
Yep, it's a classic textbook case of system hang from virtual memory disk swaping. Haven't experienced it myself in years as it has been a while since memory prices have fallen to where almsot no consumer usage scenarios would need more than the basic configuration but in the 90s I had to deal with this on a daily basis developing database on Windows 3.1.
AFAIK, Daz3D can only use CPU to set up a scene before it can render. GPU renders the data in compressed format but CPU has to de-compress the data to set them up, and in Iray both the setup and render need to have all that data at once (such restriction in memory implentation in consumer GPU is probably deliberate as Nvidia needs to protect its professional GPU profit margins). Reading the forum it seems that the decompression ratio is something like 1.5-3x of what Iray reports. So the sweet spot for DRAM is at least 2x VRAM, more if the same machine is used for multi-tasking. 99.9% of people won't ever run into this problem as almost no one who can afford a 11GB RTX2080Ti would only have 16GB of system RAM :P
I'm still of the opinion that this was all about being out of VRAM. I just tried pulling half my RAM, down to 16, and started a bunch of renders, all too big for my 2070 but that do fit on my 1080ti, and not a single one took an unusual amount of time to start. So there is definitely some other issue. Also cutting his start time in half drops it to a half hour not to anything acceptable.