Why Does It Take So Long To Load Figures? ("Solved", with GUIDE)
EDIT: I was able to reduce loading times by 90%, and I'm posting the guide (originally on page 2) here, at the beginning of the first post.
FINAL CONSIDERATIONS ON FIGURES LOADING TIMES
Why are figures loading times important?
Your Daz3D library is installed on an SSD, so you load entire apartments in less than 10 seconds, and poses in a fraction of a second, but loading a single G8F takes an eternity? And even deleting the scene makes you want to switch to cave painting? Then you have the same problem I had, when waiting for figures to load, or for a scene to clear, used to take a huge part of my workflow.
What influences loading times?
- The most important factor, by a long mile, is the number of the morphs you have installed. A G8 figure, indeed, has those morphs loaded all the time, and it has to build the relationship between those morphs, even if they're not currently used, and so each morph slows down both the loading and the clearing times.
- Morph Packages: you probably found yet another morph package for $1.99, that you've never used. Uninstall it. Many morph packages have hundreds of morphs, so they're way worse than single characters.
- HD morphs: I think they're worse, regarding loading times, than standard morphs.
- Characters: don't buy figures that are similar to other ones you already have. Most of the time t's not even worth keeping them installed for hair, dresses or tattoos that come with them.
- CPU and RAM. They work together, so I don't know what had the biggest impact. But I have seen a 25% improvement in loading times, with a 50% improvement in CPU performance.
- Storage. It's fundamental. Whether to navigate your content faster, to apply poses, or to load either figures or environments/props. The difference between SSD and HDD is huge. There is yet another difference between NVME and SATA SSDs, but it's not as big as the previous one.
Was I able to improve loading times?
In my tests, I've gone from 136 seconds of loading time for the basic Genesis 8 Female figure, down to 29 seconds (-79%).
I hadn't tested that, but the 136s already were with an SSD, they were surely even slower with an HDD. Probably almost double than that.
So I'd say I reduced the loading times by around 90%.
What have I changed to get that improvement?
- I have uninstalled almost 2/3 of my figures, and the vast majority of my morph packages.
I know, it hurts a lot to let products you have bought take dust, but, if you don't desperately need them, having them installed is a huge malus for how Daz works.
(Some people install the products they need each time they need them. That's not viable for my workflow, because I make comics with Daz, and so I need a dynamic creative process that's not compatible with browsing DIM every time I need to do anything). - I have upgraded my PC, from a Ryzen 5 1600 with 16GB of 2400MHz RAM, to an overclocked Ryzen 5 3600 with 32GB of 3800MHz RAM.
- I have upgraded my storage first from HDD to SATA III SSD, then to an NVME SSD PCIe 3.0. They got really cheap these days, and, since once installed your library you won't touch it much, for this use cheap QLC will do the trick.
This has been a month-long experience, that has taught me a lot, and that has changed my style of purchasing: now, in order for me to buy a morph package/figure, it must be really worth it.
Here is the discussion with the whole experience, step-by-step: https://www.daz3d.com/forums/discussion/411446/why-does-it-take-so-long-to-load-figures/p1
I hope my experience will be helpful to you!
LenioTG.
______________________________________________________________________________________________________________________________________________________________________
Hi everyone!
I've measured how much time my Daz takes to load each figure, here are the results (in seconds, for base figure, bought character and custom ones with around 40 morphs active):
G8M
33 base
55 character
70 customG8F
136 base
207 character
218 customG3F
8 base
6 characterG3M
9 base
6 characterGenesis
3 base
3 character
My whole library is on a fast SATA III SSD (560MB/s sequential read, 33-396MB/s random).
I have a Ryzen 5 1600 and 16GB of RAM.
I have checked the log, and there are no errors.
The loading times of entire scenes without characters are crazy fast, in the order of 10 total seconds.
I guess you can imagine, then, what a huge impact loading Genesis 8 figures has on my workflow.
I have bought many G8 figures, and 3 more times G8F than G8M, so this is the value that corresponds more to the loading time. But I'd prefer to not uninstall anything, I like them all.
While I never use other generations of figures, and I have almost no characters for those.
Is there any way to speed up these loading times?
What is the bottleneck?
Thanks in advance for your help!
Comments
There was a thread, or part thread, on this topic fairly recently, but I don't have a link to hand. If therea re no errors then it's essentialy down to lloading and configuring the morph set up - DS doesn't store the shape data but it does need the name to appear on the slider and how the different sldiers are connected, so that they can be set and behave correctly when set. The more morphs there are installed the more there is for DS to do, and so the slower the process will be.
That's interesting, thanks.
So I could try to uninstall useless packs of morphs.
Yes, lightening the load should speed the loading.
Yes, lightening the load should speed the loading.
But what is involved in that loading?
Does a faster CPU load significantly faster?
In that case, it's more a single-threaded or a multi-threaded workload?
Thank you!
But what is involved in that loading?
Does a faster CPU load significantly faster?
In that case, it's more a single-threaded or a multi-threaded workload?
Thank you!
It seems to be single-threaded, I'm not sure factors govern the time - quite a bit of data has to be read from disc (the fiels contain the shape chnages, even if DS doesn't keep them in memory), the property lists have to be scanned in memory, and the CPU has to process everything as it gets it - I would imagien all play a part in the time taken, given a fixed amount of data to process.
It seems to be single-threaded, I'm not sure factors govern the time - quite a bit of data has to be read from disc (the fiels contain the shape chnages, even if DS doesn't keep them in memory), the property lists have to be scanned in memory, and the CPU has to process everything as it gets it - I would imagien all play a part in the time taken, given a fixed amount of data to process.
Thanks!
So single-threaded performance and system memory also play a role.
I'll test that when I'll update my hardware!
So I've looked at hardware during the loading, and with only Daz working, 4 of my 12 threads were in heavy use, and all of the cores had some spikes.
The RAM was being slowly filled.
I've uninstalled just some morph packages that I had never used before, and the loading times have already decreased:
Then I've uninstalled around 1/4-1/3 of my G8F figures:
Regarding custom characters, so the ones that we actually use, here are the new times:
I guess all that's left to do to improve those times is to get better CPU and RAM.
First thing to check is just what is happening when you load G8F base, load her in DS and then check the log, compare the bottom of your log with what you see in the attached image.
I have multiple content directories, that image is the log from my test directory which only has G8F starter essentials installed, my main G8 directory wasn't live at that time. Now while I don't have a fraction of the stuff you do, my log from my main G8 directory looks the same as that image, only real difference is the time it takes to load G8F (9.4 seconds).
My log is that clean for one reason, me, I've gone through every asset that dared to show up in the log, and either deleted it for being useless to me (correctives I'm looking at you) or edited it to "fix" whatever it was that was "broken", which ranges from a non zero default value right up to typos and bad parent data.
Basically you need to micro manage the assets you use with any of the Genesis figures.
Thanks Bejaymac, but as I've written in the very first post, I've already done that!
And my whole library is on the same drive, I didn't touch anything from the standard Daz installation :)
That's normal, a process may be single-threaded but that doesn't mean it will run on only one (virtual) core - it is quite usual for the system to spread it across multiple physical/virtual cores. If you had Iray preview mode on that would have been at least oen more thread, even if it was using the GPU for the actual rendering, and of course system tasks will use one or more threads.
I didn't know that, thanks!
I've done one more test.
I've loaded a complex environment without figures, and it took few seconds, as always.
Then I've merged a file with 4 different custom characters: 3 G8F and 1 G8M.
This took 278 seconds.
So it basically has to reload everything again even if another G8F figure is already in the scene, right?
A friend has tested Genesis with and without a ton of morphs, and the loading times stayed very short, without changing at all.
What has changed in the management of morphs between the two generations?
I've bought a ton of different hardware, hoping to see an improvement in my workflow.
I'll do further testing and I'll let you know if something changes!
For example, I only have 16GB of slow RAM right now, and it's completely saturated by this scene I'm working on.
I can think of 3 things that differ between Genesis 1 and G8
1) triax vs.dual quaternion weight mapping
2) load with no textures vs. PBR textures
3) facial expressions purely through morphs vs. rigged facial bones
I'm pretty amazed that the proposed solution is to uninstall software you've paid for - and apparently people do this?!
Poor scalability coding should result in, well, fixing that before adding even more potential bog-downs.
...or is it ok for people to delete a bunch of morph packages and then get refunds for those?
For how disappointed I am as well, I think at this point it would be very hard to fix this issue.
Maybe they will take a different approach with Genesis 9? Not that I'm so eager to have to update my whole Daz library...
Now I can't uninstall what I've not uninstalled, because I've used it on previous project (and I've wasted two days loading all of my characters to check that), and I have no intention at all to purchase new figures or new morph packages. Either because I have enough, and because I don't want to slow loading further.
I haven't understood much, but that explains that the problem is complex, thank you! :)
Yes, we do uninstall and save on other media or another drive what we don't need. It's been that way forever. I leave most everything I don't need right away on DAZ servers. Download when I need it. If it's something I really like I keep it on a backup drive. I don't see any way DAZ could redo this without blowing everything thing up in DS and ticking off users. It is the way it is.
I've invested a lot of money in storage in order to not have to worry about those things! :D
I didn't know figures slowed down the loading. It's a very specific situation, and it doesn't affect other type of products.
If I would have known, I would have bought fewer figures/morph packages, but I want to have installed everything I'm gonna use in the future. If that means having slower loading times, it's not the best, but installing things when I need them is definitely not suitable to my workflow!
When tracking problematic morps, wanting to speed-up the loading process or save the RAM, you can go into the morph folder and make a zip-file out of the contents, the you can delete the morph-files. DS-doesn't find them in the zip-file and if you do need the files back, you can just unzip them back.
Whatever the reason for disabling morps, this is a good way to start, even if you wanted to remove them completely.
Another way is to rename the files individually with an extension you can easily find, ie. Sunshine.duf -> Sunshine.duf.xxx
Thank you for the info! :D
The fact is that I make stories, so I have created tons of characters, that I need to access sooner or later, and I have to have many morphs available to create new ones.
So that's not viable for me.
As I've said before, it took me days just to check if I needed the figures I had uninstalled, and I had to reinstall 1/3 of them.
My solution was to buy a 512 GB NVMe SSD that goes into an M.2 slot on the mainboard. It runs at round 2,800 MB/s instead of the standard SATA III 600 MB/s and everything loads up mighty fast. I'll never go back to spinny disks for anything other than long term large capacity storage. Heck, even Windows boots pretty quick now. The PC boots up and is ready to run in about 15 seconds. You can get a 500-512 GB NVMe driver for less than $100 and a 1 TB for $150-$200.
I also bumped my memory up from 16 GB to 32 GB to reduce swapping out to the page file.
I don't know that upgrading the CPU will help as much as you would hope for the $ you'd spend unless you're using one that has integrated graphics. Even if you're not using the integrated graphics having the built in AGP can affect performance.
512GB is not enough for my library!
I have everything on a 2TB SSD.
As I have said, everything is crazy fast to load (a whole complex scene in around 10 seconds), the only anomaly is figures.
I just moved my library to a solid state drive last night. Before, on my traditional drive, it was taking roughly 4 minutes and 45 seconds to load a G8F (with 452 characters, not counting the non-character morph packages, of which I also have a lot). Now it takes 3 minutes and 20 seconds. My next step is to identify characters I rarely use and use the zip procedure.
I'm glad this thread has been helpful to other people! :D
All that's left to do for me is to see is it'll get faster with better CPU/RAM/Mobo when they arrive!
@LenioTG, your entire Daz library should be able to fit on a NVMe SSD and load your G8 characters much faster. Your current SATA III SSD will max-out around 600 MB/sec whereas a NVMe SSD drive will max-out at over 3,500 MB/s for reading data (i.e. about 6x Faster). If your motherboard has a PCIe Gen 3.0 x4, you can install a super-fast M.2 NVMe SSD . All the big names including Samsung, Western Digtial, and Crucial sell up to 2 TB cards (around $300). I have even seen some manufacturers selling 4 TB cards (i.e. Sabrent). If available, get a NVMe card with a heatsink.
NOTES:
M.2 = Physical form factor (i.e. Shape of the card and type of connector) - available in SATA and NVMe drives.
NVMe = Non-Volatile Memory Express (i.e. Control Interface and protocol for reading/writing memory that does not lose data when powered off)
PCIe bus = Peripheral Component Interconnect express (Draws data and power straight from motherboard slot with no cables needed) - Up to 32 GB/s
PCIe 3.0 = 3rd Generation with 32 GB/s Bandwidth (Gen 2.0 has 16 GB/s bandwidth and Gen 4 has 64 GB/s bandwidth... bandwidth doubles each generation)
PCIe x4 = 4 x PCI Data Lanes (x1 = 1 lane, x16 = 16 lanes, etc.) - If you put a PCIe x4 card i a PCIe x2 slot (with an adapter), it will only work at half bandwidth.
SATA III = Serial ATA bus interface - Up to 6 GB/sec and bandwidth throughput up to 600 MB/sec
SSD = Solid-State Drive (i.e. No mechanical moving parts)
Thank you robertswww, I'm passionate about hardware, so I already knew all of that :)
I'm satisfied with my SSD performance, the bottleneck is in Daz itself at this point, and it regards G8 figures only.
A 2TB NVME PCIe 4.0 would cost way more than my GPU, and that would be a much more important upgrade :)
Indeed, the cheapest one costs 390€ (or 440$), while I'm still using a RTX 2060.
So, if I had to spent that much money, it would have more sense to upgrade my GPU, rather than to get a small boost in loading times! :)
Interesting, the 2 TB NVMe drives are roughly half that here in the US. :(
You may see some speed improvement with a new MB/CPU/memory but it won't be nearly as much as using an NVMe drive as your boot drive (even if you don't load your entire library onto it). Also there is a point of diminishing returns when it comes to RAM speed. That is to say it probably won't be worth the extra money to go over 3200 and you probably won't see much difference between 2400 and 3200. Having more cores sounds like a great idea but if the software isn't written so support it - and I doubt Daz Studio is - then you won't see any gain beyond the increased clock speed.
I have an AMD Ryzen 7 2700 8 core / 16-thread CPU on a 470 board with 32 GB RAM, a GTX 1060 Ti, and the aforementioned NVMe drive. The bottleneck IMO - at least in terms of loading - will still be the SATA speed. If the .duf files are compressed then a CPU upgrade will help somewhat.
Best of luck, and I hope you do see some better performance!
Hay all, made an interesting discovery. How fast the figures load can have nothing to do with D/S per sae. Check how much 'empty' space is on the drive holding D/S. It works better if there is more than 15GB free, even better if there is more than 20GB free. The problem is how in the world that drive keeps filling up in the first place! But, some claim that's because of all the various updates for the OS, and its parts.
And yes I too have deleted a few purchased programs from the C drive - installing some of them onto external drives. Bit of a lag but nothing super note worthy.
D/S is on the C drive and ALL its content is on the D drive. 3D Coat is entirely on an external drive. Carrara has yet to be reinstalled.
Lucky you :D
Consider that that model is PCIe Gen4, and that in Italy all prices are VAT included.
Yeah but everything other than figures loads crazy fast, so I struggle to think that the problem might be the SATA connection here!
I already have an NVME as boot drive, I use it for Windows only. That's where Daz is installed.
I've already purchased a Ryzen 5 3600, 32GB of 3600MHz C16 RAM and a X570 Motherboard (in the future I'll probably get a Ryzen 9 4000something and 2 high-end GPUs), that should allow me to overclock properly. We'll see.
I highly doubt that's it! I have 600GB of free space in my drive.
I think you're using a Hard Drive?
SATA SSDs are crazy fast for Daz: again, if I take a complex complete scene, delete the figure, and load it, it loads in around just 10 seconds. Hard Drives are much slower than that.
My problem regards Genesis 8 figures and only those!
I run the Windows 10 and Daz Studio from a 512GB M.2 PCIe x4 SSD and the Daz Studio content from a second 512GB M.2 PCIe x4 SSD but as far as I can tell there is no benefitn the content may just as well be on a normal SATA3 SSD.
This appears to be a major limitation in Daz Studio's design.
The only way is to keep a base library of essential content and then switch in and out other librarys of content as you require them.
Best Wishes
Steve.
You know this is the best idea I've heard so far?
Of course it's unpractical to install and uninstall everything, but installing all the figures on the same drive, but in a different folder, and turning it on and off might have sense.
One question: what if I created morphs from my figures, then would I be able to delete the figures I have installed just to not lose the reference?