Ridiculously long loading time........

Marmalade BoyMarmalade Boy Posts: 103

Hi...

Here is hoping someone can help me. I rarely use the genesis 8 figures because of the long loading time to add just the basic character to the scene. I purchased a new character and it took almost 27 minutes to load. Is there something I am doing wrong? Can someone tell me how to get the loading time down to a minute or 2? I have quite a few characters for genesis 8 and anyone one of them takes almost a half an hour to load.  I read some of the post on here about loading times it apparently there is a correlation to the amount of characters you have. I have 83 different characters for Genesis 2 and that sucker loades in under a minute. attached is a portion of the log file.

Capture.PNG
603 x 398 - 36K
Post edited by Marmalade Boy on
«1

Comments

  • TheKDTheKD Posts: 2,677

    I don't think it's only characters with G8, I am pretty sure expressions add time as well.

  • fastbike1fastbike1 Posts: 4,077
    edited April 2020

    @Marmalade Boy

    How much RAM does your maching have? Do you have a swap file running, and if so, does it have enough space?

    I have almost 700 G8F character and the slowest one loads in uder 3 minutes.. The is Win 7 64 bit, 32GB RAM, I7-4770K 3.5GHz.

    Post edited by fastbike1 on
  • Hey fastbike1

    My System is  AMD Ryzen 5 1600 Six -Core Processor  3.20GHz, 16 GB of memory, 64-bit, GeForce GTX 1080.

  • Sensual ArtSensual Art Posts: 641
    edited April 2020

    I am guessing the latency is due to DS trying to find and read the metadata for any relevant morph across all content libraries while loading a new character. By metadata I mean stuff like morph name, associated icon, default values, min and max range, step size which can be obtained only by reading each of the individual morph files. This operation may take more time for characters since their filesize is large. So the effect of having many characters installed is more pronounced.

    I wish Daz made it a mandatory practice to implement inject and remove morphs for characters and expressions instead of making everyting available upfront. Atleast some vendors at renderosity still follow that good old practice of inj and rem even for new characters.

    Post edited by Sensual Art on
  • vukiolvukiol Posts: 66
    fastbike1 said:

    700

    700?!

     

     

    700...

     

     

    omg!

  • I'd be running process monitor or something to find out which files are being accessed, then delete the ones causing the problem, this shouldn't be happening.

    Could be a HD morph or 2 that have default values greater than 0, a vendor mistake or something you have done, but even then the load time is still over the top, only time something like that happened to me is loading a character which set the render subD to 5 and preview subD to 4, more than my machine likes to handle. 

  • LenioTGLenioTG Posts: 2,118

    The loading times bother me a lot too. It's all wasted time.

  • TheKDTheKD Posts: 2,677

    If you really want to use a "cleaner" G8 with injection-like behavior, it is possible to do manually. Set up a library with just what you consider bare essentials g8 stuff installed, probably be a good idea to keep "base pro characters" installed in there. Create and map another "temporary" library, and drag drop the morph sets you want to use from the product zip for the current artwork into there before openning DS and starting to work. The problem with that system, is you would have to know or make note of what to reinstall in the "temporary library" if you wish to revisit an old scene you set up. Perhaps a notepad file kept with the saved scene listing what is needed. I do this method, but with props, scenery, vehicles etc etc.

  • fastbike1fastbike1 Posts: 4,077

    I may be preaching to the choir, but you might find that another 16GB RAM makes a surprising difference. You have a good GPU, but your RAM and HD are what control everything until the render starts. 

    Hey fastbike1

    My System is  AMD Ryzen 5 1600 Six -Core Processor  3.20GHz, 16 GB of memory, 64-bit, GeForce GTX 1080.

     

  • westechiewestechie Posts: 17

    OpenGL works much better with Intel over AMD, although Ryzen is not bad at all.  It relates to the CPU architecture.  IRay really needs Intel to work well.  Ryzen provides superior performance with FBX compatible file formats how ever, such as with the Unity gaming engine.

    As an alternate solution, you can probably use 3Delight stuff still just fine with out much trouble.  You will need the following product:  https://www.daz3d.com/rssy-iray-to-3delight-converter-and-merchant-resource

    1) Create a New Scene.

    2) Use the RSSY Iray to 3Delight Converter And Merchant Resource product to convert IRay stuff to 3Delight instead.

    3) Try loading your Genesis 8 figure of choice to test.

    Again, your problem is with Ryzen's compatibility with NVidia's IRay.  You still have very nice hardware.  You just can't use IRay with your setup.

  • westechie said:

    OpenGL works much better with Intel over AMD, although Ryzen is not bad at all.  It relates to the CPU architecture.  IRay really needs Intel to work well.  Ryzen provides superior performance with FBX compatible file formats how ever, such as with the Unity gaming engine.

    As an alternate solution, you can probably use 3Delight stuff still just fine with out much trouble.  You will need the following product:  https://www.daz3d.com/rssy-iray-to-3delight-converter-and-merchant-resource

    1) Create a New Scene.

    2) Use the RSSY Iray to 3Delight Converter And Merchant Resource product to convert IRay stuff to 3Delight instead.

    3) Try loading your Genesis 8 figure of choice to test.

    Again, your problem is with Ryzen's compatibility with NVidia's IRay.  You still have very nice hardware.  You just can't use IRay with your setup.

    I'm ot sure what you are saying here - Iray neesd an nVidia card for GPU rendering, OpenGL is used for previewing (and I've not heard that Intel GPUs do it well - indeed rather the reverse judging by the issues people have with the Viewport using an Intel GPU) and neither is connected to file loading. Did you post to the wrong thread?

  • TBorNotTBorNot Posts: 369

    My goodness, I am using a Mac Mini, and G8 characters only take a few seconds to load.

  • BejaymacBejaymac Posts: 1,886

    Sorry to say but that little snippet of a log file is about as much use as a chocolate fireguard, whatever issues are causing the long load times will be seen earlier in the log.

    This is for all those with long load times.

    First start up DS, once running with an empty scene, go "Help > Troubleshooting > View Log File". Once the log open click on it to make sure it's selected, then on a PC (MAC you are on your own here) use Ctrl+A to select everything in the log, then press "Backspace" or "Delete" on your keyboard to remove it all. Still in the log file go "File > Save", this will replace the log with a nice clean and empty one, you can now close the log file.

    Back in DS load the base figure you are having problems with, and I mean the base figure rather than any of the characters, so G8F/G8M rather than V8/M8.

    Once DS has finished loading the figure open the log file again, everything in that log is what DS went though loading that figure, so make a copy of that log, ZIP it and attach it to a forum post.

    That way someone like myself can get a look at it and see if we can find the issue(s).

    Now for the bad news, all we can do is give advice, it's going to be up to you to decide what do and how to tackle the products causing errors.

  • westechiewestechie Posts: 17
    westechie said:

    OpenGL works much better with Intel over AMD, although Ryzen is not bad at all.  It relates to the CPU architecture.  IRay really needs Intel to work well.  Ryzen provides superior performance with FBX compatible file formats how ever, such as with the Unity gaming engine.

    As an alternate solution, you can probably use 3Delight stuff still just fine with out much trouble.  You will need the following product:  https://www.daz3d.com/rssy-iray-to-3delight-converter-and-merchant-resource

    1) Create a New Scene.

    2) Use the RSSY Iray to 3Delight Converter And Merchant Resource product to convert IRay stuff to 3Delight instead.

    3) Try loading your Genesis 8 figure of choice to test.

    Again, your problem is with Ryzen's compatibility with NVidia's IRay.  You still have very nice hardware.  You just can't use IRay with your setup.

    I'm ot sure what you are saying here - Iray neesd an nVidia card for GPU rendering, OpenGL is used for previewing (and I've not heard that Intel GPUs do it well - indeed rather the reverse judging by the issues people have with the Viewport using an Intel GPU) and neither is connected to file loading. Did you post to the wrong thread?

    I'm saying most Intel CPUs work much better with NVidia cards than a Ryzen will.  Ryzen has issues with some of the top tier NVidia video hardware.

  • westechie said:
    westechie said:

    OpenGL works much better with Intel over AMD, although Ryzen is not bad at all.  It relates to the CPU architecture.  IRay really needs Intel to work well.  Ryzen provides superior performance with FBX compatible file formats how ever, such as with the Unity gaming engine.

    As an alternate solution, you can probably use 3Delight stuff still just fine with out much trouble.  You will need the following product:  https://www.daz3d.com/rssy-iray-to-3delight-converter-and-merchant-resource

    1) Create a New Scene.

    2) Use the RSSY Iray to 3Delight Converter And Merchant Resource product to convert IRay stuff to 3Delight instead.

    3) Try loading your Genesis 8 figure of choice to test.

    Again, your problem is with Ryzen's compatibility with NVidia's IRay.  You still have very nice hardware.  You just can't use IRay with your setup.

    I'm ot sure what you are saying here - Iray neesd an nVidia card for GPU rendering, OpenGL is used for previewing (and I've not heard that Intel GPUs do it well - indeed rather the reverse judging by the issues people have with the Viewport using an Intel GPU) and neither is connected to file loading. Did you post to the wrong thread?

    I'm saying most Intel CPUs work much better with NVidia cards than a Ryzen will.  Ryzen has issues with some of the top tier NVidia video hardware.

    Yes, I have the occasinal stuttering display that others have run into (just updated my BIOS this morning, after two in three days, in case that helps) but it has nothing to do with loading times.

  • BejaymacBejaymac Posts: 1,886

    Loading times are down to a few things all happening at the same time.

    First off is all the software you have running, they all generate software errors, even your OS, these errors sit in your CPU like boulders in a river, slowing the data flow, and will remain there for as long as the software is running.

    Second is DS has to load thousands of small asset files in a short space of time, computers would rather have the same amount of data in one or two large files, as they can process them a hell of a lot quicker.

    Third is the amount of errors being generated by those small asset files, those errors also end up in your CPU, slowing it even further.

    Fourth is errors that cause DS to freeze, those are the main problem as it can take your computer a few minutes to get DS unlocked, only for it to run into more of the same.

  • Bejaymac said:

    Loading times are down to a few things all happening at the same time.

    First off is all the software you have running, they all generate software errors, even your OS, these errors sit in your CPU like boulders in a river, slowing the data flow, and will remain there for as long as the software is running.

    Second is DS has to load thousands of small asset files in a short space of time, computers would rather have the same amount of data in one or two large files, as they can process them a hell of a lot quicker.

    Third is the amount of errors being generated by those small asset files, those errors also end up in your CPU, slowing it even further.

    Fourth is errors that cause DS to freeze, those are the main problem as it can take your computer a few minutes to get DS unlocked, only for it to run into more of the same.

    I'm not sure what you are trying to say here.

  • westechiewestechie Posts: 17
    edited April 2020
    westechie said:
    westechie said:

    OpenGL works much better with Intel over AMD, although Ryzen is not bad at all.  It relates to the CPU architecture.  IRay really needs Intel to work well.  Ryzen provides superior performance with FBX compatible file formats how ever, such as with the Unity gaming engine.

    As an alternate solution, you can probably use 3Delight stuff still just fine with out much trouble.  You will need the following product:  https://www.daz3d.com/rssy-iray-to-3delight-converter-and-merchant-resource

    1) Create a New Scene.

    2) Use the RSSY Iray to 3Delight Converter And Merchant Resource product to convert IRay stuff to 3Delight instead.

    3) Try loading your Genesis 8 figure of choice to test.

    Again, your problem is with Ryzen's compatibility with NVidia's IRay.  You still have very nice hardware.  You just can't use IRay with your setup.

    I'm ot sure what you are saying here - Iray neesd an nVidia card for GPU rendering, OpenGL is used for previewing (and I've not heard that Intel GPUs do it well - indeed rather the reverse judging by the issues people have with the Viewport using an Intel GPU) and neither is connected to file loading. Did you post to the wrong thread?

    I'm saying most Intel CPUs work much better with NVidia cards than a Ryzen will.  Ryzen has issues with some of the top tier NVidia video hardware.

    Yes, I have the occasinal stuttering display that others have run into (just updated my BIOS this morning, after two in three days, in case that helps) but it has nothing to do with loading times.

    I'm saying it totally does.  Again, you have a compatibility issue.  Ryzen has recognized trouble with IRay, even up to the NVidia RTX series.  Go try some 3Delight shaders if you want to test this.  You will see the difference.

    This following product might be your best cheap fix, until you take care of your IRay problem.  https://www.daz3d.com/rssy-iray-to-3delight-converter-and-merchant-resource

    Post edited by westechie on
  • BejaymacBejaymac Posts: 1,886
    Bejaymac said:

    Loading times are down to a few things all happening at the same time.

    First off is all the software you have running, they all generate software errors, even your OS, these errors sit in your CPU like boulders in a river, slowing the data flow, and will remain there for as long as the software is running.

    Second is DS has to load thousands of small asset files in a short space of time, computers would rather have the same amount of data in one or two large files, as they can process them a hell of a lot quicker.

    Third is the amount of errors being generated by those small asset files, those errors also end up in your CPU, slowing it even further.

    Fourth is errors that cause DS to freeze, those are the main problem as it can take your computer a few minutes to get DS unlocked, only for it to run into more of the same.

    I'm not sure what you are trying to say here.

    Basically that a CPU clogged full of Software Errors, the software design, and poorly done content are all combining to cause long load times.

    First up is the programs running in the background, a Selective Startup would help a little, but how many people here know what that is let alone how to use it.

    Second is the design of the software, tbh using thousands of little asset files is pretty much ingrained into a software designers DNA, so not much chance of that changing.

    Which leaves the content, and the part I'm trying to skirt around as I'm likely to get myself banned if I'm not careful.

    To give you some idea of the problem, in the store there is a product for G8F, I believe it's been there for a couple of years now, it's a morph pack rather than a character. This product has two "broken" morph asset files in it, by broken I mean DS isn't creating the slider or any of the ERC in those files, which means other assets in the product aren't working correctly either. The problem, DS is loading G8F but the parent data in those assets is looking for G3F, result instant failure to create the modifier, the worst part is that the log tells you those files aren't working, yet neither the vendor or DAZ QA have bothered their backside to check it, or they seen it and didn't have a clue what any of it meant.

    Then there's the products with corrective morphs for every morph pack and character with an 8 in the name, that's fine for the small minority who own all those products with an 8 in the name, for the rest of us it means a log file bloated with error messages, and extended load times.

    Things need to change before people get P'd off enough to take their money elsewhere.

    ps I don't have any of these products, but I am used as Customer Support by my friends, as a result I get to see the issues as I'm normally the person that ends up fixing the problems for them.

  • PM me the names of the actually erroneous sets and if I hve them I will look at them, and raise a ticket if need be. I am pretty sure not beng bothered isn't the issue, it would help if you would refrain from comments like that.

  • RD2ARTRD2ART Posts: 28

    I have committed on this issue before, and the only solution I have found to loading a character quicker was to remove morphs in the "data" folder and add them as you need them. It is a little of a pain, but your load times are faster and the RAM that each character uses is less. I don't know if this is caused by the number of character assets/morphs in the data folder or if it is because of they way I have my content library setup. I only have the character morphs that I use alot in the data folder and as I need another one I will add it before I launch a character. If you add it after you launch a character you will need to save your work and restart the program. The character will then add the morph the next time you load the scene, which isn't a problem because you can still pose and work with the scene and then correct uses later when you reopen the file.

  • BejaymacBejaymac Posts: 1,886

    PM me the names of the actually erroneous sets and if I hve them I will look at them, and raise a ticket if need be. I am pretty sure not beng bothered isn't the issue, it would help if you would refrain from comments like that.

    PM sent.

    The other product was more of a generalization, basically any product with correctives for other products will create errors if that other product isn't installed.
    Simple example of this is Vicky 8, she has a bunch of correctives that work on the morph assets that came with the starter essentials, but she also has a bunch that need the Head, Body and Expression morph packs. Until recently I didn't have any of those morph packs, still don't have the Expressions, so she generated twelve errors for me, I'll let you figure out how many errors that would be if I owned every G8F character DAZ made, but still didn't have the morph packs.

    A "cover the ass move" would be to go through every product and add a required products to anything with correctives.

    I'm not saying these are "bad" and causing the long load times because they aren't, but they are slowing down DS and your computer, so the more you have the slower things get.

    The long load times are down to nastier errors, basically they are causing DS to go "not responding", the more of these you have the longer it takes DS to recover, add in a boat load of the other errors to slow down DS and your computer, and it's no wonder some are talking about 30 minute load times..
    Main problem here is despite all the people complaining about long load times, they don't come back and shown their log file, so even I have no idea WTF is causing them although I do have some suspicions. Three "Failed to create alias" has locked up DS for 4 minutes for one user, problem here is the log doesn't tell which alias file it crapped out on.

  • Sensual ArtSensual Art Posts: 641
    edited April 2020
    TheKD said:

    If you really want to use a "cleaner" G8 with injection-like behavior, it is possible to do manually. Set up a library with just what you consider bare essentials g8 stuff installed, probably be a good idea to keep "base pro characters" installed in there. Create and map another "temporary" library, and drag drop the morph sets you want to use from the product zip for the current artwork into there before openning DS and starting to work. The problem with that system, is you would have to know or make note of what to reinstall in the "temporary library" if you wish to revisit an old scene you set up. Perhaps a notepad file kept with the saved scene listing what is needed. I do this method, but with props, scenery, vehicles etc etc.

     

    RD2ART said:

    I have committed on this issue before, and the only solution I have found to loading a character quicker was to remove morphs in the "data" folder and add them as you need them. It is a little of a pain, but your load times are faster and the RAM that each character uses is less. I don't know if this is caused by the number of character assets/morphs in the data folder or if it is because of they way I have my content library setup. I only have the character morphs that I use alot in the data folder and as I need another one I will add it before I launch a character. If you add it after you launch a character you will need to save your work and restart the program. The character will then add the morph the next time you load the scene, which isn't a problem because you can still pose and work with the scene and then correct uses later when you reopen the file.

    The problem with those approaches is that one has to manually maintain a directory of morph product names(for connect), filenames(for DIM/Manual) and morph display names which is a non trivial task where the benefit does not necessarily outweigh the effort. And more importantly, you lose the capability of searching the morphs by name in the content library. Well, may be, the CMS may still have those entries but the file mapping would be missing if their paths are not included in the loaded content library so you may see empty useless placeholders assuming search works in the first place.

    On the other hand, injection based morphs enable one to benefit from full content library search for morph files without requiring maintenance of an external directory and without incurring the latency overhead while loading characters.

    Post edited by Sensual Art on
  • SnugginsSnuggins Posts: 52
    edited April 2020
    TheKD said:

    If you really want to use a "cleaner" G8 with injection-like behavior, it is possible to do manually. Set up a library with just what you consider bare essentials g8 stuff installed, probably be a good idea to keep "base pro characters" installed in there. Create and map another "temporary" library, and drag drop the morph sets you want to use from the product zip for the current artwork into there before openning DS and starting to work. The problem with that system, is you would have to know or make note of what to reinstall in the "temporary library" if you wish to revisit an old scene you set up. Perhaps a notepad file kept with the saved scene listing what is needed. I do this method, but with props, scenery, vehicles etc etc.

    Yes, I copy and paste the "missing files" list for an old scene and someday will create scene or scene family library folders. Great idea.

    Post edited by Snuggins on
  • TheKDTheKD Posts: 2,677

    Yeah, but the whole point of genesis was the whole gene pool thing. I don't see daz messing with that system and I would probably be pissed if they did. I am a huge morph combiner. I rarely every use a character out of the box. Mix and match would be a lot more of a hassle if you have to chug through a massive library and inject everything you want to use. I think in the end it waould take a lot more time doing that, than waiting a few minutes to load up the fully loaded people.

  • Sensual ArtSensual Art Posts: 641
    edited April 2020

    Even I mix a lot of morphs in my characters. Instead of searching for those morphs in the parameters pane (for which we are paying with loadtime latency), with injection based morphs the same morph search can be done in the content library just like any other content so long as their metadata is available in the CMS. Statistically speaking, even if I combine 50 morph dials to define my character that would be less than 2% of all the morphs that are getting preloaded while loading each character in the scene. While the current approach works if one has only a few morph sets or characters installed, but it does not scale up with increasing number of morphs, expressions and characters that are being incrementally installed over a period of time. The approach for preloading every applicable morph unnecessarily is against the software engineering best practices such as YAGNI (you aren't gonna need it) and in this case, we are paying the technical debt of increased load times for not following it.

    Even if you consider the tiny additional delay of searching and adding injection morphs from the content library compared to directly dialing them in the parameters pane, the one-time delay more than pays for itself considering how many times a scene gets reloaded during its entire lifecycle spanning several days or even weeks of work.

    Post edited by Sensual Art on
  • Sensual ArtSensual Art Posts: 641
    edited April 2020

    Also when you look at it from the perspective of the Genesis generation lifecycle, the past generations (1, 2 and 3) had shorter lifespans than the current G8. At the risk of presuming that Daz is exploring a horizontal expansion model (more characters/morphs releases per prolonged generation) from its earlier vertical expansion model (fewer characters per 2-year generation), I would say that the current morph loading strategy does not scale with the horizontal expansion model. A sustainable strategy would be to preload only a "minimum viable set" of morphs and inject the other morphs on demand basis. In fact, the whole premise of the Genesis base is to provide the minimum viable model, a concept which needs to be extended to the character loading strategy as well.

    If I dig into past discussions, the only users who do not seem to be inconvenienced by the issue of loading times have more than 32 GB RAM. I find it ridiculous to justify that requirement to load just a 17K poly G8 character in the scene or open a scene with a G8 character. Now, most 3D professional may already have 32 GB RAM but for enthusiasts like me, even with 16 GB RAM its hard to justify a reason to upgrade when other modeling softwares like Zbrush and Blender can load these models within a fraction of time and resources that DS is demanding just for loading a 17k poly model. It seems like putting the lid on inefficient design by throwing more hardware to solve the problem.

    Post edited by Sensual Art on
  • mrinal said:

    Even I mix a lot of morphs in my characters. Instead of searching for those morphs in the parameters pane (for which we are paying with loadtime latency), with injection based morphs the same morph search can be done in the content library just like any other content so long as their metadata is available in the CMS. Statistically speaking, even if I combine 50 morph dials to define my character that would be less than 2% of all the morphs that are getting preloaded while loading each character in the scene. While the current approach works if one has only a few morph sets or characters installed, but it does not scale up with increasing number of morphs, expressions and characters that are being incrementally installed over a period of time. The approach for preloading every applicable morph unnecessarily is against the software engineering best practices such as YAGNI (you aren't gonna need it) and in this case, we are paying the technical debt of increased load times for not following it.

    Even if you consider the tiny additional delay of searching and adding injection morphs from the content library compared to directly dialing them in the parameters pane, the one-time delay more than pays for itself considering how many times a scene gets reloaded during its entire lifecycle spanning several days or even weeks of work.

    Injection poses, at least as implemented in Victoria 4 etc., still need to load the channel list and ERC links for them (for exactly the same reason as the Genesis figures do) and they are still slit across multiple files (all those pz2s in /Runtime/Libraries/!DAZ/Victoria 4/ etc.).

  • Sensual ArtSensual Art Posts: 641
    mrinal said:

    Even I mix a lot of morphs in my characters. Instead of searching for those morphs in the parameters pane (for which we are paying with loadtime latency), with injection based morphs the same morph search can be done in the content library just like any other content so long as their metadata is available in the CMS. Statistically speaking, even if I combine 50 morph dials to define my character that would be less than 2% of all the morphs that are getting preloaded while loading each character in the scene. While the current approach works if one has only a few morph sets or characters installed, but it does not scale up with increasing number of morphs, expressions and characters that are being incrementally installed over a period of time. The approach for preloading every applicable morph unnecessarily is against the software engineering best practices such as YAGNI (you aren't gonna need it) and in this case, we are paying the technical debt of increased load times for not following it.

    Even if you consider the tiny additional delay of searching and adding injection morphs from the content library compared to directly dialing them in the parameters pane, the one-time delay more than pays for itself considering how many times a scene gets reloaded during its entire lifecycle spanning several days or even weeks of work.

    Injection poses, at least as implemented in Victoria 4 etc., still need to load the channel list and ERC links for them (for exactly the same reason as the Genesis figures do) and they are still slit across multiple files (all those pz2s in /Runtime/Libraries/!DAZ/Victoria 4/ etc.).

    Can you help me understand the reason why there is a need to preload the channel list if not for populating the parameters pane while loading a character? The existing injection implemention in V4 generation may not be the ideal approach to tackle this, but injection, as in on-demand loading of morphs is still there within tools like the Morphloader.

  • The channel list and links are needed so that when a property is set other linked properties are also set correctly.

Sign In or Register to comment.