Export user data ignores many items
Psycho
Posts: 22
Hi there. I've found exporting/importing my custom categories to be very unreliable: it claims to back everything up and then to restore it again successfully without any error messages...yet when I browse through my categories, they're missing huge amounts of the material that was in the original folders. I'm not 100% sure, but from what's missing I'm wondering if it's renaming items that breaks the backup/restore. In other words, if I rename something in the original Daz 3D content library and then "categorize..." it into one of my folders (and it seems to be necessary to do it that way, because once it's categorized the software won't allow me to rename the reference in my custom folders), it seems that this causes the backup/restore process to (quietly) fail. It claims to succeed, but much of the (renamed, I think, although there could be some other cause) content just doesn't appear in the restored folders.
Does anyone know if there's a workaround for this? For example, are there any files or folders (in Windows) or chunks of registry that I can copy directly from one PC to another, so that my custom categories are restored accurately? It would take days of work to recreate everything manually: from some quick early tests I assumed the backup/restore feature was working, so have invested many, many hours into organizing content only to find that the restore process ignores large chunks of my library.
Daz 3D is part of 
Connect
DAZ Productions, Inc.
224 S 200 W, Suite #250
Salt Lake City, UT 84101
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
Sorry to hear about your nasty surprise.
Process that works for me is categorize from DazStudio Content area to categories "as is" and then rename in categories section. Sounds like you are using the folders that already exist in categories? Suggest you send up a whole new directory with subdirectories right at root of categories that makes sense for how you use it. Then this will be unlocked and you can edit names as you want. If you want to unlock what you have copy and paste to a fresh new folder you make. You'll see the folder icon change. I learned that the hard way when I restored my stuff. And that worked well, so the preceeding suggestion worked 100% for me. Was going to write DS a ticket asking if there is a magic way to unlock, without copy & paste, but for reasons did not. Was a pain to do that for 40 plus subdirectories. But worked without issue, other than time.
With you having renamed the DS content folders themselves, before categorizing, I really don't know.
Thanks Saxa. The most important thing for me is to have a process that works going forward: I can live with re-doing days of work one time, if I never have to do it again. Unfortunately your suggestion doesn't work for me: I wonder why it works for you? I can rename content while it's up in the "My DAZ 3D Library" folder structure, but as soon as it's categorized into one of my (non-standard, editable) custom folders, the "rename" option disappears from the context menu. Note that by "content" I mean actors, props, poses and so on: it allows me to rename the folders into which I organize my content as many times as I like...but the process for renaming the content itself appears to be rename first, categorise later. I'm guessing that's what's breaking the backup/restore...but the fact that everything works perfectly except for the process of transferring it to a new system means that there must be some way to get it working on the target systems too. I'm new to DAZ so have no idea where the information linking renamed icons to original content files might be stored: if anyone can point me to that, I can try copying the relevant files/registry sections/whatever across. Maybe a CMS database file/tree somewhere...although I'm reluctant to mess with that stuff as the install manager works perfectly well for re-installing all the content on the new machines: it's just the organization of it that seems weirdly difficult to backup and restore.
Ahhhh. OK. I misunderstood what you were renaming. Never organized content that way. And sounds like it doesn't work. Sadly for you.
My process is to create a network of new subfolders with names that mean something, but I never rename the product itself.
The new folders sorta take care of that.
Kinda like slotting them into a bin that I can easily find. This way original names don't need to be renamed for me. If that makes sense to you. On top of that use Visual Menus by bitwelder to make many of them accessible with quick hotkey menus and nested submenus, and in VM you can rename the product itself.
To answer your other question.
After "Exporting User Data, go to MainDazContent\Runtime\Support, and you'll find userdata_1.dsx, and maybe userdata_2.dsx if you have alot of content. Open this with notepad, and you'll see the xml listing of how the categories info is being stored.
CMS database is another matter. Probably for good reason to give a measure of protection for their content.
Yes, I wondered if you were thinking of the folders. I find renaming the content itself just as helpful, because there's no consistency (e.g. some artists write themselves into the product names, others don't; some specify what generation it's for or whether it's HD or whatever in the name, others don't; etc., etc.) so even the simple thing of having them listed in alphabetical order (within each folder) doesn't really work without some tweaking...basically I'd rather get everything well-organized up front and have a smooth workflow later on, than leave it in the chaotic mess that it arrives in, only to have my workflow interrupted by annoying frustrations later on. Search is useful but not enough: I like to be able to browse through sensibly organised folders of sensibly (to me) named stuff.
Thank you for the Visual Menus suggestion: that's great to know as I've had a look and it will definitely do what I need, if there's no easy fix for the problems with the built-in folders. In fact I've been wanting to customize icons too so I'm tempted to shell out for Visual Menus (I've already set up hotkeys for specific folders so it's really just the ability to customize content names and icons that appeals to me)...but wow, it would involve an enormous amount of work to convert my thousands of content items into those menus manually: or can you do something similar to the "categorize" context-menu item, turning Daz content folders automatically into Visual Menus? Also, can you confirm that *that* method of organizing content, at least, does successfully backup and restore (i.e. I won't waste more days organizing everything into Visual Menus only to discover that it's just as bad, or worse, when it comes to reinstalling that method of organization on a new machine)?
Am same way at organizing stuff so there's minimal chaos later when creating.
Failed to mention, that I rely on the images way more than on the names.
And a really nice thing is that DS will let you get away with about 800px images before it starts to scale them down. Or it does on my system. So made many new images. When hovering over any thumbnail image it scales up to whatever size I saved it (up to about 800px) in DS in categories or wherever (sadly not in VM).
Also, find when I customize my products ahead of time with mats and extra morphs assigned, then save them to a new totally separate root directory, where change the name of the product itself to something more meaningful. Down side is you have the asset taking up twice as much space being fully saved in a totally new directory (like you should for 3rd partystuff). But harddrive space is cheap these days. :) But asset is more ready to go.
As for VisualMenus.
When I reset CMS and reimported Userdata, the Visual Menu stuff was just there. Nothing changed.
One really cool thing with VMs is you can make your own UI's using QT (see DAZ Studio intsall dir for exe to get started). So you can create a ui that faciliatates organizing the one button click layout that works for your workflow.
Here's an example of Visual Menus and images primarily for individual poses. This is a nested submenu part of a network of menus for Poses. The smaller boxes are the mirror'd poses in most cases.
Visual Menus is a life*saver. :) And I'm not a shareholder. lol. Just a happy user.
OK, from forum discussions and videos about VisualMenus it looks as though (a) they are basically stored as scripts that can be backed up with an "actions" export (and so presumably restored on new installations), and (b) it's not as arduous and time-consuming as I'd imagined to set them up. Thanks again for making me aware of this product: I'm going to put some thought into how best to arrange my content so that it can be transferred reliably to new installs (before investing the days of work it's going to take me, either way, to set it all up again), but I'm probably going to purchase VisualMenus regardless, because even if I stick with the categories system for organizing most of my content, VisualMenus looks like a great way to speed things up for things I use a lot.
If the asset's location (relative path or the fiel name) does not match that in the database then the link between asset and file is broken. If DS can't resole the path when importing metadata (that is can't find the named file in the specified location) then it will skip the asset entirely, rather than create a bad entry.
OK, that explains what I'm seeing. I think what's happening is that the "rename" function doesn't do the renaming as a form of user data that's backed up and restored along with the category information, but instead makes the changes right down at the level of file names. As a result, my backed up categories are stored as a collection of links to renamed files...and, since these files have their original names on freshly installed systems, this effectively means that all my user data related to any asset I've renamed is lost. This seems to me to be a very weird design (i.e. why would it not keep the .duf file identically named, so that everything works as expected, and simply rename a pointer to that in the user data so that the UI reflects the user's edit?)...and even weirder still is that, from a browse through the "My DAZ 3D Library" files (under "Public Documents"), it looks as though the files have actually been duplicated rather than just renamed (so for example if I've renamed something like "ABC Bob HD for Genesis 8" to "Bob", there are now two .duf files in the folder, both around a MB in size, one with the original and one with the new name!). If this is how it's meant to work rather than the result of something I've done wrong, it seems quite clumsy and inefficient to me (I had no idea I was making my data take up more disk space too, simply by renaming stuff)...but I assume there's a good reason for it.
Anyway, I'm hopeful now that simply copying across the whole "Public Documents/My DAZ 3D Library" tree (well over 100GB of data), and then re-importing the UserData backup, will magically make all the missing stuff reappear on the new system. It's a very long way from being an ideal scenario, because it means that I always have to have a local copy of my content backed up somewhere (I can't rely on a fresh install from the Install Manager to work with a backup of my user data), but if it works I'll have a look at scripting a better solution. It should be possible to identify all the renamed assets programmatically and then backup just those files (probably less than a thousandth of the overall data), in which case it should be possible to realise my goal of being able to reproduce a Daz setup on a new installation from just a small efficient backup. I'll have a go today and see if the documents+userdata transfer works.
If you rename or move the file then yes, you are doing that literally - that's why Daz advises just recategorsing. Tracking file name/location chnages and reapplying them isn't really the job of a content database.
Well, based on the experience I've just had with broken user data backups, it seems to me that if content can be renamed at all by a user, then those changes should be tracked so that the whole system is aware of them and manages them sensibly. Not necessarily in the core CMS database, but at least as some form of metadata rather than as direct edits to filenames that aren't backed up when users attempt to back up their customizations. I can't think of any substantial technical difference between a UI's presenting a "category" that's stored as metadata that can be backed up, and presenting a "name" that's stored as metadata that can be backed up...so to have users' changes applied directly to asset filenames rather than simply to create custom names that are backed up and restored along with the categories doesn't make sense to me. However, I'm new to Daz and largely ignorant of how it works so accept that there may be scenarios I can't think of where it's important to apply users' name changes as filename changes rather than just as changes visible in the UI. Also, apologies: I missed the part where Daz advised against renaming...had I known that, I wouldn't have done it...perhaps I'd have found the VisualMenus product instead.
Fortunately, clumsy or not, the existing setup works: I can now confirm that the combination of restoring the UserData backup *and* copying the whole "My DAZ 3D Library" tree to a new system works as a way of reproducing heavily customized content on another computer...all my edits have reappeared in their appropriate folders, so the days of organizing everything weren't wasted...whew! Having invested so much time in doing things this way, I've decided to stick with it (now that it's clear I'll be able to reproduce it on new installs) rather than converting all my custom categories and names into VisualMenus menus and icons. Thank you for the help and advice along the way: I guess my renaming so much content could produce other issues later on, so it's not something I'd do if I were starting from scratch again...but for now it all seems to be working as I'd hoped it would.
Ha!
Congrats! Can imagine that was a huge relief!
Kudos too, that you persisted to find a way. Had a few DazStudio oddities where had to figure out of my own too.
Happy eventual creating.