Connect Library Size Doesn't Add Up
brucek5
Posts: 39
I'm starting to hace hard disk capcity issues and found the DAZ Connect Library being the largest directory on my drive, over 700 GB. I went into the DAZ Instal Manager application, Installed tab. It states I have 666 products installed. Only 12 products are 1 to 1.5 GB, and only 20 are 500 to 999 MB. The remaining 600 items are 1MB or much less. This clearly doesn't come close to adding up to 700 GB.
About a year ago I performed a fresh install of DAZ and contents, this included deleting the DAZ Connect Library directory. When I downloaded all my content using the DAZ Instal Manager application, I noticed in the DAZ Studio my content had an icon indicating it needed to be installed, Thinking I need to "install again" within DAZ Studio to use my content, I selected all my content and had it install. I took hours to complete that I thought was strange since the DAZ Instal Manager application already spent hours "installing", I resently purchased additional content and only used the DAZ Instal Manager application, and nocticed even though DAZ Studio indicates it needs to be installed, the content was still fully usable. Is DAZ Studio duplicating the content?
How do I fix this? Is there a tool that "cleans up" the DAZ Connect Library?
Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
Yes, Studio is duplicating the content. Uninstall everything you installed with Connect. If you install everything with DIM, there is no need to use Connect. In fact, there is no need to log in from Studio other than to use Connect. Go to Preferences and remove your credentials and uncheck the option to log in automatically. The icon shows that the product is available to install, not that it must be. DAZ has been very vague on the concept.
Hi NorthOf45,
I did as you suggested and now I'm down to 450 GB. But now, all my content is all gray scale, and when I double click on any product, it wants a log-in. I tried one by giving it my log-in, and it re-loaded it. In addition, there's a disconnect between the product count in the DIM (660), and in DAZ Sudio there was only 416 items to unintall.
This what what it looks like in DAZ Studio (picture attached)
Which tab is that? Smart Content, yes, but Installed or Available? If you load, say, a figure, does it show compatible items (Filter By Context ON)? What about the Content Library? Does it show all your content, and does it work?
That looks like DAZ Connect still has control over your metadata
Hi All,
I've attached larger captures, Notice the tab "Installed" the items (only 29) are in color, and does alow me to use the producr. These are the items I didn't install in DAZ Studio, only DIM. The available tab has all the gray icons that I just in-installed within DAZ Sudio and wants me to install them to use them. DIM is unchanged and still shows 666 intems installed.
No, it looks as if the installed-with-DIM/DazCentral content is not in a mapped directory and so is not being found. (Of coruse the Connect data is still available, otherwise it wouldn't be showing the download arrow, but if it was seeing the other files the icon would be in colour and usable.
See additional captures from DIM and DAZ Studio. You can see the path to content is the same. Plus my directory file size won't have gone down if they were in two locations,
Your Base Content directory for DIM is pointing to the "E:/Documents/DAZ 3D/Studio/My Daz Connect Library" folder. That is also where Connect is installing the content. Not a good idea, but I do not know if the two will co-exist, or if Connect deletes anything it didn't put there.
You have three folders configured for DAZ Studio Formats and Poser Formats in Content Directory Manager. The first one of each is what you probably should be using for DIM installations.
You will have to explore the folder contents outside of Studio to see where things are, or use DIM to locate the installed files. Right-click any product in the installed tab, select "Show Installed Files..." The popup will show the base folder, then the relative paths of all installed files as blue hyperlinks. If they are plain italic text, they are not where they are supposed to be.
Thanks NorthOf45 for all your help.
I guess I’m not grasping the whole “content” terminology. I thought that when I purchased products from “DAZ”, and installed it, I was adding “content”. But this forum thread reads to me like DAZ Studio and the DAZ Installation Manager were created by two different companies doing there own thing on how and where products are installed with no visibility of each other. I thought the “Smart Content Tab” was simply a different means to display and organize and group products purchased and the “Contents Tab” was more of a “raw” way to see all the induvial product files. I had no idea that they are completely different systems, hence why I had most of my purchases installed twice using up hard disk space because DAZ Studio was displaying an icon it needed to be installed. But stupid me, that’s the other DAZ company written software.
As far as the directory’s paths setup in both DIM and DAZ, they are what they were when DAZ installed. I changed them both to same directory, because I logically thought content was content. I created the “My Library” path to DAZ Studio so it could find products I bought elsewhere and simply came as files and no installer information. This I understand, that DAZ doesn’t want third-party products messing with DAZ’s data files.
I attached a capture of a DIM install product file location. I checked multiple products and they all have blue hyperlinks. In addition, when I go to the “Content Tab” in DAZ Studio it looks like all my products are there and usable, just not very organized and hard to find anything.
I’m not sure what my options are to enable the use of the organized “Smart Tab” and how the file paths need to be different for the two DAZ company software products (DAZ Studio and DAZ Installation Manager) and what software to use in the future. I’m guessing you’re going to say I’ll need to delete everything, and re-download all 650 products to restore the “Smart Content’s” data file?
Again, thanks for your help, it’s just very frustrating and makes no sense, unless I’m missing something.
Bruce
The "Smart" part of the Smart Content is derived from the metadata. The metadata can come from any of the installation methods (Connect, DAZ Central, DIM), they are all supposed to be identical. DAZ added the ability to install directly inside Studio, clearly their preferred method, but many prefer DIM. There will always be dissenters to any of the methods, but that is another kettle of fish. You are not wrong in believing that Smart Content and Content Library are different ways of displaying the same information, but it is not explicitly clear. DAZ has been rather vague on that point, whether by design or omission. You do not have to install content from Studio to use the Smart Content system. (Again, there are those who would disagree, and with some merit, it is not foolproof).
Third party data isn't necessarily unwelcome, as long as it follows the conventions used by DAZ vendors (which is sometimes a problem anyway). However, it would be a good idea to keep it separate just in case. By default, those products have to be installed manually unless you make a DIM-compatible package (like with Content Package Assist)
The content is smart if there are compatibility relationships established between items, like a figure or prop and material presets or preset poses. With the figure selected in the scene, Smart Content would show you only those products and files that would be compatible with it. Sometimes the net is pretty wide, but you can narrow down the results by drilling down in the categories on the left side.
If you are sure everything is installed properly, you can re-import the metadata from the Content DB Manager -> Re-Import Metadata... (NOT in Smart Content, that will go and use Connect). Select all products and accept. It could take a while to complete. This action pulls in the metadata from the DIM-installed manifests for the products. Hopefully you won't have to re-download and re-install everything. My own personal concern is leaving it in the Connect folder. That might mess with Studio's head, thinking you should download it when using that folder. The content can be moved to a different folder with little effort (really, just move it) without breaking anything. (Well, almost nothing. There is a path in each product's manifest that says where it was originally installed, but that can be edited to point to the new location with a batch replace using something like notepad++ in 5 minutes).
You can still have more than one library folder, just remember that Studio scans them in the order they are defined looking for files, taking the first one it finds (in case of multiple versions).
Thanks NorthOf45,
What would DAZ normally use for a directory name for items installed with DIM if I move everything?
BTW - I'm assuming no one has written a script to "re-create" the "Smart Content" file, SQL based database or whatever it's using?
"Normal" would have been the original folder path, but that doesn't really matter. First, determine which drive you want to use. Space will get used up, and you don't want to move it too often. I use a second internal drive with nothing but the DAZ Studio libraries. The name of the root folder doesn't really matter, but since you have "E:/Documents/DAZ 3D/Studio/My Library", you could use that, unless it has your third party content. Maybe "E:/Documents/DAZ 3D/Studio/My DAZ Library"? It is important to maintain the same relative path, so, in a file explorer, just create the new root folder, and drag the top content folders (Animals, Architecture, Environments, People, etc.) onto it. Do one or two small ones at a time to make sure you get it right. You'll know if you see it in the Content Library tab pane (refresh the view first, and don't forget to add that new path to the Content Directory Manager).
The CMS database uses the metadata from the installed products, so if it is really borked, you can reset the database and re-import all the installed products' metadata (Content DB Maintenance - Reset Database, then Re-Import Metadata...). Nothing else to do, really, but wait for it to finish. Studio will use the database to figure out the compatibilities, or at least, what the metadata gave it.
If one is using non-default disks, there is no reason to make the path too long.
If the Content library is put on the E-drive, the root-directory could be something as simple as E:\My Daz3D\
So to make sure there's no miss-understanding, the directory name "My Daz Connect Library" should be reserved for "Smart for Content", and anyother name I come up with should be used for DIM installed content? And a thrid direrectory name of my liking used for my third-party content?
Or, is just "My Daz Connect Library" and only use one install method? I'm debating on just re-installing everything.
It's not about the name of the directory, it's about the whole path.
Whatever path that is specified as "DAZ Connect Data" in Content Directory Manager, would better be left for just DAZ Connect installations.
Save the "My Daz Connect Library" folder for Connect installations, which you will likely never use.
You shouldn't need to re-install everything if it is already installed somewhere else, just move it, reset the database and re-import the metadata.
Don't use the terms "Smart Content" and "Connect" interchangeably, they are not the same thing.
Sorry for bugging you guys on this subject. It wasn't until Northof45 last post that I noticed the directory had the word "Connect", I've been seeing it (blindly) as "Content", So is "Connect Installtions" what I've been calling "Smart Content" installs perfromed within DAZ Studio? If so, is the proper name for installing using DIM (DAZ Instal Manager) called "DIM Installtions"?
Yes, effectively. Smart Content is how Studio handles the compatibilities and the relationships between items, as defined in the metadata. Connect is a method of installing the content directly from within Studio. In the end, the content and metadata are the same whichever method is used. Connect's main advantage is the ability to update individual pieces, unlike DIM, which must update the entire product package for even the most trivial change. Thankfully for DIM users, that doesn't happen too often, but is that because there are no errors to be corrected, or are the errors are not being corrected? Are there no errors reported, or are there too many to handle? That is a subject for another discussion.
NorthOf45, I a comment you made above "The content can be moved to a different folder with little effort (really, just move it) without breaking anything. (Well, almost nothing. There is a path in each product's manifest that says where it was originally installed, but that can be edited to point to the new location with a batch replace using something like notepad++ in 5 minutes)."
1. Where are these files located?
2. Is there any special Notepad++ text formating setting I need to know so the files don't become unreadable?
3. What do I search for, the entire root path "E:\Documents\DAZ 3D\Studio\My Daz Connect Library" or is a relitive path?
4. Do the paths use double back-slash "E:\\Documents..." or single and are they forward slash "//"
Or do you know off an article or posting on performing this task?
The content is referred to by relative path, so you can change the root folder to anything, or anywhere, you want, as long as you make the same change in DIM (Advanced Settings -> Content Path Shortcuts, add or change to the list below) and Studio Content Directory Manager.
The content in your original DIM installation path (under "E:/Documents/DAZ 3D/Studio/My Daz Connect Library" that is not Connect installed, i.e., 1_*) can be moved to the new root folder directly in a file explorer. If you are not comfortable moving, you could always copy first, then delete it after confirming it still works from the new location (after adding the new path to Studio).
When DIM installs a product, it creates a record of that in the Manifest Archive (DIM -> Advanced Settings -> Installation). All those manifests (.dsx files) contain the information about the product, installation date, installation location (absolute path), user, etc. It is an XML file, which is simple text and human readable. The location can be changed for future use, and the manifests themselves moved there now. I keep them next to my Library folder and back them up, just in case.
The line of interest in the manifest is:
<UserInstallPath VALUE="Drive:/LibraryPath"/>
Change the "Drive:/LibraryPath" to your new content library. With Notepad++ use Search -> Find In Files, put the exact text from the .dsx file in the Find What: box, the replacement text in the Replace with: box, then Filters: *.dsx, and the directory to search. Make sure search mode is Normal, not Extended or Regular Expressions. Try a few manually to get the idea. This bit of info is used when DIM refreshes to see what is actually installed on your system. If not found, the product will appear in the Ready to Install tab, or, if you did not keep the .zips, in the ready to download tab.
Hi NorthOf45, Thanks for spending all this time to help me!!! Wish me luck.
Happy to help. Hopefully others can use it, too. Let us know how it goes, and if any procedures need to be corrected (it has been a while since I moved content around).
Hi NorthOf45,
Good news! Installed a new 8TB SSD and copied the all my DAZ products to a new directory and renamed the old directory (to backup just in case) so the DAZ application couldn't find it. See pictures of DIM and DAZ Studio path settings.
1) Did the Notepad++ thing, ran the DAZ Studio DB Reset and DB Import.
2) Restarted DIM and everything shows installed.
3) Restarted DAZ Studio, and all the Smart Contents shows installed (Color icons and no install icon) and grouped by products and files (see screen capture).
4) Deleted all the old direcories to make sure none of the DAZ apps were finding them. Everything still works.
One thing I can't explain is what made my content directory drop down to 150 GB (which adds up). I randomlly loaded different products and all the contents loaded.
I started with 700 GB and now I'm down to 150 GB. There must have been a bunch of duplication of files somewhere.
Do you see any problems with my path settings (even so it looks like everything is fixed). The new "My Daz Connect Library" directory now only has a directory "Data", with a "Cloud" inside it which is empty. The directory "My_Daz_Private_Library" is where I put products not from DAZ that are manually installed. Did you indicate there was a software tool or process that would enable me to install products that don't have installers, so I could get them organized?
Thanks again. Bruce
Congrats! Looks good! All I would do is make the Poser Content libraries use the same folders as the DAZ Studio Formats. You might not have any Poser content now, but some day you might. Nothing wrong keeping it all together. You might want to turn off "Create Shortcuts", that could clog up the destinations pretty quick (not even sure if that still does anything, or maybe use it for third-party tools and utilities that operate outside Studio). You can see how much space DIM has used up by selecting all Products from the Installed tab. At the bottom of the window there will be a line that says "Selected: nn (Disk: xxx.x GB / Pkg xxx.x GB / Total xxx.x GB). Might even get into the TB.
As for a tool to make DIM-compatible installers, there is Content Package Assist, available here, or, I have seen some freebie applications that basically do the same thing (well, one). I can say CPA works very well, but cannot speak for any others. You can even make the DIM thumbnail files by naming them the same as the Product ID.
I use a different DIM account for non-DAZ3D sources (dummy e-mail, not that it matters; always offline, because it can not use any other stores anyway), and a different archive folder. I install to the same library, but you can use a different one, that is not critical. I use a different account because, even if you have all the content path shortcuts defined, you will forget to change it sometimes, and, to keep the archive packages (downloads) separate. Depends on how organized you want to be.