Creating your own DIM install package?

Skiriki

Hi, so I was wondering -- now that there's DIM, which can keep track of what has been installed and where, and I happen to do lots of freebies...

...is there a way to do your own DIM-compatible packages? I think it might be handy for people who want to install and uninstall stuff. Just chuck a DIM-compatible .zip to right folder and Bob's your uncle.

I looked at the Manifest.dsx file, and basically it looks like all you have to do is create appropriate structure, but the puzzles me.

Am I able to put in empty value or, say, my name or something else?

Is there a howto in wiki or something? (I failed to spot it, if there is.)

Is it even OK to attempt non-DAZ created DIM-compatible packages?

Lissa_xyz

Good question. I opened up the IM******.dsx and Manifest.dsx from FM Jeans for Genesis just to take a peak. The Manifest seems to map the install paths in a relatively easy format (although a pain in the ass for lots of files), but I'm not sure what you would plug in for the GlobalID at the top of the file. IM******.dsx looks a little more complicated than the Manifest in that it has info regarding the order number, date, installer date, etc, and some other long ID that seems to come out of nowhere.

I wonder if they plan on coming out with a DIM installer packager to make it easy. *hint hint nudge nudge*

fixmypcmike

I would think you could -- the GlobalID is basically a checksum which DIM can use to tell if a product has been updated, but since there's no matching item in the store (assuming you give it a name like IM00099999-myproduct.zip), I don't think it would make a difference. The rest of the manifest is just install instructions for each file in the zip.

Lindsey

Global ID and .dsx (and .dsa) files in the support folder are the end product when you create your own metadata with the Content DB Editor. Here's DAZ's Adding Metadata to Products tutorial in the Documentation Wiki. There has been other tutorials posted in the forums. I'm sure HeraldOfFire will post a link to his forum tutorial.

Once you grasp the concepts from the tutorial and can associate cause and effect, you'll find yourself adding metadata mindlessly. Start with something simple like a hair product.

One benefit of DIM is that it creates an entry in Content Library>Products so that step is already done.

Skiriki

Grrrroooovy. :) I need to read all this and then package up my nice widdle freebie :)

wels13

you could create your own DIM to install and un instal your popes or figures , and put them all in a DIM file to instal and uninstall or redistribute to other users ,
but it would require quite and extensive work to prepare all what you need for the creation of such a DIM file
DAZ does allow users to become art publishers and provide a great detail ( a little complicated ) about creating your own product
this documentation is dealing with the steps of becoming a daz3d publisher with step by step instruction on how to create your bundle

http://docs.daz3d.com/doku.php/public/publishing/start

patrickwilliamwalker

The documentation is lacking and out of date bordering on sheer uselessness.

I've been trying to create my own DIM install files and can't seem to get them to work at all.

Failed to open file : /Volumes/Macintosh HD/Downloads/DAZ3D/IM00099999-01_Test.zip
Processed 138 installed products, totaling 7.8 GB
User Data Refresh: Elapsed Time 0 sec 335 msec
Product failed check, moving to next product : IM00099999-01_Test.zip
Install Queue finished in 0 min 0.0 sec.

I've tried playing with filenames, added stuff to the Manifest and support .dsx files. The only thing I don't have added is the GlobalID (removed the lines and have tried a zero'd out GlobalID as well). Can't get the globalID anymore because I can't find the Content DB Editor anywhere (used to be in the context menu).

I know it can open the zip file because inside the Runtime/Support .dsx file, I changed the product name and the product name inside the DIM description changes.

Totally stumped. I'm baffled as to why there needs to be TWO .dsx files in the archive that essentially duplicate the same information. Are both .dsx files needed for basic install? I don't know.

The only thing I can't eliminate is the built-in "compress" function in Mac making a zip file that DIM doesn't like, in the same way you have to play with the zip command line to get epub files to work.

LordScorpion13

Here's a way to check your manifest.. when you pull up DIM.. right-click on your product, and choose "show packaged files" if it's blank.. then there's an issue in the manifest. also.. if you have your /Support items .. the metadata (the *.dsa, *.dsx and *.jpg) then rewriting the manifest from that is simple..

patrickwilliamwalker

LordScorpion13 said:

Here's a way to check your manifest.. when you pull up DIM.. right-click on your product, and choose "show packaged files" if it's blank.. then there's an issue in the manifest. also.. if you have your /Support items .. the metadata (the *.dsa, *.dsx and *.jpg) then rewriting the manifest from that is simple..

So, it's blank, but all files that need to be installed are in there.

Could those hidden .DS_Store files be causing DIM to puke since they're probably in all those folders yet not in the manifest?

Okay, so I think removing the .DS_Store files did something because now it's saying the Manifest is corrupted or missing. Manifest is there so it's definitely the problem.

Can you create a DIM product without having to have Runtime/Support files? I gather then those aren't really needed for DIM but for D|S database entries.

LordScorpion13

as for the supplement.DSX.. that's really not needed.. makes the product listing in DIM pretty :-\

You can write a manifest without the metadata files (runtime/support) without a problem.. you won't have it in smart library though.

as for the Content DB editor...

LordScorpion13

DIM might be kicking out the manifest because it doesn't have the global ID...

let me PM you.

Scavenger

YES. IT IS POSSIBLE.
I do it.

I have hopes to universalize my method to put on sharecg..right now it's a bit code monkey and only works in one location of my computer, but my scripting friend and I are working to make it more open.

Basically, I put the files in a Content folder. I run the script which reads thru and spits out a manifest file in the proper format. Zip the two together and give it the name.

------------------------------------------------------------------

For Patrick: You're on a mac. The .DS_store files and the other mac invisible files WILL mess things up...

Don't use the Mac's built in zip program. YEMUZIP is a free zip program, that has a "PC" mode..it'll remove the mac stuff.

Also, don't use Mac's TextEdit for any editing of the manifest..it makes it unreadable. I use Text Wrangler.

patrickwilliamwalker

Scavenger said:

YES. IT IS POSSIBLE.
I do it.

I have hopes to universalize my method to put on sharecg..right now it's a bit code monkey and only works in one location of my computer, but my scripting friend and I are working to make it more open.

Basically, I put the files in a Content folder. I run the script which reads thru and spits out a manifest file in the proper format. Zip the two together and give it the name.

------------------------------------------------------------------

For Patrick: You're on a mac. The .DS_store files and the other mac invisible files WILL mess things up...

Don't use the Mac's built in zip program. YEMUZIP is a free zip program, that has a "PC" mode..it'll remove the mac stuff.

Also, don't use Mac's TextEdit for any editing of the manifest..it makes it unreadable. I use Text Wrangler.

Regarding zip, I use the command line (zip -r archive.zip archive -x "*.DS_Store") I broke down and bought YEMUZIP from the AppStore. Only $6.00. Had no effect.

Says that the Manifest is missing or corrupt. Using command line I confirm that the dsx file is actually plain text:

AA00000000-01_LBW patrickwilliamwalker$ file Manifest.dsx
Manifest.dsx: ASCII text

TextEdit can be used but you have to ensure that it writes out in plain text and not rich text. I use TextWrangler too, actually. Easier to have multiple files anymore.

I feel like I should write a program in Xcode once I figure out this stuff out.

Scavenger

I never explored if command line zip would work to remove the dsstore's.

My script is in Perl. It just makes the manifest. If you'd like to see it I can pop it onto Pastebin. I haven't messed with it in months, :( (I can run it on the folder with 2 cut and paste lines into terminal, so it hasn't been onerous enough to work on it further.

Ideally, I'd like an app where you point it to or drop onto it, a Folder that has the Content folder in it. It then generates the manifest file and zips up the file.

Being able to set the file in the right format (IM########-##_WhateverTheNameIsWithNoSpaces.zip) would be a bonus.

patrickwilliamwalker

Scavenger said:

I never explored if command line zip would work to remove the dsstore's.

My script is in Perl. It just makes the manifest. If you'd like to see it I can pop it onto Pastebin. I haven't messed with it in months, :( (I can run it on the folder with 2 cut and paste lines into terminal, so it hasn't been onerous enough to work on it further.

Ideally, I'd like an app where you point it to or drop onto it, a Folder that has the Content folder in it. It then generates the manifest file and zips up the file.

Being able to set the file in the right format (IM########-##_WhateverTheNameIsWithNoSpaces.zip) would be a bonus.

I suppose I could take a look at it but never did do much of anything in Perl...

I removed the globalID from the manifest and still no luck but getting a different error message this time from DIM's error log:

Product 0:IM00000000_LBW.zip has no tags

I also noticed when I try to export the product from DB editor to make the support DSX file, it's sized at zero bytes, seems to be because of the empty token field in the DB editor.

DAZ really needs to have better documentation for all this...

I don't think the problem is the Manifest.dsx because I've done over them with a fine tooth comb and used the DB editor dsx output into TextWrangler and did a find/replace to make it in Manifest.dsx XML format. Tried with, without and DAZ Studio generated GlobalIDs. I think there is some issue with ZIP (and or DIM itself constantly throwing hissy fits). I made an extremely simply 2 object "package" so there are only six lines in the manifest file. Saved the manifest with both CR/LF and LF formats. Didn't have any effect.

patrickwilliamwalker

I realized something last night to help figure out where my problems are.

What I did is take a download from DIM that I know works. I decompressed it then recompressed it under an altered filename. This pretty much eliminates the Manifest.dsx as the culprit.

So, it's pretty clear the issue is the creation of the zip file and DIM's inability to handle quirks in zip files.

I've tried YEMUZIP (v2.4.8). I'm using PC for type, with no compression (set in Preferences). I've been going through command line options as well.

After fiddling around for ages (and modern technology is supposed to free us from drudgery, am I right?), it's pretty clear the problem centres around the directory structure inside the zip file (as I suspected):

Installing product : Test
File: Content/People/Genesis/Poses/Test/Test.duf was in manifest but not found in zip : Test
File: Content/People/Genesis/Poses/Test/Test.duf.png was in manifest but not found in zip : Test
Product successfully installed : Test

I can't seem to win.

Twilight76

I did create DIM Install Packages with this tool.
Its a little older and no longer updated but it works :)

Windows Tool:
http://www.sharecg.com/v/75813/browse/3/YouTube-Video-Tutorial/Daz-3D-Studio-Making-Installer-zip-Package-Detailed-Version

fixmypcmike

patrickwilliamwalker said:

I realized something last night to help figure out where my problems are.

What I did is take a download from DIM that I know works. I decompressed it then recompressed it under an altered filename. This pretty much eliminates the Manifest.dsx as the culprit.

So, it's pretty clear the issue is the creation of the zip file and DIM's inability to handle quirks in zip files.

I've tried YEMUZIP (v2.4.8). I'm using PC for type, with no compression (set in Preferences). I've been going through command line options as well.

After fiddling around for ages (and modern technology is supposed to free us from drudgery, am I right?), it's pretty clear the problem centres around the directory structure inside the zip file (as I suspected):

Installing product : Test
File: Content/People/Genesis/Poses/Test/Test.duf was in manifest but not found in zip : Test
File: Content/People/Genesis/Poses/Test/Test.duf.png was in manifest but not found in zip : Test
Product successfully installed : Test

I can't seem to win.

And are those files in the zip?

patrickwilliamwalker

fixmypcmike said:

patrickwilliamwalker said:

I realized something last night to help figure out where my problems are.

What I did is take a download from DIM that I know works. I decompressed it then recompressed it under an altered filename. This pretty much eliminates the Manifest.dsx as the culprit.

So, it's pretty clear the issue is the creation of the zip file and DIM's inability to handle quirks in zip files.

I've tried YEMUZIP (v2.4.8). I'm using PC for type, with no compression (set in Preferences). I've been going through command line options as well.

After fiddling around for ages (and modern technology is supposed to free us from drudgery, am I right?), it's pretty clear the problem centres around the directory structure inside the zip file (as I suspected):

Installing product : Test
File: Content/People/Genesis/Poses/Test/Test.duf was in manifest but not found in zip : Test
File: Content/People/Genesis/Poses/Test/Test.duf.png was in manifest but not found in zip : Test
Product successfully installed : Test

I can't seem to win.

And are those files in the zip?

Yes, they are there. The differences are whether the paths are in the zip or not.

fixmypcmike

patrickwilliamwalker said:

fixmypcmike said:

patrickwilliamwalker said:

I realized something last night to help figure out where my problems are.

What I did is take a download from DIM that I know works. I decompressed it then recompressed it under an altered filename. This pretty much eliminates the Manifest.dsx as the culprit.

So, it's pretty clear the issue is the creation of the zip file and DIM's inability to handle quirks in zip files.

I've tried YEMUZIP (v2.4.8). I'm using PC for type, with no compression (set in Preferences). I've been going through command line options as well.

After fiddling around for ages (and modern technology is supposed to free us from drudgery, am I right?), it's pretty clear the problem centres around the directory structure inside the zip file (as I suspected):

Installing product : Test
File: Content/People/Genesis/Poses/Test/Test.duf was in manifest but not found in zip : Test
File: Content/People/Genesis/Poses/Test/Test.duf.png was in manifest but not found in zip : Test
Product successfully installed : Test

I can't seem to win.

And are those files in the zip?

Yes, they are there. The differences are whether the paths are in the zip or not.

I'm not sure I follow -- are you saying the path in the manifest is not the path inside the .zip?

patrickwilliamwalker

fixmypcmike said:

I'm not sure I follow -- are you saying the path in the manifest is not the path inside the .zip?

DIM finds the Manifest and appears correctly (when you do Show Package), but DIM says that the files pointed to by the entries in it cannot be found in the zip when you go in and look at the error log. Which leads me to believe it's the folder structure causing problems.

I think I may know a solution but I'll try it tomorrow. I'm wondering when I use zip on the product's parent folder, if that's adding that layer to file paths inside the zip (it doesn't appear to be doing so as the extra layer isn't happening when I decompress the zip file). All that would mean is that I just have to recursively zip up the content folder and make sure the manifest.dsx gets swept up too. Just way too tired at the moment to try. This probably explains why the Manifest.dsx file can only be seen when I use the -D or -j options in zip (removes directories and junks file paths) while DIM can't find the files. If there is an extra portion added to the path, then DIM won't find the file.

I shall see tomorrow. :)

patrickwilliamwalker

Yes, that was the problem. As soon as I change the target from the folder contain Content and Manifest.dsx to Content and Manifest.dsx directly, things seem to not give errors.

Now, to figure out an easier workflow for it.

Here's the quick-and-dirty Unix sh script I'm using on Mac OS X 10.10.3. It generates the manifest file as well as zipping up the content into a new Archive.zip (it deletes any older version) with a new Manifest.dsx (any existing will have their contents overwritten).

#!/bin/sh
echo "______________________________________________"
echo "\nManifest Generator Script, v0.0 (29 Jun 2015)"
echo "Written by Patrick Walker, [email protected]"

# This will overwrite any existing Manifest.dsx present and designed to be this way.
# For use in Mac OS X.  Ensures that .DS_Store files do not appear in archive.
# Generates manifest, then zips files up into archive.

rm -f Archive.zip  # If file doesn't exist, don't squawk about it.

find Content -type f \! -name "*.DS_Store" > files.txt

echo "" > Manifest.dsx

while read line
do
    echo "\t" >> Manifest.dsx
done < files.txt

echo "" >> Manifest.dsx

echo "Manifest.dsx" >> files.txt # Add the manifest to the list of files.

echo "\nCreating archive..."
cat files.txt | zip Archive -@ -Z store

echo "\nCleaning up..."
rm files.txt

echo "Done.\n"

DesignSlave

Thanks Patrick.

I'll give this a shot...

hsor1