Render Farm: New project need advice.

paul_ae567ab9

I really hate a few things about DAZ Studio despite it's awesome power! 

1. While rendering you take a nap or eat more than you should. Your computer is a brick until done.

2. There are no easy options other than lux render (which I have issues with aside from this topic) to offload renders to the network and let another machine do the work.

3. DAZ seems un-interested in releasing a native Linux version which would FLY much faster that windows.

1&2 are obviously related.

My work around for 1 & 2 has been to install DS on an i7 on another table. That DS copy's library points to the same library on the main machine via network so the assets are always the same even if I add product.  I copy the DUF over to the i7 (or open from the network) and render on the second machine. Convoluted but it works.

I'm lazy.

I want more speed!

I bought on eBay pretty cheap, a Xeon server I want to use just as a render farm. Using the same process as above that works already and it is pretty quick, a single charactor, single light no background renders in 3Delight in abouot 30 seconds.  Iray renders it in about 90 seconds or so. Both remote machines have same Nvidia 730 graphics cards. The main difference of course is CPU and RAM.  Xeon is a dual quad for total 8 cores and 32GB ram. i7 is a single quad with 16BG ram.

But using DS as the render program limits me to rendering only one scene per cpu at one time. I cannot share the work between two machines at once ( or three for that matter )

To be clear I have absolutly no excuse to justify doing any of this as I am just playing with 3d for fun. At almost 71 years old I'm not looking for work, just goofing off.

Did I mention I'm lazy?

So: Tried lux not happy with results but I'm sure that is more to do with my knowlede of settings and how to use it so that is another subject I will address on their forum later.

The meat of my topic is to seek advice, feedback and "gotchas" of setting up my new Xeon render farm (if one machine can be a farm, guess I'll call it a render garden for now)

It has a 4GB Nvidia 730 card, DDR3 RAM.  Obviously CUDA irrelevant to 3Delight but the Xeon's CPUs have horsepower and 32 GB fast server ram and a server grade HDD 7200 RPM with 32MB cache. I may trade cards with my desktop which has a Nvidia with 8GB DDR5 ram and outperforms the Xeon in Novabenc on that factor. That would better serve my needs in the Xeon machine than on the desktop.

I can get Tesla cards on eBay for around $35 each with adds 3GB Ram and a couple hundred more cuda cores. Again, irrelevant to 3Delight but a boost for iRay and Lux.

Part of my inquiry to those who have tried addind a Tesla card...I hear reports of issues getting drivers for Tesla to peacefully coexist along side the main GPU cards which is a problem because Tesla's lack video connections.

I also downloaded ( now expired ) a trial of Nvidia's server but never got anywhere because I could not figure out how to send work from DS to their server product. Any thoughts?

This is where I am starting.  Hoping others with prior experience can offer some sound advice on a visual product!

 

DrunkMonkeyProductions

Unfortunately the only way to do a true render farm with daz studio as your base program,currently, is to purchase the 3delight standalone, currently 1200 usd a year for unlimited cores and exporting from daz as a rib file, or the iray server at 295 per year per machine(if i'm reading their page correctly), 2 machines, 590 per year, and exporting as an mi file.

3delight:http://www.3delight.com/en/index.php?page=3DSP_pricing

iray:http://store.nvidia.com/store/nvidiamr/en_US/Content/pbPage.products#_ga=1.174358314.92775092.1483174138

 

now on to your initial points/questions

1. for cpu rendering, if you set the affinity(task manager>select program>right click>set affinity) and reduce it by one core your computer won't be a total brick and you'll still be able to do other things, although it will increase render time, and other programs may be a bit sluggish.

You'll need to do this each time you start daz studio, but it's an option.

2. refer to the start of this response for a couple options other than lux. there are a lot of other render engines which you can export your scene to, keyshot, octane, etc. and simply reapply texture maps and adjust shader settings, but those have their own inherent issues, as you've partly learned with lux.

I actually don't like lux much, mostly due to being lazy myself, and the need to change every single stinking shader everytime i work in it is annoying, (yes i know i can save presets, but it's easier with 3delight and iray in program) plus the lights are an absolute PIA to get right without a massive amount of fireflies in it.

3. i wouldn't hold my breath on native linux support. but the 3delight standalone does have a linux version.

 

For a dual box render farm i'll just make a couple suggestions to make your life a bit easier.

The big one is , get another drive or create a partition on your main work box drive, if you only have one drive. install all your content to that drive/partition, as well as saving scenes to that drive/partition and "share" that with the other computer, mapping the shared drive/partition with the same drive letter as the main computer.

this will negate the copy/paste step you mentioned going through. you will of course have to instal all plugins etc on both machines.

For CPU only rendering, for raw speed,  you'll either need to look at 4u rack mounts, such as the proliant 580/585 series, which can support upto 4 processors and upto 2TB of ram, or something else that supports 4 physical cpus.

The one thing about cpu rendering you should be aware of for iray is that at render time it can take upto 3-4 times the ram used while working. A scene that takes 2GB while working, will take around 6GB at render time.

For gpu rendering, before you go purchasing any cards, take a look at this list,https://developer.nvidia.com/cuda-gpus,  from the nvidia site. Anything less than a 2.0, and it won't be viable. be sure to check out the legacy cards list as well.

While i currently haven't tested any i'd shy away from those 3gb cards you mentioned.

Unlike ram on your motherboard, Gpu ram doesn't stack and iray uses the ram of the smallest card. So if you have a card with 3GB and one with 4GB, your scene would need to fit into the 3gb, or it would kick over to cpu only mode.

The lowest cost option i've seen, that might work haven't tested, is the tesla m2090, at around 90 usd on ebay, 512 cuda and 6gb of ram. For ram size it's about the cheapest i've seen. A consumer card with comparable specs is around 300 usd. And from there the costs explode with a k20 running around 1k with 6Gb ram

The big hurdle with servers, and some desktops, is the wiring harness, just because you can fit the card in it, doesn't mean the powersupply is going to support it.

In my case i have an old proliand dl385 g7, 2u rack mount i use for a render box, 16 cores on two opterons. while i can mount an x16 card in it, the problem is that there are no power connections, so it has to be unpowered. To mount a powered card I'd need to get an extension cable for the x16 slot and externally mount it with a secondary powersupply.

My workstation on the other hand wouldn't have a problem adding in a couple tesla cards.

The last time i check the largest number of x16 slots on a single motherboard was 8, but with the size of many of these types of gpus, you'd max at about 4, without going the extension cable route, and of course deminishing returns as the lane width drops.

there are some 4u rack mounts that support upto 16 gpu cards, but those require a 220volt line, a HIC, and run about 8k on ebay, without cards.

so there's just a few menadering thoughts on the subject.

 

 

prixat

Just a minor correction...

There is no longer a need to export .mi files. (That was what you had to do a couple of Betas ago) 

You can even join servers together into a cluster and build yourself a "Render Prarie" 

paul_ae567ab9

Update (HAPPY NEW YEAR)

I got the faster drive in the Xeon machine along with the AHIC drivers. Novabench dropped one point!  But it is 780GB 3 partitions vs. 238 1 partition. Not sure that matters. One point no biggie.

I put Novabench on the other "render" machine, 4 core i7 and it blows the doors off both the AMD and the Xeon machine.  I have attached the details for all three machines.  I struggled a bit getting an OS on the Xeon machine.  Keep in mind this is just hobby and the small cost for the Xeon machine ($150) was expendable, in retrospect another of the same referbished i7 machines would have made more sense as it seems quite a bit faster than the older dual Xeon box.

The AMD is my main desktop where I do all my work but I off load renders to other boxes and both of the remote are faster (except for GPU which AMD appears nearly twice the speed despite half the GPU RAM of the other machines. ( Since the AMD is normally not rendering anything unless the other two are busy and I want a nap!) I may swap cards and move the 740 card to the Xeon machine. Cannot put in the i7 as that is slim case and card won't physically fit.  That may bring the Xeon machine closer to the i7 in speed. Plus+ I can then move my AMD back to smaller case and power supply to save space.

Again. Hobby. I get bored and want to play. This is play and I have zero rational reason to do any of this!

( no way will pay for stand alone render software, will just send a separate DUF to each of the other machines and let them chug them out. )

But this is still fun !

paul_ae567ab9

Stand alone 3Dlight.  I may register a couple free licenses so I can test one on Windows and one on a lInux machine.  I guess if I dual boot one license is all I really need since it is same physical machine.