Crypto currency mining

[moltuae]

Wow!

The Dash mining difficulty is going to shoot up when those things start coming online!

Very tempting but at $10K-$13K each that a lot of eggs in one basket.

[ainarssems]

It is a lot but I cannot ignore potential returns. it's not going to be what it could potentially be at the moment but still should be pretty decent. $13k + VAT would be a bit too much for me to risk. But I would be interested to share the cost, risk and returns if somebody else joined in. 50/50 is probably a bit out of my comfort zone as well but if there was possibility to split it 3 ways I would probably go for it.

[ainarssems]

I have also been working towards frame that could potentially hold 13 video cards and 4 PSU's. I had to change couple of things along the way and it's not what I first evisioned but should work. As far as I unerstand there is 6 AMDS GPU limit in Windows drivers but I have seen somebody running 6 AMD and 7 nVidia GPU's on 1 motherboard. Linux should support up to 32 but custom settings like clock, voltage only work for first 10 so would need to flash BIOS to get right settings for all GPUs.

[ainarssems]

Another thing is that while I considered returning 1080Ti's, I have now decided to keep them. For a moment they were actually earning more than 4x RX480 and after tweaking settings a bit and changing PSU they are lower on power as well. I was going to make a frame for them and PSU and connect them to my HP Microserver. Unfortunately one of hard drives have decided to give up on that so I need to change that first. it's not an emergency as I am running RAID 1+0 so I can have any 2 drives out of 4 fail

[moltuae]

Originally Posted by ainarssems

Another thing is that while I considered returning 1080Ti's, I have now decided to keep them. For a moment they were actually earning more than 4x RX480 and after tweaking settings a bit and changing PSU they are lower on power as well.

I much prefer them to the RX480 now. The Nvidias aren't as good value for money as the AMD cards but they just seem to have much less issues. The GPU management/tweaking software is generally less buggy, plus it lets you make adjustments without having a monitor attached, unlike the AMDs for which you need a monitor or 'dummy load' connected just to make adjustments. The 5-card rig I'm presently working on has 4 1080Ti cards in it and an RX480, though I'm probably going to replace the 480 with MJ's 980Ti when it arrives

[paulrstaylor]

Originally Posted by ainarssems

I am running RAID 1+0 so I can have any 2 drives out of 4 fail

That most likely isn't correct, Raid 1+0 is often described as supporting "at least" one drive failure, but that isn't the same as two drives.

It is a mirrored stripe set, so you could have two drives fail and be fine, if they are both the same side of the mirror, but a second drive failure could also take everything out!

Get it changed ASAP is my advice

[HPsauce]

I've forgotten (having long left the corporate IT world) all the RAID variations, but we used to run systems that would automatically fail over (and keep running) in the case of a single drive failure and rebuild a resilient array automatically onto a "hot spare" sitting there already powered up and spinning.

Plus all drives would be "hot swap" so you just pulled the faulty one out and stuffed a new one in.

But that was at least 15 years ago so maybe things have improved a bit since then?

Though the mainframes I worked on decades before that were much cleverer, with resilient load-balanced controllers, dual access paths to all peripherals, multiple processors and memory banks as well; they basically had almost no single point of failure in the whole system.

[paulrstaylor]

Originally Posted by HPsauce

I've forgotten (having long left the corporate IT world) all the RAID variations, but we used to run systems that would automatically fail over (and keep running) in the case of a single drive failure and rebuild a resilient array automatically onto a "hot spare" sitting there already powered up and spinning.

Plus all drives would be "hot swap" so you just pulled the faulty one out and stuffed a new one in.

But that was at least 15 years ago so maybe things have improved a bit since then?

Though the mainframes I worked on decades before that were much cleverer, with resilient load-balanced controllers, dual access paths to all peripherals, multiple processors and memory banks as well; they basically had almost no single point of failure in the whole system.

RAID hasn't moved on that much to be honest, you probably used RAID5 with a hot spare, which didn't really scale to current disk sizes. RAID10 (1+0) is the current norm (big drives are cheap now), again some controllers allow a hot spare, but not usual in a microserver!

http://www.raid-calculator.com/raid-...reference.aspx gives a bit of background as to how each mode works.

http://www.zdnet.com/article/why-rai...rking-in-2009/ explains why RAID5 didn't really scale....

[ainarssems]

It's entry level for me so no hot swap but it does run happily with 1 drive failed and I have a new one in the post.

[moltuae]

Originally Posted by ainarssems

I have also been working towards frame that could potentially hold 13 video cards and 4 PSU's. I had to change couple of things along the way and it's not what I first evisioned but should work. As far as I unerstand there is 6 AMDS GPU limit in Windows drivers but I have seen somebody running 6 AMD and 7 nVidia GPU's on 1 motherboard. Linux should support up to 32 but custom settings like clock, voltage only work for first 10 so would need to flash BIOS to get right settings for all GPUs.

Looking good

Have you made that frame from scratch or did you buy it?

It can certainly be a challenge getting larger rigs to run stably but it's somehow more rewarding to see them running if you do. Good luck!

I suspect you've probably seen most of them already, but there's quite a few Youtube videos of 13 GPU miners built using that motherboard. It does seem that Linux is the best option for such a large rig:
https://www.youtube.com/watch?v=beCjF9taFdU

I'm currently working on a much smaller rig; just 5 cards. I'm using extruded aluminium and aiming for compactness and simplicity this time. I've got it down to 500mm (high) x370mm x370mm, though I could probably go a little smaller. I'm looking to standardise a compact, stackable and robust design that I can easily make more of to use as room heaters this winter. I'll post some photos when it's finished.