Water cooling build in my wife’s computer – Part V

Build Log:

Yes the build log continues, but this iteration does not begin on a high note. And just to warn you, this is going to be a long article, but I cover a lot of things that happened in a short period of time with a lot of explanatory information as well.

About 8 weeks post-installation of my wife’s loop, we had a catastrophic failure of one of her components, and I think it’s the CPU block that failed. Recall from previous iterations of this series that the CPU block is the AlphaCool NexXxoS XP³ Light – Plexi Edition. In the loop we’d been using the Mayhem’s XT1 coolant diluted to about 1.5L, about a 10% dilution.

Prior to the failure, I had acquired an AlphaCool XT45 240mm radiator to install in the floor of my wife’s case. My intent with adding the second radiator was to also flush out the coolant and replace it with just straight distilled water and a few drops of copper sulfate. Her reservoir came with a silver plug so there would be no need for a kill coil. And I had intended on doing the installation the weekend following the catastrophic failure, after I had acquired the last of what I needed to do the installation.

Needless to say, those plans got scrapped.

The failure was beyond catastrophic to the point of it being lucky I was home. The leaking coolant fell onto the first graphics card and ignited, while some additional coolant fell to the second graphics card and caused some corrosion but did not ignite. Thankfully the flame didn’t last long, but I cut power and pulled the plug to be sure while also grabbing the fire extinguisher as an extra precaution – you do have one of these in your home, right? Had neither my wife or I been home – the failure occurred about 20 min after she left for work – what would we have been coming home to?

The material safety data sheet for the coolant specifies that the main ingredient is mono-ethylene glycol, which is classified as a flammable liquid – it likely shorted out several of the components on the graphics card where it made contact, causing it to ignite.

Once I was sure nothing was going to flare up, my first move was to drain the loop – that all-important drainage system I made sure to install and test comes in handy yet again. Then it was a matter of pulling things out, starting with the graphics cards. Things didn’t look good.

What led me to believe the CPU block was the cause of the failure was the drips of coolant that were hanging off it after I had drained the loop. You can see from the pictures that there were not any trails of coolant leading from the fittings, leading me to believe the perimeter of the block is what failed, likely from the o-ring inside it. This picture below shows drops of water still sitting on the lower edge of the CPU water block.

DSC_0662.jpg

Under the graphics cards was a horror story as well, as you can tell by the above pictures. This is what happens when you mix flames with PCBs. The fact several of the electricity-regulating components got fried, literally, led me to declare all the singed components to be a total loss.

My concern then fell to what else might have been lost beyond the mainboard and graphics cards. The memory chips looked perfectly fine, but the scorching reached to alongside the CPU socket. Getting the block off the CPU, I was able to inspect it and didn’t see anything obvious wrong with it. There wasn’t any scorching or anything like that on the CPU, so I was confident it survived – though I would need to acquire another mainboard to know for certain to set up a bench test on both the CPU and memory chips with a spare video card.

So yet another trip to my local Microcenter was in order. In the mean time I needed to also get the larger radiator out of the system so I could flush it out, get all traces of the glycol coolant out of what remained of the loop, along with the pump and reservoir.

In the mean time, I also took some time to see if I could figure out what happened. I built a makeshift loop with another pump and reservoir and some spare fittings to get the CPU block back under pressure. One thing I noticed right away was the thickness of the water coming out of the loop, as if the glycol solution had separated and gelled up inside the block. How that could lead to the catastrophic failure I’m unsure, but I could definitely notice the water coming out of the block and back toward the reservoir was definitely not the same as the water that originally went in and it seemed a lot of the concentrate was being rinsed out.

Unfortunately I was not able to reproduce a leak, so I’m not sure what happened, and in hindsight I realized that what I could’ve done was hooked the pump to an external power supply to keep the loop under pressure to see if I could observe a leak. But when your system just caught fire, you’re probably not going to be thinking along those lines.

But I was still of the conclusion that the CPU block was to blame, either directly through a component failure, or indirectly by way of the coolant separating and gelling up. So the next task on the list was getting ahold of AlphaCool.

* * * * *

Contacting AlphaCool

Now trying to figure out how to contact AlphaCool was initially not easy, as their main website is a mix of English and German and I’m an English-only kind of person. So I contacted Performance-PCs, the retailer through which I ordered the block, for assistance in contacting AlphaCool to help get this or at the least investigated. Performance-PCs forwarded my e-mail of to their sales representative with AlphaCool UK who replied back to them with permission to give me his e-mail address. So I sent an e-mail directly to the sales rep:

Recall from the previous e-mail chain with Performance-PCs that the component in question is the AlphaCool NexXxoS XP3 Light – Plexi Edition. It was installed on an AMD FX-8350 processor on an ASRock 990FX Extreme4 mainboard.

How to begin on this… I believe the AlphaCool CPU water block that was being used in my wife’s computer failed in some way and leaked coolant inside her computer, causing components to literally catch fire. Thankfully the fire was small and did not last long, but it still caused the loss of two graphics cards plus her mainboard. I’m unsure if the memory and CPU were lost as well as I have not yet had an opportunity to test those, as I need to acquire a new mainboard first.

Getting back to the water block. It was installed about eight (8) weeks ago. The installation was smooth and the leak testing went smoothly as well and I had no reason to believe there was a leak from any of the components or fittings in the loop. Everything ran smoothly until yesterday afternoon when the aforementioned leak occurred. I’m unsure if the failure was due to the water block being faulty, or if the issue was caused or exacerbated by the coolant I chose for her system — Mayhem’s XT-1 Clear, which according to its material safety data sheet is made from monoethylene glycol (MSDS is available here: http://www.mayhems.co.uk/front/download/XT1-150ml-MSDS.pdf).

I also have pictures of the components following the failure, and I can forward those to you if you desire — they are several megabytes a piece which is why I have not just directly attached them here.

So the first thing I believe needs to happen is I need to get the CPU block back to AlphaCool to be examined. So starting with that, how can I get the process started to do that?

Again my intent with initiating contact was to find out if the CPU block actually failed, and AlphaCool would be best able to examine this. They can determine if the block did actually fail due to some fault in its manufacture, or if the coolant was the issue. If the coolant was the concern, then I would have to take things up with Mayhem’s. As it was a Friday afternoon my time when I contacted him, meaning Friday night in the UK, I realized it might be till the following Monday before I heard back.

And the following Monday I received a reply from AquaCool’s UK representative, pointing me to the details on how I can start a defective item claim through the Aqua Tuning store, which I started that evening after getting home as the support request required attaching pictures for defective items. Good thing I have a high quality camera.

* * * * *

Microcenter and Testing the hardware

At Microcenter, I considered getting her the same mainboard I run in my system, the Gigabyte 990FXA-UD3, which Microcenter had for a shelf-price of $135. Unfortunately they did not have her original mainboard in stock anymore. But in searching their site the night of the fire, I discovered they had an ASUS Sabertooth R2.0 mainboard for $148 as an open box – it’s typically $185 on the shelf right now. So for only $13 more, I decided to spring for that given this is for a predominantly gaming machine that will, in the end, be overclocked and water cooled. Unfortunately the mainboard did not have the back-panel I/O shield, but I was able to order a replacement through ASUS’s eStore.

When I got home, I decided to break down the computer more out of the case. I discovered that the smell from the burnt PCB had gotten into the radiator, and I wasn’t sure how to get that smell out. I decided that the best attempt would be to run the fans that were on it continually to see if that would work. The smell was also on the case, and I wasn’t sure if I’d have to scrap it or if I could recover it in similar fashion. I was hoping I wouldn’t have to scrap either.

First order of business though would be testing the CPU and memory chips.

I set up a test workbench on the mainboard’s box, using the stock CPU cooler I still had around and a spare graphics card along with a spare power supply. Once I got everything seated and double-checked (the DRAM_LED on the mainboard kept trying to tell me the memory chips weren’t seated properly), I powered it up and watched as everything came to life. So far, so good, and meaning I wasn’t out yet another $300 in hardware for the processor and memory chips, or so I hoped.

But merely getting a response from the mainboard wouldn’t tell me if the CPU and chips suffered any damage, only that they were responding when the mainboard pinged them. To fully test them, I booted up an Ubuntu USB drive and pulled Prime95 off the web. Now for the CPU cooler I was using IC Diamond thermal compound, and the temperature maxed out at 53C according to Psensor, running Small FFT on all 8 cores to have them maxed out at 100%. I left that running for about an hour before declaring it done – if there were any major issues, I figured it would’ve been found within that time frame.

After that came the memory test with MemTest86+ – press F6 while the nearly-blank purple screen is displayed as the live drive is booting and select “Test Memory”. That test came back clean as well.

So thankfully the only losses were the graphics cards and the mainboard – the latter of which is already replaced simply because I needed it to make sure the CPU and memory chips weren’t dead as well, as well as set my wife up with a working system until I can start building out a water cooling loop again. That will happen when I get another CPU water block, as I’ll build out a CPU water loop at least since the stock AMD cooler sounds like a blow drier.

* * * * *

Smelly components

If you’ve ever had the wonderful opportunity to be in the vicinity of burning PCBs, you probably know the smell very readily. Prior to this, the last time I encountered that smell that I can readily recall was a little over 15 years ago when I attempted to upgrade a computer with an AT mainboard and power supply. Some of you probably know where this is going. I connected the power supply to the mainboard incorrectly, and fried the power supply and the mainboard as a result – literally there was smoke coming out of the power supply via the exhaust fan. My father was sitting in the living room at the time – we lived in a 3 bedroom apartment – and he recognized the smell straight away.

Now thankfully that smoke wasn’t accompanied by any flames, that I could readily tell at least, so the smell didn’t really linger. Unfortunately for my wife’s computer, the smell got into her radiator and also settled quite readily into her case. I wasn’t ready to declare them to be a loss, though, though I feared I would be doing so.

After getting her case virtually cleared out, I got the Bitspower X-Station out as well and used it for powering the fans on her radiator (Corsair SP120s) with an external power supply and let the fans blow overnight before checking on them. The smell was almost completely gone from the radiator after that. There was a slight lingering odor where there was a gap between two fans, but I wasn’t concerned as the vast majority of the smell was gone, though I did leave the fans blowing for another couple hours to clear out as much of the last as possible.

Taking that as a good sign, I set up a large fan to blow on her case. My wife and I had a concert in Omaha to see that day (Lindsey Stirling at the Sumtur Amphitheater in Papillion, NE), so there’d be plenty of time for the fan to do its work, hopefully. Now I knew the case would likely take longer simply because it’s larger but the fan did do the trick. We got home about 3am following the concert, and when I checked on the case I could not smell any of that burnt scent on it.

Another $225 in components saved.

* * * * *

Following up with AlphaCool

The reply to my e-mail to the AlphaCool support e-mail address would be from the UK rep. I had no idea the e-mail would be going to the same person, and worded my e-mail as such. The reply included two follow-up questions: was the pump still working and were the temperature alarms turned off on the mainboard?

The manual for the mainboard doesn’t specify any kind of configurable alarms, so the answer to the latter is No by default. I do know that many modern mainboards have temperature fail-safes: they will power the system off if the CPU gets too hot. And if you attempt to turn the system back on, it’ll either power down quickly or the BIOS will issue a beep code saying the CPU is overheating.

You might be surprised to know that this feature wasn’t implemented until relatively recently in computer years. This meant that prior to such an implementation, it was quite easy to cook your processor, and the mainboard with it. Computer processors get very, very hot – hotter than you can touch without burning your skin – even with the kind of air cooling solutions available. It is easy for the processor temperature to jump into the 40s Celsius in just the first 15 seconds after you turn on the computer, and that’s with the processor fan on. Without the fan on, or without a cooling solution on the processor at all, temperatures can fast climb over 100 Celsius and higher, getting hot enough to literally fry the processor and mainboard – search YouTube for ‘cpu no heatsink’ and you’ll find some videos showing this.

Recently I saw a video of a guy building a computer where he didn’t even put the cooling solution onto his processor before getting it powered on. He had a somewhat good reason for doing so: he wanted to get the computer to POST before continuing further, as if it didn’t POST, then he would’ve had reason to believe something was faulty. Without those failsafes, though, he could’ve easily fried his machine doing that. And as in that video he was using an ASUS Rampage IV Extreme mainboard with an Intel i7 3930K processor – i.e. $1,000 worth of components according to NewEgg’s current pricing – it was one costly risk to take that thankfully worked out in the end.

Personally, I was waiting for the thing to practically burst into flame watching the video.

And I certainly understand the nature of the question regarding the pump, given that in my e-mail to the support address and the earlier e-mail, I’m alleging that the failure of the component that is still under warranty and should not have failed caused the loss of several hundred dollars in components. So naturally he’s going to want to ask a few additional questions to determine if another failure in the system led to the block failing. And as CPUs can get pretty hot, if the pump failed, then the coolant would start heating up, possibly causing a build-up of steam pressure inside the loop, with coolant leaking at the first point of failure.

But I tested the pump by setting up a makeshift loop and all was well. So far everything still points to the CPU block.

* * * * *

And that is where I’ll call this iteration. I’ll be sending the block into AlphaCool later this week, which will be interesting since I’ve never actually mailed anything international before – AlphaCool is in Germany. I’ll continue with another iteration once I hear back from AlphaCool on the status of the water block or if I have anything new to add with regard to getting my wife’s system back up and running.