A side effect of setting up your own VMWare lab is that, after all is said and done, you end up with a little furnace in a closet. With Orlando weather, this is mostly unnecessary.
Having recently setup the ELK stack to monitor this VM, I searched for the string ‘temperature’ not knowing what I would find.
Bingo, the image above clearly shows temperature warnings from the three hard drives in VM.
Note – the threshold crossed is from a range not a temperature value.
This VMWare box has 3 identical WD Caviar Green drives and these values are within the tolerances of the drive. The range for these drives is from 99 – 125. This roughly translates to an operating range of 140 to 32 Fahrenheit. See specs here.
When setting up this box, one concern was that using an AMD processor was going to result in a hotter running box but the cost savings were worth the trade-off. I never imagined that the hard drives would be the ones throwing warnings at me. After reading up on this, however, it makes sense that we would care more for the temperature of the drives than for the temperature of other components in a system. If the CPU or motherboard gets hot, we get diminished performance, if the drives get hot, we get shortened lifespan and loosing drives is way more critical.
From the command line, we can see more details on each drive. For example:
~ # esxcli storage core device smart get -d t10.ATA_____WDC_WD10EZRX2D00A8LB0__________________WD2DWCC1U3018884 Parameter Value Threshold Worst ---------------------------- ----- --------- ----- Health Status OK N/A N/A Media Wearout Indicator N/A N/A N/A Write Error Count N/A N/A N/A Read Error Count 200 51 200 Power-on Hours 85 0 85 Power Cycle Count 100 0 100 Reallocated Sector Count 200 140 200 Raw Read Error Rate 200 51 200 Drive Temperature 103 0 92 Driver Rated Max Temperature N/A N/A N/A Write Sectors TOT Count 200 0 200 Read Sectors TOT Count 200 0 200 Initial Bad Block Count N/A N/A N/A
The number reported on the logs is the ‘Drive Temperature’ in bold above. It is worth also noting that the worst this particular drive has experienced is 92. Remember our acceptable range starts at 99 so this extreme is going to affect the life of these drive. This translates to about 164 degrees Fahrenheit!
This box is running with one fan in the rear or the case. In addition to this fan location, there are two locations on the top of the case, one in the front and one in the bottom. This gives us up to 5 fans at a time to play with. I had 4 fans to play with and, in combination with making each an intake or an exhaust, had plenty of permutations to try. In the end, it took about two weeks of trying different settings to collect enough data for these combinations. You can download the data here.
Here is a graphical representation of the test time-span for the three drives. I’ve annotated a few of the changes made throughout the test. More fans, cooler temps.
It’s hard to tell from this image but fan count is not the only thing that counts. Different combination of fans produce different temperature averages as well.
Perhaps this is a better way to check all the combinations for the fans tried over time.
For the three hard drives on the case, I plot a chart per fan. The number zero indicates the fan is off and the number one indicates the fan is on.
The endpoints of the lines mark the value when said fain is off or on. Higher values are better. A positive slope, therefore, indicates the temperature went down and a negative slope indicates the temperature went up. The steeper the slope, with respect to the other slopes in the same drive (horizontally) indicates a better improvement by turning this fan on than the next steepest slope.
With so many permutations of fan combinations available, it is peculiar that, the default fan location, when combined with other fans, proves detrimental. Since I do not have as many fans as I have places for them, this is the fan I decide to do without.
In the end, I went from one exhaust fan in the back of the computer to four fans. Two exhausts in the top, one intake in the front and one intake in the bottom. The average temperature for the hard-drives went from 120 degrees to 80 degrees Fahrenheit. The total cost was about $20 for three additional fans and most of the time was spent trying different fan location combinations and operating modes (whether intake or exhaust).
Here’s a test shot for happy server mid test.
That’s it. Thanks for reading.