Reverse Engineering Dell iDRAC6 Express: Fan Control

As previously mentioned, the Dell PowerEdge T710's stock Pulse Width Modulated (PWM) fans are controlled by the iDRAC6 & BMC firmware data files. I haven't had time to properly address these components but luckily there is another quick method that we can use to silence the stock fans. Later on I also tried swapping out the stock fans (YouTube video) with much quieter Arctic F9 PWM models but this introduced other problems.

As it turns out, running /avct/util/pwm_test will override the iDRAC firmware fan control (mostly determined by BMC firmware and the /bin/fullfw processes) and allow us to set the fan RPM to a reasonable level, somewhere around 1,300 - 1,500 RPM, down from an average of 2,000 - 2,400 RPM. This is well above the 1,080 RPM "critical threshold" set by the iDRAC6 BMC. As you can imagine, the lower spin rate causes the noise level to drop significantly while still keeping the server cool at my regular work load.

Run pwm_test as root via iDRAC SSH or Telnet connection. These commands can be inserted into the init sequence if you would like to have them persist across RAC resets.

Note that we have to keep this process running in the background or fullfw will resume control.

$ cd /avct/util
$ ./pwm_test 0 0 0 1 255 &
$ ./pwm_test 0 1 0 1 255 &
$ ./pwm_test 0 2 0 1 255 &
$ ./pwm_test 0 3 0 1 255 &

That's it!

Swapping stock fans for Arctic F9s

The Arctic F9 fans spin at a much lower rate than the stock fans rate of 1,000 - 6,000 RPM. With Arctic F9 you can expect anywhere between 550 - 1,800 RPM. Without running pwm_test the fans spin down below the critical threshold mentioned above and the iDRAC reacts to this by cycling the fan speed up before returning it to regular levels. As soon as the RPMs return to "regular levels" they fall below the threshold which causes a vicious cycle.

To combat this I used /avct/util/FanTachTestAP to adjust the interpreted tachometer output frequency at 1 pulse per revolution. As the spec for PWM fans is typically 2 pulses per revolution this effectively causes the software to double the calculated RPM rate. With the "numbers" back into the 2,000+ RPM range I was able to avoid the vicious cycling caused by the iDRAC firmware.

I found an acceptable fan speed/noise level using the following commands:

$ cd /avct/util
$ ./pwm_test 0 0 4 -1 50 &
$ ./pwm_test 0 1 4 -1 50 &
$ ./pwm_test 0 2 4 -1 50 &
$ ./pwm_test 0 3 4 -1 50 &
$ ./FanTachTestAP config all 1

To incorporate these commands into the iDRAC firmware, I created an init script at /etc/sysapps_script/S_0000_local.sh (link to my script) and then placed it into the init sequence by editing /etc/init.d/postinit_adv.sh and placing it into the list immediately after the line that reads /etc/sysapps_script/S_5378_idrac_fully_ready.sh. Unfortunately it takes the iDRAC a while after booting to reach this point and you'll hear the fans cycle up and down until then, creating error entries in the DRAC system log.

You'll notice a departure from the previous article in my init script. I've moved the netcat startup to this script instead of including it in the stock Telnet init script. This wasn't for any particular reason except that I wanted as many of my customizations to appear in one place should I have to incorporate these changes into a newer version of the iDRAC6 firmware.

I'll cover customizing the fan sensor and PWM control kernel modules in the next part.

Related reading

• Introduction to PWM
• PWM Tutorial
• Application of PWM speed control
• PWM Controlled Fans Specification

Wikipedia articles

• Computer fan control
• Pulse-width modulation
• Duty cycle

Comments