Fancontrol

[xnuken]

Do you have any documentation or files regarding fancontrol want to make my own T3-1 less noisy.

[antoinecarme]

Hi @xnuken

Thanks for your message. A nice machine which supports debian and runs all the data /machine learning software I need with 128 threads.

Seems the noise level is part of the design specifications of these servers !! The noise reduces considerably (fan speed by 50%) after 6 minutes of uptime. Some kind of automatic fan speed setting is installed but even with that the server is still noisy for a home usage.

I tried the following :

1. Updated the firmware. The latest one is available somewhere on this repository (152738-02.zip).
2. Tweak the service processor parameters/sensor thresholds. No luck here for the moment.
3. Ipmitool. Some version has a "sunoem fan speed" command, no luck. Raw IPMI commands are also possible.
4. Reduce the number of installed fans : removed every other fan. works but can be dangerous. Not very risky when doing some less cpu intensive tasks (coding).
5. Even tried to get inside the service processor (linux@ARM) restricted shell. Some command /usr/local/sbin/fanctrl can be used. Nice software solution if one can help. No luck (for the moment ;)
6. Find a good location far from my desk, behind two doors. Works, this is what I use for the moment + solution 4 above.

Anyway, any help or feedback is welcome. I did not try physical solution like changing the fans or the cabling.

Please, don't hesitate to share your ideas here. I may have missed something that you can improve/correct. More and more people are interested.

cheers.

[antoinecarme]

Added a few notes here : Notes.md

[xnuken]

Alright think im going to look into the SP firmware since I opened it last night and like you said its running linux on a arm soc. Going to see if maybe there is something there that can be changed. If it requires the FW update package to be repacked and flashed it might be a bit risky since I dont want to brick the SP/ILOM. Will also dig around in the ILOM commands and dirs to see if anything could be changed without changing the FW.

[antoinecarme]

I tried at some time to repackage my own firmware. I disassembled the firmware (squashfs file system) and updated a file etc/static_conf/ifc_config.conf which contain something like this

# This is a comment
# ALGO_TYPE - defines the type of fan algotrihm to use
# valid algorithm types are:
#    0 - IFC, dynamically control fan speed to meet temperature threshold.
#    1 - FIXED, put fan at fixed speed, the actuall speed is in the SPEED section of this file
ALGO_TYPE 0

# MIN_SPEED - defines minimal fan speed allowed
MIN_SPEED 128

# MAX_SPEED - defines max fan speed allowed
MAX_SPEED 255

This is probably the config file for the automatic fan speed setting library. I replaced 128 and 255 with smaller speeds/values.

The service processor refused at that time to flash the re-assemebled modified firmware, a protection against bricking ?

If you are motivated enough, go ahead, you may have more luck than I had.

[xnuken]

Probably checking the checksum of the image might be some way to bypass that

[antoinecarme]

Sure.

There is also a point I tested, is to get inside the service processor pre-boot and change this file. Was not able to get to that point.
It relies on a "xyzzy" keyboard combination in the first two seconds of power-on, strange enough but harmless.

https://docs.oracle.com/cd/E19045-01/blade.x6220/820-0048-18/sp.html

[avocado2002]

My server is T3-2 with 6 hot-swap fans (hope they're the same as in T3-1). The letter being cut and wired via 8.5Ohm resistors (mind red wire - 12V) I've reduced its speed from 5800 to 2900RPM. Not enough to reduce overall noise though (but I admit it's 30% lower now).

The major impact comes from PSU fans. I've dismantled one PSU and it has 2 DFTA0456B2U. Specs (https://www.vialetter.com/product/dfta0456b2u/) say it's speed about 18K RPM with sound pressure 63dB, that's why the server itself is so noisy.

Unfortunately SysFW doesn't report PSU fans' speed. Only the temperature. The space is so tight there, that I have doubts now how could I place any resistor ....

[antoinecarme]

@avocado2002

Thanks for your message. Your feedback is useful. I however focus on software only solutions for this task.

In the case of T3-1 here, the PSU fan is very noisy on power-on and stops almost completely after 5 minutes. It does not restart on reboot (or reset /SYS from service processor).

Indeed, there is no control on the PSU fan.

T3-1 (2U) and T3-2 (3U) chassis are not similar and the mainboard fans are very different :

T3-1 :

T3-2 :

[avocado2002]

Mine are always on. Maybe 9000RPM or so. PSUs are my next endeavour. Very risky however.

Antoine, can you show your power management settings from ilom? Maybe I can slow down my box either ...

Systems fans for T3-1 are smaller. That's why they produce a higher noise (T3-2 with 92 mm vs 60mm for T3-1) -- https://www.vibrant.com/SUN-371-4613.html

However T3-1 has 2 fans totally in 2 PSUs. Whilst T3-2 has 4 fans totally in 2 PSUs. And its design is more complex.

[antoinecarme]

I made a dump of all service processor settings (ILOM) here :

https://github.com/antoinecarme/sparc-t3-data/blob/master/debian-sparc64/service_processor_output.txt

You can make your own (ls -t /) and compare the power settings.

[avocado2002]

your systems fans sound like mine last week :)
but system itself works in a performance mode. mine - as elastic.

From those all messages I read in logs and settings so far, I'm afraid there is no way to change RPM programmatically...

[antoinecarme]

@avocado2002

Fan speed on sparc machines can be changed through the IPMI protocol. There is a command ipmitool to set the fan speed as part of SUN OEM commands.

Roughly speaking, something like : ipmitool sunoem fan speed 30%

This is the case of almost all servers (Dell, etc) which have their specific fan speed control commands. I use ipmitool to control an ASUS server here :

https://github.com/antoinecarme/xeon-phi-data/blob/master/fan_control/fan_control.py

and I have full software control on RPMs.

For some unknown/cryptic reasons (security / safety/ ...) , ipmitool does not work out of the box on SUN machines, There is some missing info to get a full control on fan speeds.

You probably have more info than what I have,

By the way, if some Larry Ellison teams read this, please help if you have more info on setting IPMI.