Ubuntu with Grub2 + LUKS encrypted LVM root + hidden USB keyfile

Posted: January 2nd, 2012 | Author: | Filed under: computing | Tags: , , , , , | 2 Comments »

STANDARD DISCLAIMER APPLIES. USE AT YOUR OWN RISK. This is what worked for me. Your system may be different. IF THINGS GO WRONG YOU’LL BE STUCK DOING MANUAL SYSTEM RECOVERY. This is a high-level guide, not an exact step-by-step and assumes you are familiar with linux administration and command line.

Started with a fresh install of Ubuntu Server (Ubuntu Lucid 10.04.3 LTS) and wanted the same USB keyfile setup as used previously in the links below, but needed to adapt for Grub2.
Debian Lenny + LUKS encrypted root + hidden USB keyfile
Debian Lenny + LUKS encrypted root + hidden USB keyfile (part 2)

I’m too embarrassed to publish how long it took me to figure this out. Let’s just say it’s working now:

1) System setup using the manual partitioner to create crypt container partition and LVM partitions. Use the same process as described here to create and test the usb key and the udev rule file.

2) The keyscript has changed (including modifying the keyscript filename to remove the hyphens, update-initramfs would not copy in the script with hyphens in the filename). Create the following keyscript as “/usr/local/sbin/unlockusbkey.sh”


#!/bin/sh
TRUE=0
FALSE=1

# flag tracking key-file availability
OPENED=$FALSE

# check and modprobe the USB driver if not already loaded
cat /proc/modules | busybox grep usb_storage >/dev/null 2>&1
USBLOAD=0$?
if [ $USBLOAD -gt 0 ]; then
modprobe usb_storage >/dev/null 2>&1
fi

# give the system time to settle and open the USB device
sleep 10

# check for the specifc /dev/usbkey device created by udev using /etc/udev/rules.d/99-unlock-luks.rules
if [ -b /dev/usbkey ]; then
# if device exists then output the keyfile from the usb key (hidden key is 4096 bytes long starting at 2048 bytes)
dd if=/dev/usbkey bs=512 skip=4 count=8 | cat
OPENED=$TRUE
fi

if [ $OPENED -ne $TRUE ]; then
echo "FAILED to get USB key file ..." >&2
if [ -x /bin/plymouth ] && plymouth --ping; then
plymouth ask-for-password --prompt "Enter passphrase"
else
/lib/cryptsetup/askpass "Enter passphrase"
fi
else
echo "Success loading key file. Moving on." >&2
fi

sleep 1
exit 0

Make the script executable:

# chmod a+x /usr/local/sbin/unlockusbkey.sh

3) Create/edit the following files (edit sda2_crypt and the UUID for your system):

/etc/crypttab

sda2_crypt /dev/disk/by-uuid/uuid-goes-here none luks,keyscript=/usr/local/sbin/unlockusbkey.sh

/etc/initramfs-tools/conf.d/cryptroot

CRYPTROOT=target=sda2_crypt,source=/dev/disk/by-uuid/uuid-goes-here

/etc/initramfs-tools/modules

usb_storage

4) Create this file (and make executable) to ensure your custom udev rule gets copied into the initrd image when running update-initramfs.

/etc/initramfs-tools/hooks/udevusbkey


#!/bin/sh
# udev-usbkey script

PREREQ="udev"
prereqs()
{
echo "$PREREQ"
}

case $1 in
prereqs)
prereqs
exit 0
;;
esac

. /usr/share/initramfs-tools/hook-functions

# Copy across relevant rules

cp /etc/udev/rules.d/99-unlock-luks.rules ${DESTDIR}/lib/udev/rules.d/

exit 0

5) Edit your Grub2 config. The key line is “GRUB_CMDLINE_LINUX_DEFAULT”. Notice the initrd keyscript location is different than the previous setups (using cryptsetup 2:1.1.0~rc2-1ubuntu13 and initramfs-tools 0.92bubuntu78).

/etc/default/grub


# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.

GRUB_DEFAULT=0
#GRUB_HIDDEN_TIMEOUT=0
#GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="cryptopts=target=sda2_crypt,source=/dev/disk/by-uuid/uuid-goes-here,lvm=vg-your-root,keyscript=/lib/cryptsetup/scripts/unlockusbkey.sh"
GRUB_CMDLINE_LINUX=""

# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
GRUB_GFXMODE=1280x1024
GRUB_GFXPAYLOAD=1280x1024
GRUB_GFXPAYLOAD_LINUX=1280x1024

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_LINUX_RECOVERY="true"

# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"

6) Run “update-grub” and check the generated /boot/grub/grub.cfg file to verify the cryptopts kernel options were added.

7) Update the initrd image for your current kernel (or just run “update-initramfs -u” to update all kernels):

# update-initramfs -u -k 2.6.XX-XX-server

8) If you want to verify that everything has copied correctly, you can unpack your current initrd.img to the tmp directory and look through the extracted files. The keyscript should have been copied into the “lib/cryptsetup/scripts” directory and and udev rule into “lib/udev/rules.d/”. The grub keyscript line above should match the location of the keyscript in the initrd image.

# cd /tmp
# zcat /boot/initrd.img-2.6.XX-X-amd64 | cpio -iv

9) If all looks in order then reboot. With the USB stick inserted the system should boot all the way to the login prompt. Without the USB stick it should prompt you for your crypt password to mount the root filesystem.

If something goes wrong you’ll likely get dropped to the “(initramfs)” prompt. From here you can manually unlock the crypt partition using “cryptsetup luksOpen /dev/sda2 sda2_crypt” and entering your crypt password. If this unlocks successfully then typing “exit” should drop you back into the automated boot sequence.

————-

Steps for creating & adding additional crypt disks:

1) Use fdisk to create the new partition (using /dev/sdx1 for example) with Type 83, Linux

2) Optional -- good idea to check for badblocks and fill partition with pseudo-random data:
badblocks -c 10240 -s -w -t random -v /dev/sdx1

Or, for truly paranoid types (verrry slow):
dd if=/dev/urandom of=/dev/sdx1

3) Create new luks partition
# cryptsetup --verify-passphrase --verbose --hash=sha256 --cipher=aes-cbc-essiv:sha256 --key-size=256 luksFormat /dev/sdx1
# cryptsetup luksOpen /dev/sdx1 sdx1_crypt

4) create LVM
# pvcreate /dev/mapper/sdx1_crypt
# vgcreate vgname /dev/mapper/sdx1_crypt
# lvcreate -LXXG -n lvname vgname

5) format new LVM partition (example uses ext4 fs)
# mkfs.ext4 /dev/vgname/lvname

6) mount new LV (add to /etc/fstab to make persistent)
# mount /dev/mapper/vgX-nameX /mnt/nameX

7) find the uuid of the encrypted physical partition
# ls -al /dev/disk/by-uuid
uuid-goes-here -> ../../sdx1

or

cryptsetup luksDump /dev/sdx1

8) add new line to /etc/crypttab
sdx1_crypt /dev/disk/by-uuid/uuid-goes-here none luks,keyscript=/usr/local/sbin/unlockusbkey.sh

9) Add the USB key to the new crypt partition:
cryptsetup luksAddKey /dev/sdx1 /root/luks-secret.key --key-slot 1

10) Update the initrd for good measure:
update-initramfs -u

11) profit.


Debian Lenny + LUKS encrypted root + hidden USB keyfile

Posted: May 22nd, 2010 | Author: | Filed under: computing | Tags: , , | 16 Comments »

The setup:

* Recently built up a new 1U Xeon quad core to take over home server duties.

* Installed a fresh copy of Debian Lenny using a USB stick install and the debian-504-amd64-netinst.iso image.

* Setup and partitioned disks manually with the Debian installer:

/dev/sda1 as /boot
/dev/sda2 as encrypted LUKS partition sda2_crypt
sda2_crypt as an LVM physical volume (technically /dev/dm-0)
/dev/mapper/vg0-swap as swap
/dev/mapper/vg0-root as root

* Finished standard install, configured system basics and tested working LUKS setup using normal console password entry during boot.

* Converted system to Proxmox VE running proxmox-ve-2.6.24 kernel. (Shouldn’t be relevant to this how-to).

The Challenge:

All is working well, but since this will be a headless server sitting down in the utility room it’s going to be a PITA to have to physically enter the LUKS password at each reboot. I could setup a remote LUKS passphrase over ssh, but really I want the machine to be able to survive a reboot without my intervention to get it running. So, for my needs a USB key sounds like the ticket.

I found a few nice how-to’s via the Google, but I wanted a few tweaks so I ended up using a blend of the following:
* The passwordless disk encryption in Debian Etch how-to is an excellent guide and provided much of what I needed.
* The Unlocking a luks volume with a USB key how-to doesn’t work for encrypting root, but I liked the idea of hiding the key between the MBR and first partition of the USB stick (yes, I know, security through obscurity is bad.. blah, blah).
* I found a nice udev config in section 2.1 of this post.
* Found the solution of adding kopts parameters to grub from somewhere else I can’t seem to find again.

Getting it done:

This assumes you already have a working LUKS setup using console password entry.

Creating the key

1) Insert your usb stick and use dmesg to identify the device file. We’ll assume /dev/sdx.

2) Fill the entire usb stick with random data (this will erase all data on the usb stick). If you’re extra paranoid and have lots of spare time use /dev/random instead.

# dd if=/dev/urandom of=/dev/sdx bs=1

3) If you still want the usb stick to be usable for storing data you’ll need to recreate the partition table and filesystem. Use fdisk to create a new single partition and mark the partition as type “W95 FAT32” then use mkfs.vfat to format the new partition.

4) Extract 4096 bits of random data off the usb stick that will become the new keyfile from in between the MBR and the first partition.

# dd if=/dev/sdx of=/root/luks-secret.key bs=512 skip=4 count=8

5) Add the key to your LUKS encrypted partition in key slot 1 (your current LUKS password should already be in slot 0). You’ll be prompted for you current password when running this.

# cryptsetup luksAddKey /dev/sda2 /root/luks-secret.key --key-slot 1

Your usb stick is now ready to go.

6) If you don’t want to keep a backup copy of this key on your filesystem then use shred.

# shred --remove --zero /root/luks-secret.key

Creating a udev rule

This will make your specific usb stick available at /dev/usbkey when inserted. Other usb sticks will be ignored even if they contained the same keyfile.

1) Run the following command to get the necessary information about the usb stick.

# udevadm info -a -p $(udevadm info -q path -n /dev/sdx)

2) In the output, look for the section that contains SUBSYSTEMS==”usb”, DRIVERS==”usb”, ATTR{manufacturer}, ATTR{product} and ATTR{serial}. Use this information for creating the rule in the next step.

3) Create a file in “/etc/udev/rules.d/99-unlock-luks.rules” that contains the following (all on one line):

SUBSYSTEMS=="usb", DRIVERS=="usb", ATTRS{manufacturer}=="XYZ Corporation", ATTRS{product}=="Flash Thingy", ATTRS{serial}=="0123456789abc", SYMLINK+="usbkey%n"

4) Reload udev rules with:

# udevadm control --reload-rules

5 ) Test that /dev/usbkey is created when the usb stick is inserted.

Create the keyscript

This will be the shell script responsible for reading the keyfile from the usb stick and passing it via cat to cryptsetup when called upon during boot. If the keyfile is not available or valid then it will revert to asking for your normal LUKS password on the console. The nice part about this keyscript is that by reading the keyfile directly using dd we get to skip worrying about drivers for mounting the usb stick’s filesystem.

1) Create a keyscript containing the following as “/usr/local/sbin/unlock-usb-key.sh”


#!/bin/sh
TRUE=0
FALSE=1

# flag tracking key-file availability
OPENED=$FALSE

# check and modprobe the USB driver if not already loaded
cat /proc/modules | busybox grep usb_storage >/dev/null 2>&1
USBLOAD=0$?
if [ $USBLOAD -gt 0 ]; then
modprobe usb_storage >/dev/null 2>&1
fi

# give the system time to settle and open the USB device
sleep 10

# check for the specifc /dev/usbkey device created by udev using /etc/udev/rules.d/99-unlock-luks.rules
if [ -b /dev/usbkey ]; then
# if device exists then output the keyfile from the usb key (hidden key is 4096 bytes long starting at 2048 bytes)
dd if=/dev/usbkey bs=512 skip=4 count=8 | cat
OPENED=$TRUE
fi

if [ $OPENED -ne $TRUE ]; then
echo "FAILED to get USB key file ..." >&2
/lib/cryptsetup/askpass "Try LUKS password: "
else
echo "Success loading key file. Moving on." >&2
fi

sleep 2

2) Make the script executable:

# chmod a+x /usr/local/sbin/unlock-usb-key.sh

Configure cryptsetup & initramfs-tools

1) Install initramfs-tools package.

2) Add the keyscript parameter to /etc/crypttab

sda2_crypt /dev/sda2 none luks,keyscript=/usr/local/sbin/unlock-usb-key.sh

3) Add the following modules to /etc/initramfs-tools/modules

sha256
aes-x86_64 (or aes-i586 if running 32-bit)
aes_generic
crypto_api
dm-crypt
dm-mod
scsi_dh
usbcore
usbhid
usb_storage

4) Add the following to /etc/initramfs-tools/conf.d/cryptroot

CRYPTROOT=target=sda2_crypt,source=/dev/sda2

5) Ensure “MODULES=most” and “BUSYBOX=y” are set in /etc/initramfs-tools/initramfs.conf

Update grub config & initrd images

Now we need to build a new boot initrd.img that contains the scripts and modules we configured above.
1) First, create a “safe” backup copy of your current initrd image.


# cd /boot

Copy your current initrd version.
# cp initrd.img-2.6.XX-X-amd64 initrd.img-2.6.XX-X-amd64-safe

2) Edit /boot/grub/menu.lst and create a duplicate of your current boot option that utilizes the “safe” initrd.img.


title Debian GNU/Linux, kernel 2.6.XX-X-amd64
root (hd0,0)
kernel /vmlinuz-2.6.XX-X-amd64 root=/dev/mapper/vg0-root ro
initrd /initrd.img-2.6.XX-X-amd64

title Debian GNU/Linux, kernel 2.6.XX-X-amd64-safe
root (hd0,0)
kernel /vmlinuz-2.6.XX-X-amd64 root=/dev/mapper/vg0-root ro
initrd /initrd.img-2.6.XX-X-amd64-safe

3) For some reason I still haven’t figured out (and haven’t spent much time further researching), I needed to add the “cryptopts” to my kernel boot options to make everything work. Otherwise, instead of the init scripts mounting root, I would get errors similar to “LVM driver is detected but LVM is not configured” during boot.
Again, edit /boot/grub/menu.lst and add the “cryptopts” parameters to your current kopts line.

# kopt=root=/dev/mapper/vg0-root ro cryptopts=target=sda2_crypt,source=/dev/sda2,lvm=vg0-root,keyscript=/keyscripts/unlock-usb-key.sh

4) Run “update-grub”. This will update all your boot kernels and the kopts line from the previous step will ensure it’s added to new kernels as well.


title Debian GNU/Linux, kernel 2.6.XX-X-amd64
root (hd0,0)
kernel /vmlinuz-2.6.XX-X-amd64 root=/dev/mapper/vg0-root ro cryptopts=target=sda2_crypt,source=/dev/sda2,lvm=vg0-root,keyscript=/keyscripts/unlock-usb-key.sh
initrd /initrd.img-2.6.XX-X-amd64

5) Now that you have a backup “safe” initrd.img, it’s time to update your current one to include the scripts and modules configured in the previous steps so that they are available at boot. Thanks to the initramfs-tools package, this is as simple as:

# update-initramfs -u -k 2.6.XX-X-amd64

6) If you want to verify that everything has copied correctly, you can unpack your current initrd.img to the tmp directory and look through the extracted files. The keyscript should have been copied into the “keyscripts” folder.

# cd /tmp
# zcat /boot/initrd.img-2.6.XX-X-amd64 | cpio -iv
# ls -al keyscripts/

Testing

1) Reboot with the usb stick installed and select the boot option that uses the new initrd.img you created. The system should boot all the way to the login prompt.
2) Reboot without the usb stick and it should stop at the prompt for your LUKS password.
3) If you run into issues you can reboot with your “safe” kernel.

Advantages

In my opinion there are a couple advantages to this setup:
1) The udev script approach makes it a tiny bit more difficult for someone to use an alternate usb stick even if they had the keyfile.
2) The dd method of “hiding” the keyfile in random deadspace means that even if someone got their hands on your usb stick they wouldn’t know they had a keyfile.

Why?

So.. what’s the point to encrypting if you’re just going to leave the key sitting in the machine? For me, the drive encryption isn’t about protecting against the NSA or uber-l33t hackers. I just want to ensure if the hardware gets ripped off during a break-in that the data is secure. My home setup will allow me to physically secure the usb key nearby and run a usb extension cable from the key to the server. I’ll take my chances that the average criminal would just unplug any cables when taking the machine and regardless, it would be extremely difficult to physically get to the secured usb stick even if they knew to look for it. Of course if you value security over the convenience of unattended reboots then just pull usb stick when not in use.

Enjoy.

Update 10/14/2010 -- corrected several typos.
Update 11/24/2010 -- added part 2: http://www.oxygenimpaired.com/debian-lenny-luks-encrypted-root-hidden-usb-keyfile-part-2
Update 01/02/2011 -- Updated version for Ubuntu/Grub2: http://www.oxygenimpaired.com/ubuntu-with-grub2-luks-encrypted-lvm-root-hidden-usb-keyfile


New hardware is good.

Posted: May 20th, 2010 | Author: | Filed under: computing | Tags: , , , | 1 Comment »

The time has come for a new machine to take over duties as the home server. Instead of stuffing together the normal patchwork of old workstations I’ve decided to pick up some proper hardware that can reliably support a few concurrent virtual machines running on Proxmox VE.

A few combo deals on Newegg later…
* 1x ASUS RS100-E5/PI2 1U Barebone Server (P5BV-M/RS100-E5 mobo)
* 1 x Intel Xeon X3220 Kentsfield 2.4GHz LGA 775 Quad-Core (SLACT, G0 stepping)
* 4 x Kingston ValueRAM 2GB 240-Pin DDR2 SDRAM DDR2 800 (PC2 6400) ECC
* 2 x Western Digital Caviar Black WD1001FALS 1TB 7200 RPM SATA 3.0Gb/s 3.5″ Internal HDDs
* 1 x Intel EXPI9402PT 10/100/1000Mbps PCI-Express PRO/1000 PT Dual Port Server Adapter
* 1 x Nippon Labs eSATA Bracket Two SATA II internal cable to 2 port ESATA bracket for SATA I and SATA II Hard Drive Model ESATAB-2P

That leaves me with 4-cores, 8G of ECC memory, 2 TBs of storage and 4 Gigabit NICs for a little over a grand. Hardware commoditization is a Good Thing™.

Assembly begins:
ASUS RS100-E5/PI2 barebones

 

1 beer. Everything fits easily. Note the 90 degree PCI riser will allow you to install a 16x PCI-e card but the riser only provides an 8x connection to the MB.
ASUS RS100-E5/PI2 partial build

 

2 beers. A little mod to remove the Sata2/eSata cables from their stock bracket and install them neatly with a little cutting to the slim-cd cover plate on the front of the server.
ASUS RS100-E5/PI2 built

Hardware is done and initial testing goes smoothly. BIOS was already up to date for me.

Grabbed a fresh copy of Debian Lenny and installed via USB stick with the debian-504-amd64-netinst.iso image.

lm-sensors 3.1.2 shows me running a steady 57-60°C at idle which seems high to me, but checking the Intel specs it appears normal for the G0 stepping version of this chip. On the plus side, the G0 is actually a 95W thermal design instead of the 105W advertised on Newegg. Under load it climbs well into the 60’s but the fans spin up a little and keep it holding steady within range.

So far I’m impressed. This thing has some nice features for the price point:
* The MB provides 4 SATAII ports, so using the Nippon Sata/eSata connectors I can toss on an additional 2 external mounted drives as long as they have their own power supply.
* Easily expandable for KVM/IPMI with purchase of a module
* The serial console supports remote management so you can skip IPMI if you choose
* 4 (2 front/2 rear) USB connections and legacy PS/2 ports
* Onboard GPU for VGA display
* Quiet for a 1U rack mount. Not silent, but certainly not distracting. Volume picks up with the fans under heavy load, but nothing unexpected.

A few quirks:
* The slim-cd/dvd connector looks like an old style molex. I haven’t bothered to look for a drive.
* Doesn’t include rails and it’s a bit heavy for mounting just by the ears

Following a short burn-in with Debian I converted the system Proxmox VE running the proxmox-ve-2.6.24 kernel. I started with proxmox-ve-2.6.18 but couldn’t get lm-sensors to work due to the outdated “w83627ehf” module. So far everything is running smooth with a mix of a few KVM VMs and OpenVZ appliances. (Edit 10/15/2010 -- I’ve since reformatted this machine to to a bare metal Untangle box as I was able to accomplish everything I’ve needed by hacking up the Untangle/Debian distribution without the overhead of virtualization).

lspci output for the curious:


00:00.0 Host bridge: Intel Corporation 3200/3210 Chipset DRAM Controller (rev 01)
00:01.0 PCI bridge: Intel Corporation 3200/3210 Chipset Host-Primary PCI Express Bridge (rev 01)
00:1c.0 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 1 (rev 01)
00:1c.4 PCI bridge: Intel Corporation 82801GR/GH/GHM (ICH7 Family) PCI Express Port 5 (rev 01)
00:1c.5 PCI bridge: Intel Corporation 82801GR/GH/GHM (ICH7 Family) PCI Express Port 6 (rev 01)
00:1d.0 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #1 (rev 01)
00:1d.1 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #2 (rev 01)
00:1d.2 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #3 (rev 01)
00:1d.7 USB Controller: Intel Corporation 82801G (ICH7 Family) USB2 EHCI Controller (rev 01)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev e1)
00:1f.0 ISA bridge: Intel Corporation 82801GB/GR (ICH7 Family) LPC Interface Bridge (rev 01)
00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE Controller (rev 01)
00:1f.2 SATA controller: Intel Corporation 82801GR/GH (ICH7 Family) SATA AHCI Controller (rev 01)
00:1f.3 SMBus: Intel Corporation 82801G (ICH7 Family) SMBus Controller (rev 01)
01:03.0 VGA compatible controller: XGI Technology Inc. (eXtreme Graphics Innovation) Volari Z7
02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet PCI Express (rev 21)
03:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet PCI Express (rev 21)
05:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06)
05:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06)

My homemade /etc/sensors3.conf file for lm-sensors. Lots of guesswork in here. I welcome suggestions or validation from someone more versed on lm-sensors.

# Winbond W83627EHF configuration
# This is for an Asus P5BV-M mobo.
# w83627dhg-isa-0290

chip "w83627ehf-*" "w83627dhg-*"

label in0 "VCore"
label in1 "+12V"
label in2 "AVCC"
label in3 "3VCC"
label in6 "+5V"
label in7 "VSB"
label in8 "VBAT"

# +12V is in1 and +5V is in6 as recommended by datasheet
compute in1 @*(1+(56/10)), @/(1+(56/10))
compute in6 @*(1+(22/10)), @/(1+(22/10))
set in1_min 12.0*0.9
set in1_max 12.0*1.1
set in6_min 5.0*0.8
set in6_max 5.0*1.2

# Set others
set in4_min 1.75*0.95
set in4_max 1.75*1.05
set in5_min 1.4*0.9
set in5_max 1.4*1.1

# Set the 3.3V
set in2_min 3.3*0.95
set in2_max 3.3*1.05
set in3_min 3.3*0.95
set in3_max 3.3*1.05
set in7_min 3.3*0.95
set in7_max 3.3*1.05
set in8_min 3.1*0.95
set in8_max 3.1*1.05

# Fans
label fan1 "Case Fan"
label fan2 "CPU Fan"
label fan3 "Aux Fan"
set fan2_min 3020
ignore fan1
ignore fan3
ignore fan4
ignore fan5

# Temperatures
label temp1 "Sys Temp"
label temp2 "CPU Temp"
label temp3 "AUX Temp"
set temp1_max 95
set temp1_max_hyst 80
set temp2_max 80
set temp2_max_hyst 75
ignore temp3