利用 ZFS 和 Proxmox VE 自建 NAS
參考下列幾篇
1https://forum.level1techs.com/t/how-to-create-a-nas-using-zfs-and-proxmox-with-pictures/117375
4.https://newtoypia.blogspot.tw/2017/03/zfs.html
5.https://www.freebsd.org/doc/zh_TW/books/handbook/zfs.html#zfs-differences
6.https://github.com/lxc/lxd/blob/master/doc/storage.md
7.https://pthree.org/2012/04/17/install-zfs-on-debian-gnulinux/ ZFS 介紹
8.https://datahunter.org/lxc_mount LXC mount
9.http://wanggen.myweb.hinet.net/index.html 阿旺的 Linux 開竅手冊
先照第一篇跟著做到 part10
zpool list -v, zpool iostat -v 確定pool 有建成功
pvesm zfsscan 確定 pve有找到zfs
遇到
E:Failed to fetch https://enterprise.proxmox.com/…/pve-…/binary-amd64/Packages401 Unauthorized
執行
deb http://download.proxmox.com/debianstretch pve-no-subscription
rm -f /etc/apt/sources.list.d/pve-enterprise.list
因為我們要裝omv 所以跳到第2篇
注意 OMV 3 只能裝在 Debian8 不能用 Debian9
如果遇到 NO_PUBKEY 7E7A6C592EF35D13
apt-get update
apt-get install openmediavault-keyring postfix
apt-get update
apt-get install openmediavault
omv-initsystem
一直做到 5.1 記得 stop container
5.2 We shoult mount LXC container virtual hard drive for example into /mnt directory in ProxmoxVE
mount /dev/mapper/pve-vm–102–disk–1 /mnt/
大部分新手都卡在這步 ，我也是，跟據下面 c0mputerking 的回文，這步不用做
用 df 找subvol-100-disk1 在那來代換下列的命令
mount -t proc none /apool/subvol-150-disk-1/proc/
mount --rbind /dev/ /apool/subvol-150-disk-1/dev/
mount --rbind /sys/ /apool/subvol-150-disk-1/sys/
chroot /apool/subvol-150-disk-1/ /bin/bash
如果以我的例子來做就變成
mount -t proc none /storage/vmstorage/limited/subvol-100-disk-1/proc/
mount --rbind /dev/ /storage/vmstorage/limited/subvol-100-disk-1/dev/
mount --rbind /sys/ /storage/vmstorage/limited/subvol-100-disk-1/sys/
chroot /storage/vmstorage/limited/subvol-100-disk-1/ /bin/bash
這樣就繼續下面的步驟， 我是遇到3個 error，文章只有2個，不過照學它 mv掉
這樣就可以裝好 omv了
總結安裝omv in container 指令過程
1.在host更新pve container
pveam update
2.建立CT
3.在container /etc/pve/lxc/100.conf 加入
lxc.apparmor.profile: unconfined
lxc.mount.auto: cgroup:rw
lxc.mount.auto: proc:rw
lxc.mount.auto: sys:rw
4.更新 container
apt-get update && apt-get dist-upgrade
5.加入 omv repository
echo "deb http://packages.openmediavault.org/public erasmus main" > /etc/apt/sources.list.d/openmediavault.list
apt-get update
apt-get install openmediavault-keyring postfix
apt-get update
apt-get install openmediavault
這時遇到 error stop container, 用 df 找 subvol-100-disk1 在那
mount -t proc none /storage/vmstorage/limited/subvol-100-disk-1/proc/
mount --rbind /dev/ /storage/vmstorage/limited/subvol-100-disk-1/dev/
mount --rbind /sys/ /storage/vmstorage/limited/subvol-100-disk-1/sys/
chroot /storage/vmstorage/limited/subvol-100-disk-1/ /bin/bash
mv /usr/share/openmediavault/initsystem/20hostname /root/
mv /usr/share/openmediavault/initsystem/60rootfs /root/
omv-initsystem
ctrl+D
nano /storage/vmstorage/limited/subvol-100-disk-1/usr/share/openmediavault/mkconf/avahi-daemon
replace rlimit-nproc=3 by #rlimit-nproc=3
建立 visual disk
建立zvol
zfs create -V 8G storage/vmstorage/omv-disk
ls -al /dev/zvol/storage/vmstorage/ ## find omv-disk node
fdisk /dev/zd32 ## create omv-disk partition /dev/zd32p1
mkfs.ext4 /dev/zd32p1
mount /dev/zd32p1 /mnt/omv-disk
找device id
ls -al /dev/zd*
nano /etc/pve/lxc/100.conf
加入
#mp0: volume=/storage/share,mp=/dev/sda1/storage
lxc.cgroup.devices.allow: b 230:32 rwm
lxc.cgroup.devices.allow: b 230:33 rwm
lxc.mount.entry: /dev/zd32p1 /dev/sda1 none bind,create=dir,optional 0 0
---]
lxc.autodev: 1
lxc.hook.autodev: /var/lib/lxc/100/mount-hook.sh
這行為設定 host上 zfs pool storage 到omv container
nano /var/lib/lxc/100/mount-hook.sh
加入
mknod -m 777 ${LXC_ROOTFS_MOUNT}/dev/sda b 230 32
mknod -m 777 ${LXC_ROOTFS_MOUNT}/dev/sda1 b 230 33
chmod +x /var/lib/lxc/100/mount-hook.sh
開機自動mount zvol
在pve的/etc/fstab 加入
/dev/zd16p1 /mnt/omv-disk ext4 defaults 0 0
讓omv 在 web gui 自動mount /dev/sda1
touch /etc/systemd/system/omvmount.service
chmod 664 /etc/systemd/system/omvmount.service
nano /etc/systemd/system/omvmount.service
systemctl daemon-reload
systemctl start omvmount.service
systemctl enable omvmount
nano omvmount.service
[Unit]
Description=Stupid Storage Mount for OMV in lxc
[Service]
Type=oneshot
ExecStart=/bin/mount -a
[Install]
WantedBy=multi-user.target