概要描述

本文描述了，在安装或扩容TDH节点时，部分节点出现报错 Error starting daemon：Devices cgroup isn’t mounted 或Your kernel does not support cgroup memory limit 的解决方法

详细描述

1 报错信息

在安装或扩容TDH节点时，部分节点出现docker无法启动的情况

报错信息与cgroup相关：类似于报错
journalctl -xe docker查看日志报错：

docker Your kernel does not support cgroup memory limit

Error starting daemon：Devices cgroup isn't mounted;

journalctl -xe kubelet查看日志报错：

main process exited, code=exited status=219/cgroup

2 检查cgroup

findmnt | grep cgroup

以下是正常的cgroup挂载

以下是非正常的cgroup挂载

3 解决方法

方案1

如果步骤2里没有cgroup的相关挂载，可以重启服务器进行解决。
一般来说，如果节点安装时候正常，运行一段时间后出现cgroup挂载点丢失，可能是误操作了 umount -a 导致；这种情况下，重启节点后，大概率 cgroupfs 会自动挂载。

方案2

如果客户不能重启，可以考虑执行以下脚本

#!/bin/sh
# Copyright 2011 Canonical, Inc
#           2014 Tianon Gravi
# Author: Serge Hallyn 
#         Tianon Gravi 
set -e

# for simplicity this script provides no flexibility

# if cgroup is mounted by fstab, don't run
# don't get too smart - bail on any uncommented entry with 'cgroup' in it
if grep -v '^#' /etc/fstab | grep -q cgroup; then
    echo 'cgroups mounted from fstab, not mounting /sys/fs/cgroup'
    exit 0
fi

# kernel provides cgroups?
if [ ! -e /proc/cgroups ]; then
    exit 0
fi

# if we don't even have the directory we need, something else must be wrong
if [ ! -d /sys/fs/cgroup ]; then
    exit 0
fi

# mount /sys/fs/cgroup if not already done
if ! mountpoint -q /sys/fs/cgroup; then
    mount -t tmpfs -o uid=0,gid=0,mode=0755 cgroup /sys/fs/cgroup
fi

cd /sys/fs/cgroup

# get/mount list of enabled cgroup controllers
for sys in $(awk '!/^#/ { if ($4 == 1) print $1 }' /proc/cgroups); do
    mkdir -p $sys
    if ! mountpoint -q $sys; then
        if ! mount -n -t cgroup -o $sys cgroup $sys; then
            rmdir $sys || true
        fi
    fi
done

# example /proc/cgroups:
#  #subsys_name hierarchy   num_cgroups enabled
#  cpuset   2   3   1
#  cpu  3   3   1
#  cpuacct  4   3   1
#  memory   5   3   0
#  devices  6   3   1
#  freezer  7   3   1
#  blkio    8   3   1

# enable cgroups memory hierarchy, like systemd does (and lxc/docker desires)
# https://github.com/systemd/systemd/blob/v245/src/core/cgroup.c#L2983
# https://bugs.debian.org/940713
if [ -e /sys/fs/cgroup/memory/memory.use_hierarchy ]; then
    echo 1 > /sys/fs/cgroup/memory/memory.use_hierarchy
fi

if [ -e /sys/fs/cgroup/cpu,cpuacct ]; then
    mount -n -t cgroup -o cpu,cpuacct cgroup cpu,cpuacct
fi

exit 0