리λ μ€ κΈ°λ° μ»¨ν μ΄λ 격리 κΈ°μ (chroot, Namespace, cGroups, Union Filesystem)
μ¬μ©νκ²½
- Azure VM
- Ubuntu 22.04 / 30GM / vCPU 2
컨ν μ΄λ κΈ°λ° κΈ°μ
ν΄λΌμ°λ μλ, νΉν 컨ν
μ΄λ κΈ°λ°μ μ ν리μΌμ΄μ
μ΄μμμλ "μ΄λμλ λκ°μ΄ λμνλ κ°λ²Όμ΄ μ€ν νκ²½"μ΄ ν΅μ¬μ΄λ€.
νμ§λ§ βκ°λ³λ€βλ λ§μ κΈ°μ μ μΌλ‘ λ€μ λ κ°μ§λ₯Ό μλ―Ένλ€:
- μ΄μ체μ λ₯Ό ν΅μ§Έλ‘ 볡μ¬νμ§ μλλ€. β κ°μλ¨Έμ (VM)κ³Όμ κ°μ₯ ν° μ°¨μ΄
- μμκ³Ό νκ²½μ 격리νλ€. β μ¬λ¬ μ±μ΄ μΆ©λ μμ΄ κ°μ OS μμμ λμκ° μ μκ² ν¨
μ΄κ±Έ κ°λ₯νκ² ν΄μ£Όλ 리λ μ€ μ»€λ κΈ°μ μ΄ λ°λ‘ μλμ κ°λ€
1. chroot
2. Namespace
3. cGroups
4. Union filesystem
1. chroot
chroot : 리λ μ€μμ μ 곡νλ νμΌ μμ€ν 격리 κΈ°μ
- νΉμ λλ ν 리λ₯Ό 루νΈ(/)λ‘ κ°μ£Όνκ² ν¨
- λͺ©μ : νλ‘μΈμ€κ° μμ€ν μ λλ¨Έμ§ μμμ μ κ·Ό λͺ» νλλ‘ κ²©λ¦¬
μ€μ΅ μμ : bash μμ /root/newrootλ₯Ό λ£¨νΈ λλ ν 리μ²λΌ μΈμνκ² μ€ν
β chroot /root/newroot /bin/bash
1λ¨κ³: chroot μ€μΉ νμΈ λ° κΈ°λ³Έ λλ ν 리 ꡬ쑰 λ§λ€κΈ°
# chroot λͺ
λ Ήμ΄ νμΈ
chroot --version
# 격리 νκ²½ λλ ν 리 μμ±
mkdir -p newroot/{bin,lib,lib64}
# νΈλ¦¬ ꡬ쑰 νμΈ λꡬ μ€μΉ
apt update
apt install -y tree
tree newroot
2λ¨κ³: 첫 μλ β μ€ν¨ (κ·Έλ¦¬κ³ μ΄μ )
# 격리λ νκ²½μμ bash μ€ν μλ
chroot newroot /bin/bash
# => μ€ν¨: No such file or directory
π§ μμΈ: λ¨μν /bin/bash μ€ννμΌλ§ μλ€κ³ λλ κ² μλ
bashλ μ€νλ λ νμν λμ λΌμ΄λΈλ¬λ¦¬λ€μ λ‘λ©ν΄μΌ νκΈ° λλ¬Έ
3λ¨κ³: μμ‘΄ λΌμ΄λΈλ¬λ¦¬ 볡μ¬
# bash μ€ννμΌ λ³΅μ¬
cp /bin/bash newroot/bin/
# μ΄λ€ λΌμ΄λΈλ¬λ¦¬λ₯Ό νμλ‘ νλμ§ νμΈ
ldd /bin/bash
=> linux-vdso.so.1 (0x00007ffd8957e000)
libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (0x00007eab997ce000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007eab99400000)
/lib64/ld-linux-x86-64.so.2 (0x00007eab99969000)
# λΌμ΄λΈλ¬λ¦¬ λλ ν 리 μμ±
mkdir -p newroot/lib/x86_64-linux-gnu
mkdir -p newroot/lib64
# νμν λΌμ΄λΈλ¬λ¦¬ 볡μ¬
cp /lib/x86_64-linux-gnu/libtinfo.so.6 newroot/lib/x86_64-linux-gnu/
cp /lib/x86_64-linux-gnu/libc.so.6 newroot/lib/x86_64-linux-gnu/
cp /lib64/ld-linux-x86-64.so.2 newroot/lib64/
μ¬κΈ°κΉμ§ νλ©΄, μ΅μν bashκ° μ€νλ μ μλ μ΅μ μ€ν νκ²½μ΄ κ°μΆ°μ§λ€
μλλ νΈλ¦¬ ꡬ쑰
root@master:~# tree
newroot
βββ bin
β βββ bash
βββ lib
β βββ x86_64-linux-gnu
β βββ libc.so.6
β βββ libtinfo.so.6
βββ lib64
βββ ld-linux-x86-64.so.2
4λ¨κ³: chroot μ¬μλ β μ±κ³΅
root@master:~# chroot newroot /bin/bash
bash-5.1#
μμ νλ©΄μ²λΌ bashλ‘ μ μλλ©΄ chroot μ μ μλ
bash-5.1# pwd
/
bash-5.1# ls
bash: ls: command not found
μμΉλ₯Ό νμΈν΄λ³΄λ©΄ /, 루νΈμ μμΉν¨
/bin/bash/ λ§ λ³΅μ¬νκΈ°μ lsμ κ°μ λͺ λ Ήμ΄λ 격리λ 루νΈμμ μ€νλμ§ μλλ€
μμ κ°μ λ°©λ²λλ‘ μ°Έμ‘°ν λΌμ΄λΈλ¬λ¦¬λ€μ 볡μ¬νλ©΄ μ€ν κ°λ₯νλ€
bash-5.1# exit
exit
root@master:~#
exitλ₯Ό μ€ννλ©΄ μλ 루νΈλ‘ 볡κ·
2. Namepace
Namespace : νλ‘μΈμ€μ μμμ 격리
- μ¬λ¬ μ’ λ₯κ° μμ: PID, UTS(hostname), NET, MNT, USER λ±
- λͺ©μ : νλμ 리λ μ€ OS μμμλ λ§μΉ μ¬λ¬ OSκ° λμκ°λ κ²μ²λΌ 격리
μ€μ΅ μμ : unshare --fork --pid --mount-proc /bin/sh
β μ΄ μ μμμ λ΄κ° PID 1λ² νλ‘μΈμ€!
PID, UTS, NET Namepace 3κ°μ§ μ’ λ₯μ λν΄ μ€μ΅ μ§ν
PID Namespace
- νλ‘μΈμ€ ID (PID) λ₯Ό λΆλ¦¬νμ¬ κ΄λ¦¬νλ€
- νλ‘μΈμ€ ID 곡κ°μ λΆλ¦¬ν΄μ, κ° λ€μμ€νμ΄μ€λ§λ€ PID 1λ²λΆν° λ 립μ μΌλ‘ μμν μ μκ² ν΄μ€λ€.
- μ¦, νλμ μμ€ν μμ μλ‘ λ€λ₯Έ PIDλ₯Ό κ°μ§ "μμ 리λ μ€ νκ²½"μ λ§λ€ μ μλ€.
# νμ¬ μμ PID νμΈ
echo $$
# μλ‘μ΄ PID λ€μμ€νμ΄μ€ μμ± + /proc λ§μ΄νΈ
unshare --fork --pid --mount-proc=/proc /bin/sh
- unshare --fork --pid --mount-proc=/proc /bin/sh
- μ΄ λͺ
λ Ήμ΄λ νμ¬ μλ‘λΆν° λΆλ¦¬λ μλ‘μ΄ PID λ€μμ€νμ΄μ€λ₯Ό λ§λ€κ³ ,
κ·Έ μμμ λ³λλ‘ /proc νμΌμμ€ν μ λ§μ΄νΈν μνλ‘ μ μμ μ€ννλ κ²
μ΅μ | μλ―Έ |
--pid | μλ‘μ΄ PID λ€μμ€νμ΄μ€ μμ± |
--fork | μμ νλ‘μΈμ€λ‘ μλ‘μ΄ μμ μ€ν (νμ¬ μμ κ·Έλλ‘ μ μ§) |
--mount-proc=/proc | μλ‘μ΄ /proc νμΌμμ€ν λ§μ΄νΈ (PID λ€μμ€νμ΄μ€μ λ§κ²) |
/bin/sh | μ€νν μ νλ‘μΈμ€ |
μ€μ λ‘ μ€νν΄λ³΄λ©΄
root@master:~# echo $$
16818
root@master:~# unshare --fork --pid --mount-proc=/proc /bin/sh
# ps
PID TTY TIME CMD
1 pts/1 00:00:00 sh
2 pts/1 00:00:00 ps
π― μ΄κ² ν΅μ¬!
sh νλ‘μΈμ€κ° PID 1λ², psλ 2λ²μ΄λ€.
λ§μΉ μ΄ μμμ λ΄κ° "리λ
μ€λ₯Ό λΆν
ν΄μ μ΅μ΄λ‘ μ€νλ νλ‘μΈμ€"μ²λΌ 보μΈλ€.
μ€μ μμ€ν
μμ μ΄ νλ‘μΈμ€κ° PID 16818μ΄μμ§λ§, λ΄λΆμ μΌλ‘λ μμ ν λ€λ₯Έ νλ‘μΈμ€ ID 곡κ°μ΄ λ§λ€μ΄μ§ κ².
# exit
/bin/sh: 7: Cannot set tty process group (No such process)
λ§μ°¬κ°μ§λ‘ exitλ‘ νμΆ κ°λ₯
UTS Namespace
νΈμ€νΈ μ΄λ¦(hostname) κ³Ό λλ©μΈ μ΄λ¦μ 격리νλ 곡κ°μ΄λ€.
λ€μμ€νμ΄μ€ μμμ μμ€ν
μ΄λ¦μ λ°κΏλ λ€λ₯Έ νκ²½μμ μν₯μ λ°μ§ μλλ€.
μ¦, νλμ 리λ
μ€μμ μ¬λ¬ κ°μ λ
립λ μ»΄ν¨ν° μ΄λ¦μ κ°μ§ νκ²½μ λ§λ€ μ μλ€.
ν΄λΉ μ€μ΅μ νλμ μλ²λ₯Ό 2κ°μ ν°λ―Έλ(terminal1, 2) λ‘ λμ°κ³ λΉκ΅νλ©° μ§ννλ€
β μ€μ΅: νΈμ€νΈ μ΄λ¦ 격리 νμΈνκΈ°
# (λ κ°μ ν°λ―Έλ μ΄κΈ°) κ°κ° terminal1, terminal2
# κ° ν°λ―Έλμμ νμ¬ hostname νμΈ
uname -n
hostname
----
root@master:~# uname -n
master
root@master:~# hostname
master
β λ ν°λ―Έλ λͺ¨λ κ°μ μ΄λ¦ μΆλ ₯λ¨
β μ΄μ terminal1μμ μλ‘μ΄ UTS λ€μμ€νμ΄μ€ μμ
unshare -u /bin/bash
hostname happy.life.com
-----
root@master:~# unshare -u /bin/bash
root@master:~# hostname happy.life.com
root@master:~# hostname
happy.life.com
root@master:~# uname -n
happy.life.com
β μ΄ μ Έ λ΄λΆμμλ hostnameμ΄ happy.life.comμΌλ‘ λ°λ.
π κ²°κ³Ό λΉκ΅: μ§μ§ 격리λμμκΉ?
# terminal1 (μ λ€μμ€νμ΄μ€ μ)
hostname # β happy.life.com
uname -n # β happy.life.com
# terminal2 (κΈ°μ‘΄ μμ€ν
νκ²½)
hostname # β master
uname -n # β master
κ°μ κ°μ λ¨Έμ μμ μλ‘ λ€λ₯Έ νΈμ€νΈλ€μμ μΆλ ₯νλ κ²μ νμΈν μ μλ€
π λ΄λΆ ꡬ쑰 νμΈ β /procμ ν΅ν΄ λ€μμ€νμ΄μ€ ID λΉκ΅
# terminal1
ls -l /proc/$$/ns/uts
# β uts:[4026532377]
---
root@master:~# uname -n
happy.life.com
root@master:~# ls -l /proc/$$/ns/uts
lrwxrwxrwx 1 root root 0 Jun 18 12:10 /proc/17039/ns/uts -> 'uts:[4026532377]'
# terminal2
ls -l /proc/$$/ns/uts
# β uts:[4026531838]
---
root@master:~# uname -n
master
root@master:~# ls -l /proc/$$/ns/uts
lrwxrwxrwx 1 root root 0 Jun 18 12:10 /proc/17024/ns/uts -> 'uts:[4026531838]'
uts:[μ«μ] κ°μ΄ λ€λ₯΄λ€ β μλ‘ λ€λ₯Έ UTS λ€μμ€νμ΄μ€μ μλ€λ μ¦κ±°
NET Namespace
NET λ€μμ€νμ΄μ€λ 리λ μ€μμ λ€νΈμν¬ μμμ 격리νλ ν΅μ¬ κΈ°μ λ‘, λ€μκ³Ό κ°μ νΉμ§μ κ°μ§λ€:
- κ° λ€μμ€νμ΄μ€λ§λ€ κ³ μ ν μΈν°νμ΄μ€μ IP ꡬμ±μ κ°λλ€
β μλ‘ λ€λ₯Έ 컨ν μ΄λλ λ 립λ λ€νΈμν¬ κ³΅κ°μμ λμνλ€ - λμΌν ν¬νΈλ₯Ό λμμ λ°μΈλ©ν΄λ μΆ©λμ΄ μλ€
β κ° λ€μμ€νμ΄μ€ μμμ ν¬νΈκ° λ 립μ μΌλ‘ κ΄λ¦¬λκΈ° λλ¬Έ - λΌμ°ν
ν
μ΄λΈ, λ°©νλ²½ κ·μΉ(iptables), λ€νΈμν¬ μ₯μΉ λ±λ μμ ν λΆλ¦¬λλ€
β νλμ μμ€ν μ΄ μ¬λ¬ λ€νΈμν¬λ₯Ό λμμ κ°λ κ²μ²λΌ κ΅¬μ± κ°λ₯
μ€μ΅ λͺ©ν
- μλ‘μ΄ λ€νΈμν¬ λ€μμ€νμ΄μ€(guestnet)λ₯Ό λ§λ€κ³
- κΈ°λ³Έ λ€μμ€νμ΄μ€(default)μ veth νμ΄ μΈν°νμ΄μ€λ‘ μ°κ²°
- κ° μΈν°νμ΄μ€μ IPλ₯Ό ν λΉν λ€, μλ‘ ping ν΅μ μ΄ κ°λ₯νμ§ νμΈν΄λ³Έλ€
μλ μ°Έκ³ νλ©΄ μ’μ λ΄μ©λ€
https://devocean.sk.com/blog/techBoardDetail.do?ID=163803
10λΆλ§μ μ΄ν΄νλ 컨ν μ΄λ λ€νΈμν¬
devocean.sk.com
https://80000coding.oopy.io/a6a43912-cd6e-485e-9031-fbc3969aa335
리λ μ€ λ€νΈμν¬ λ€μμ€νμ΄μ€ μ΄ν΄λ³΄κΈ°
리λ μ€ λ€νΈμν¬ λ€μμ€νμ΄μ€λ?
80000coding.oopy.io
https://malwareanalysis.tistory.com/249
μΏ λ²λ€ν°μ€ λ€νΈμν¬ μ€ν°λ 1μ£Όμ°¨ 2νΈ: λ€νΈμν¬ λ€μμ€νμ΄μ€
μ€ν°λ λͺ©μ°¨ 1μ£Όμ°¨ 컨ν μ΄λ 격리 - https://malwareanalysis.tistory.com/248 λ€νΈμν¬ λ€μμ€νμ΄μ€ - https://malwareanalysis.tistory.com/249 2μ£Όμ°¨ Flannel CNI: https://malwareanalysis.tistory.com/254 pause 컨ν μ΄λ: https://malwar
malwareanalysis.tistory.com
β Step 1. λ€μμ€νμ΄μ€ μμ± λ° loopback νμ±ν
# μλ‘μ΄ NET λ€μμ€νμ΄μ€ guestnet μμ±
ip netns add guestnet
# λ΄λΆ loopback μΈν°νμ΄μ€ νμ±ν
ip netns exec guestnet ip link
ip netns exec guestnet ip link set lo up
root@master:~# ip netns exec guestnet ip link
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
root@master:~# ip netns exec guestnet ip link set lo up
root@master:~# ip netns exec guestnet ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
lo: <LOOPBACK> => lo: <LOOPBACK,UP,LOWER_UP>
loλ κΈ°λ³Έμ μΌλ‘ down μνμ΄λ―λ‘ μλμΌλ‘ μ¬λ €μ€μΌ ν¨
β Step 2. veth νμ΄ μΈν°νμ΄μ€ μμ± λ° λΆλ¦¬
μΈν°νμ΄μ€ | λ€μμ€νμ΄μ€ |
host | default |
guest | guestnet |
# veth νμ΄ μμ±: host <-> guest
ip link add host type veth peer name guest
# guest μΈν°νμ΄μ€λ₯Ό guestnet λ€μμ€νμ΄μ€λ‘ μ΄λ
ip link set guest netns guestnet
# νμΈ
ip link # guest μ¬λΌμ§ β μ΄λλ¨
ip netns exec guestnet ip link # guest λ±μ₯
μ€ν κ²°κ³Ό
root@master:~# ip link add host type veth peer name guest
root@master:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 00:22:48:05:3f:0d brd ff:ff:ff:ff:ff:ff
3: enP61412s1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master eth0 state UP mode DEFAULT group default qlen 1000
link/ether 00:22:48:05:3f:0d brd ff:ff:ff:ff:ff:ff
altname enP61412p0s2
4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
link/ether 1a:58:0f:2f:2a:b0 brd ff:ff:ff:ff:ff:ff
29: veth97b1361@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
link/ether 6a:00:54:d9:dc:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 0
30: veth2729f05@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
link/ether 8e:bf:84:65:34:cc brd ff:ff:ff:ff:ff:ff link-netnsid 1
33: veth3dc75bf@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
link/ether 86:79:fa:e7:a4:2c brd ff:ff:ff:ff:ff:ff link-netnsid 2
34: guest@host: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether f6:87:fa:a4:91:bb brd ff:ff:ff:ff:ff:ff
35: host@guest: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 26:4c:e4:cd:ee:c3 brd ff:ff:ff:ff:ff:ff
34: guest@host:...
35: host@guest:...
μ΄λ κ² host <-> guest κ°μ veth νμ΄ μμ± νμΈ
root@master:~# ip link set guest netns guestnet
root@master:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 00:22:48:05:3f:0d brd ff:ff:ff:ff:ff:ff
3: enP61412s1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master eth0 state UP mode DEFAULT group default qlen 1000
link/ether 00:22:48:05:3f:0d brd ff:ff:ff:ff:ff:ff
altname enP61412p0s2
4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
link/ether 1a:58:0f:2f:2a:b0 brd ff:ff:ff:ff:ff:ff
29: veth97b1361@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
link/ether 6a:00:54:d9:dc:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 0
30: veth2729f05@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
link/ether 8e:bf:84:65:34:cc brd ff:ff:ff:ff:ff:ff link-netnsid 1
33: veth3dc75bf@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
link/ether 86:79:fa:e7:a4:2c brd ff:ff:ff:ff:ff:ff link-netnsid 2
35: host@if34: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 26:4c:e4:cd:ee:c3 brd ff:ff:ff:ff:ff:ff link-netns guestnet
guestλ₯Ό μλ‘ μμ±ν guestnetμΌλ‘ μ΄λμμΌ°λλ
- μμλ μμλ 34: guest@host:... κ° μ¬λΌμ§ => κΈ°λ³Έ λ€μμ€νμ΄μ€(default)μμλ guest μΈν°νμ΄μ€κ° μ¬λΌμ§λ€
- guestnet λ€μμ€νμ΄μ€ λ΄λΆλ‘ λ€μ΄κ°μ νμΈνλ©΄, κ·Έ μμ guest μΈν°νμ΄μ€κ° μκΈ΄λ€.
- default μͺ½μ λ¨μ host μΈν°νμ΄μ€λ μ΄λ κ² λ°λλ€:
- 35: host@guest:... => 35: host@if34:...
- μ¬κΈ°μ if34λ guest μΈν°νμ΄μ€κ° μ΄μ μ μ°λ μΈν°νμ΄μ€ μΈλ±μ€λ₯Ό μλ―Ένλ€.
- hostλ μ¬μ ν guestμ μ°κ²°λμ΄ μμ§λ§, μ΄μ κ·Έ μ°κ²° μλλ guestnetμ΄λΌλ λ€λ₯Έ λ€μμ€νμ΄μ€ μμ μ‘΄μ¬νλ κ²μ΄λ€.
β Step 3. κ° μΈν°νμ΄μ€μ IP μ£Όμ ν λΉ
host(veth) IP (2.2.2.1/24) ν λΉ
# default λ€μμ€νμ΄μ€ (host)
ip link set host up
ip address add 2.2.2.1/24 dev host
---
root@master:~# ip link set host up
root@master:~# ip address add 2.2.2.1/24 dev host
root@master:~# ip address show dev host
35: host@if34: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state LOWERLAYERDOWN group default qlen 1000
link/ether 26:4c:e4:cd:ee:c3 brd ff:ff:ff:ff:ff:ff link-netns guestnet
inet 2.2.2.1/24 scope global host
valid_lft forever preferred_lft forever
guest (veth) IP (2.2.2.2/24) ν λΉ
# guestnet λ€μμ€νμ΄μ€ (guest)
ip netns exec guestnet ip link set guest up
ip netns exec guestnet ip address add 2.2.2.2/24 dev guest
---
root@master:~# ip netns exec guestnet ip link set guest up
root@master:~# ip netns exec guestnet ip address add 2.2.2.2/24 dev guest
root@master:~# ip netns exec guestnet ip address show dev guest
34: guest@if35: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether f6:87:fa:a4:91:bb brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 2.2.2.2/24 scope global guest
valid_lft forever preferred_lft forever
inet6 fe80::f487:faff:fea4:91bb/64 scope link
valid_lft forever preferred_lft forever
μ¬κΈ°κΉμ§ μ§ννλ€λ©΄ μλμ κ°μ ꡬ쑰λ₯Ό κ°κ² λλ€
[ default namespace ] [ guestnet namespace ]
----------------------- -----------------------
| | | |
| host (veth) | <==============> | guest (veth) |
| IP: 2.2.2.1/24 | veth pair | IP: 2.2.2.2/24 |
| | | |
----------------------- -----------------------
β Step 4. λ€μμ€νμ΄μ€ κ° ν΅μ ν μ€νΈ
# default β guestnet
ping 2.2.2.2
# guestnet β default
ip netns exec guestnet ping 2.2.2.1
root@master:~# ping 2.2.2.2
PING 2.2.2.2 (2.2.2.2) 56(84) bytes of data.
64 bytes from 2.2.2.2: icmp_seq=1 ttl=64 time=0.112 ms
64 bytes from 2.2.2.2: icmp_seq=2 ttl=64 time=0.057 ms
64 bytes from 2.2.2.2: icmp_seq=3 ttl=64 time=0.102 ms
64 bytes from 2.2.2.2: icmp_seq=4 ttl=64 time=0.066 ms
64 bytes from 2.2.2.2: icmp_seq=5 ttl=64 time=0.058 ms
--- 2.2.2.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4131ms
rtt min/avg/max/mdev = 0.057/0.079/0.112/0.023 ms
root@master:~# ip netns exec guestnet ping 2.2.2.1
PING 2.2.2.1 (2.2.2.1) 56(84) bytes of data.
64 bytes from 2.2.2.1: icmp_seq=1 ttl=64 time=0.063 ms
64 bytes from 2.2.2.1: icmp_seq=2 ttl=64 time=0.078 ms
64 bytes from 2.2.2.1: icmp_seq=3 ttl=64 time=0.063 ms
64 bytes from 2.2.2.1: icmp_seq=4 ttl=64 time=0.081 ms
64 bytes from 2.2.2.1: icmp_seq=5 ttl=64 time=0.061 ms
--- 2.2.2.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4133ms
rtt min/avg/max/mdev = 0.061/0.069/0.081/0.008 ms
β κ²°κ³Ό: μλ‘ λ€λ₯Έ λ€μμ€νμ΄μ€μ μμΌλ©΄μλ vethλ‘ μ°κ²°λμκΈ° λλ¬Έμ ν΅μ κ°λ₯!
3. cGropus
cgroups: μμ(CPU, λ©λͺ¨λ¦¬ λ±)μ μ ννκ³ ν λΉ
- 리λ μ€ μ»€λμ΄ μ 곡νλ κΈ°λ₯μΌλ‘, νΉμ νλ‘μΈμ€(λλ κ·Έλ£Ή)κ° CPU, λ©λͺ¨λ¦¬, λ€νΈμν¬, λμ€ν¬ λ±μ μΌλ§λ μ¬μ©ν μ§ μ ννκ±°λ λͺ¨λν°λ§ν μ μλ€.
- λͺ©μ : μ΄λ€ 컨ν μ΄λκ° μμμ λ μνμ§ λͺ»νκ² λ§μ.
- μλ² μμμ λκ° μΌλ§λ μΈ μ μλμ§ μ ν΄μ£Όλ κ΄λ¦¬μ λꡬλ‘, CPU μ¬μ©λ₯ , λ©λͺ¨λ¦¬ μ ν λ±μ μ€μ κ°λ₯
ꡬ쑰λ μ΄λ»κ² μκ²Όλ?
- /sys/fs/cgroup μλμ ν΄λμ²λΌ κ³μΈ΅ κ΅¬μ‘°λ‘ κ΅¬μ±λμ΄ μλ€
- μ΄ ν΄λ ꡬ쑰 μμ μ€μ νμΌλ€μ μμ ν΄μ μμμ μ μ΄ν¨
- λΆλͺ¨ κ·Έλ£Ήμ΄ μ€μ ν μμ μ νμ μμ κ·Έλ£Ήμ΄ λ¬Όλ €λ°μ μ μμ (μμ ꡬ쑰)
cGroupμ μ μ΄νλ λ°©λ²
1. μ§μ νμΌ μμ (VFS λ°©μ)
- λλ ν 리 λ§λ€κ³ , μ€μ νμΌ νΈμ§
- μ: echo "50000 100000" > cpu.max
2. λꡬ μ¬μ©
- Ubuntu: cgroup-bin
- RHEL: libcgroup
- β λͺ λ Ήμ΄ κΈ°λ°μΌλ‘ μμ ν λΉ λ° κ·Έλ£Ή κ΅¬μ± κ°λ₯
μ€μ΅μ μ§μ νμΌ μμ νμ¬ μ§ν
β Step 1. CPUλ₯Ό 무νν μ¬μ©νλ νλ‘κ·Έλ¨ λ§λ€κΈ°
# μ
λ°μ΄νΈ
apt update
apt install -y gcc
# vimμΌλ‘ a.c νμΌ μμ± λ° μλ μ½λ μΆκ°
vi a.c
# C μ½λ μμ± (a.c)
void main() {
for(;;);
}
# μ»΄νμΌ ν μ€ν (λ°±κ·ΈλΌμ΄λλ‘)
gcc a.c
./a.out &

a.c μ»΄νμΌ ν a.out μ΄λΌλ νμΌ μμ±λλ€
μ΄ νμΌμ μ€νμν€κ³ λ€λ₯Έ ν°λ―Έλμμ topμΌλ‘ CPU μ¬μ©λμ 체ν¬ν΄λ³΄λ©΄ a.outμ΄ 100%λ₯Ό μ°¨μ§νλ κ²μ νμΈν μ μλ€
β Step 2. cGroup λλ ν 리 μμ± λ° μ§μ
sudo mkdir /sys/fs/cgroup/mygroup
cd /sys/fs/cgroup/mygroup
mygroup, μ΄ λλ ν 리λ μ°λ¦¬κ° κ΄λ¦¬ν "μμ κ·Έλ£Ή"μ΄λΌκ³ 보면 λλ€
μ΄ μμ μ€μ νμΌμ λ£μ΄μ μ ν 쑰건μ μ§μ κ±Έ μ μλ€

λΆλͺ¨ κ·Έλ£Ήμ΄ μ€μ ν μμ μ νμ μμ κ·Έλ£Ήμ΄ λ¬Όλ €λ°μ μ μμ (μμ ꡬ쑰)
β‘οΈ μμ±λ mygroupμ λ€μ΄κ°λ³΄λ©΄ λΉ λλ ν λ¦¬κ° μλ λ§μ νμΌλ€μ΄ μ‘΄μ¬νλλ° μ΄λ /sys/fs/cgroup μ νμΌλ€μ μμλ°μκΈ° λλ¬Έμ΄λ€
cgroupμλμ μλ λλ ν 리λ€μ νμΌλ€κΉμ§ λ€ κ°μ Έμ€κΈ° λλ¬Έμ cgroup λ³΄λ€ νμΌλ€μ΄ μ’ λ λ§μ보μΈλ€
β Step 3. CPU μ¬μ©λ μ ν μ€μ
# CPU μ ν μ€μ : 100ms μ£ΌκΈ° μ€ 50msλ§ μ€ν κ°λ₯ (μ¦, 50% μ ν)
echo "50000 100000" | sudo tee cpu.max
μ€μ νλͺ© | μλ―Έ |
50000 | μ¬μ©ν μ μλ μκ° (microsecond λ¨μ, 50ms) |
100000 | μ 체 μ£ΌκΈ° (100ms) β λ°λΌμ 50% μ¬μ© μ ν |
β Step 4. νλ‘μΈμ€λ₯Ό cGroupμ λ±λ‘
# a.outμ PIDλ₯Ό ν΄λΉ κ·Έλ£Ήμ λ±λ‘
echo $(pgrep a.out) | sudo tee cgroup.procs
μ΄ μκ°λΆν° ν΄λΉ νλ‘μΈμ€λ cpu.maxμμ μ μν μ μ½ μ‘°κ±΄μ μ μ©μ λ°λλ€
μ€ν κ²°κ³Ό
root@master:/sys/fs/cgroup/mygroup# echo "50000 100000" | sudo tee cpu.max
50000 100000
root@master:/sys/fs/cgroup/mygroup# echo $(pgrep a.out) | sudo tee cgroup.procs
17453

μ΄μ μ 100%λ‘ μ¬μ©λλ CPU μ¬μ©λμ΄ 50%λ‘ μ νλ κ²μ νμΈν μ μλ€
β cgroup μ€μ μ κΈ°λ³Έκ°(μ ν μμ)μΌλ‘ λλ리λ λ°©λ²
κΈ°λ³Έκ°μΌλ‘ λλ리기
# 무μ νμΌλ‘ μ€μ
echo "max 100000" | sudo tee cpu.max
κ·Έλ£Ή ν΄μ (νλ‘μΈμ€λ₯Ό κ·Έλ£Ήμμ λΉΌλ΄κΈ°)
# ν΄λΉ PIDλ₯Ό λ€λ₯Έ κ·Έλ£Ή(cgroup.procs)μΌλ‘ μ΄λ
# μ: root λν΄νΈ κ·Έλ£ΉμΌλ‘ μ΄λ
echo <PID> | sudo tee /sys/fs/cgroup/cgroup.procs
cgroup λλ ν 리 μμ (μ£Όμ νμ)
cd /sys/fs/cgroup
sudo rmdir mygroup
4. Union Filesystem
Union Filesystem: λ μ΄μ΄λ‘ μμ ν¨μ¨μ μ΄λ―Έμ§ ꡬμ±
- μ΄λ―Έμ§λ€μ λ μ΄μ΄λ‘ κ΄λ¦¬ β μ€λ³΅ μ μ₯ μμ
- λνμ μΌλ‘ AUFS, OverlayFS μ¬μ©
μ€μ΅ μμ : overlayfsλ₯Ό μ§μ mount ν΄μ μ¬λ¬ μ΄λ―Έμ§ λ μ΄μ΄ κ²°ν©νλ λ°©μ νμΈ
μ€μ΅ λͺ©ν
- OverlayFSλ‘ λ€μ€ μ΄λ―Έμ§ λ μ΄μ΄λ₯Ό λ³ν©ν΄μ κ°μμ ν΅ν© λλ ν 리λ₯Ό λ§λ€κ³ ,
- κ·Έ μμμ νμΌ μμ /μΆκ°κ° μ΄λ»κ² λμνλμ§λ₯Ό νμΈνλ€.
β Step 1. μ€μ΅ νκ²½ ꡬμ±
# μ€μ΅ λλ ν 리 μμ±
mkdir overlayfs && cd overlayfs
mkdir container image1 image2 work merge
# κ°κ°μ λ μ΄μ΄μ νμΌ μμ±
touch image1/{a,b} image2/c
---
βββ container
βββ image1
β βββ a
β βββ b
βββ image2
β βββ c
βββ merge
βββ work
λλ ν 리 | μ©λ |
image1, image2 | μ½κΈ° μ μ© μ΄λ―Έμ§ κ³μΈ΅ (lower) |
container | μ°κΈ° κ°λ₯ν upper layer |
work | OverlayFS λμμ μν μν¬ λλ ν 리 |
merge | μ°λ¦¬κ° μ κ·Όν ν΅ν© λλ ν 리 (mount point) |
β Step 2. OverlayFS λ§μ΄νΈ
mount -t overlay overlay \
-o lowerdir=image2:image1,upperdir=container,workdir=work \
merge
π μ€λͺ :
- lowerdir = image2:image1 β μ΄λ―Έμ§1 μμ μ΄λ―Έμ§2κ° μΉν ꡬ쑰
- upperdir = container β μ°κΈ° λ μ΄μ΄. λ³κ²½μ μ¬κΈ°μ μ μ₯
- workdir = work β λ΄λΆμ μΌλ‘ OverlayFSκ° μ¬μ©νλ μμ μ μ₯μ
- κ²°κ³Όμ μΌλ‘ merge λλ ν 리λ₯Ό 보면, 3κ° λ μ΄μ΄κ° ν©μ³μ§ νλμ νμΌμμ€ν μ²λΌ 보μ
Filesystem | 1K-blocks | Used | Available | Use% | Mounted on |
overlay | 30298176 | 2998448 | 27283344 | 10% | /root/overlayfs/merge |
β Step 3. κ²°κ³Ό νμΈ
tree
βββ container
βββ image1
β βββ a
β βββ b
βββ image2
β βββ c
βββ merge
β βββ a
β βββ b
β βββ c
βββ work
βββ work
β‘οΈ mergeμλ a, b, cκ° λͺ¨λ 보μ β λ μ΄μ΄ λ³ν© μ±κ³΅
β Step 4. νμΌ μμ λ° μΆκ° μ€μ΅
rm merge/a
touch merge/d
tree . -I work
---
βββ container
β βββ a
β βββ d
βββ image1
β βββ a
β βββ b
βββ image2
β βββ c
βββ merge
βββ b
βββ c
βββ d