곡뢀기둝/Docker & Kubernetes

λ¦¬λˆ…μŠ€ 기반 μ»¨ν…Œμ΄λ„ˆ 격리 기술 (chroot, Namespace, cGroups, Union Filesystem)

kaizen_bh 2025. 6. 18. 22:58

 

 

 

 

μ‚¬μš©ν™˜κ²½

- Azure VM

- Ubuntu 22.04 / 30GM / vCPU 2

 

 

μ»¨ν…Œμ΄λ„ˆ 기반 기술

 

ν΄λΌμš°λ“œ μ‹œλŒ€, 특히 μ»¨ν…Œμ΄λ„ˆ 기반의 μ• ν”Œλ¦¬μΌ€μ΄μ…˜ μš΄μ˜μ—μ„œλŠ” "μ–΄λ””μ„œλ‚˜ λ˜‘κ°™μ΄ λ™μž‘ν•˜λŠ” κ°€λ²Όμš΄ μ‹€ν–‰ ν™˜κ²½"이 핡심이닀.
ν•˜μ§€λ§Œ β€˜κ°€λ³λ‹€β€™λŠ” 말은 기술적으둠 λ‹€μŒ 두 κ°€μ§€λ₯Ό μ˜λ―Έν•œλ‹€:

  • 운영체제λ₯Ό ν†΅μ§Έλ‘œ λ³΅μ‚¬ν•˜μ§€ μ•ŠλŠ”λ‹€. β†’ 가상머신(VM)과의 κ°€μž₯ 큰 차이
  • μžμ›κ³Ό ν™˜κ²½μ„ κ²©λ¦¬ν•œλ‹€. β†’ μ—¬λŸ¬ 앱이 좩돌 없이 같은 OS μœ„μ—μ„œ λŒμ•„κ°ˆ 수 있게 함

이걸 κ°€λŠ₯ν•˜κ²Œ ν•΄μ£ΌλŠ” λ¦¬λˆ…μŠ€ 컀널 기술이 λ°”λ‘œ μ•„λž˜μ™€ κ°™λ‹€

 

1. chroot

2. Namespace

3. cGroups

4. Union filesystem

 

 

 

1. chroot

chroot : λ¦¬λˆ…μŠ€μ—μ„œ μ œκ³΅ν•˜λŠ” 파일 μ‹œμŠ€ν…œ 격리 기술

  • νŠΉμ • 디렉토리λ₯Ό 루트(/)둜 κ°„μ£Όν•˜κ²Œ 함
  • λͺ©μ : ν”„λ‘œμ„ΈμŠ€κ°€ μ‹œμŠ€ν…œμ˜ λ‚˜λ¨Έμ§€ μ˜μ—­μ— μ ‘κ·Ό λͺ» ν•˜λ„λ‘ 격리

μ‹€μŠ΅ μš”μ : bash μ‰˜μ„ /root/newrootλ₯Ό 루트 λ””λ ‰ν† λ¦¬μ²˜λŸΌ μΈμ‹ν•˜κ²Œ μ‹€ν–‰
β†’ chroot /root/newroot /bin/bash

 

 

1단계: chroot μ„€μΉ˜ ν™•인 λ° κΈ°λ³Έ λ””렉토리 κ΅¬μ‘° λ§Œλ“€κΈ°

# chroot λͺ…λ Ήμ–΄ 확인
chroot --version

# 격리 ν™˜κ²½ 디렉토리 생성
mkdir -p newroot/{bin,lib,lib64}

# 트리 ꡬ쑰 확인 도ꡬ μ„€μΉ˜
apt update
apt install -y tree
tree newroot

 

 

2단계: 첫 μ‹œλ„ – μ‹€νŒ¨ (그리고 이유)

# 격리된 ν™˜κ²½μ—μ„œ bash μ‹€ν–‰ μ‹œλ„
chroot newroot /bin/bash
# => μ‹€νŒ¨: No such file or directory

 

🧠 원인: λ‹¨μˆœνžˆ /bin/bash μ‹€ν–‰νŒŒμΌλ§Œ μžˆλ‹€κ³  λ˜λŠ” 게 μ•„λ‹˜
bashλŠ” 싀행될 λ•Œ ν•„μš”ν•œ 동적 λΌμ΄λΈŒλŸ¬λ¦¬λ“€μ„ λ‘œλ”©ν•΄μ•Ό ν•˜κΈ° λ•Œλ¬Έ

 

 

3단계: 의쑴 라이브러리 볡사

# bash μ‹€ν–‰νŒŒμΌ 볡사
cp /bin/bash newroot/bin/

# μ–΄λ–€ 라이브러리λ₯Ό ν•„μš”λ‘œ ν•˜λŠ”μ§€ 확인
ldd /bin/bash
  =>    linux-vdso.so.1 (0x00007ffd8957e000)
        libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (0x00007eab997ce000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007eab99400000)
        /lib64/ld-linux-x86-64.so.2 (0x00007eab99969000)


# 라이브러리 디렉토리 생성
mkdir -p newroot/lib/x86_64-linux-gnu
mkdir -p newroot/lib64

# ν•„μš”ν•œ 라이브러리 볡사
cp /lib/x86_64-linux-gnu/libtinfo.so.6 newroot/lib/x86_64-linux-gnu/
cp /lib/x86_64-linux-gnu/libc.so.6     newroot/lib/x86_64-linux-gnu/
cp /lib64/ld-linux-x86-64.so.2         newroot/lib64/

 

μ—¬κΈ°κΉŒμ§€ ν•˜λ©΄, μ΅œμ†Œν•œ bashκ°€ 싀행될 수 μžˆλŠ” μ΅œμ†Œ μ‹€ν–‰ ν™˜κ²½μ΄ κ°–μΆ°μ§„λ‹€

μ•„λž˜λŠ” 트리 ꡬ쑰

root@master:~# tree
newroot
β”œβ”€β”€ bin
β”‚   └── bash
β”œβ”€β”€ lib
β”‚   └── x86_64-linux-gnu
β”‚       β”œβ”€β”€ libc.so.6
β”‚       └── libtinfo.so.6
└── lib64
    └── ld-linux-x86-64.so.2

 

 

 

4단계: chroot μž¬μ‹œλ„ – 성곡

root@master:~# chroot newroot /bin/bash
bash-5.1#

 

μœ„μ˜ ν™”λ©΄μ²˜λŸΌ bash둜 μ ‘μ†λ˜λ©΄ chroot 정상 μž‘λ™

 

bash-5.1# pwd
/
bash-5.1# ls
bash: ls: command not found

 

μœ„μΉ˜λ₯Ό 확인해보면 /, λ£¨νŠΈμ— μœ„μΉ˜ν•¨

/bin/bash/ 만 λ³΅μ‚¬ν–ˆκΈ°μ— ls와 같은 λͺ…λ Ήμ–΄λŠ” 격리된 λ£¨νŠΈμ—μ„œ μ‹€ν–‰λ˜μ§€ μ•ŠλŠ”λ‹€

μœ„μ™€ 같은 λ°©λ²•λŒ€λ‘œ μ°Έμ‘°ν•œ λΌμ΄λΈŒλŸ¬λ¦¬λ“€μ„ λ³΅μ‚¬ν•˜λ©΄ μ‹€ν–‰ κ°€λŠ₯ν•˜λ‹€

 

bash-5.1# exit
exit
root@master:~#

 

exitλ₯Ό μ‹€ν–‰ν•˜λ©΄ μ›λž˜ 루트둜 볡귀

 

 

 


 

 

 

2. Namepace

Namespace : ν”„λ‘œμ„ΈμŠ€μ™€ μžμ›μ„ 격리

  • μ—¬λŸ¬ μ’…λ₯˜κ°€ 있음: PID, UTS(hostname), NET, MNT, USER λ“±
  • λͺ©μ : ν•˜λ‚˜μ˜ λ¦¬λˆ…μŠ€ OS μ•ˆμ—μ„œλ„ 마치 μ—¬λŸ¬ OSκ°€ λŒμ•„κ°€λŠ” κ²ƒμ²˜λŸΌ 격리

μ‹€μŠ΅ μš”μ : unshare --fork --pid --mount-proc /bin/sh
β†’ 이 μ‰˜ μ•ˆμ—μ„  λ‚΄κ°€ PID 1번 ν”„λ‘œμ„ΈμŠ€!

 

PID, UTS, NET Namepace 3κ°€μ§€ μ’…λ₯˜μ— λŒ€ν•΄ μ‹€μŠ΅ μ§„ν–‰

 

 

 

PID Namespace

  • ν”„λ‘œμ„ΈμŠ€ ID (PID) λ₯Ό λΆ„λ¦¬ν•˜μ—¬ κ΄€λ¦¬ν•œλ‹€
  • ν”„λ‘œμ„ΈμŠ€ ID 곡간을 λΆ„λ¦¬ν•΄μ„œ, 각 λ„€μž„μŠ€νŽ˜μ΄μŠ€λ§ˆλ‹€ PID 1λ²ˆλΆ€ν„° λ…λ¦½μ μœΌλ‘œ μ‹œμž‘ν•  수 있게 ν•΄μ€€λ‹€.
  • 즉, ν•˜λ‚˜μ˜ μ‹œμŠ€ν…œμ—μ„œ μ„œλ‘œ λ‹€λ₯Έ PIDλ₯Ό κ°€μ§„ "μž‘μ€ λ¦¬λˆ…μŠ€ ν™˜κ²½"을 λ§Œλ“€ 수 μžˆλ‹€.
# ν˜„μž¬ μ‰˜μ˜ PID 확인
echo $$

# μƒˆλ‘œμš΄ PID λ„€μž„μŠ€νŽ˜μ΄μŠ€ 생성 + /proc 마운트
unshare --fork --pid --mount-proc=/proc /bin/sh

 

  • unshare --fork --pid --mount-proc=/proc /bin/sh
  • 이 λͺ…λ Ήμ–΄λŠ” ν˜„μž¬ μ‰˜λ‘œλΆ€ν„° λΆ„λ¦¬λœ μƒˆλ‘œμš΄ PID λ„€μž„μŠ€νŽ˜μ΄μŠ€λ₯Ό λ§Œλ“€κ³ ,
    κ·Έ μ•ˆμ—μ„œ λ³„λ„λ‘œ /proc νŒŒμΌμ‹œμŠ€ν…œμ„ λ§ˆμš΄νŠΈν•œ μƒνƒœλ‘œ μƒˆ μ‰˜μ„ μ‹€ν–‰ν•˜λŠ” 것
μ˜΅μ…˜ 의미
--pid μƒˆλ‘œμš΄ PID λ„€μž„μŠ€νŽ˜μ΄μŠ€ 생성
--fork μžμ‹ ν”„λ‘œμ„ΈμŠ€λ‘œ μƒˆλ‘œμš΄ μ‰˜μ„ μ‹€ν–‰ (ν˜„μž¬ μ‰˜μ€ κ·ΈλŒ€λ‘œ μœ μ§€)
--mount-proc=/proc μƒˆλ‘œμš΄ /proc νŒŒμΌμ‹œμŠ€ν…œ 마운트 (PID λ„€μž„μŠ€νŽ˜μ΄μŠ€μ— 맞게)
/bin/sh μ‹€ν–‰ν•  μ‰˜ ν”„λ‘œμ„ΈμŠ€

 

 

 

μ‹€μ œλ‘œ 싀행해보면

root@master:~# echo $$
16818
root@master:~# unshare --fork --pid  --mount-proc=/proc /bin/sh
# ps
    PID TTY          TIME CMD
      1 pts/1    00:00:00 sh
      2 pts/1    00:00:00 ps

 

🎯 이게 핡심!
sh ν”„λ‘œμ„ΈμŠ€κ°€ PID 1번, psλŠ” 2λ²ˆμ΄λ‹€.
마치 이 μ•ˆμ—μ„œ λ‚΄κ°€ "λ¦¬λˆ…μŠ€λ₯Ό λΆ€νŒ…ν•΄μ„œ 졜초둜 μ‹€ν–‰λœ ν”„λ‘œμ„ΈμŠ€"처럼 보인닀.
μ‹€μ œ μ‹œμŠ€ν…œμ—μ„  이 ν”„λ‘œμ„ΈμŠ€κ°€ PID 16818μ΄μ—ˆμ§€λ§Œ, λ‚΄λΆ€μ μœΌλ‘œλŠ” μ™„μ „νžˆ λ‹€λ₯Έ ν”„λ‘œμ„ΈμŠ€ ID 곡간이 λ§Œλ“€μ–΄μ§„ 것.

 

# exit
/bin/sh: 7: Cannot set tty process group (No such process)

 

λ§ˆμ°¬κ°€μ§€λ‘œ exit둜 νƒˆμΆœ κ°€λŠ₯

 

 

 

 

 

 

 

UTS Namespace

호슀트 이름(hostname) κ³Ό 도메인 이름을 κ²©λ¦¬ν•˜λŠ” 곡간이닀.
λ„€μž„μŠ€νŽ˜μ΄μŠ€ μ•ˆμ—μ„œ μ‹œμŠ€ν…œ 이름을 바꿔도 λ‹€λ₯Έ ν™˜κ²½μ—μ„  영ν–₯을 λ°›μ§€ μ•ŠλŠ”λ‹€.
즉, ν•˜λ‚˜μ˜ λ¦¬λˆ…μŠ€μ—μ„œ μ—¬λŸ¬ 개의 λ…λ¦½λœ 컴퓨터 이름을 κ°€μ§„ ν™˜κ²½μ„ λ§Œλ“€ 수 μžˆλ‹€.

 

ν•΄λ‹Ή μ‹€μŠ΅μ€ ν•˜λ‚˜μ˜ μ„œλ²„λ₯Ό 2개의 터미널(terminal1, 2) 둜 λ„μš°κ³  λΉ„κ΅ν•˜λ©° μ§„ν–‰ν•œλ‹€

 

βœ… μ‹€μŠ΅: 호슀트 이름 격리 ν™•μΈν•˜κΈ°

# (두 개의 터미널 μ—΄κΈ°) 각각 terminal1, terminal2

# 각 ν„°λ―Έλ„μ—μ„œ ν˜„μž¬ hostname 확인
uname -n
hostname
----
root@master:~# uname -n
master
root@master:~# hostname
master

 

β†’ 두 터미널 λͺ¨λ‘ 같은 이름 좜λ ₯됨 

 

 

βœ… 이제 terminal1μ—μ„œ μƒˆλ‘œμš΄ UTS λ„€μž„μŠ€νŽ˜μ΄μŠ€ μ‹œμž‘

unshare -u /bin/bash
hostname happy.life.com
-----
root@master:~# unshare -u /bin/bash
root@master:~# hostname happy.life.com
root@master:~# hostname
happy.life.com
root@master:~# uname -n
happy.life.com

 

β†’ 이 μ…Έ λ‚΄λΆ€μ—μ„œλŠ” hostname이 happy.life.com으둜 λ°”λ€œ.

 

πŸ”Ž κ²°κ³Ό 비ꡐ: μ§„μ§œ κ²©λ¦¬λ˜μ—ˆμ„κΉŒ?

# terminal1 (μƒˆ λ„€μž„μŠ€νŽ˜μ΄μŠ€ μ•ˆ)
hostname       # β†’ happy.life.com
uname -n       # β†’ happy.life.com

# terminal2 (κΈ°μ‘΄ μ‹œμŠ€ν…œ ν™˜κ²½)
hostname       # β†’ master
uname -n       # β†’ master

 

같은 가상 λ¨Έμ‹ μ—μ„œ μ„œλ‘œ λ‹€λ₯Έ ν˜ΈμŠ€νŠΈλ„€μž„μ„ 좜λ ₯ν•˜λŠ” 것을 확인할 수 μžˆλ‹€

 

πŸ“‚ λ‚΄λΆ€ ꡬ쑰 확인 – /proc을 톡해 λ„€μž„μŠ€νŽ˜μ΄μŠ€ ID 비ꡐ

# terminal1
ls -l /proc/$$/ns/uts
# β†’ uts:[4026532377]
---
root@master:~# uname -n
happy.life.com
root@master:~# ls -l /proc/$$/ns/uts
lrwxrwxrwx 1 root root 0 Jun 18 12:10 /proc/17039/ns/uts -> 'uts:[4026532377]'


# terminal2
ls -l /proc/$$/ns/uts
# β†’ uts:[4026531838]
---
root@master:~# uname -n
master
root@master:~# ls -l /proc/$$/ns/uts
lrwxrwxrwx 1 root root 0 Jun 18 12:10 /proc/17024/ns/uts -> 'uts:[4026531838]'

 

uts:[숫자] 값이 λ‹€λ₯΄λ‹€ β†’ μ„œλ‘œ λ‹€λ₯Έ UTS λ„€μž„μŠ€νŽ˜μ΄μŠ€μ— μžˆλ‹€λŠ” 증거

 

 

 

 

 

NET Namespace

 

NET λ„€μž„μŠ€νŽ˜μ΄μŠ€λŠ” λ¦¬λˆ…μŠ€μ—μ„œ λ„€νŠΈμ›Œν¬ μžμ›μ„ κ²©λ¦¬ν•˜λŠ” 핡심 기술둜, λ‹€μŒκ³Ό 같은 νŠΉμ§•μ„ κ°€μ§„λ‹€:

  • 각 λ„€μž„μŠ€νŽ˜μ΄μŠ€λ§ˆλ‹€ κ³ μœ ν•œ μΈν„°νŽ˜μ΄μŠ€μ™€ IP ꡬ성을 κ°–λŠ”λ‹€
    β†’ μ„œλ‘œ λ‹€λ₯Έ μ»¨ν…Œμ΄λ„ˆλŠ” λ…λ¦½λœ λ„€νŠΈμ›Œν¬ κ³΅κ°„μ—μ„œ λ™μž‘ν•œλ‹€
  • λ™μΌν•œ 포트λ₯Ό λ™μ‹œμ— 바인딩해도 좩돌이 μ—†λ‹€
    β†’ 각 λ„€μž„μŠ€νŽ˜μ΄μŠ€ μ•ˆμ—μ„œ ν¬νŠΈκ°€ λ…λ¦½μ μœΌλ‘œ κ΄€λ¦¬λ˜κΈ° λ•Œλ¬Έ
  • λΌμš°νŒ… ν…Œμ΄λΈ”, λ°©ν™”λ²½ κ·œμΉ™(iptables), λ„€νŠΈμ›Œν¬ μž₯치 등도 μ™„μ „νžˆ λΆ„λ¦¬λœλ‹€
    β†’ ν•˜λ‚˜μ˜ μ‹œμŠ€ν…œμ΄ μ—¬λŸ¬ λ„€νŠΈμ›Œν¬λ₯Ό λ™μ‹œμ— κ°–λŠ” κ²ƒμ²˜λŸΌ ꡬ성 κ°€λŠ₯

μ‹€μŠ΅ λͺ©ν‘œ

  • μƒˆλ‘œμš΄ λ„€νŠΈμ›Œν¬ λ„€μž„μŠ€νŽ˜μ΄μŠ€(guestnet)λ₯Ό λ§Œλ“€κ³ 
  • κΈ°λ³Έ λ„€μž„μŠ€νŽ˜μ΄μŠ€(default)와 veth νŽ˜μ–΄ μΈν„°νŽ˜μ΄μŠ€λ‘œ μ—°κ²°
  • 각 μΈν„°νŽ˜μ΄μŠ€μ— IPλ₯Ό ν• λ‹Ήν•œ λ’€, μ„œλ‘œ ping 톡신이 κ°€λŠ₯ν•œμ§€ 확인해본닀

 

μ•„λž˜ μ°Έκ³ ν•˜λ©΄ 쒋은 λ‚΄μš©λ“€

https://devocean.sk.com/blog/techBoardDetail.do?ID=163803

 

10λΆ„λ§Œμ— μ΄ν•΄ν•˜λŠ” μ»¨ν…Œμ΄λ„ˆ λ„€νŠΈμ›Œν¬

 

devocean.sk.com

 

https://80000coding.oopy.io/a6a43912-cd6e-485e-9031-fbc3969aa335

 

λ¦¬λˆ…μŠ€ λ„€νŠΈμ›Œν¬ λ„€μž„μŠ€νŽ˜μ΄μŠ€ μ‚΄νŽ΄λ³΄κΈ°

λ¦¬λˆ…μŠ€ λ„€νŠΈμ›Œν¬ λ„€μž„μŠ€νŽ˜μ΄μŠ€λž€?

80000coding.oopy.io

 

https://malwareanalysis.tistory.com/249

 

μΏ λ²„λ„€ν‹°μŠ€ λ„€νŠΈμ›Œν¬ μŠ€ν„°λ”” 1μ£Όμ°¨ 2편: λ„€νŠΈμ›Œν¬ λ„€μž„μŠ€νŽ˜μ΄μŠ€

μŠ€ν„°λ”” λͺ©μ°¨ 1μ£Όμ°¨ μ»¨ν…Œμ΄λ„ˆ 격리 - https://malwareanalysis.tistory.com/248 λ„€νŠΈμ›Œν¬ λ„€μž„μŠ€νŽ˜μ΄μŠ€ - https://malwareanalysis.tistory.com/249 2μ£Όμ°¨ Flannel CNI: https://malwareanalysis.tistory.com/254 pause μ»¨ν…Œμ΄λ„ˆ: https://malwar

malwareanalysis.tistory.com

 

 

 

 

 

βœ… Step 1. λ„€μž„μŠ€νŽ˜μ΄μŠ€ 생성 및 loopback ν™œμ„±ν™”

# μƒˆλ‘œμš΄ NET λ„€μž„μŠ€νŽ˜μ΄μŠ€ guestnet 생성
ip netns add guestnet

# λ‚΄λΆ€ loopback μΈν„°νŽ˜μ΄μŠ€ ν™œμ„±ν™”
ip netns exec guestnet ip link
ip netns exec guestnet ip link set lo up

 

root@master:~# ip netns exec guestnet ip link
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
root@master:~# ip netns exec guestnet ip link set lo up
root@master:~# ip netns exec guestnet ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

 

lo: <LOOPBACK> => lo: <LOOPBACK,UP,LOWER_UP>

loλŠ” 기본적으둜 down μƒνƒœμ΄λ―€λ‘œ μˆ˜λ™μœΌλ‘œ μ˜¬λ €μ€˜μ•Ό 함

 

 

βœ… Step 2. veth νŽ˜μ–΄ μΈν„°νŽ˜μ΄μŠ€ 생성 및 뢄리

 

μΈν„°νŽ˜μ΄μŠ€ λ„€μž„μŠ€νŽ˜μ΄μŠ€
host default
guest guestnet

 

# veth νŽ˜μ–΄ 생성: host <-> guest
ip link add host type veth peer name guest

# guest μΈν„°νŽ˜μ΄μŠ€λ₯Ό guestnet λ„€μž„μŠ€νŽ˜μ΄μŠ€λ‘œ 이동
ip link set guest netns guestnet
# 확인
ip link                                # guest 사라짐 β†’ 이동됨
ip netns exec guestnet ip link         # guest λ“±μž₯

 

 

 

μ‹€ν–‰ κ²°κ³Ό

root@master:~# ip link add host type veth peer name guest
root@master:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:22:48:05:3f:0d brd ff:ff:ff:ff:ff:ff
3: enP61412s1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master eth0 state UP mode DEFAULT group default qlen 1000
    link/ether 00:22:48:05:3f:0d brd ff:ff:ff:ff:ff:ff
    altname enP61412p0s2
4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
    link/ether 1a:58:0f:2f:2a:b0 brd ff:ff:ff:ff:ff:ff
29: veth97b1361@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
    link/ether 6a:00:54:d9:dc:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 0
30: veth2729f05@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
    link/ether 8e:bf:84:65:34:cc brd ff:ff:ff:ff:ff:ff link-netnsid 1
33: veth3dc75bf@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
    link/ether 86:79:fa:e7:a4:2c brd ff:ff:ff:ff:ff:ff link-netnsid 2
34: guest@host: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether f6:87:fa:a4:91:bb brd ff:ff:ff:ff:ff:ff
35: host@guest: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 26:4c:e4:cd:ee:c3 brd ff:ff:ff:ff:ff:ff

 

34: guest@host:...

35: host@guest:...

μ΄λ ‡κ²Œ host <-> guest κ°„μ˜ veth νŽ˜μ–΄ 생성 확인

 

root@master:~# ip link set guest netns guestnet
root@master:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:22:48:05:3f:0d brd ff:ff:ff:ff:ff:ff
3: enP61412s1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master eth0 state UP mode DEFAULT group default qlen 1000
    link/ether 00:22:48:05:3f:0d brd ff:ff:ff:ff:ff:ff
    altname enP61412p0s2
4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
    link/ether 1a:58:0f:2f:2a:b0 brd ff:ff:ff:ff:ff:ff
29: veth97b1361@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
    link/ether 6a:00:54:d9:dc:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 0
30: veth2729f05@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
    link/ether 8e:bf:84:65:34:cc brd ff:ff:ff:ff:ff:ff link-netnsid 1
33: veth3dc75bf@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
    link/ether 86:79:fa:e7:a4:2c brd ff:ff:ff:ff:ff:ff link-netnsid 2
35: host@if34: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 26:4c:e4:cd:ee:c3 brd ff:ff:ff:ff:ff:ff link-netns guestnet

 

guestλ₯Ό μƒˆλ‘œ μƒμ„±ν•œ guestnet으둜 μ΄λ™μ‹œμΌ°λ”λ‹ˆ

  • μœ„μ—λŠ” μžˆμ—ˆλ˜ 34: guest@host:... κ°€ 사라짐 => κΈ°λ³Έ λ„€μž„μŠ€νŽ˜μ΄μŠ€(default)μ—μ„œλŠ” guest μΈν„°νŽ˜μ΄μŠ€κ°€ 사라진닀
  • guestnet λ„€μž„μŠ€νŽ˜μ΄μŠ€ λ‚΄λΆ€λ‘œ λ“€μ–΄κ°€μ„œ ν™•μΈν•˜λ©΄, κ·Έ μ•ˆμ— guest μΈν„°νŽ˜μ΄μŠ€κ°€ 생긴닀.
  • default μͺ½μ— 남은 host μΈν„°νŽ˜μ΄μŠ€λŠ” μ΄λ ‡κ²Œ 바뀐닀:
    • 35: host@guest:... => 35: host@if34:... 
    • μ—¬κΈ°μ„œ if34λŠ” guest μΈν„°νŽ˜μ΄μŠ€κ°€ 이전에 μ“°λ˜ μΈν„°νŽ˜μ΄μŠ€ 인덱슀λ₯Ό μ˜λ―Έν•œλ‹€.
    • hostλŠ” μ—¬μ „νžˆ guest와 μ—°κ²°λ˜μ–΄ μžˆμ§€λ§Œ, 이제 κ·Έ μ—°κ²° μƒλŒ€λŠ” guestnetμ΄λΌλŠ” λ‹€λ₯Έ λ„€μž„μŠ€νŽ˜μ΄μŠ€ μ•ˆμ— μ‘΄μž¬ν•˜λŠ” 것이닀.

 

 

 

 

βœ… Step 3. 각 μΈν„°νŽ˜μ΄μŠ€μ— IP μ£Όμ†Œ ν• λ‹Ή

 

host(veth) IP (2.2.2.1/24) ν• λ‹Ή

# default λ„€μž„μŠ€νŽ˜μ΄μŠ€ (host)
ip link set host up
ip address add 2.2.2.1/24 dev host
---
root@master:~# ip link set host up
root@master:~# ip address add 2.2.2.1/24 dev host
root@master:~# ip address show dev host
35: host@if34: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state LOWERLAYERDOWN group default qlen 1000
    link/ether 26:4c:e4:cd:ee:c3 brd ff:ff:ff:ff:ff:ff link-netns guestnet
    inet 2.2.2.1/24 scope global host
       valid_lft forever preferred_lft forever

 

 

 

guest (veth) IP (2.2.2.2/24) ν• λ‹Ή

# guestnet λ„€μž„μŠ€νŽ˜μ΄μŠ€ (guest)
ip netns exec guestnet ip link set guest up
ip netns exec guestnet ip address add 2.2.2.2/24 dev guest
---
root@master:~# ip netns exec guestnet ip link set guest up
root@master:~# ip netns exec guestnet ip address add 2.2.2.2/24 dev guest
root@master:~# ip netns exec guestnet ip address show dev guest
34: guest@if35: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether f6:87:fa:a4:91:bb brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 2.2.2.2/24 scope global guest
       valid_lft forever preferred_lft forever
    inet6 fe80::f487:faff:fea4:91bb/64 scope link
       valid_lft forever preferred_lft forever

 

 

μ—¬κΈ°κΉŒμ§€ μ§„ν–‰ν•œλ‹€λ©΄ μ•„λž˜μ™€ 같은 ꡬ쑰λ₯Ό κ°–κ²Œ λœλ‹€

[ default namespace ]                    [ guestnet namespace ]
-----------------------                  -----------------------
|                     |                  |                     |
|     host (veth)     | <==============> |    guest (veth)     |
|   IP: 2.2.2.1/24    |    veth pair     |  IP: 2.2.2.2/24     |
|                     |                  |                     |
-----------------------                  -----------------------

 

 

βœ… Step 4. λ„€μž„μŠ€νŽ˜μ΄μŠ€ κ°„ 톡신 ν…ŒμŠ€νŠΈ

# default β†’ guestnet
ping 2.2.2.2

# guestnet β†’ default
ip netns exec guestnet ping 2.2.2.1

 

root@master:~# ping 2.2.2.2
PING 2.2.2.2 (2.2.2.2) 56(84) bytes of data.
64 bytes from 2.2.2.2: icmp_seq=1 ttl=64 time=0.112 ms
64 bytes from 2.2.2.2: icmp_seq=2 ttl=64 time=0.057 ms
64 bytes from 2.2.2.2: icmp_seq=3 ttl=64 time=0.102 ms
64 bytes from 2.2.2.2: icmp_seq=4 ttl=64 time=0.066 ms
64 bytes from 2.2.2.2: icmp_seq=5 ttl=64 time=0.058 ms
--- 2.2.2.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4131ms
rtt min/avg/max/mdev = 0.057/0.079/0.112/0.023 ms


root@master:~# ip netns exec guestnet ping 2.2.2.1
PING 2.2.2.1 (2.2.2.1) 56(84) bytes of data.
64 bytes from 2.2.2.1: icmp_seq=1 ttl=64 time=0.063 ms
64 bytes from 2.2.2.1: icmp_seq=2 ttl=64 time=0.078 ms
64 bytes from 2.2.2.1: icmp_seq=3 ttl=64 time=0.063 ms
64 bytes from 2.2.2.1: icmp_seq=4 ttl=64 time=0.081 ms
64 bytes from 2.2.2.1: icmp_seq=5 ttl=64 time=0.061 ms
--- 2.2.2.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4133ms
rtt min/avg/max/mdev = 0.061/0.069/0.081/0.008 ms

 

 

βœ… κ²°κ³Ό: μ„œλ‘œ λ‹€λ₯Έ λ„€μž„μŠ€νŽ˜μ΄μŠ€μ— μžˆμœΌλ©΄μ„œλ„ veth둜 μ—°κ²°λ˜μ—ˆκΈ° λ•Œλ¬Έμ— 톡신 κ°€λŠ₯!

 

 

 

 

 


 

 

 

 

3. cGropus

 

cgroups: μžμ›(CPU, λ©”λͺ¨λ¦¬ λ“±)을 μ œν•œν•˜κ³  ν• λ‹Ή

  • λ¦¬λˆ…μŠ€ 컀널이 μ œκ³΅ν•˜λŠ” κΈ°λŠ₯으둜, νŠΉμ • ν”„λ‘œμ„ΈμŠ€(λ˜λŠ” κ·Έλ£Ή)κ°€ CPU, λ©”λͺ¨λ¦¬, λ„€νŠΈμ›Œν¬, λ””μŠ€ν¬ 등을 μ–Όλ§ˆλ‚˜ μ‚¬μš©ν• μ§€ μ œν•œν•˜κ±°λ‚˜ λͺ¨λ‹ˆν„°λ§ν•  수 μžˆλ‹€.
  • λͺ©μ : μ–΄λ–€ μ»¨ν…Œμ΄λ„ˆκ°€ μžμ›μ„ λ…μ‹ν•˜μ§€ λͺ»ν•˜κ²Œ λ§‰μŒ.
  • μ„œλ²„ μžμ›μ„ λˆ„κ°€ μ–Όλ§ˆλ‚˜ μ“Έ 수 μžˆλŠ”μ§€ μ •ν•΄μ£ΌλŠ” κ΄€λ¦¬μž λ„κ΅¬λ‘œ, CPU μ‚¬μš©λ₯ , λ©”λͺ¨λ¦¬ μ œν•œ 등을 μ„€μ • κ°€λŠ₯

 

κ΅¬μ‘°λŠ” μ–΄λ–»κ²Œ μƒκ²Όλ‚˜?

  • /sys/fs/cgroup μ•„λž˜μ— ν΄λ”μ²˜λŸΌ 계측 ꡬ쑰둜 κ΅¬μ„±λ˜μ–΄ μžˆλ‹€
  • 이 폴더 ꡬ쑰 μ•ˆμ˜ μ„€μ • νŒŒμΌλ“€μ„ μˆ˜μ •ν•΄μ„œ μžμ›μ„ μ œμ–΄ν•¨
  • λΆ€λͺ¨ 그룹이 μ„€μ •ν•œ μžμ› μ œν•œμ„ μžμ‹ 그룹이 물렀받을 수 있음 (상속 ꡬ쑰)

 

cGroup을 μ œμ–΄ν•˜λŠ” 방법

 

1. 직접 파일 μˆ˜μ • (VFS 방식)

  1. 디렉토리 λ§Œλ“€κ³ , μ„€μ • 파일 νŽΈμ§‘
  2. 예: echo "50000 100000" > cpu.max

2. 도ꡬ μ‚¬μš©

  • Ubuntu: cgroup-bin
  • RHEL: libcgroup
  • β†’ λͺ…λ Ήμ–΄ 기반으둜 μžμ› ν• λ‹Ή 및 κ·Έλ£Ή ꡬ성 κ°€λŠ₯

μ‹€μŠ΅μ€ 직접 파일 μˆ˜μ •ν•˜μ—¬ μ§„ν–‰

 

 

 

 

βœ… Step 1. CPUλ₯Ό λ¬΄ν•œνžˆ μ‚¬μš©ν•˜λŠ” ν”„λ‘œκ·Έλž¨ λ§Œλ“€κΈ°

 

# μ—…λ°μ΄νŠΈ
apt update
apt install -y gcc

 

# vim으둜 a.c 파일 생성 및 μ•„λž˜ μ½”λ“œ μΆ”κ°€
vi a.c

# C μ½”λ“œ μž‘μ„± (a.c)
void main() {
  for(;;);
}

 

# 컴파일 ν›„ μ‹€ν–‰ (λ°±κ·ΈλΌμš΄λ“œλ‘œ)
gcc a.c
./a.out &

 

 

 

a.c 컴파일 ν›„ a.out μ΄λΌλŠ” 파일 μƒμ„±λœλ‹€

이 νŒŒμΌμ„ μ‹€ν–‰μ‹œν‚€κ³  λ‹€λ₯Έ ν„°λ―Έλ„μ—μ„œ top으둜 CPU μ‚¬μš©λŸ‰μ„ 체크해보면  a.out이 100%λ₯Ό μ°¨μ§€ν•˜λŠ” 것을 확인할 수 μžˆλ‹€

 

 

 

βœ… Step 2. cGroup 디렉토리 생성 및 μ§„μž…

 

sudo mkdir /sys/fs/cgroup/mygroup
cd /sys/fs/cgroup/mygroup

 

mygroup, 이 λ””λ ‰ν† λ¦¬λŠ” μš°λ¦¬κ°€ 관리할 "μžμ› κ·Έλ£Ή"이라고 보면 λœλ‹€
이 μ•ˆμ— μ„€μ • νŒŒμΌμ„ λ„£μ–΄μ„œ μ œν•œ 쑰건을 직접 κ±Έ 수 μžˆλ‹€

 

 

 

 

 

λΆ€λͺ¨ 그룹이 μ„€μ •ν•œ μžμ› μ œν•œμ„ μžμ‹ 그룹이 물렀받을 수 있음 (상속 ꡬ쑰)

➑️ μƒμ„±λœ mygroup에 듀어가보면 빈 디렉토리가 μ•„λ‹Œ λ§Žμ€ νŒŒμΌλ“€μ΄ μ‘΄μž¬ν•˜λŠ”λ° μ΄λŠ” /sys/fs/cgroup 의 νŒŒμΌλ“€μ„ μƒμ†λ°›μ•˜κΈ° 떄문이닀

cgroupμ•„λž˜μ— μžˆλŠ” λ””λ ‰ν† λ¦¬λ“€μ˜ νŒŒμΌλ“€κΉŒμ§€ λ‹€ κ°€μ Έμ˜€κΈ° λ•Œλ¬Έμ— cgroup 보닀 νŒŒμΌλ“€μ΄ μ’€ 더 λ§Žμ•„λ³΄μΈλ‹€

 

 

 

 

βœ… Step 3. CPU μ‚¬μš©λŸ‰ μ œν•œ μ„€μ •

# CPU μ œν•œ μ„€μ •: 100ms μ£ΌκΈ° 쀑 50ms만 μ‹€ν–‰ κ°€λŠ₯ (즉, 50% μ œν•œ)
echo "50000 100000" | sudo tee cpu.max

 

 

μ„€μ • ν•­λͺ© 의미
50000 μ‚¬μš©ν•  수 μžˆλŠ” μ‹œκ°„ (microsecond λ‹¨μœ„, 50ms)
100000 전체 μ£ΌκΈ° (100ms) β†’ λ”°λΌμ„œ 50% μ‚¬μš© μ œν•œ

 

 

 

 

 

βœ… Step 4. ν”„λ‘œμ„ΈμŠ€λ₯Ό cGroup에 등둝

# a.out의 PIDλ₯Ό ν•΄λ‹Ή 그룹에 등둝
echo $(pgrep a.out) | sudo tee cgroup.procs

 

이 μˆœκ°„λΆ€ν„° ν•΄λ‹Ή ν”„λ‘œμ„ΈμŠ€λŠ” cpu.maxμ—μ„œ μ •μ˜ν•œ μ œμ•½ 쑰건의 μ μš©μ„ λ°›λŠ”λ‹€

 

 

μ‹€ν–‰ κ²°κ³Ό

root@master:/sys/fs/cgroup/mygroup# echo "50000 100000" | sudo tee cpu.max
50000 100000
root@master:/sys/fs/cgroup/mygroup# echo $(pgrep a.out) | sudo tee cgroup.procs
17453

 

 

이전에 100%둜 μ‚¬μš©λ˜λ˜ CPU μ‚¬μš©λŸ‰μ΄ 50%둜 μ œν•œλœ 것을 확인할 수 μžˆλ‹€

 

 

 

βœ… cgroup 섀정을 κΈ°λ³Έκ°’(μ œν•œ μ—†μŒ)으둜 λ˜λŒλ¦¬λŠ” 방법

 

κΈ°λ³Έκ°’μœΌλ‘œ 되돌리기

# λ¬΄μ œν•œμœΌλ‘œ μ„€μ •
echo "max 100000" | sudo tee cpu.max

 

 

κ·Έλ£Ή ν•΄μ œ (ν”„λ‘œμ„ΈμŠ€λ₯Ό κ·Έλ£Ήμ—μ„œ λΉΌλ‚΄κΈ°)

# ν•΄λ‹Ή PIDλ₯Ό λ‹€λ₯Έ κ·Έλ£Ή(cgroup.procs)으둜 이동
# 예: root λ””ν΄νŠΈ 그룹으둜 이동
echo <PID> | sudo tee /sys/fs/cgroup/cgroup.procs

 

 

cgroup 디렉토리 μ‚­μ œ (주의 ν•„μš”)

cd /sys/fs/cgroup
sudo rmdir mygroup

 

 

 

 

 

 


 

 

 

 

4. Union Filesystem

Union Filesystem: λ ˆμ΄μ–΄λ‘œ μŒ“μ•„ 효율적 이미지 ꡬ성

  • 이미지듀을 λ ˆμ΄μ–΄λ‘œ 관리 β†’ 쀑볡 μ €μž₯ μ—†μŒ
  • λŒ€ν‘œμ μœΌλ‘œ AUFS, OverlayFS μ‚¬μš©

μ‹€μŠ΅ μš”μ : overlayfsλ₯Ό 직접 mount ν•΄μ„œ μ—¬λŸ¬ 이미지 λ ˆμ΄μ–΄ κ²°ν•©ν•˜λŠ” 방식 확인

 

 

μ‹€μŠ΅ λͺ©ν‘œ

  • OverlayFS둜 닀쀑 이미지 λ ˆμ΄μ–΄λ₯Ό λ³‘ν•©ν•΄μ„œ κ°€μƒμ˜ 톡합 디렉토리λ₯Ό λ§Œλ“€κ³ ,
  • κ·Έ μ•ˆμ—μ„œ 파일 μ‚­μ œ/μΆ”κ°€κ°€ μ–΄λ–»κ²Œ λ™μž‘ν•˜λŠ”μ§€λ₯Ό ν™•μΈν•œλ‹€.

 

 

βœ… Step 1.  μ‹€μŠ΅ ν™˜κ²½ ꡬ성

# μ‹€μŠ΅ 디렉토리 생성
mkdir overlayfs && cd overlayfs
mkdir container image1 image2 work merge

# 각각의 λ ˆμ΄μ–΄μ— 파일 생성
touch image1/{a,b} image2/c
---
β”œβ”€β”€ container
β”œβ”€β”€ image1
β”‚   β”œβ”€β”€ a
β”‚   └── b
β”œβ”€β”€ image2
β”‚   └── c
β”œβ”€β”€ merge
└── work

 

 

디렉토리 μš©λ„
image1, image2 읽기 μ „μš© 이미지 계측 (lower)
container μ“°κΈ° κ°€λŠ₯ν•œ upper layer
work OverlayFS λ™μž‘μ„ μœ„ν•œ μ›Œν¬ 디렉토리
merge μš°λ¦¬κ°€ μ ‘κ·Όν•  톡합 디렉토리 (mount point)

 

 

βœ… Step 2.  OverlayFS 마운트

mount -t overlay overlay \
-o lowerdir=image2:image1,upperdir=container,workdir=work \
merge

 

πŸ” μ„€λͺ…:

  • lowerdir = image2:image1 β†’ 이미지1 μœ„μ— 이미지2κ°€ μ–ΉνžŒ ꡬ쑰
  • upperdir = container β†’ μ“°κΈ° λ ˆμ΄μ–΄. 변경은 여기에 μ €μž₯
  • workdir = work β†’ λ‚΄λΆ€μ μœΌλ‘œ OverlayFSκ°€ μ‚¬μš©ν•˜λŠ” μž„μ‹œ μ €μž₯μ†Œ
  • 결과적으둜 merge 디렉토리λ₯Ό 보면, 3개 λ ˆμ΄μ–΄κ°€ 합쳐진 ν•˜λ‚˜μ˜ νŒŒμΌμ‹œμŠ€ν…œμ²˜λŸΌ λ³΄μž„
Filesystem  1K-blocks Used Available Use% Mounted on
overlay 30298176 2998448 27283344 10% /root/overlayfs/merge

 

 

 

βœ… Step 3.  κ²°κ³Ό 확인

tree
β”œβ”€β”€ container
β”œβ”€β”€ image1
β”‚   β”œβ”€β”€ a
β”‚   └── b
β”œβ”€β”€ image2
β”‚   └── c
β”œβ”€β”€ merge
β”‚   β”œβ”€β”€ a
β”‚   β”œβ”€β”€ b
β”‚   └── c
└── work
    └── work

 

➑️ mergeμ—λŠ” a, b, cκ°€ λͺ¨λ‘ λ³΄μž„ β†’ λ ˆμ΄μ–΄ 병합 성곡

 

 

 

 

βœ… Step 4.  파일 μ‚­μ œ 및 μΆ”κ°€ μ‹€μŠ΅

rm merge/a
touch merge/d
tree . -I work
---
β”œβ”€β”€ container
β”‚   β”œβ”€β”€ a
β”‚   └── d
β”œβ”€β”€ image1
β”‚   β”œβ”€β”€ a
β”‚   └── b
β”œβ”€β”€ image2
β”‚   └── c
└── merge
    β”œβ”€β”€ b
    β”œβ”€β”€ c
    └── d