Minikube 설치

● 파워셀에서 노트패드로 Vagrantfile 실행

start notepad++ .\Vagrantfile

 

 

 

● Vagrantfile

Vagrant.configure("2") do |config|
	config.vm.define "minikube-new" do |cfg|
		config.vm.box = "generic/ubuntu2204"
		cfg.vm.provider "virtualbox" do |vb|
			vb.name = "minikube-new" 
			vb.memory = "8192"
			vb.cpus = "2"
		end
		cfg.vm.host_name = "minikube.example.com"
		cfg.vm.network "private_network", ip: "192.168.77.10"
	end
end

 

 

https://minikube.sigs.k8s.io/docs/start/

minikube start

 

 

 

 

 

● minikube 를 컨테이너 형식으로 VM 으로 설치하는 방법  or  host 에 바로 설치하는 방법이 있다

 

○ 컨테이너 형식으로 VM 으로 설치

#! /usr/bin/env bash
# minikube version 1.30 based on ubuntu22.04
# last modified : 2023-09-27
# edited by lee
# referenced site => https://minikube.sigs.k8s.io/docs/start/
# -------------------------------------------------------------------------------------------------------------------------------------------------
# What you’ll need
# 2 CPUs or more
# 2GB of free memory
# 20GB of free disk space
# Internet connection
# Container or virtual machine manager, such as: Docker, QEMU, Hyperkit, Hyper-V, KVM, Parallels, Podman, VirtualBox, or VMware Fusion/Workstation
# -------------------------------------------------------------------------------------------------------------------------------------------------

# There are two ways to install: binary file installation and rpm package installation.
# In this case, it was created by installing using an rpm package.
# CNI used calico built into minikube
# For container run-time, cir-dockerd was used.

# docker install
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# install editor and utilities
sudo apt install -y vim nano wget
# swap off
sudo swapoff -a
sudo sed -e '/swap/s/^/#/' -i /etc/fstab

# disable service firewalld
sudo systemctl disable --now ufw

# disable selinux
sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing/SELINUX=permissive/' /etc/selinux/config

# Install iptables related packages
sudo apt install -y conntrack

# k8s.conf - configure iptables for bridge network.
echo "net.bridge.bridge-nf-call-ip6tables = 1" | sudo tee -a /etc/sysctl.d/k8s.conf
echo "net.bridge.bridge-nf-call-iptables = 1" | sudo tee -a  /etc/sysctl.d/k8s.conf
sudo sysctl --system

# install minikube dependencies
sudo apt install -y curl wget apt-transport-https

# download binary minkube file
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64

sudo install minikube-linux-amd64 /usr/local/bin/minikube

# install kubectl
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl

chmod +x kubectl
sudo mv kubectl /usr/local/bin

# give  user vagrant permission to run minikube
# To set the driver type to Docker when running minikube, the user who will run minikube must belong to the docker group.
sudo usermod -aG docker $LOGNAME
# It is necessary to have added group permissions without having to log in again.
sudo sed -i 's/(ALL)/(ALL:ALL)/g' /etc/ssh/sshd_config

# In addition to docker, there are several driver types such as kvm2,virtualbox,podman,ssh
sudo -g docker minikube start --driver=docker

# give user vagrant permission to run kubectl commands
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown ${LOGNAME}:$(groups|awk '{print $1}') /home/vagrant/.kube/config
echo "source <(kubectl completion bash)" >> ~/.bashrc

# -- The following is a setting for conveniently using the yaml file with vim
#    and nano editors It's not absolutely necessary.  --

# .vimrc config for yaml syntax
echo "autocmd FileType yaml setlocal ts=2 sts=2 sw=2 expandtab autoindent" > $HOME/.vimrc

# .nanorc config yaml syntax
tee $HOME/.nanorc <<EOF
# nano editor config for yaml syntax highliting
syntax "default"
color white,black ".*"
## Keys
color magenta "^\s*[\$A-Za-z0-9_-]+\:"
color brightmagenta "^\s*@[\$A-Za-z0-9_-]+\:"

# Values
color white ":\s.+$"
# Booleans
icolor brightcyan " (y|yes|n|no|true|false|on|off)$"
## Numbers
color brightred " [[:digit:]]+(\.[[:digit:]]+)?"
## Arrays
color red "\[" "\]" ":\s+[|>]" "^\s*- "
## Reserved
color green "(^| )!!(binary|bool|float|int|map|null|omap|seq|set|str) "

## Comments
color brightwhite "#.*$"

## Errors
color ,red ":\w.+$"
color ,red ":'.+$"
color ,red ":".+$"
color ,red "\s+$"

## Non closed quote
color ,red "['\"][^['\"]]*$"

## Closed quotes
color yellow "['\"].*['\"]"

## Equal sign
color brightgreen ":( |$)"
# tab size
set tabsize 2
set tabstospaces
EOF

# disconnect after five seconds
echo "The connection will be disconnected after 3 seconds. Log in again to use"
for i in {3..1}; do echo "count down : $i"; sleep 1; done
sudo pkill -f "sshd: $LOGNAME"

 

○ ./mini

 

>> 스크립트 내에 sudo usermod -aG docker $LOGNAME 이 부분 때문에 재로그인이 필요하다

>> 마지막 부분에 로그아웃 스크립트가 들어가있다

 

 

 

○ docker 컨테이너 안에 minikube 가 생성된다

 

 

 

 

○ 미니큐브 내부에서 pod 가 생성된다

vagrant@minikube:~$ kubectl run httpd --image=httpd:2.4

pod/httpd created

 

vagrant@minikube:~$ kubectl get pods -o wide
NAME    READY   STATUS    RESTARTS   AGE   IP           NODE       NOMINATED NODE   READINESS GATES
httpd   1/1     Running   0          35s   10.244.0.3   minikube   <none>           <none>

 

 

vagrant@minikube:~$ docker ps
CONTAINER ID   IMAGE                                 COMMAND                  CREATED         STATUS         PORTS                                                                                                                                  NAMES
e8333c1a89f8   gcr.io/k8s-minikube/kicbase:v0.0.42   "/usr/local/bin/entr…"   6 minutes ago   Up 6 minutes   127.0.0.1:32772->22/tcp, 127.0.0.1:32771->2376/tcp, 127.0.0.1:32770->5000/tcp, 127.0.0.1:32769->8443/tcp, 127.0.0.1:32768->32443/tcp   minikube

 

 

○ 미니큐브 컨테이너에 직접 접속

docker exec -it minikube /bin/bash

 

 

● 컨테이너 안에 컨테이너가 있는 구조이다

root@minikube:/# docker ps
CONTAINER ID   IMAGE                       COMMAND                  CREATED              STATUS              PORTS     NAMES
9813b84d34bd   httpd                       "httpd-foreground"       About a minute ago   Up About a minute             k8s_httpd_httpd_default_cc8f3275-1812-4c2a-a030-2179cf4f495a_0
8f9fbeafaebc   registry.k8s.io/pause:3.9   "/pause"                 About a minute ago   Up About a minute             k8s_POD_httpd_default_cc8f3275-1812-4c2a-a030-2179cf4f495a_0
07c2edc25a0f   6e38f40d628d                "/storage-provisioner"   5 minutes ago        Up 5 minutes                  k8s_storage-provisioner_storage-provisioner_kube-system_d95ad7a5-86ec-4cfa-8604-1ae3ac969ea0_1
5903520e40e5   ead0a4a53df8                "/coredns -conf /etc…"   6 minutes ago        Up 6 minutes                  k8s_coredns_coredns-5dd5756b68-bk2gv_kube-system_e7e66123-bc35-4efa-924f-1bee3e47f443_0
458baf9d3b7b   bfc896cf80fb                "/usr/local/bin/kube…"   6 minutes ago        Up 6 minutes                  k8s_kube-proxy_kube-proxy-55477_kube-system_36dcb027-d3c1-44c6-8e5f-5f1aac140361_0
0edc4b739902   registry.k8s.io/pause:3.9   "/pause"                 6 minutes ago        Up 6 minutes                  k8s_POD_coredns-5dd5756b68-bk2gv_kube-system_e7e66123-bc35-4efa-924f-1bee3e47f443_0
3b41b5ab220e   registry.k8s.io/pause:3.9   "/pause"                 6 minutes ago        Up 6 minutes                  k8s_POD_kube-proxy-55477_kube-system_36dcb027-d3c1-44c6-8e5f-5f1aac140361_0
a0903435e019   registry.k8s.io/pause:3.9   "/pause"                 6 minutes ago        Up 6 minutes                  k8s_POD_storage-provisioner_kube-system_d95ad7a5-86ec-4cfa-8604-1ae3ac969ea0_0
2a0accb34771   6d1b4fd1b182                "kube-scheduler --au…"   6 minutes ago        Up 6 minutes                  k8s_kube-scheduler_kube-scheduler-minikube_kube-system_75ac196d3709dde303d8a81c035c2c28_0
4f90b5f50018   537434729123                "kube-apiserver --ad…"   6 minutes ago        Up 6 minutes                  k8s_kube-apiserver_kube-apiserver-minikube_kube-system_55b4bbe24dac3803a7379f9ae169d6ba_0
4c03e1e21593   73deb9a3f702                "etcd --advertise-cl…"   6 minutes ago        Up 6 minutes                  k8s_etcd_etcd-minikube_kube-system_9aac5b5c8815def09a2ef9e37b89da55_0
446b2fc1bbca   10baa1ca1706                "kube-controller-man…"   6 minutes ago        Up 6 minutes                  k8s_kube-controller-manager_kube-controller-manager-minikube_kube-system_7da72fc2e2cfb27aacf6cffd1c72da00_0
deb6eaf3b153   registry.k8s.io/pause:3.9   "/pause"                 6 minutes ago        Up 6 minutes                  k8s_POD_kube-scheduler-minikube_kube-system_75ac196d3709dde303d8a81c035c2c28_0
f6ca8d949a41   registry.k8s.io/pause:3.9   "/pause"                 6 minutes ago        Up 6 minutes                  k8s_POD_kube-controller-manager-minikube_kube-system_7da72fc2e2cfb27aacf6cffd1c72da00_0
16eadd7a54d1   registry.k8s.io/pause:3.9   "/pause"                 6 minutes ago        Up 6 minutes                  k8s_POD_kube-apiserver-minikube_kube-system_55b4bbe24dac3803a7379f9ae169d6ba_0
98b16ee7b840   registry.k8s.io/pause:3.9   "/pause"                 6 minutes ago        Up 6 minutes                  k8s_POD_etcd-minikube_kube-system_9aac5b5c8815def09a2ef9e37b89da55_0

 

 

 

 

 

● minikube 컨테이너 안에서 파드에 있는 컨테이너에 접속이 가능하다

vagrant@minikube:~$ kubectl run myapache --image httpd

 

vagrant@minikube:~$ kubectl get pods -o wide
NAME       READY   STATUS    RESTARTS   AGE   IP           NODE       NOMINATED NODE   READINESS GATES
httpd      1/1     Running   0          21m   10.244.0.3   minikube   <none>           <none>
myapache   1/1     Running   0          10s   10.244.0.4   minikube   <none>           <none>

 

 

○ minikube 컨테이너 들어가야 curl 로 볼 수 있다

 

 

 

● 밖에서 접속 방법

 

>> apache.yaml

apiVersion: v1
kind: Pod
metadata:
  name: apache-pod
  labels:
    app: myweb
spec:
  containers:
  - name: myweb-container
    image: httpd:2.4
    ports:
    - containerPort: 80

 

 

>> myweb-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: myweb-service
spec:
  ports:
  - port: 8001
    targetPort: 80
  selector:
    app: myweb
  type: NodePort

 

 

 

○ Node 에서 2개의 파일 적용

 

vagrant@minikube:~/work$ kubectl get pods --show-labels
NAME         READY   STATUS    RESTARTS   AGE     LABELS
apache-pod   1/1     Running   0          4m      app=myweb
httpd        1/1     Running   0          29m     run=httpd
myapache     1/1     Running   0          8m31s   run=myapache

 

# service 는 -o wide 로 라벨 확인한다

vagrant@minikube:~/work$ kubectl get svc -o wide
NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE     SELECTOR
kubernetes      ClusterIP   10.96.0.1        <none>        443/TCP          34m     <none>
myweb-service   NodePort    10.108.110.161   <none>        8001:30107/TCP   4m19s   app=myweb

 

 

◎ 하지만 NodePort 로 외부 접속이 불가능하다

 

 

 

● minikube 컨테이너의 IP 로 접속해야한다 [ Host IP 로 보는 것이 아니다 ]

 

 

 

 

 

 

 

○ Host 에 직접 설치

#! /usr/bin/env bash
# minikube version 1.30 based on ubuntu22.04
# last modified : 2023-09-27
# edited by lee
# referenced site => https://minikube.sigs.k8s.io/docs/start/
# -------------------------------------------------------------------------------------------------------------------------------------------------
# What you’ll need
# 2 CPUs or more
# 2GB of free memory
# 20GB of free disk space
# Internet connection
# Container or virtual machine manager, such as: Docker, QEMU, Hyperkit, Hyper-V, KVM, Parallels, Podman, VirtualBox, or VMware Fusion/Workstation
# -------------------------------------------------------------------------------------------------------------------------------------------------

# There are two ways to install: binary file installation and rpm package installation.
# In this case, it was created by installing using an rpm package.
# CNI used calico built into minikube
# For container run-time, cir-dockerd was used.

# docker install
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# install editor and utilities
sudo apt install -y vim nano wget
# swap off
sudo swapoff -a
sudo sed -e '/swap/s/^/#/' -i /etc/fstab

# disable service firewalld
sudo systemctl disable --now ufw

# disable selinux, if selinux package is installed
sudo setenforce 0 2> /dev/null
sudo sed -i 's/^SELINUX=enforcing/SELINUX=permissive/' /etc/selinux/config 2> /dev/null

# Install iptables related packages
sudo apt install -y conntrack

# k8s.conf - configure iptables for bridge network.
echo "net.bridge.bridge-nf-call-ip6tables = 1" | sudo tee -a /etc/sysctl.d/k8s.conf
echo "net.bridge.bridge-nf-call-iptables = 1" | sudo tee -a  /etc/sysctl.d/k8s.conf
sudo sysctl --system

# install minikube dependencies
sudo apt install -y curl wget apt-transport-https

# download binary minkube file
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64

sudo install minikube-linux-amd64 /usr/local/bin/minikube

# install kubectl
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl

chmod +x kubectl
sudo mv kubectl /usr/local/bin

# install Go and cri-dockerd
sudo apt-get -y install golang
wget https://storage.googleapis.com/golang/getgo/installer_linux
chmod 755 installer_linux
./installer_linux
git clone https://github.com/Mirantis/cri-dockerd.git
cd cri-dockerd/
mkdir bin
go get && go build -o bin/cri-dockerd
sudo install -o root -g root -m 0755 bin/cri-dockerd /usr/local/bin/cri-dockerd
sudo cp bin/cri-dockerd /usr/bin/
sudo cp -a packaging/systemd/* /etc/systemd/system
sudo sed -i -e 's,/usr/bin/cri-dockerd,/usr/local/bin/cri-dockerd,' /etc/systemd/system/cri-docker.service
sudo systemctl daemon-reload
sudo systemctl enable --now docker cri-docker.service cri-docker.socket

cd $HOME
# install container networking-plugins for none driver(requried if you run minikube --driver=none)
# referenced site url : https://minikube.sigs.k8s.io/docs/faq/#how-do-i-install-containernetworking-plugins-for-none-driver
CNI_PLUGIN_VERSION="v1.3.0"
CNI_PLUGIN_TAR="cni-plugins-linux-amd64-$CNI_PLUGIN_VERSION.tgz"
CNI_PLUGIN_INSTALL_DIR="/opt/cni/bin"
curl -LO "https://github.com/containernetworking/plugins/releases/download/$CNI_PLUGIN_VERSION/$CNI_PLUGIN_TAR"
sudo mkdir -p "$CNI_PLUGIN_INSTALL_DIR"
sudo tar -xf "$CNI_PLUGIN_TAR" -C "$CNI_PLUGIN_INSTALL_DIR"
rm "$CNI_PLUGIN_TAR"

#install crictl(requried if you run minikube --driver=none)
VERSION="v1.26.0"
wget https://github.com/kubernetes-sigs/cri-tools/releases/download/$VERSION/crictl-$VERSION-linux-amd64.tar.gz
sudo tar -xf crictl-v1.26.0-linux-amd64.tar.gz -C /usr/local/bin
sudo mv /usr/local/bin/crictl /usr/bin


# Give user vagrant permission to run minikube
# To set the driver type to Docker when running minikube, the user who will run minikube must belong to the docker group.
sudo usermod -aG docker $LOGNAME
# It is necessary to have added group permissions without having to log in again.
sudo sed -i 's/(ALL)/(ALL:ALL)/g' /etc/ssh/sshd_config

# In addition to none, there are several driver types such as docker,kvm2,virtualbox,podman,ssh
#To run minikube without a new login, the -g docker option is requiredG
sudo -g docker minikube start --driver=none 2> /dev/null
# give user vagrant permission to run kubectl commands
# When minikube is run with --driver=none, the $HOME/.kube/config file is automatically created.
#mkdir -p $HOME/.kube
#sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
#Gudo chown ${LOGNAME}:$(groups|awk '{print $1}') /home/vagrant/.kube/config
echo "source <(kubectl completion bash)" >> ~/.bashrc
#source ~/.bashrc

# -- The following is a setting for conveniently using the yaml file with vim
#    and nano editors It's not absolutely necessary.  --

# .vimrc config for yaml syntax
echo "autocmd FileType yaml setlocal ts=2 sts=2 sw=2 expandtab autoindent" > $HOME/.vimrc

# .nanorc config yaml syntax
tee $HOME/.nanorc <<EOF
# nano editor config for yaml syntax highliting
syntax "default"
color white,black ".*"
## Keys
color magenta "^\s*[\$A-Za-z0-9_-]+\:"
color brightmagenta "^\s*@[\$A-Za-z0-9_-]+\:"

# Values
color white ":\s.+$"
# Booleans
icolor brightcyan " (y|yes|n|no|true|false|on|off)$"
## Numbers
color brightred " [[:digit:]]+(\.[[:digit:]]+)?"
## Arrays
color red "\[" "\]" ":\s+[|>]" "^\s*- "
## Reserved
color green "(^| )!!(binary|bool|float|int|map|null|omap|seq|set|str) "

## Comments
color brightwhite "#.*$"

## Errors
color ,red ":\w.+$"
color ,red ":'.+$"
color ,red ":".+$"
color ,red "\s+$"

## Non closed quote
color ,red "['\"][^['\"]]*$"

## Closed quotes
color yellow "['\"].*['\"]"

## Equal sign
color brightgreen ":( |$)"
# tab size
set tabsize 2
set tabstospaces
EOF

# disconnect after five seconds
echo "The connection will be disconnected after 3 seconds. Log in again to use"
for i in {3..1}; do echo "count down : $i"; sleep 1; done
sudo pkill -f "sshd: $LOGNAME"