728x90
✍ Posted by Immersive Builder Seong
1. Kubernetes Authentication & Authorization
Service Account
어플리케이션 파드에서 AWS 리소스에 접근하기 위해 사용하는 인증정보입니다.
파드마다 별도의 Service Account를 할당하고 IAM Role을 바인딩해서 필요한 리소스에만 접근하도록 할 수 있습니다.
API Server 접근 과정
- kubectl, HTTP/HTTPS, CLI 등 다양한 방법으로 API Server에 접근이 가능합니다.
- Authentication → Authorizaion -> Admission Control 순으로 진행됩니다.
Authentication
- 인증은 사용자를 식별하고 신원을 증명하는 프로세스입니다.
- X.509 기반 CA crt(발급기관 인증서), Client crt(클라이언트 인증서), Client key(클라이언트 개인키)를 사용하여 인증합니다.
Authorization
- 인가는 인증된 사용자가 특정한 작업이나 리소스에 대해 수행할 수 있는 권한을 결정하는 프로세스입니다.
- 인가 방식에는 RBAC, ABAC, Webhook, Node Authorizaion 등이 있습니다.
- RBAC(Role-Based Access Control): 역할 기반의 권한 관리
- 클러스터 수준의 자원 권한 : Cluster - ClusterRole - ClusterRoleBinding - Service Account
- 네임스페이스 내 자원 권한 : Namespace - Role - RoleBinding - Service Account
Admission Controller
- 쿠버네티스 클러스터 내에서 발생하는 API 요청을 검사하고 조작하는 역할을 수행하는 컴포넌트입니다.
- 권한이 있는 사용자에 한정해서 관리자의 정책에 따라 추가로 특정 작업을 제한(Validate)하거나 변경(Mutate)할 수 있습니다.
- Dynamic Admission Controller: MutatingWebhook / ValidatingWebhook
- MutatingWebhook: 사용자가 요청한 Request에 대해서 관리자가 임의로 값을 변경하는 작업입니다.
- ValidatingWebhook: 사용자가 요청한 Request에 대해서 관리자가 허용을 막는 작업입니다.
* .kube/config 파일에는 무슨 내용이 있을까요 ?
- clusters: kubectl이 사용할 쿠버네티스 API Server의 접속 정보 목록
- users : 쿠버네티스의 API Server에 접속하기 위한 사용자 인증 정보 목록
- contexts: clusters 항목과 users 항목에 정의된 값을 조합하여 최종으로 사용할 쿠버네티스 클러스터 정보를 설정함.
RoleBinding을 이용한 인증/인가 실습
- 네임스페이스 생성: dev-team / infra-team
- Service Account(SA) 생성: dev-k8s / infra-k8s
- 각각 별도의 kubectl 파드를 생성하고 SA를 지정하여 권한을 부여합니다. (RoleBinding)
# 네임스페이스 생성
$ kubectl create namespace dev-team
namespace/dev-team created
$ kubectl create ns infra-team
namespace/infra-team created
# 네임스페이스 확인
$ kubectl get ns
NAME STATUS AGE
default Active 52m
dev-team Active 45s
infra-team Active 42s
kube-node-lease Active 52m
kube-public Active 52m
kube-system Active 52m
monitoring Active 5m51s
# 서비스 어카운트 생성
$ kubectl create sa dev-k8s -n dev-team
serviceaccount/dev-k8s created
$ kubectl create sa infra-k8s -n infra-team
serviceaccount/infra-k8s created
# 서비스 어카운트 정보 확인
$ kubectl get sa -n dev-team
NAME SECRETS AGE
default 0 2m1s
dev-k8s 0 47s
$ kubectl get sa dev-k8s -n dev-team -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
creationTimestamp: "2024-04-13T11:27:53Z"
name: dev-k8s
namespace: dev-team
resourceVersion: "13361"
uid: b8f1235d-c296-4e76-be75-db8cd1efa8d6
$ kubectl get sa -n infra-team
NAME SECRETS AGE
default 0 2m16s
infra-k8s 0 56s
$ kubectl get sa infra-k8s -n infra-team -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
creationTimestamp: "2024-04-13T11:28:02Z"
name: infra-k8s
namespace: infra-team
resourceVersion: "13407"
uid: 5ae66dc2-22b9-460c-a832-4d0474186909
# kubectl 파드 생성
# dev-team
$ cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
name: dev-kubectl
namespace: dev-team
spec:
serviceAccountName: dev-k8s
containers:
- name: kubectl-pod
image: bitnami/kubectl:1.28.5
command: ["tail"]
args: ["-f", "/dev/null"]
terminationGracePeriodSeconds: 0
EOF
# infra-team
$ cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
name: infra-kubectl
namespace: infra-team
spec:
serviceAccountName: infra-k8s
containers:
- name: kubectl-pod
image: bitnami/kubectl:1.28.5
command: ["tail"]
args: ["-f", "/dev/null"]
terminationGracePeriodSeconds: 0
EOF
# 확인
$ kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
dev-team dev-kubectl 1/1 Running 0 40s
infra-team infra-kubectl 1/1 Running 0 31s
$ kubectl get pod -o dev-kubectl -n dev-team -o yaml
serviceAccount: dev-k8s
serviceAccountName: dev-k8s
$ kubectl get pod -o infra-kubectl -n infra-team -o yaml
serviceAccount: infra-k8s
serviceAccountName: infra-k8s
# 파드에 기본 적용되는 서비스 어카운트(토큰) 정보 확인
$ kubectl exec -it dev-kubectl -n dev-team -- ls /run/secrets/kubernetes.io/serviceaccount
ca.crt namespace token
$ kubectl exec -it dev-kubectl -n dev-team -- cat /run/secrets/kubernetes.io/serviceaccount/token
$ kubectl exec -it dev-kubectl -n dev-team -- cat /run/secrets/kubernetes.io/serviceaccount/namespace
$ kubectl exec -it dev-kubectl -n dev-team -- cat /run/secrets/kubernetes.io/serviceaccount/ca.crt
# Alias 설정
$ alias k1='kubectl exec -it dev-kubectl -n dev-team -- kubectl'
$ alias k2='kubectl exec -it infra-kubectl -n infra-team -- kubectl'
# 권한 테스트
$ k1 get pods
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:dev-team:dev-k8s" cannot list resource "pods" in API group "" in the namespace "dev-team"
command terminated with exit code 1
$ k1 run nginx --image nginx:1.20-alpine
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:dev-team:dev-k8s" cannot create resource "pods" in API group "" in the namespace "dev-team"
command terminated with exit code 1
$ k1 get pods -n kube-system
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:dev-team:dev-k8s" cannot list resource "pods" in API group "" in the namespace "kube-system"
command terminated with exit code 1
$ k2 get pods
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:infra-team:infra-k8s" cannot list resource "pods" in API group "" in the namespace "infra-team"
command terminated with exit code 1
$ k2 run nginx --image nginx:1.20-alpine
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:infra-team:infra-k8s" cannot create resource "pods" in API group "" in the namespace "infra-team"
command terminated with exit code 1
$ k2 get pods -n kube-system
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:infra-team:infra-k8s" cannot list resource "pods" in API group "" in the namespace "kube-system"
command terminated with exit code 1
# kubectl 사용자의 특정 권한 여부 확인
$ k1 auth can-i get pods
no
$ k2 auth can-i get pods
no
# 모든 권한 Role 생성
$ cat <<EOF | kubectl create -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: role-dev-team
namespace: dev-team
rules:
- apiGroups: ["*"]
resources: ["*"]
verbs: ["*"]
EOF
role.rbac.authorization.k8s.io/role-dev-team created
$ cat <<EOF | kubectl create -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: role-infra-team
namespace: infra-team
rules:
- apiGroups: ["*"]
resources: ["*"]
verbs: ["*"]
EOF
role.rbac.authorization.k8s.io/role-infra-team created
# Role 생성 확인
$ kubectl get roles -n dev-team
$ kubectl get roles -n infra-team
$ kubectl get roles -n dev-team -o yaml
$ kubectl describe roles role-dev-team -n dev-team
```
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
*.* [] [] [*]
```
$ kubectl describe roles role-infra-team -n infra-team
```
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
*.* [] [] [*]
```
# RoleBinding 생성
$ cat <<EOF | kubectl create -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: roleB-dev-team
namespace: dev-team
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: role-dev-team
subjects:
- kind: ServiceAccount
name: dev-k8s
namespace: dev-team
EOF
rolebinding.rbac.authorization.k8s.io/roleB-dev-team created
$ cat <<EOF | kubectl create -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: roleB-infra-team
namespace: infra-team
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: role-infra-team
subjects:
- kind: ServiceAccount
name: infra-k8s
namespace: infra-team
EOF
rolebinding.rbac.authorization.k8s.io/roleB-infra-team created
# RoleBinding 생성 확인
$ kubectl get rolebindings -n dev-team
$ kubectl get rolebindings -n infra-team
$ kubectl get rolebindings -n dev-team -o yaml
$ kubectl describe rolebindings roleB-dev-team -n dev-team
```
Role:
Kind: Role
Name: role-dev-team
Subjects:
Kind Name Namespace
---- ---- ---------
ServiceAccount dev-k8s dev-team
```
$ kubectl describe rolebindings roleB-infra-team -n infra-team
Role:
Kind: Role
Name: role-infra-team
Subjects:
Kind Name Namespace
---- ---- ---------
ServiceAccount infra-k8s infra-team
# 권한 테스트
$ k1 get pods
NAME READY STATUS RESTARTS AGE
dev-kubectl 1/1 Running 0 50m
$ k1 run nginx --image nginx:1.20-alpine
pod/nginx created
$ k1 delete pods nginx
pod "nginx" deleted
$ k1 get pods -n kube-system
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:dev-team:dev-k8s" cannot list resource "pods" in API group "" in the namespace "kube-system"
command terminated with exit code 1
$ k1 get nodes
Error from server (Forbidden): nodes is forbidden: User "system:serviceaccount:dev-team:dev-k8s" cannot list resource "nodes" in API group "" at the cluster scope
command terminated with exit code 1
$ k2 get pods
NAME READY STATUS RESTARTS AGE
infra-kubectl 1/1 Running 0 52m
$ k2 run nginx --image nginx:1.20-alpine
pod/nginx created
$ k2 delete pods nginx
pod "nginx" deleted
$ k2 get pods -n kube-system
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:infra-team:infra-k8s" cannot list resource "pods" in API group "" in the namespace "kube-system"
command terminated with exit code 1
$ k2 get nodes
Error from server (Forbidden): nodes is forbidden: User "system:serviceaccount:infra-team:infra-k8s" cannot list resource "nodes" in API group "" at the cluster scope
command terminated with exit code 1
# kubectl 사용자의 특정 권한 여부 확인
$ k1 auth can-i get pods
yes
$ k2 auth can-i get pods
yes
2. EKS Authentication & Authorization
RBAC 관련 krew 플러그인
# Krew 플러그인 설치
$ kubectl krew install access-matrix rbac-tool rbac-view rolesum whoami
# kubectl 인증 주체 확인
$ kubectl whoami
arn:aws:iam::732659419746:user/admin
# Show an RBAC access matrix for server resources
$ kubectl access-matrix --namespace default
NAME LIST CREATE UPDATE DELETE
alertmanagerconfigs.monitoring.coreos.com ✔ ✔ ✔ ✔
alertmanagers.monitoring.coreos.com ✔ ✔ ✔ ✔
bindings ✔
configmaps ✔ ✔ ✔ ✔
controllerrevisions.apps ✔ ✔ ✔ ✔
cronjobs.batch ✔ ✔ ✔ ✔
csistoragecapacities.storage.k8s.io ✔ ✔ ✔ ✔
daemonsets.apps ✔ ✔ ✔ ✔
deployments.apps ✔ ✔ ✔ ✔
endpoints ✔ ✔ ✔ ✔
endpointslices.discovery.k8s.io ✔ ✔ ✔ ✔
events ✔ ✔ ✔ ✔
events.events.k8s.io ✔ ✔ ✔ ✔
horizontalpodautoscalers.autoscaling ✔ ✔ ✔ ✔
ingresses.networking.k8s.io ✔ ✔ ✔ ✔
jobs.batch ✔ ✔ ✔ ✔
leases.coordination.k8s.io ✔ ✔ ✔ ✔
limitranges ✔ ✔ ✔ ✔
localsubjectaccessreviews.authorization.k8s.io ✔
networkpolicies.networking.k8s.io ✔ ✔ ✔ ✔
persistentvolumeclaims ✔ ✔ ✔ ✔
poddisruptionbudgets.policy ✔ ✔ ✔ ✔
podmonitors.monitoring.coreos.com ✔ ✔ ✔ ✔
pods ✔ ✔ ✔ ✔
pods.metrics.k8s.io ✔
podtemplates ✔ ✔ ✔ ✔
policyendpoints.networking.k8s.aws ✔ ✔ ✔ ✔
probes.monitoring.coreos.com ✔ ✔ ✔ ✔
prometheusagents.monitoring.coreos.com ✔ ✔ ✔ ✔
prometheuses.monitoring.coreos.com ✔ ✔ ✔ ✔
prometheusrules.monitoring.coreos.com ✔ ✔ ✔ ✔
replicasets.apps ✔ ✔ ✔ ✔
replicationcontrollers ✔ ✔ ✔ ✔
resourcequotas ✔ ✔ ✔ ✔
rolebindings.rbac.authorization.k8s.io ✔ ✔ ✔ ✔
roles.rbac.authorization.k8s.io ✔ ✔ ✔ ✔
scrapeconfigs.monitoring.coreos.com ✔ ✔ ✔ ✔
secrets ✔ ✔ ✔ ✔
securitygrouppolicies.vpcresources.k8s.aws ✔ ✔ ✔ ✔
serviceaccounts ✔ ✔ ✔ ✔
servicemonitors.monitoring.coreos.com ✔ ✔ ✔ ✔
services ✔ ✔ ✔ ✔
statefulsets.apps ✔ ✔ ✔ ✔
targetgroupbindings.elbv2.k8s.aws ✔ ✔ ✔ ✔
thanosrulers.monitoring.coreos.com ✔ ✔ ✔ ✔
# RBAC Lookup by subject(user/group/serviceaccount)
$ kubectl rbac-tool lookup system:masters
SUBJECT | SUBJECT TYPE | SCOPE | NAMESPACE | ROLE | BINDING
+----------------+--------------+-------------+-----------+---------------+---------------+
system:masters | Group | ClusterRole | | cluster-admin | cluster-admin
$ kubectl rbac-tool lookup system:nodes
SUBJECT | SUBJECT TYPE | SCOPE | NAMESPACE | ROLE | BINDING
+--------------+--------------+-------------+-----------+-----------------------+-----------------------+
system:nodes | Group | ClusterRole | | eks:node-bootstrapper | eks:node-bootstrapper
$ kubectl rbac-tool lookup system:bootstrappers
SUBJECT | SUBJECT TYPE | SCOPE | NAMESPACE | ROLE | BINDING
+----------------------+--------------+-------------+-----------+-----------------------+-----------------------+
system:bootstrappers | Group | ClusterRole | | eks:node-bootstrapper | eks:node-bootstrapper
$ kubectl describe ClusterRole eks:node-bootstrapper
Name: eks:node-bootstrapper
Labels: eks.amazonaws.com/component=node
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
certificatesigningrequests.certificates.k8s.io/selfnodeserver [] [] [create]
# RBAC List Policy Rules for subject(user/group/serviceaccount)
$ kubectl rbac-tool policy-rules -e '^system:authenticated'
TYPE | SUBJECT | VERBS | NAMESPACE | API GROUP | KIND | NAMES | NONRESOURCEURI | ORIGINATED FROM
+-------+----------------------+--------+-----------+-----------------------+--------------------------+-------+------------------------------------------------------------------------------------------+-----------------------------------------+
Group | system:authenticated | create | * | authentication.k8s.io | selfsubjectreviews | | | ClusterRoles>>system:basic-user
Group | system:authenticated | create | * | authorization.k8s.io | selfsubjectaccessreviews | | | ClusterRoles>>system:basic-user
Group | system:authenticated | create | * | authorization.k8s.io | selfsubjectrulesreviews | | | ClusterRoles>>system:basic-user
Group | system:authenticated | get | * | | | | /api,/api/*,/apis,/apis/*,/healthz,/livez,/openapi,/openapi/*,/readyz,/version,/version/ | ClusterRoles>>system:discovery
Group | system:authenticated | get | * | | | | /healthz,/livez,/readyz,/version,/version/ | ClusterRoles>>system:public-info-viewer
# Generate ClusterRole with all available permissions from the target cluster
$ kubectl rbac-tool show
# Shows the subject for the current context with which one authenticates with the cluster
$ kubectl rbac-tool whoami
{Username: "arn:aws:iam::732*****:user/admin",
UID: "aws-iam-authenticator:732*****:AIDA*****",
Groups: ["system:authenticated"],
Extra: {accessKeyId: ["AKIA*******"],
arn: ["arn:aws:iam::732*****:user/admin"],
canonicalArn: ["arn:aws:iam::732*****:user/admin"],
principalId: ["AIDA*****"],
sessionName: [""]}}
# Summarize RBAC roles for subjects : ServiceAccount(default), User, Group
$ kubectl rolesum -k Group system:authenticated
Group: system:authenticated
Policies:
• [CRB] */system:basic-user ⟶ [CR] */system:basic-user
Resource Name Exclude Verbs G L W C U P D DC
selfsubjectaccessreviews.authorization.k8s.io [*] [-] [-] ✖ ✖ ✖ ✔ ✖ ✖ ✖ ✖
selfsubjectreviews.authentication.k8s.io [*] [-] [-] ✖ ✖ ✖ ✔ ✖ ✖ ✖ ✖
selfsubjectrulesreviews.authorization.k8s.io [*] [-] [-] ✖ ✖ ✖ ✔ ✖ ✖ ✖ ✖
• [CRB] */system:discovery ⟶ [CR] */system:discovery
• [CRB] */system:public-info-viewer ⟶ [CR] */system:public-info-viewer
# A tool to visualize your RBAC permissions
$ kubectl rbac-view
INFO[0000] Getting K8s client
INFO[0000] serving RBAC View and http://localhost:8800
$ netstat -lntp
tcp6 0 0 :::8800 :::* LISTEN 10667/kubectl-rbac_
# RBAC 웹 페이지 접속
$ echo -e "RBAC View Web http://$(curl -s ipinfo.io/ip):8800"
RBAC View Web http://43.203.204.127:8800
EKS Authentication & Authorization
- 핵심: 인증은 AWS IAM에서 처리하고, 인가는 K8s RBAC에서 처리합니다.
1. kubectl 명령 수행 시, 'aws eks get-token'으로 EKS Service endpoint(STS)에 토큰을 요청합니다.
# sts caller id의 ARN 확인
$ aws sts get-caller-identity --query Arn
"arn:aws:iam::732659419746:user/admin"
# kubeconfig 정보 확인
$ cat ~/.kube/config
```
- name: admin@myeks.ap-northeast-2.eksctl.io
user:
exec:
apiVersion: client.authentication.k8s.io/v1beta1
args:
- eks
- get-token
- --output
- json
- --cluster-name
- myeks
- --region
- ap-northeast-2
command: aws
env:
- name: AWS_STS_REGIONAL_ENDPOINTS
value: regional
interactiveMode: IfAvailable
provideClusterInfo: false
```
$ aws eks get-token help
DESCRIPTION
Get a token for authentication with an Amazon EKS cluster. This can be
used as an alternative to the aws-iam-authenticator.
# STS 토큰 요청
$ aws eks get-token --cluster-name $CLUSTER_NAME
{
"kind": "ExecCredential",
"apiVersion": "client.authentication.k8s.io/v1beta1",
"spec": {},
"status": {
"expirationTimestamp": "2024-04-13T13:30:30Z",
"token": "k8s-aws-v1.aHR0cHM6Ly9*******************NzcwYjU0OTU4Yjc"
}
}
$ aws eks get-token --cluster-name $CLUSTER_NAME | jq -r '.status.token'
2. Pre-Signed URL을 Bearer Token으로 EKS API Cluster Endpoint로 요청을 보냅니다.
"https://sts.ap-northeast-2.amazonaws.com/?
Action=GetCallerIdentity&
Version=2011-06-15&
X-Amz-Algorithm=AWS4-HMAC-SHA256&
X-Amz-Credential=AKIA2VF*****/20240413/ap-northeast-2/sts/aws4_request&
X-Amz-Date=20240413T131630Z&
X-Amz-Expires=60&
X-Amz-SignedHeaders=host;x-k8s-aws-id&
X-Amz-Signature=176c46d5275f6*****"
3. EKS API는 Token Review를 Webhook token authenticator에 요청합니다.
그리고 AWS IAM 인증 완료 이후 User/Role에 대한 ARN을 반환합니다.
# tokenreviews api 리소스 확인
$ kubectl api-resources | grep authentication
selfsubjectreviews authentication.k8s.io/v1 false SelfSubjectReview
tokenreviews authentication.k8s.io/v1 false TokenReview
# List the fields for supported resources.
$ kubectl explain tokenreviews
...
DESCRIPTION:
TokenReview attempts to authenticate a token to a known user. Note:
TokenReview requests may be cached by the webhook token authenticator
plugin in the kube-apiserver.
4. K8s RBAC 인가를 처리합니다.
- 해당 IAM User/Role 확인이 되면 k8s aws-auth configmap에서 mapping 정보를 확인하게 됩니다.
- aws-auth 컨피그맵에 'IAM 사용자, 역할 arm, K8S 오브젝트' 로 권한 확인 후 k8s 인가 허가가 되면 최종적으로 동작 실행을 합니다.
- 참고로 EKS를 생성한 IAM principal은 aws-auth 와 상관없이 kubernetes-admin Username으로 system:masters 그룹에 권한을 가집니다.
# Webhook api 리소스 확인
$ kubectl api-resources | grep Webhook
mutatingwebhookconfigurations admissionregistration.k8s.io/v1 false MutatingWebhookConfiguration
validatingwebhookconfigurations admissionregistration.k8s.io/v1 false ValidatingWebhookConfiguration
# validatingwebhookconfigurations 리소스 확인
$ kubectl get validatingwebhookconfigurations
NAME WEBHOOKS AGE
aws-load-balancer-webhook 3 130m
eks-aws-auth-configmap-validation-webhook 1 173m
kube-prometheus-stack-admission 1 126m
vpc-resource-validating-webhook 2 173m
$ kubectl get validatingwebhookconfigurations eks-aws-auth-configmap-validation-webhook -o yaml | kubectl neat | yh
# aws-auth Configmap 확인
$ kubectl get cm -n kube-system aws-auth -o yaml | kubectl neat | yh
apiVersion: v1
data:
mapRoles: |
- groups:
- system:bootstrappers
- system:nodes
rolearn: arn:aws:iam::732659419746:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-z0wkA5ivz992
username: system:node:{{EC2PrivateDNSName}}
kind: ConfigMap
metadata:
name: aws-auth
namespace: kube-system
# system:masters , system:authenticated 그룹의 정보 확인
$ kubectl rbac-tool lookup system:masters
SUBJECT | SUBJECT TYPE | SCOPE | NAMESPACE | ROLE | BINDING
+----------------+--------------+-------------+-----------+---------------+---------------+
system:masters | Group | ClusterRole | | cluster-admin | cluster-admin
$ kubectl rbac-tool lookup system:authenticated
SUBJECT | SUBJECT TYPE | SCOPE | NAMESPACE | ROLE | BINDING
+----------------------+--------------+-------------+-----------+---------------------------+---------------------------+
system:authenticated | Group | ClusterRole | | system:discovery | system:discovery
system:authenticated | Group | ClusterRole | | system:basic-user | system:basic-user
system:authenticated | Group | ClusterRole | | system:public-info-viewer | system:public-info-viewer
$ kubectl rolesum -k Group system:masters
Group: system:masters
Policies:
• [CRB] */cluster-admin ⟶ [CR] */cluster-admin
Resource Name Exclude Verbs G L W C U P D DC
*.* [*] [-] [-] ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔
$ kubectl rolesum -k Group system:authenticated
Group: system:authenticated
Policies:
• [CRB] */system:basic-user ⟶ [CR] */system:basic-user
Resource Name Exclude Verbs G L W C U P D DC
selfsubjectaccessreviews.authorization.k8s.io [*] [-] [-] ✖ ✖ ✖ ✔ ✖ ✖ ✖ ✖
selfsubjectreviews.authentication.k8s.io [*] [-] [-] ✖ ✖ ✖ ✔ ✖ ✖ ✖ ✖
selfsubjectrulesreviews.authorization.k8s.io [*] [-] [-] ✖ ✖ ✖ ✔ ✖ ✖ ✖ ✖
• [CRB] */system:discovery ⟶ [CR] */system:discovery
• [CRB] */system:public-info-viewer ⟶ [CR] */system:public-info-viewer
# system:masters 그룹이 사용 가능한 클러스터 롤 확인 : cluster-admin
$ kubectl describe clusterrolebindings.rbac.authorization.k8s.io cluster-admin
Name: cluster-admin
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
Role:
Kind: ClusterRole
Name: cluster-admin
Subjects:
Kind Name Namespace
---- ---- ---------
Group system:masters
# cluster-admin 의 PolicyRule 확인 : 모든 리소스 사용 가능
$ kubectl describe clusterrole cluster-admin
Name: cluster-admin
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
*.* [] [] [*]
[*] [] [*]
# system:authenticated 그룹이 사용 가능한 클러스터 롤 확인
$ kubectl describe ClusterRole system:discovery
Name: system:discovery
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
[/api/*] [] [get]
[/api] [] [get]
[/apis/*] [] [get]
[/apis] [] [get]
[/healthz] [] [get]
[/livez] [] [get]
[/openapi/*] [] [get]
[/openapi] [] [get]
[/readyz] [] [get]
[/version/] [] [get]
[/version] [] [get]
$ kubectl describe ClusterRole system:public-info-viewer
Name: system:public-info-viewer
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
[/healthz] [] [get]
[/livez] [] [get]
[/readyz] [] [get]
[/version/] [] [get]
[/version] [] [get]
$ kubectl describe ClusterRole system:basic-user
Name: system:basic-user
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
selfsubjectreviews.authentication.k8s.io [] [] [create]
selfsubjectaccessreviews.authorization.k8s.io [] [] [create]
selfsubjectrulesreviews.authorization.k8s.io [] [] [create]
$ kubectl describe ClusterRole eks:podsecuritypolicy:privileged
Error from server (NotFound): clusterroles.rbac.authorization.k8s.io "eks:podsecuritypolicy:privileged" not found
Devops 신입을 위한 인증/인가 설정해보기
1. Devops 신입에게 발급할 'testuser' 사용자를 생성합니다.
# testuser 사용자 생성
$ aws iam create-user --user-name testuser
{
"User": {
"Path": "/",
"UserName": "testuser",
"UserId": "AIDA2*****",
"Arn": "arn:aws:iam::732659419746:user/testuser",
"CreateDate": "2024-04-13T14:02:30+00:00"
}
}
# 사용자에게 프로그래밍 방식 액세스 권한 부여
$ aws iam create-access-key --user-name testuser
{
"AccessKey": {
"UserName": "testuser",
"AccessKeyId": "AKIA2*****",
"Status": "Active",
"SecretAccessKey": "uqIF3O*****",
"CreateDate": "2024-04-13T14:04:10+00:00"
}
}
# testuser 사용자에 정책을 추가
$ aws iam attach-user-policy --policy-arn arn:aws:iam::aws:policy/AdministratorAccess --user-name testuser
# get-caller-identity 확인
$ aws sts get-caller-identity --query Arn
"arn:aws:iam::732659419746:user/admin"
2. 'testuser' 자격증명을 설정합니다.
# get-caller-identity 확인
$ aws sts get-caller-identity --query Arn
Unable to locate credentials. You can configure credentials by running "aws configure".
# testuser 자격증명 설정
$ aws configure
AWS Access Key ID [None]: AKIA2*****
AWS Secret Access Key [None]: uqIF3*****
Default region name [None]: ap-northeast-2
# get-caller-identity 다시 확인
$ aws sts get-caller-identity --query Arn
"arn:aws:iam::732659419746:user/testuser"
# kubectl 시도
$ kubectl get node -v6
I0413 23:09:28.406298 2044 round_trippers.go:553] GET http://localhost:8080/api?timeout=32s in 0 milliseconds
E0413 23:09:28.406501 2044 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
I0413 23:09:28.406521 2044 cached_discovery.go:120] skipped caching discovery info due to Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
I0413 23:09:28.406827 2044 round_trippers.go:553] GET http://localhost:8080/api?timeout=32s in 0 milliseconds
E0413 23:09:28.406871 2044 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
I0413 23:09:28.408017 2044 cached_discovery.go:120] skipped caching discovery info due to Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
I0413 23:09:28.408044 2044 shortcut.go:100] Error loading discovery information: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
I0413 23:09:28.408276 2044 round_trippers.go:553] GET http://localhost:8080/api?timeout=32s in 0 milliseconds
E0413 23:09:28.408317 2044 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
I0413 23:09:28.408332 2044 cached_discovery.go:120] skipped caching discovery info due to Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
I0413 23:09:28.408534 2044 round_trippers.go:553] GET http://localhost:8080/api?timeout=32s in 0 milliseconds
E0413 23:09:28.408571 2044 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
I0413 23:09:28.409656 2044 cached_discovery.go:120] skipped caching discovery info due to Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
I0413 23:09:28.409851 2044 round_trippers.go:553] GET http://localhost:8080/api?timeout=32s in 0 milliseconds
E0413 23:09:28.409889 2044 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
I0413 23:09:28.409904 2044 cached_discovery.go:120] skipped caching discovery info due to Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
I0413 23:09:28.409941 2044 helpers.go:264] Connection error: Get http://localhost:8080/api?timeout=32s: dial tcp 127.0.0.1:8080: connect: connection refused
$ ls ~/.kube
ls: cannot access /root/.kube: No such file or directory
3. 'testuser'에 system:masters 그룹을 부여하여 EKS 관리자 수준의 권한을 설정합니다.
# Creates a mapping from IAM role or user to Kubernetes user and groups
$ eksctl get iamidentitymapping --cluster $CLUSTER_NAME
ARN USERNAME GROUPS ACCOUNT
arn:aws:iam::732659419746:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-z0wkA5ivz992 system:node:{{EC2PrivateDNSName}} system:bootstrappers,system:nodes
$ eksctl create iamidentitymapping --cluster $CLUSTER_NAME --username testuser --group system:masters --arn arn:aws:iam::$ACCOUNT_ID:user/testuser
2024-04-13 23:12:13 [ℹ] checking arn arn:aws:iam::732659419746:user/testuser against entries in the auth ConfigMap
2024-04-13 23:12:13 [ℹ] adding identity "arn:aws:iam::732659419746:user/testuser" to auth ConfigMap
# ConfigMap 확인
$ kubectl get cm -n kube-system aws-auth -o yaml | kubectl neat | yh
apiVersion: v1
data:
mapRoles: |
- groups:
- system:bootstrappers
- system:nodes
rolearn: arn:aws:iam::732659419746:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-z0wkA5ivz992
username: system:node:{{EC2PrivateDNSName}}
mapUsers: |
- groups:
- system:masters
userarn: arn:aws:iam::732659419746:user/testuser
username: testuser
kind: ConfigMap
metadata:
name: aws-auth
namespace: kube-system
$ kubectl get validatingwebhookconfigurations eks-aws-auth-configmap-validation-webhook -o yaml | kubectl neat | yh
$ eksctl get iamidentitymapping --cluster $CLUSTER_NAME
ARN USERNAME GROUPS ACCOUNT
arn:aws:iam::732659419746:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-z0wkA5ivz992 system:node:{{EC2PrivateDNSName}} system:bootstrappers,system:nodes
arn:aws:iam::732659419746:user/testuser testuser system:masters
4. 'testuser' kubeconfig 생성 후 kubectl 명령어를 실행합니다.
# testuser kubeconfig 생성
$ aws eks update-kubeconfig --name $CLUSTER_NAME --user-alias testuser
Added new context testuser to /root/.kube/config
# kubectl 사용 확인
$ kubectl ns default
Context "testuser" modified.
Active namespace is "default".
$ kubectl get node -v6
I0413 23:16:51.164168 2246 loader.go:395] Config loaded from file: /root/.kube/config
I0413 23:16:51.971739 2246 round_trippers.go:553] GET https://8D687B71EBE1B368120C1A1450DD3178.gr7.ap-northeast-2.eks.amazonaws.com/api/v1/nodes?limit=500 200 OK in 796 milliseconds
NAME STATUS ROLES AGE VERSION
ip-192-168-1-162.ap-northeast-2.compute.internal Ready <none> 3h32m v1.28.5-eks-5e0fdde
ip-192-168-2-115.ap-northeast-2.compute.internal Ready <none> 3h32m v1.28.5-eks-5e0fdde
ip-192-168-3-166.ap-northeast-2.compute.internal Ready <none> 3h32m v1.28.5-eks-5e0fdde
$ kubectl krew install rbac-tool && kubectl rbac-tool whoami
```
{Username: "testuser",
UID: "aws-iam-authenticator:732659419746:AIDA2*****",
Groups: ["system:masters",
"system:authenticated"],
Extra: {accessKeyId: ["AKIA2V*****"],
arn: ["arn:aws:iam::732659419746:user/testuser"],
canonicalArn: ["arn:aws:iam::732659419746:user/testuser"],
principalId: ["AIDA2*****"],
sessionName: [""]}}
```
5. Configmap에서 'testuser'의 그룹을 system:authenticated로 변경하여 RBAC 동작을 확인합니다.
노드 정보를 조회하였을 때 권한 거부되는 것을 확인할 수 있습니다.
$ kubectl edit cm -n kube-system aws-auth
- groups:
- system:authenticated
# 그룹 변경 확인
$ eksctl get iamidentitymapping --cluster $CLUSTER_NAME
ARN USERNAME GROUPS ACCOUNT
arn:aws:iam::732659419746:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-z0wkA5ivz992 system:node:{{EC2PrivateDNSName}} system:bootstrappers,system:nodes
arn:aws:iam::732659419746:user/testuser testuser system:authenticated
# kubectl 실행
$ kubectl get node -v6
I0413 23:22:14.246382 2644 loader.go:395] Config loaded from file: /root/.kube/config
I0413 23:22:15.127064 2644 round_trippers.go:553] GET https://8D687B71EBE1B368120C1A1450DD3178.gr7.ap-northeast-2.eks.amazonaws.com/api/v1/nodes?limit=500 403 Forbidden in 868 milliseconds
I0413 23:22:15.127415 2644 helpers.go:246] server response object: [{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "nodes is forbidden: User \"testuser\" cannot list resource \"nodes\" in API group \"\" at the cluster scope",
"reason": "Forbidden",
"details": {
"kind": "nodes"
},
"code": 403
}]
Error from server (Forbidden): nodes is forbidden: User "testuser" cannot list resource "nodes" in API group "" at the cluster scope
$ kubectl api-resources -v5
```
NAME SHORTNAMES APIVERSION NAMESPACED KIND
bindings v1 true Binding
componentstatuses cs v1 false ComponentStatus
configmaps cm v1 true ConfigMap
endpoints ep v1 true Endpoints
events ev v1 true Event
limitranges limits v1 true LimitRange
namespaces ns v1 false Namespace
nodes no v1 false Node
```
6. Configmap에서 'testuser'의 IAM 매핑 정보를 삭제하고 RBAC 동작을 확인합니다.
'testuser'로 어떠한 리소스 정보도 조회할 수 없음을 알 수 있습니다.
이렇게 Configmap을 잘못 수정하면 EKS에 대한 접근 권한을 잃게 됩니다.
# testuser IAM 매핑 정보 삭제
$ eksctl delete iamidentitymapping --cluster $CLUSTER_NAME --arn arn:aws:iam::$ACCOUNT_ID:user/testuser
2024-04-13 23:25:06 [ℹ] removing identity "arn:aws:iam::732659419746:user/testuser" from auth ConfigMap (username = "testuser", groups = ["system:authenticated"])
# Get IAM identity mapping(s)
$ eksctl get iamidentitymapping --cluster $CLUSTER_NAME
ARN USERNAME GROUPS ACCOUNT
arn:aws:iam::732659419746:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-z0wkA5ivz992 system:node:{{EC2PrivateDNSName}} system:bootstrappers,system:nodes
$ kubectl get cm -n kube-system aws-auth -o yaml
```
mapUsers: |
[]
kind: ConfigMap
```
# kubectl 실행
$ kubectl get node -v6
I0413 23:26:58.146916 3068 loader.go:395] Config loaded from file: /root/.kube/config
I0413 23:26:59.766244 3068 round_trippers.go:553] GET https://8D687B71EBE1B368120C1A1450DD3178.gr7.ap-northeast-2.eks.amazonaws.com/api/v1/nodes?limit=500 401 Unauthorized in 1603 milliseconds
I0413 23:26:59.766608 3068 helpers.go:246] server response object: [{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "Unauthorized",
"reason": "Unauthorized",
"code": 401
}]
$ kubectl api-resources -v5
E0413 23:27:22.582233 3125 memcache.go:265] couldn't get current server API group list: the server has asked for the client to provide credentials
I0413 23:27:22.582262 3125 cached_discovery.go:120] skipped caching discovery info due to the server has asked for the client to provide credentials
NAME SHORTNAMES APIVERSION NAMESPACED KIND
I0413 23:27:22.582489 3125 helpers.go:246] server response object: [{
"metadata": {},
"status": "Failure",
"message": "the server has asked for the client to provide credentials",
"reason": "Unauthorized",
"details": {
"causes": [
{
"reason": "UnexpectedServerResponse",
"message": "unknown"
}
]
},
"code": 401
}]
error: You must be logged in to the server (the server has asked for the client to provide credentials)
EKS Access Management Controls
AWS IAM 보안 주체가 EKS 클러스터에 액세스하는 방법을 제어하는 기능입니다.
클러스터 액세스 인증 모드로 EKS API, ConfigMap을 지원하고 있으며,
EKS API를 사용하도록 수정한 후에는 ConfigMap 전용으로 다시 변경할 수 없습니다.
두 가지 방법을 모두 사용할 경우, EKS API 정책이 ConfigMap 정책보다 우선합니다.
- EKS API : 클러스터가 인증된 IAM 보안 주체를 EKS API에서만 소싱합니다.
- EKS API 및 ConfigMap : 클러스터가 인증된 IAM 보안 주체를 EKS API와 aws-auth ConfigMap 모두에서 소싱합니다.
- ConfigMap : 클러스터가 인증된 IAM 보안 주체를 aws-auth ConfigMap에서만 소싱합니다.
액세스 인증 모드를 EKS API로 변경합니다. 적용되기까지 수 분 소요됩니다.
액세스 정책으로 AmazonEKSAdminPolicy, AmazonEKSClusterAdminPolicy, AmazonEKSEditPolicy, AmazonEKSViewPolicy를 제공하고 있습니다.
# Access Policy List
$ aws eks list-access-policies
{
"accessPolicies": [
{
"name": "AmazonEKSAdminPolicy",
"arn": "arn:aws:eks::aws:cluster-access-policy/AmazonEKSAdminPolicy"
},
{
"name": "AmazonEKSClusterAdminPolicy",
"arn": "arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy"
},
{
"name": "AmazonEKSEditPolicy",
"arn": "arn:aws:eks::aws:cluster-access-policy/AmazonEKSEditPolicy"
},
{
"name": "AmazonEKSViewPolicy",
"arn": "arn:aws:eks::aws:cluster-access-policy/AmazonEKSViewPolicy"
},
{
"name": "AmazonEMRJobPolicy",
"arn": "arn:aws:eks::aws:cluster-access-policy/AmazonEMRJobPolicy"
}
]
}
# 매핑된 ClusterRole 정보 확인
$ kubectl get clusterroles -l 'kubernetes.io/bootstrapping=rbac-defaults' | grep -v 'system:'
NAME CREATED AT
admin 2024-04-13T10:35:05Z
cluster-admin 2024-04-13T10:35:05Z
edit 2024-04-13T10:35:05Z
view 2024-04-13T10:35:05Z
EKS를 설치한 주체인 'Admin' 계정은 AmazonEKSClusterAmdinPolicy 정책과 연결되어 있는 모습입니다.
# Admin 계정에 연결된 Access Policy 정보 확인
$ aws eks list-associated-access-policies --cluster-name $CLUSTER_NAME --principal-arn arn:aws:iam::$ACCOUNT_ID:user/admin | jq
{
"associatedAccessPolicies": [
{
"policyArn": "arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy",
"accessScope": {
"type": "cluster",
"namespaces": []
},
"associatedAt": "2024-04-13T19:29:42.634000+09:00",
"modifiedAt": "2024-04-13T19:29:42.634000+09:00"
}
],
"clusterName": "myeks",
"principalArn": "arn:aws:iam::732659419746:user/admin"
}
IAM 액세스 항목에서 IAM 보안 주체에 적용된 유형도 확인할 수 있습니다.
# Access Entries
$ aws eks list-access-entries --cluster-name $CLUSTER_NAME | jq(admin@myeks:default) [root@myeks-bastion ~]# aws eks list-access-entries --cluster-name $CLUSTER_NAME
{
"accessEntries": [
"arn:aws:iam::732659419746:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-z0wkA5ivz992",
"arn:aws:iam::732659419746:user/admin"
]
}
# Access Entry Type
$ aws eks describe-access-entry --cluster-name $CLUSTER_NAME --principal-arn arn:aws:iam::$ACCOUNT_ID:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-z0wkA5ivz992
{
"accessEntry": {
"clusterName": "myeks",
"principalArn": "arn:aws:iam::732659419746:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-z0wkA5ivz992",
"kubernetesGroups": [
"system:nodes"
],
"accessEntryArn": "arn:aws:eks:ap-northeast-2:732659419746:access-entry/myeks/role/732659419746/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-z0wkA5ivz992/dec76b86-f5ab-bc96-80e9-4c0acefbe4f9",
"createdAt": "2024-04-13T19:43:17.540000+09:00",
"modifiedAt": "2024-04-13T19:43:17.540000+09:00",
"tags": {},
"username": "system:node:{{EC2PrivateDNSName}}",
"type": "EC2_LINUX"
}
}
$ aws eks describe-access-entry --cluster-name $CLUSTER_NAME --principal-arn arn:aws:iam::$ACCOUNT_ID:user/admin
{
"accessEntry": {
"clusterName": "myeks",
"principalArn": "arn:aws:iam::732659419746:user/admin",
"kubernetesGroups": [],
"accessEntryArn": "arn:aws:eks:ap-northeast-2:732659419746:access-entry/myeks/user/732659419746/admin/8ec76b80-bdce-3d8f-9fcb-8a4edb6f9117",
"createdAt": "2024-04-13T19:29:42.498000+09:00",
"modifiedAt": "2024-04-13T19:29:42.498000+09:00",
"tags": {},
"username": "arn:aws:iam::732659419746:user/admin",
"type": "STANDARD"
}
}
그럼 신규 기능을 이용하여 기존 'testuser' 계정에 권한을 부여해보겠습니다.
# testuser 의 access entry 생성
$ aws eks create-access-entry --cluster-name $CLUSTER_NAME --principal-arn arn:aws:iam::$ACCOUNT_ID:user/testuser
{
"accessEntry": {
"clusterName": "myeks",
"principalArn": "arn:aws:iam::732659419746:user/testuser",
"kubernetesGroups": [],
"accessEntryArn": "arn:aws:eks:ap-northeast-2:732659419746:access-entry/myeks/user/732659419746/testuser/d2c76c26-0eec-f87e-025b-2432eadf93d1",
"createdAt": "2024-04-14T01:30:50.941000+09:00",
"modifiedAt": "2024-04-14T01:30:50.941000+09:00",
"tags": {},
"username": "arn:aws:iam::732659419746:user/testuser",
"type": "STANDARD"
}
}
$ aws eks list-access-entries --cluster-name $CLUSTER_NAME | jq -r .accessEntries[]
arn:aws:iam::732659419746:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-z0wkA5ivz992
arn:aws:iam::732659419746:user/admin
arn:aws:iam::732659419746:user/testuser
# testuser에 AmazonEKSClusterAdminPolicy 연동
$ aws eks associate-access-policy --cluster-name $CLUSTER_NAME --principal-arn arn:aws:iam::$ACCOUNT_ID:user/testuser \
--policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy --access-scope type=cluster
{
"clusterName": "myeks",
"principalArn": "arn:aws:iam::732659419746:user/testuser",
"associatedAccessPolicy": {
"policyArn": "arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy",
"accessScope": {
"type": "cluster",
"namespaces": []
},
"associatedAt": "2024-04-14T01:31:40.549000+09:00",
"modifiedAt": "2024-04-14T01:31:40.549000+09:00"
}
}
# Access Policy 연동 확인
$ aws eks list-associated-access-policies --cluster-name $CLUSTER_NAME --principal-arn arn:aws:iam::$ACCOUNT_ID:user/testuser | jq
{
"associatedAccessPolicies": [
{
"policyArn": "arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy",
"accessScope": {
"type": "cluster",
"namespaces": []
},
"associatedAt": "2024-04-14T01:31:40.549000+09:00",
"modifiedAt": "2024-04-14T01:31:40.549000+09:00"
}
],
"clusterName": "myeks",
"principalArn": "arn:aws:iam::732659419746:user/testuser"
}
# Access Entry 확인
$ aws eks describe-access-entry --cluster-name $CLUSTER_NAME --principal-arn arn:aws:iam::$ACCOUNT_ID:user/testuser | jq
{
"accessEntry": {
"clusterName": "myeks",
"principalArn": "arn:aws:iam::732659419746:user/testuser",
"kubernetesGroups": [],
"accessEntryArn": "arn:aws:eks:ap-northeast-2:732659419746:access-entry/myeks/user/732659419746/testuser/d2c76c26-0eec-f87e-025b-2432eadf93d1",
"createdAt": "2024-04-14T01:30:50.941000+09:00",
"modifiedAt": "2024-04-14T01:30:50.941000+09:00",
"tags": {},
"username": "arn:aws:iam::732659419746:user/testuser",
"type": "STANDARD"
}
}
'testuser' 계정으로 kubectl 명령어를 실행해 봅니다.
# testuser 정보 확인
$ aws sts get-caller-identity --query Arn
"arn:aws:iam::732659419746:user/testuser"
$ kubectl whoami
arn:aws:iam::732659419746:user/testuser
# kubectl 명령어 실행
$ kubectl get node -v6
I0414 01:37:05.996913 3924 loader.go:395] Config loaded from file: /root/.kube/config
I0414 01:37:06.762059 3924 round_trippers.go:553] GET https://8D687B71EBE1B368120C1A1450DD3178.gr7.ap-northeast-2.eks.amazonaws.com/api/v1/nodes?limit=500 200 OK in 755 milliseconds
NAME STATUS ROLES AGE VERSION
ip-192-168-1-162.ap-northeast-2.compute.internal Ready <none> 5h53m v1.28.5-eks-5e0fdde
ip-192-168-2-115.ap-northeast-2.compute.internal Ready <none> 5h53m v1.28.5-eks-5e0fdde
ip-192-168-3-166.ap-northeast-2.compute.internal Ready <none> 5h53m v1.28.5-eks-5e0fdde
$ kubectl api-resources -v5
```
NAME SHORTNAMES APIVERSION NAMESPACED KIND
bindings v1 true Binding
componentstatuses cs v1 false ComponentStatus
configmaps cm v1 true ConfigMap
endpoints ep v1 true Endpoints
events ev v1 true Event
limitranges limits v1 true LimitRange
namespaces ns v1 false Namespace
nodes no v1 false Node
```
$ kubectl rbac-tool whoami
{Username: "arn:aws:iam::732*****:user/testuser",
UID: "aws-iam-authenticator:732*****:AIDA2*****",
Groups: ["system:authenticated"],
Extra: {accessKeyId: ["AKIA*****"],
arn: ["arn:aws:iam::732*****:user/testuser"],
canonicalArn: ["arn:aws:iam::732*****:user/testuser"],
principalId: ["AIDA2*****"],
sessionName: [""]}}
$ kubectl auth can-i delete pods --all-namespaces
yes
# ConfigMap 정보 없음
$ kubectl get cm -n kube-system aws-auth -o yaml | kubectl neat | yh
apiVersion: v1
data:
mapRoles: |
- groups:
- system:bootstrappers
- system:nodes
rolearn: arn:aws:iam::732659419746:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-z0wkA5ivz992
username: system:node:{{EC2PrivateDNSName}}
mapUsers: |
[]
kind: ConfigMap
metadata:
name: aws-auth
namespace: kube-system
# iamidentitymapping을 더이상 사용하지 않음
$ eksctl get iamidentitymapping --cluster $CLUSTER_NAME
ARN USERNAME GROUPS ACCOUNT
arn:aws:iam::732659419746:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-z0wkA5ivz992 system:node:{{EC2PrivateDNSName}} system:bootstrappers,system:nodes
이번에는 기본 제공되는 액세스 정책이 아닌 커스터마이징 정책을 연결해보겠습니다.
'pod-viewer', 'pod-admin' 두 가지를 생성하고 'pod-viewer'를 먼저 적용해봅니다.
# 기존 testuser access entry 제거
$ aws eks delete-access-entry --cluster-name $CLUSTER_NAME --principal-arn arn:aws:iam::$ACCOUNT_ID:user/testuser
# 커스터마이징 ClusterRole 배포
$ cat <<EoF> ~/pod-viewer-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: pod-viewer-role
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["list", "get", "watch"]
EoF
$ cat <<EoF> ~/pod-admin-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: pod-admin-role
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["*"]
EoF
$ kubectl apply -f ~/pod-viewer-role.yaml
clusterrole.rbac.authorization.k8s.io/pod-viewer-role created
$ kubectl apply -f ~/pod-admin-role.yaml
clusterrole.rbac.authorization.k8s.io/pod-admin-role created
# ClusterRoleBinding 생성
$ kubectl create clusterrolebinding viewer-role-binding --clusterrole=pod-viewer-role --group=pod-viewer
clusterrolebinding.rbac.authorization.k8s.io/viewer-role-binding created
$ kubectl create clusterrolebinding admin-role-binding --clusterrole=pod-admin-role --group=pod-admin
clusterrolebinding.rbac.authorization.k8s.io/admin-role-binding created
# 커스터마이징 정책을 적용하여 testuser 계정 생성
$ aws eks create-access-entry --cluster-name $CLUSTER_NAME --principal-arn arn:aws:iam::$ACCOUNT_ID:user/testuser --kubernetes-group pod-viewer
{
"accessEntry": {
"clusterName": "myeks",
"principalArn": "arn:aws:iam::732659419746:user/testuser",
"kubernetesGroups": [
"pod-viewer"
],
"accessEntryArn": "arn:aws:eks:ap-northeast-2:732659419746:access-entry/myeks/user/732659419746/testuser/42c76c2f-5b97-f5ba-cd2d-9771cbe18cf7",
"createdAt": "2024-04-14T01:51:10.243000+09:00",
"modifiedAt": "2024-04-14T01:51:10.243000+09:00",
"tags": {},
"username": "arn:aws:iam::732659419746:user/testuser",
"type": "STANDARD"
}
}
# 확인
$ aws eks list-associated-access-policies --cluster-name $CLUSTER_NAME --principal-arn arn:aws:iam::$ACCOUNT_ID:user/testuser
{
"associatedAccessPolicies": [],
"clusterName": "myeks",
"principalArn": "arn:aws:iam::732659419746:user/testuser"
}
$ aws eks describe-access-entry --cluster-name $CLUSTER_NAME --principal-arn arn:aws:iam::$ACCOUNT_ID:user/testuser | jq
{
"accessEntry": {
"clusterName": "myeks",
"principalArn": "arn:aws:iam::732659419746:user/testuser",
"kubernetesGroups": [
"pod-viewer"
],
"accessEntryArn": "arn:aws:eks:ap-northeast-2:732659419746:access-entry/myeks/user/732659419746/testuser/42c76c2f-5b97-f5ba-cd2d-9771cbe18cf7",
"createdAt": "2024-04-14T01:51:10.243000+09:00",
"modifiedAt": "2024-04-14T01:51:10.243000+09:00",
"tags": {},
"username": "arn:aws:iam::732659419746:user/testuser",
"type": "STANDARD"
}
}
kubectl 명령어를 실행해보면 조회만 가능하고 삭제는 되지 않습니다.
# testuser 정보 확인
$ aws sts get-caller-identity --query Arn
"arn:aws:iam::732659419746:user/testuser"
$ kubectl whoami
arn:aws:iam::732659419746:user/testuser
# kubectl 명령어 실행
$ kubectl get pod -v6
I0414 01:59:05.967726 4843 loader.go:395] Config loaded from file: /root/.kube/config
I0414 01:59:06.736585 4843 round_trippers.go:553] GET https://8D687B71EBE1B368120C1A1450DD3178.gr7.ap-northeast-2.eks.amazonaws.com/api/v1/namespaces/default/pods?limit=500 200 OK in 754 milliseconds
No resources found in default namespace.
$ kubectl api-resources -v5
```
NAME SHORTNAMES APIVERSION NAMESPACED KIND
bindings v1 true Binding
componentstatuses cs v1 false ComponentStatus
configmaps cm v1 true ConfigMap
endpoints ep v1 true Endpoints
events ev v1 true Event
limitranges limits v1 true LimitRange
namespaces ns v1 false Namespace
nodes no v1 false Node
```
$ kubectl auth can-i get pods --all-namespaces
yes
$ kubectl auth can-i delete pods --all-namespaces
no
kubernetsGroups를 업데이트하여 'pod-admin'을 적용해보겠습니다.
# kubernetesGroups 업데이트 적용
$ aws eks update-access-entry --cluster-name $CLUSTER_NAME --principal-arn arn:aws:iam::$ACCOUNT_ID:user/testuser --kubernetes-group pod-admin | jq -r .accessEntry
{
"clusterName": "myeks",
"principalArn": "arn:aws:iam::732659419746:user/testuser",
"kubernetesGroups": [
"pod-admin"
],
"accessEntryArn": "arn:aws:eks:ap-northeast-2:732659419746:access-entry/myeks/user/732659419746/testuser/42c76c2f-5b97-f5ba-cd2d-9771cbe18cf7",
"createdAt": "2024-04-14T01:51:10.243000+09:00",
"modifiedAt": "2024-04-14T02:04:56.842000+09:00",
"tags": {},
"username": "arn:aws:iam::732659419746:user/testuser",
"type": "STANDARD"
}
# 적용 확인
$ aws eks describe-access-entry --cluster-name $CLUSTER_NAME --principal-arn arn:aws:iam::$ACCOUNT_ID:user/testuser | jq
{
"accessEntry": {
"clusterName": "myeks",
"principalArn": "arn:aws:iam::732659419746:user/testuser",
"kubernetesGroups": [
"pod-admin"
],
"accessEntryArn": "arn:aws:eks:ap-northeast-2:732659419746:access-entry/myeks/user/732659419746/testuser/42c76c2f-5b97-f5ba-cd2d-9771cbe18cf7",
"createdAt": "2024-04-14T01:51:10.243000+09:00",
"modifiedAt": "2024-04-14T02:04:56.842000+09:00",
"tags": {},
"username": "arn:aws:iam::732659419746:user/testuser",
"type": "STANDARD"
}
}
kubectl 명령어를 실행하면 조회, 삭제 등 모든 작업을 수행할 수 있습니다.
# testuser 정보 확인
$ aws sts get-caller-identity --query Arn
"arn:aws:iam::732659419746:user/testuser"
$ kubectl whoami
arn:aws:iam::732659419746:user/testuser
# kubectl 명령어 실행
$ kubectl get pod -v6
I0414 02:09:22.527150 5352 loader.go:395] Config loaded from file: /root/.kube/config
I0414 02:09:23.267790 5352 round_trippers.go:553] GET https://8D687B71EBE1B368120C1A1450DD3178.gr7.ap-northeast-2.eks.amazonaws.com/api/v1/namespaces/default/pods?limit=500 200 OK in 727 milliseconds
No resources found in default namespace.
$ kubectl api-resources -v5
```
NAME SHORTNAMES APIVERSION NAMESPACED KIND
bindings v1 true Binding
componentstatuses cs v1 false ComponentStatus
configmaps cm v1 true ConfigMap
endpoints ep v1 true Endpoints
events ev v1 true Event
limitranges limits v1 true LimitRange
namespaces ns v1 false Namespace
nodes no v1 false Node
```
$ kubectl auth can-i get pods --all-namespaces
yes
$ kubectl auth can-i delete pods --all-namespaces
yes
[출처]
1) CloudNet@, AEWS 실습 스터디
2) AWS Summit Korea 2022 - Amazon EKS 마이그레이션 요점정리, 강인호 Solution Architect
3) https://kubernetes.io/blog/2019/03/21/a-guide-to-kubernetes-admission-controllers/
A Guide to Kubernetes Admission Controllers
Author: Malte Isberner (StackRox) Kubernetes has greatly improved the speed and manageability of backend clusters in production today. Kubernetes has emerged as the de facto standard in container orchestrators thanks to its flexibility, scalability, and ea
kubernetes.io
4) https://docs.aws.amazon.com/ko_kr/IAM/latest/UserGuide/id_credentials_temp_request.html
임시 보안 자격 증명 요청 - AWS Identity and Access Management
AssumeRoleWithWebIdentity에 대한 호출이 서명(암호화)되지 않았습니다. 따라서 요청이 신뢰할 수 있는 중개자를 통해 전송된 경우에만 선택적 세션 정책을 포함해야 합니다. 이러한 경우 누군가가 정
docs.aws.amazon.com
A deep dive into simplified Amazon EKS access management controls | Amazon Web Services
Introduction Since the initial Amazon Elastic Kubernetes Service (Amazon EKS) launch, it has supported AWS Identity and Access Management (AWS IAM) principals as entities that can authenticate against a cluster. This was done to remove the burden—from ad
aws.amazon.com
6) https://catalog.workshops.aws/eks-immersionday/en-US/access-management
EKS Immersion Day
EKS Immersion Day
catalog.workshops.aws
7) https://docs.aws.amazon.com/eks/latest/userguide/access-policies.html#access-policy-permissions
Associating and disassociating access policies to and from access entries - Amazon EKS
If you later want to change the scope of an associated access policy, run the previous command again with the new scope. For example, if you wanted to remove my-namespace2, you'd run the command again using type=namespace,namespaces=my-namespace1 only. If
docs.aws.amazon.com
8) https://catalog.workshops.aws/eks-immersionday/en-US/access-management/4-migrate
EKS Immersion Day
EKS Immersion Day
catalog.workshops.aws
728x90
'AWS > EKS' 카테고리의 다른 글
| [AEWS2] 6-4. OWASP Kubernetes Top 10 (0) | 2024.04.13 |
|---|---|
| [AEWS2] 6-3. EKS IRSA & Pod Identity (0) | 2024.04.13 |
| [AEWS2] 6-1. JWT 란? (0) | 2024.04.13 |
| [AEWS2] 5-2. Karpenter (1) | 2024.04.07 |
| [AEWS2] 5-1. EKS AutoScaling (1) | 2024.04.07 |
