K8s中使用jenkins-部署jenkins

前言

现在容器越来越普及，很多公司都在往这个方向转变，我们也不列外，项目容器化已经走了好久，但是我们的项目也仅仅是容器化，还未上到k8s，这条路得慢慢走细细摸。。。我们存在一个自建k8s环境，资源有限，该环境目前主要用来做测试环境，近期又争取到了一些资源费用，总算可以搞点平台工具上去了，这个系列主要讲下我们在k8s中是如何使用jenkins

K8s中部署Jenkins

本篇内容不多，主要描述下如何部署我们的jenkins，需求如下

需要持久化存储
需要ingress访问入口
安装常用插件

jenkins部署文件

[root@node2 jenkins-deploy]# cat jenkins.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins
  labels:
    name: jenkins
spec:
  replicas: 1
  selector:
    matchLabels:
      name: jenkins
  template:
    metadata:
      name: jenkins
      labels:
        name: jenkins
    spec:
      nodeSelector:
        paas: cicd
      terminationGracePeriodSeconds: 10
      serviceAccountName: jenkins
      containers:
        - name: jenkins
          image: jenkins/jenkins
          imagePullPolicy: Always
          ports:
            - containerPort: 8080
            - containerPort: 50000
          resources:
            limits:
              cpu: 2
              memory: 2Gi
            requests:
              cpu: 2
              memory: 2Gi
          env:
            - name: JAVA_OPTS
              value: -Xmx1g
          volumeMounts:
            - name: jenkins-home
              mountPath: /var/jenkins_home
      securityContext:
        fsGroup: 1000
        runAsUser: 0
      volumes:
        - name: jenkins-home
          persistentVolumeClaim:
            claimName: jenkins-home
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-home
spec:
  storageClassName: nfs-storage
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 10Gi
---
apiVersion: v1
kind: Service
metadata:
  name: jenkins
spec:
  selector:
    name: jenkins
  ports:
    - name: http
      port: 8080
      targetPort: 8080
      protocol: TCP
    - name: agent
      port: 50000
      targetPort: 50000
      protocol: TCP
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: jenkins
rules:
- apiGroups: [""]
  resources: ["pods","events"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets","events"]
  verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: jenkins
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins
subjects:
- kind: ServiceAccount
  name: jenkins

[root@node2 jenkins-deploy]# kubectl get -f jenkins.yaml
NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/jenkins   1/1     1            1           13m

NAME                                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/jenkins-home   Bound    pvc-e0f39e1f-5d09-4a68-b5d3-e1dc938ef07e   10Gi       RWO            nfs-storage    13m

NAME              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)              AGE
service/jenkins   ClusterIP   10.96.169.148   <none>        8080/TCP,50000/TCP   13m

NAME                     SECRETS   AGE
serviceaccount/jenkins   1         13m

NAME                                     CREATED AT
role.rbac.authorization.k8s.io/jenkins   2020-08-20T18:17:02Z

NAME                                            ROLE           AGE
rolebinding.rbac.authorization.k8s.io/jenkins   Role/jenkins   13m

ingress访问入口

[root@node2 jenkins-deploy]# cat jenkins-ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: jenkins-ingress
  namespace: default
spec:
  rules:
  - host: 'jenkins.xadocker.cn'
    http:
      paths:
      - path: /
        backend:
          serviceName: jenkins
          servicePort: 8080

[root@node2 jenkins-deploy]# kubectl get -f jenkins-ingress.yaml
NAME              CLASS    HOSTS                 ADDRESS   PORTS   AGE
jenkins-ingress   <none>   jenkins.xadocker.cn             80      10m

登录测试jenkins

获取jenkins初始化密码

[root@node2 jenkins-deploy]# kubectl get pod -l name=jenkins
NAME                       READY   STATUS    RESTARTS   AGE
jenkins-577bcfb457-h54x6   1/1     Running   0          16m

[root@node2 jenkins-deploy]# kubectl exec -it jenkins-577bcfb457-h54x6 -- cat /var/jenkins_home/secrets/initialAdminPassword
fabeff06efdb443baffb22f6fd9d9874

输入密码选择安装默认插件，最终效果

配置Jenkins

因为官方的jenkins镜像默认是debian系统的，而我们内部业务系统平台都是centos，为了保证业务基础系统一致我们决定改一下，并且也添加一些我们业务中常用的构建工具maven/node/git/composser。。。

更换自制的jenkins镜像

[root@node5 jenkins-image]# cat Dockerfile
FROM jenkins/jenkins:centos7
LABEL maintainer xadocker
ENV MAVEN_VERSION 3.2.5
USER root
RUN  cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \
    yum install -y  wget && \
    rm -rf /etc/yum.repo/*.repo && \
    wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo && \
    wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo && \
    yum install -y git python36 nodejs npm composer ansible && \
    curl -sSL http://archive.apache.org/dist/maven/maven-3/$MAVEN_VERSION/binaries/apache-maven-$MAVEN_VERSION-bin.tar.gz  tar xzf - -C /usr/share \
  && mv /usr/share/apache-maven-$MAVEN_VERSION /usr/share/maven \
  && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
ENV MAVEN_HOME /usr/share/maven
USER 1000

# 自行登录并验证
[root@node5 jenkins-image]# docker build . -t myjenkins:v3
[root@node5 jenkins-image]# docker run -tid --name myjenkins -h jenkins -p 8680:8080 myjenkins:v3