Jenkins进阶之分布式服务器环境配置

[TOC]

0x01 Jenkins 常用插件

Jenkins 持续集成常用插件一览表:

# 权限管理
Role-based Authorization Strategy  - 3.1 ： http://wiki.jenkins-ci.org/display/JENKINS/Role+Strategy+Plugin
Authorize Project Plugin - 项目权限管控

# ssh
SSH Credentials
SSH Slaves

# Gitlab
Git (4.5.2) - This plugin integrates Git with Jenkins. 
Git client (3.6.0) - Utility plugin for Git support in Jenkins
Gitlab Authentication  (1.10) - This is the an authentication plugin using gitlab OAuth.
GitLab  (1.5.13) - This plugin allows GitLab to trigger Jenkins builds and display their results in the GitLab UI.
Credentials (2.3.14) - This plugin allows you to store credentials in Jenkins.
GitLab Branch Source - 提供分支源代码和文件夹组织功能的GitLab仓库在詹金斯

# Svn
Subversion Plug-in  - svn 仓库专用

# pipeline 编写
Pipeline 
Pipeline: Groovy - 2.87
Groovy  - 2.3 
Groovy Postbuild  - 2.5

# 质量检测
SonarQube Scanner - 2.13 

# 上下游构建
Parameterized Trigger 
build-pipeline-plugin  - 可视化 build pipeline 插件

# 构建工具
Maven Integration - 3.8 
Node and Label parameter 

# 消息通知
Qy Wechat Notification Plugin - 1.0.2
Email Extension Plugin  # 自带通知功能相对单一

# 容器管理
Kubernetes plugin - Jenkins插件可在Kubernetes集群中运行动态代理。


# 节点添加
SSH Agent - This plugin allows you to provide SSH credentials to builds via a ssh-agent in Jenkins

0x02 使用范例

(1) Ssh-steps-Plugin

项目描述: Jenkins流水线步骤，提供SSH工具，如命令执行或文件传输，以实现持续交付。
项目地址: https://github.com/jenkinsci/ssh-steps-plugin

1
2
3

Version: 2.0.0
Released: 2 years ago
Requires Jenkins 2.121.1

1) 在Jenkins中设置一个安全文本(Secret text)票据, 不建议使用明文票据;

1	# Secret text a2bc53c0-0b68-4fce-9f1f-d04815ae52c1 k8s-192.168.12.215 Secret text k8s-192.168.12.215

Delactive Pipeline Script:

pipeline {
  stages {
    stage ('ssh-steps-plugin') {
      steps {
        script {
            try {
              def remote = [:]
              remote.name = 'K8s-Master'
              remote.user = 'root'
              remote.host = '192.168.12.215'
              //remote.password = 'password'  // 明文不安全
              remote.allowAnyHosts = true     // 必须要运行所有主机
              withCredentials([string(credentialsId: 'a2bc53c0-0b68-4fce-9f1f-d04815ae52c1', variable: 'sshpassword')]) {
                remote.password = sshpassword
                writeFile file: 'abc.sh', text: 'ls'
                sshPut remote: remote, from: 'abc.sh', into: '.'  // 非常注意 不能为其它目录只能在当前路径之中
                sshGet remote: remote, from: 'abc.sh', into: 'bac.sh', override: true
                sshScript remote: remote, script: 'abc.sh'
                sshRemove remote: remote, path: 'abc.sh'
                sshCommand remote: remote, command: ' whoami && for i in {1..5}; do echo -n \"Loop \$i \"; date ; sleep 1; done'
              }
            }catch(Exception err) {
                echo err.toString() 
                error "[-Error] : ssh-steps-plugin测试失败\n [-Msg] : ${err.getMessage()} "
            }
        }
      }
    }
  }
}

输出结果

# SSH Steps: sshCommand - Execute command on remote node.7s
# Host key checking is off. It may be vulnerable to man-in-the-middle attacks.
[jsch] Permanently added '192.168.12.215' (RSA) to the list of known hosts.
Connected to 192.168.12.215[192.168.12.215:22] (SSH-2.0-OpenSSH_7.4)
Started command 192.168.12.215#0: whoami
root
Success command 192.168.12.215#0: whoami
Disconnected from 192.168.12.215[192.168.12.215:22]

2) 官网示例中采用密钥进行验证操作
描述: 利用Jenkins凭据存储区读取私钥之后再进行ssh主机验证操作，但是需要注意高版本的Openssh的影响。

def remote = [:]
remote.name = "node-1"
remote.host = "10.000.000.153"
remote.allowAnyHosts = true

withCredentials([sshUserPrivateKey(credentialsId: 'sshUser', keyFileVariable: 'identity', passphraseVariable: '', usernameVariable: 'userName')]) {
  remote.user = userName
  remote.identityFile = identity
  stage("SSH Steps Rocks!") {
    writeFile file: 'abc.sh', text: 'ls'
    sshCommand remote: remote, command: 'for i in {1..5}; do echo -n \"Loop \$i \"; date ; sleep 1; done'
    sshPut remote: remote, from: 'abc.sh', into: '.'
    sshGet remote: remote, from: 'abc.sh', into: 'bac.sh', override: true
    sshScript remote: remote, script: 'abc.sh'
    sshRemove remote: remote, path: 'abc.sh'
  }
}

3) 远程主机文件上传与删除
执行脚本:

try {
  def projectProduct = sh returnStdout: true, script: "find ${APP_FILE}"  // returnStatus: true
  def productName = sh returnStdout: true, script: "echo ${JOB_NAME}-${RELEASE_VERSION}${APP_SUFFIX}"
  def remote = [:]
  remote.name = "K8s-Master#${DEPLOY_HOST}"
  remote.user = "weiyigeek" // "${DEPLOY_USER}"
  remote.host = "192.168.12.107" //"${DEPLOY_HOST}"
  remote.port = 20211
  remote.allowAnyHosts = true     // 必须要运行所有主机
  withCredentials([string(credentialsId: "${DEPLOY_CREDENTIALSID}", variable: 'sshpassword')]) {
    remote.password = sshpassword
  }

  if (productName != '' && projectProduct != ''){
    println(projectProduct + "-" +  productName)
    writeFile file: 'projectProduct.sh', text: "${productName}"
  } else {
    error "[-Error] : projectProduct 不能为空!"
    
  }
      
  sshPut remote: remote, from: "/home/jenkins/agent/workspace/HelloWorld/target/hello-world.war" , into: "${APP_DIR}"         
  sshCommand remote: remote, command: "ls -alh ${APP_DIR}"
}

输出结果:

target/hello-world.war - hello-world-v1.31.war

projectProduct.sh — Write file to workspace < 1s
+ pwd
/home/jenkins/agent/workspace/HelloWorld

SSH Steps: sshPut - Put a file/directory on remote node.6s
Sending a file/directory to `K8s-Master#192.168.12.215[192.168.12.107]`: from: `/home/jenkins/agent/workspace/HelloWorld/target/hello-world.war` into: /tmp/

SSH Steps: sshCommand - Execute command on remote node.1s

Executing command on K8s-Master#192.168.12.215[192.168.12.107]: ls -alh /tmp/ sudo: false
total 5.9M
-rw-rw-r--  1 weiyigeek weiyigeek 2.0M Feb 23 09:02 hello-world-v1.31.war

4) 采用函数调用的方式

!groovy
def getHost(){
    def remote = [:]
    remote.name = 'mysql'
    remote.host = '192.168.8.108'
    remote.user = 'root'
    remote.port = 22
    remote.password = 'qweasd'
    remote.allowAnyHosts = true
    return remote
}
pipeline {
    agent {label 'master'}
    environment{
        def server = ''
    }   
    stages {
        stage('init-server'){
            steps {
                script {                 
                   server = getHost()                                   
                }
            }
        }
        stage('use'){
            steps {
                script {
                  sshCommand remote: server, command: """                 
                  if test ! -d aaa/ccc ;then mkdir -p aaa/ccc;fi;cd aaa/ccc;rm -rf ./*;echo 'aa' > aa.log
                  """
                }
            }
        }
    }
}

入坑出坑
问题1.使用ssh-steps-plugin插件并且使用Jenkins shh Private 凭据时jsch密钥连接远程Linux报错 com.jcraft.jsch.JSchException: invalid privatekey: [[email protected]
报错信息:

exception in thread "main" com.jcraft.jsch.JSchException: invalid privatekey: [[email protected]
  at com.jcraft.jsch.KeyPair.load(KeyPair.java:664)
  at com.jcraft.jsch.KeyPair.load(KeyPair.java:561)
  at com.jcraft.jsch.IdentityFile.newInstance(IdentityFile.java:40)
  at com.jcraft.jsch.JSch.addIdentity(JSch.java:407)
  at com.jcraft.jsch.JSch.addIdentity(JSch.java:388)
  at com.scc.nanny.ssh.SSH.<init>(SSH.java:59)
  at com.scc.nanny.ssh.SSH.main(SSH.java:124)

问题原因: 主要原因是生成密钥的时候使用的openssh版本过高导致

# 高版本生成的密钥
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABDRomEs9d
........
Av8l4J6WvCkc3NAAAAEAAAAAEAAAEXAAAAB3NzaC1yc2EAAAADAQABAAABAQDD1C48/OJ3
-----END OPENSSH PRIVATE KEY-----

# ssh-steps-plugin 插件支持的密钥样式
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,749E5AFD6F06C22EF6FD3AECCB6E540B
  
FwOlFOtCM+JH3EG7gzDOffkwLysiDCucdUeDZaK08rFSWzKpMwfPD/AZKNi0dqZR
....
-----END RSA PRIVATE KEY-----

解决办法: 不能在ssh-steps-plugin插件的版本使用Private验证只能使用密码验证,参考上述1步骤;

(2) Gitlab-Plugin

官方地址:
帮助文档: https://github.com/jenkinsci/gitlab-plugin#pipeline-jobs

Gitlab Triggers Setting
如果您想在声明式构建中配置插件支持的任何可选作业触发器请使用触发器块，完整的可配置触发选项列表如下:

// 触发器
triggers {
  // gitlab 相关触发
  gitlab(
    triggerOnPush: false,
    triggerOnMergeRequest: true, triggerOpenMergeRequestOnPush: "never",
    triggerOnNoteRequest: true,
    noteRegex: "Jenkins please retry a build",
    skipWorkInProgressMergeRequest: true,
    ciSkip: false,
    setBuildDescription: true,
    addNoteOnMergeRequest: true,
    addCiMessage: true,
    addVoteOnMergeRequest: true,
    acceptMergeRequestOnSuccess: false,
    branchFilterType: "NameBasedFilter",
    includeBranchesSpec: "release/qat",
    excludeBranchesSpec: "",
    pendingBuildName: "Jenkins",
    cancelPendingBuildsOnUpdate: false,
    secretToken: "abcdefghijklmnopqrstuvwxyz0123456789ABCDEF")
}

Tips : 注意:您将需要手动运行此作业一次，以便Jenkins读取和设置触发器配置。否则 webhook 将无法触发作业。

Gitlab PipeLine Status Sync
描述: Jenkins 创建的 Pipeline 同步到 Gitlab-CI 流水线之中;

pipeline {
  agent any

  options {
    gitLabConnection('your-gitlab-connection-name')
    gitlabBuilds(builds: ['build', 'test', 'deploy'])
  }

  stage {
    stage("build") {
        gitlabCommitStatus("build") {
            // your build steps
        }
      }

      stage("test") {
        gitlabCommitStatus("test") {
            // your test steps
        }
      }
  }

  post {
    failure {
      updateGitlabCommitStatus name: 'build', state: 'failed'
    }
    success {
      updateGitlabCommitStatus name: 'build', state: 'success'
    }
  }
}

options {
gitLabConnection(‘your-gitlab-connection-name’)
gitlabBuilds(builds: [‘build’, ‘test’, ‘deploy’])
}

Help - 帮助声明:

Kubernetes 插件Github : https://github.com/jenkinsci/kubernetes-plugin
Kubernetes 插件使用示例: https://github.com/jenkinsci/kubernetes-plugin/tree/master/src/main/resources/org/csanchez/jenkins/plugins/kubernetes/pipeline/samples


pipeline {
  //确认使用主机/节点机
  agent any /*{ node { label 'master'} }*/

  // 声明参数 
  // parameters{
  //   //SVN代码路径
  //   string(name:'repoUrl', defaultValue: 'http://svn.com/svn/server/job', description: ' SVN代码路径')
  //   // 部署内容的相对路径
  //   string(name:'deployLocation', defaultValue: 'target/*.jar,target/alternateLocation/*.*,'+'target/classes/*.*,target/classes/i18n/*.*,target/classes/rawSQL/*.*,'+'target/classes/rawSQL/mapper/*.*,target/classes/rawSQL/mysql/*.*,'+'target/classes/rawSQL/sqlserver/*.*', description: '部署内容的相对路径 ')
  //   //服务器参数采用了组合方式，避免多次选择
  //   string(name:'dev_server', defaultValue: 'IP,Port,Name,Passwd', description: '开发服务器(IP,Port,Name,Passwd)')
  //   string(name:'ZHtest_server', defaultValue: 'IP,Port,Name,Passwd', description: '中文测试服务器(IP,Port,Name,Passwd)')
  //   string(name:'alT19_server', defaultValue: 'IP,Port,Name,Passwd', description: ' 生产服务器T1(IP,Port,Name,Passwd)')
  //   string(name:'alT20_server', defaultValue: 'IP,Port,Name,Passwd', description: ' 生产服务器T2(IP,Port,Name,Passwd)')
  // }

  // 声明使用的工具(便于不同环境构建)
  // tools {
  //   maven 'maven_env'
  //   jdk   'jdk_k1.8.0_211'
  // }

  stages {
    stage ("代码拉取") {
     steps {
      // (1) 代码拉取  
      git credentialsId: 'b4c8b4e9-2777-44a1-a1ed-e9dc21d37f4f', url: '[email protected]:ci-cd/java-maven.git'
      // (2) 超时时间两分钟
      timeout(time: 2, unit: 'MINUTES') {
          script {
            env.git_version=input message: '选择版本', ok: '通过',parameters: [string (description: '应用版本', name: 'git_version', trim: true)];
            env.deploy_option = input message: '选择操作', ok: 'deploy', parameters: [description: '选择部署环境',choice(name: 'deploy_option', choices: ['deploy', 'rollback', 'redeploy'])];
          }
      }
      // (3) 切换版本
      sh label: 'build', script: '[ -n "${git_version}" ] && git checkout ${git_version} || { echo -e "切换的tag的版本${git_version} 不存在或为空请检查输入的tag!" && exit 1; }'
     }
   }

    stage ("代码质检") {
     steps {
       sh label: 'sonar', returnStatus: true, script: '''/usr/local/bin/sonar-scanner -Dsonar.projectName=${JOB_NAME} -Dsonar.projectKey=Hello-World -Dsonar.sources=.'''
     }
    }

    stage ("项目构建") {
     steps {
      sh label: 'build', script: 'cd /var/lib/jenkins/workspace/maven-pipeline-helloword/ && /usr/local/maven/bin/mvn clean package -Dmaven.test.skip=true '
     }
    }

    stage ("项目部署") {
     steps {

         sh label: 'deploy', script: '/tmp/script/maven-jenkins-ci-script-pipeline-webhook.sh'
   
      }
    }
  }

  //消息通知: POST阶段当所有任务执行后触发
  post {
    always {
      echo "clear workspace......"
      deleteDir()
      echo "Wechat notification......"
      // 企业微信
      qyWechatNotification aboutSend: true, failNotify: true, failSend: true, mentionedId: 'ALL', mentionedMobile: '', startBuild: true, successSend: true, unstableSend: true, webhookUrl: 'https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=c222f3fc-f645-440a-ad24-0ce8d9626fa0'
    }
  }
}

<br>


PS : 钉钉机器人在Pipeline使用示例 https://jenkinsci.github.io/dingtalk-plugin/examples/link.html

PS : 企业微信插件开源代码(在Pipeline无法获取Job_Name) https://github.com/jenkinsci/qy-wechat-notification-plugin

0x01 Jenkins 常用插件

0x02 使用范例

(1) Ssh-steps-Plugin

(2) Gitlab-Plugin

感谢读者打赏，在未来的路我们一起前行!