Restarting Gitlab via Ansible and Kestra

One of the great things about running Kestra is that, that if I do a job more than a few times, I can script it and make it re-occur very quickly and easily. If you have read my other posts, you may have noticed that I update my servers via an ansible script. Gitlab, when it changes major versions, doesn't appear to like that. Also, the service doesn't appear to want to be enabled, no matter how often I set it to be.

Tackling the 2 issues in reverse order allowed me to try out a new code block in ansible. The plan is to try to set the service to start and see if it fails. The block and rescue setup in ansible is perfect for this. The main reason it may fail is that gitlab need to be reconfigured after a package upgrade, so that's going to be the first thing I will do after a fail, then try the restart.

The first job to restart gitlab looks like this:

- name: Enable service gitlab, and not touch the state
  ansible.builtin.service:
    name: gitlab-runsvdir
    enabled: yes        
    state: started

This block starts the service "gitlab-runsvdir" as well as makes sure that the service is enabled. So, cool, first objective complete. Yay.

The next code block runs a command

- name: Execute the command gitlab-ctl reconfigure on the remote
  ansible.builtin.shell: gitlab-ctl reconfigure

So, now we have all the actual jobs that we need to run. The only thing that is missing, is the part that executes the task, and then runs something else on failure.

The code block in ansible looks like this:

     - name: Run task then run the rescue set on failure
        block:
          - name: Initial task
            ansible.builtin.shell: false

        rescue:
          - name: Task/s to run on failure
            ansible.builtin.shell: echo "there was an error"

The final set up looks like this:

# Are the services running?
      - name: Checking the service is running and reconfiging if not
        block:
          - name: Enable service gitlab, and not touch the state
            ansible.builtin.service:
              name: gitlab-runsvdir
              enabled: yes        
              state: started

        rescue:
          - name: Execute the command gitlab-ctl reconfigure on the remote
            ansible.builtin.shell: gitlab-ctl reconfigure
          - name: Enable service gitlab, and not touch the state
            ansible.builtin.service:
              name: gitlab-runsvdir
              enabled: yes        
              state: started

The only think we are missing at this point is to now get Kestra to run this job. Fortunately, we have done this lots of time with Kestra before. I won't go in to the ins and outs of this, as I have covered running ansible playbooks from kestra before.

The full playbooks are included below for completeness:

---
  - name: "Gitlab Server Playbook"
    hosts: gitlab.host
    become: yes
    become_method: sudo

    tasks:

# Are the services running?
      - name: Checking the service is running and reconfiging if not
        block:
          - name: Enable service gitlab, and not touch the state
            ansible.builtin.service:
              name: gitlab-runsvdir
              enabled: yes        
              state: started

        rescue:
          - name: Execute the command gitlab-ctl reconfigure on the remote
            ansible.builtin.shell: gitlab-ctl reconfigure
          - name: Enable service gitlab, and not touch the state
            ansible.builtin.service:
              name: gitlab-runsvdir
              enabled: yes        
              state: started

id: gitlab_checker
namespace: ansible
description: Restart and redeploy homepage when required.

labels:
  env: prod
  project: ansible    

tasks:
  - id: gitlab
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
      - id: ansible_task
        namespaceFiles:
          enabled: true
          include:
          - server-gitlab.yaml
          - hosts
          - host.key
        type: io.kestra.plugin.ansible.cli.AnsibleCLI
        taskRunner:
          type: io.kestra.plugin.scripts.runner.docker.Docker
          pullPolicy: IF_NOT_PRESENT
        containerImage: cytopia/ansible:latest-tools
        env:
          "ANSIBLE_HOST_KEY_CHECKING": "false"
        commands:
          - chmod 0400 host.key
          - ansible-playbook -i hosts server-gitlab.yaml 

concurrency:
  behavior: CANCEL
  limit: 1

Hopefully, this will come in handy for you, just as much as it does for me.