Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support bootc #830

Closed
wants to merge 15 commits into from
3 changes: 3 additions & 0 deletions playbooks/bootstrap.yml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we will also need to guard the task Download packages from roles/edpm_download_cache/tasks/main.yml with the fact

to workaround this I dropped download-cache from my NodeSet services

Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
name: osp.edpm.edpm_kernel
tags:
- edpm_kernel
when: not ansible_local.bootc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is the fact ansible_local.bootc initially gathered given that gather_facts defaults to false in the playbook?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to figure out what we want this to look like still. I have just set gather_facts: true in my deployment Ansible vars.

Maybe we would need to change the default for it to gather_subset: local at a minimum. It's not ideal that we would gather all facts for every service, but at the moment, that's what my gather_facts: true is doing until I come up with something better.

- name: Import edpm_tuned
ansible.builtin.import_role:
name: osp.edpm.edpm_tuned
Expand All @@ -53,9 +54,11 @@
tasks_from: kernelargs.yml
tags:
- edpm_kernel
when: not ansible_local.bootc
- name: Configure KSM for kernel
ansible.builtin.import_role:
name: osp.edpm.edpm_kernel
tasks_from: ksm.yml
tags:
- edpm_kernel
when: not ansible_local.bootc
54 changes: 54 additions & 0 deletions roles/edpm_bootstrap/tasks/bootstrap.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,57 @@
- name: Import FIPS tasks
ansible.builtin.import_tasks: fips.yml
when: edpm_bootstrap_fips_mode != 'check'

- name: Execute bootstrap command
ansible.builtin.import_tasks: bootstrap_command.yml

- name: Import packages tasks
ansible.builtin.import_tasks: packages.yml
when: not ansible_local.bootc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to update this change based on the commit I made earlier to handle bootc. packages.yml is already included earlier at line 24 using the other variable I had used "bootc". So that needs to be undone so we can go forward with what you're proposing here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also should we switch this to include_tasks so that the when will skip all the tasks at once instead of individually?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also see that the only reason ansible_local.bootc is set here is because of the task in bootstrap_command.yml to read local facts. I think we need that to be more explicit. Probably add something directly in bootstrap.yml.

I believe this answers my earlier comment on playbooks/bootstrap.yml on how the fact is initially set. We should make it more explicit.


- name: Set selinux state
ansible.posix.selinux:
policy: targeted
state: "{{ edpm_bootstrap_selinux_mode }}"
become: true

- name: Stop NetworkManager from updating resolv.conf
when: ( edpm_bootstrap_network_service == 'NetworkManager' ) and ( not edpm_bootstrap_network_resolvconf_update )
become: true
block:
- name: Set 'dns=none' in /etc/NetworkManager/NetworkManager.conf
community.general.ini_file:
path: /etc/NetworkManager/NetworkManager.conf
state: present
no_extra_spaces: true
section: main
option: dns
value: none
backup: true
mode: '0644'
- name: Set 'rc-manager=unmanaged' in /etc/NetworkManager/NetworkManager.conf
community.general.ini_file:
path: /etc/NetworkManager/NetworkManager.conf
state: present
no_extra_spaces: true
section: main
option: rc-manager
value: unmanaged
backup: true
mode: '0644'
- name: Reload NetworkManager
ansible.builtin.systemd:
name: NetworkManager
state: reloaded

- name: Stop dhclient from updating resolv.conf
become: true
ansible.builtin.copy:
dest: /etc/dhcp/dhclient-enter-hooks
mode: "0755"
content: |
#!/bin/sh
make_resolv_conf() { : ; }

- name: Configure swap
ansible.builtin.import_tasks: swap.yml
1 change: 1 addition & 0 deletions roles/edpm_bootstrap/tasks/packages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@
- name: Install and enable network service
when:
- (edpm_bootstrap_legacy_network_packages | length) > 0
- not ansible_local.bootc
become: true
block:
- name: Deploy network-scripts required for deprecated network service
Expand Down
1 change: 1 addition & 0 deletions roles/edpm_bootstrap/tasks/swap.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

- name: Configure swap file
when:
- not ansible_local.bootc
- not edpm_bootstrap_swap_partition_enabled|bool
- edpm_bootstrap_swap_size_megabytes|int > 0
become: true
Expand Down
24 changes: 16 additions & 8 deletions roles/edpm_container_manage/tasks/shutdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,17 +27,25 @@
loop:
- 'edpm-container-shutdown'
- 'edpm-start-podman-container'
- name: "Create /usr/lib/systemd/system/edpm-container-shutdown.service"
when: not ansible_local.bootc

- name: "Create systemd preset dir"
ansible.builtin.file:
state: "directory"
path: "/etc/systemd/system-preset"
mode: 0644

- name: "Create /etc/systemd/system/edpm-container-shutdown.service"
ansible.builtin.copy:
src: "{{ role_path }}/files/edpm-container-shutdown-service"
dest: "/usr/lib/systemd/system/edpm-container-shutdown.service"
dest: "/etc/systemd/system/edpm-container-shutdown.service"
mode: '0644'
owner: root
group: root
- name: "Create /usr/lib/systemd/system-preset/91-edpm-container-shutdown.preset"
- name: "Create /etc/systemd/system-preset/91-edpm-container-shutdown.preset"
ansible.builtin.copy:
src: "{{ role_path }}/files/91-edpm-container-shutdown-preset"
dest: "/usr/lib/systemd/system-preset/91-edpm-container-shutdown.preset"
dest: "/etc/systemd/system-preset/91-edpm-container-shutdown.preset"
mode: '0644'
owner: root
group: root
Expand All @@ -47,17 +55,17 @@
state: started
enabled: true
daemon_reload: true
- name: "Create /usr/lib/systemd/system/netns-placeholder.service"
- name: "Create /etc/systemd/system/netns-placeholder.service"
ansible.builtin.copy:
src: "{{ role_path }}/files/netns-placeholder-service"
dest: "/usr/lib/systemd/system/netns-placeholder.service"
dest: "/etc/systemd/system/netns-placeholder.service"
mode: '0644'
owner: root
group: root
- name: "Create /usr/lib/systemd/system-preset/91-netns-placeholder.preset"
- name: "Create /etc/systemd/system-preset/91-netns-placeholder.preset"
ansible.builtin.copy:
src: "{{ role_path }}/files/91-netns-placeholder-preset"
dest: "/usr/lib/systemd/system-preset/91-netns-placeholder.preset"
dest: "/etc/systemd/system-preset/91-netns-placeholder.preset"
mode: '0644'
owner: root
group: root
Expand Down
31 changes: 30 additions & 1 deletion roles/edpm_iscsid/tasks/run.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,32 @@

- name: Manage iscsid containers
ansible.builtin.include_role:
name: edpm_container_standalone
name: edpm_container_standalone
vars:
edpm_container_standalone_service: iscsid
edpm_container_standalone_container_defs:
iscsid: "{{ lookup('template', 'iscsid.yaml.j2') | from_yaml }}"
edpm_container_standalone_kolla_config_files:
iscsid: "{{ lookup('file', 'files/iscsid.yaml') | from_yaml }}"
register: manage_iscsid_stat
when: not ansible_local.bootc

- name: Configure iscsid for bootc
when: ansible_local.bootc
become: true
block:
- name: Render kolla config file
ansible.builtin.template:
src: templates/iscsid.yaml.j2
dest: /var/lib/kolla/config_files/iscsid.json
mode: 0644

- name: Manage iscsid crond containers
ansible.builtin.systemd_service:
name: edpm-compute@iscsid
enabled: true
state: started
register: manage_iscsid_stat

- name: Check if the iscsid container restart is required
ansible.builtin.stat:
Expand All @@ -49,12 +67,23 @@
- name: Restart iscsid container to refresh /etcd/iscsid.conf
become: true
when:
- not ansible_local.bootc
- not manage_iscsid_stat.changed|bool
- iscsi_restart_stat.stat.exists|bool
ansible.builtin.systemd:
name: edpm_iscsid
state: restarted

- name: Restart iscsid container to refresh /etcd/iscsid.conf
become: true
when:
- ansible_local.bootc
- not manage_iscsid_stat.changed|bool
- iscsi_restart_stat.stat.exists|bool
ansible.builtin.systemd:
name: edpm-compute@iscsid
state: restarted

- name: Remove iscsid container restart sentinel file
become: true
ansible.builtin.file:
Expand Down
4 changes: 3 additions & 1 deletion roles/edpm_kernel/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,9 @@
name: "{{ edpm_kernel_extra_packages }}"
state: present
register: _install_packages_result
when: edpm_kernel_extra_packages | length > 0
when:
- edpm_kernel_extra_packages | length > 0
- not ansible_local.bootc
until: _install_packages_result is succeeded
retries: "{{ edpm_kernel_download_retries }}"
delay: "{{ edpm_kernel_download_delay }}"
Expand Down
1 change: 1 addition & 0 deletions roles/edpm_libvirt/tasks/install.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
delay: "{{ edpm_libvirt_download_delay }}"
notify:
- Restart libvirt
when: not ansible_local.bootc

- name: Ensure monolithic libvirt and tcp socket activation is not enabled or running
tags:
Expand Down
18 changes: 17 additions & 1 deletion roles/edpm_logrotate_crond/tasks/configure.yml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like the Insert cronjob in root crontab task requires the cronie rpm. We might need to add that to the image build.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,25 @@
- name: Configure tmpwatch on the host
become: true
block:
- name: Create /var/lib/openstack
ansible.builtin.file:
path: "/var/lib/openstack"
mode: 0755
state: directory
owner: "{{ ansible_user }}"
group: "{{ ansible_user }}"
setype: "container_file_t"

- name: Create /var/lib/openstack/cron
ansible.builtin.file:
path: "/var/lib/openstack/cron"
mode: 0644
state: directory
setype: "container_file_t"

- name: Push script
ansible.builtin.copy:
dest: /usr/local/sbin/containers-tmpwatch
dest: /var/lib/openstack/cron/containers-tmpwatch
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/usr is immutable with bootc deployments. So I've proposed doing this in two different ways. 1, we bake the scripts into the container file:
https://github.com/openstack-k8s-operators/install_yamls/pull/950/files#diff-f8fb9af5355b45b9ca8936bf0d721c6f0e37e13b637f5598e2be19995dea23e7R45-R46

And 2. Which is this method of writing to /var/lib/openstack. I personally prefer doing it this way if we can agree on a common place for any scripts that we want to use. That saves us baking things into images and then trying to keep them in sync. Better imo to have them in edpm-ansible for now.

owner: root
group: root
mode: "0755"
Expand Down
2 changes: 2 additions & 0 deletions roles/edpm_logrotate_crond/tasks/install.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
until: edpm_logrotate_crond_cronie_package_download is succeeded
retries: "{{ edpm_logrotate_crond_download_retries }}"
delay: "{{ edpm_logrotate_crond_download_delay }}"
when:
- not ansible_local.bootc

- name: Gather SELinux fact if needed
when:
Expand Down
18 changes: 18 additions & 0 deletions roles/edpm_logrotate_crond/tasks/logrotate_crond.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,21 @@
logrotate_crond: "{{ lookup('template', 'logrotate_crond.yaml.j2') | from_yaml }}"
edpm_container_standalone_kolla_config_files:
logrotate_crond: "{{ lookup('file', 'files/logrotate_crond.yaml') | from_yaml }}"
when: not ansible_local.bootc

- name: Configure logrotate for bootc
when: ansible_local.bootc
block:
- name: Render kolla config file
ansible.builtin.copy:
src: files/logrotate_crond.yaml
dest: /var/lib/kolla/config_files/logrotate_crond.json
mode: 0644

- name: Manage logrotate crond containers
ansible.builtin.systemd_service:
name: edpm-compute@logrotate_crond
enabled: true
state: started
become: true

17 changes: 17 additions & 0 deletions roles/edpm_logrotate_crond/tasks/run.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,20 @@
logrotate_crond: "{{ lookup('template', 'logrotate_crond.yaml.j2') | from_yaml }}"
edpm_container_standalone_kolla_config_files:
logrotate_crond: "{{ lookup('file', 'files/logrotate_crond.yaml') | from_yaml }}"
when: not ansible_local.bootc

- name: Configure logrotate for bootc
when: ansible_local.bootc
become: true
block:
- name: Render kolla config file
ansible.builtin.copy:
src: files/logrotate_crond.yaml
dest: /var/lib/kolla/config_files/logrotate_crond.json
mode: 0644

- name: Manage logrotate crond containers
ansible.builtin.systemd_service:
name: edpm-compute@logrotate_crond
enabled: true
state: started
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to separate out this initial bootc support from the logically bound containers PR, openstack-k8s-operators/edpm-image-builder#39

This PR has a strong dep on the logically bound PR, and that complicates things. Let's just get a base bootc working with how we manage containers presently. We can move to logically bound and all the quadlet/systemd stuff as a next step.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, ok. Let's decouple them. I'll just submit a new PR to edpm-image-builder to change the Quadlet files over to using .container instead of .kube. Then a new one here to work with those instead of the .kube files.

19 changes: 18 additions & 1 deletion roles/edpm_multipathd/tasks/run.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,24 @@
multipathd: "{{ lookup('template', 'multipathd.yaml.j2') | from_yaml }}"
edpm_container_standalone_kolla_config_files:
multipathd: "{{ lookup('template', 'kolla_multipathd.yaml.j2') | from_yaml }}"
register: manage_multipathd_stat
when: not ansible_local.bootc

- name: Configure multipathd for bootc
when: ansible_local.bootc
become: true
block:
- name: Render kolla config file
ansible.builtin.template:
src: templates/kolla_multipathd.yaml.j2
dest: /var/lib/kolla/config_files/multipathd.json
mode: 0644

- name: Manage logrotate crond containers
ansible.builtin.systemd_service:
name: edpm-compute@multipathd
enabled: true
state: started
register: manage_multipathd_stat

- name: Check if the multipathd container restart is required
ansible.builtin.stat:
Expand Down
8 changes: 6 additions & 2 deletions roles/edpm_network_config/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@
# under the License.

- name: Configure network with network role from system roles [nmstate]
when: edpm_network_config_tool == 'nmstate'
when:
- edpm_network_config_tool == 'nmstate'
become: true
block:
- name: Install OVS NetworkManager plugin [nmstate]
Expand All @@ -26,11 +27,14 @@
until: nm_ovs_status is succeeded
retries: "{{ edpm_network_config_download_retries }}"
delay: "{{ edpm_network_config_download_delay }}"
when: not ansible_local.bootc
- name: Restart NetworkManager after plugin installation [nmstate]
ansible.builtin.systemd:
name: NetworkManager
state: restarted
when: nm_ovs_status.changed # noqa: no-handler
when:
- nm_ovs_status.changed # noqa: no-handler
- not ansible_local.bootc
- name: Render network_state variable
ansible.builtin.set_fact:
network_state: "{{ edpm_network_config_template | from_yaml }}"
Expand Down
1 change: 1 addition & 0 deletions roles/edpm_network_config/tasks/network_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
until: edpm_network_config_onc_download is succeeded
retries: "{{ edpm_network_config_download_retries }}"
delay: "{{ edpm_network_config_download_delay }}"
when: not ansible_local.bootc

- name: Ensure /var/lib/edpm-config directory exists
become: true
Expand Down
17 changes: 17 additions & 0 deletions roles/edpm_neutron_metadata/tasks/run.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,20 @@
ovn_metadata_agent: "{{ lookup('template', 'ovn_metadata_agent.yaml.j2') | from_yaml }}"
edpm_container_standalone_kolla_config_files:
ovn_metadata_agent: "{{ lookup('template', 'kolla_ovn_metadata_agent.yaml.j2') | from_yaml }}"
when: not ansible_local.bootc

- name: Configure ovn_metadata_agent for bootc
when: ansible_local.bootc
become: true
block:
- name: Render kolla config file
ansible.builtin.template:
src: templates/kolla_ovn_metadata_agent.yaml.j2
dest: /var/lib/kolla/config_files/ovn_metadata_agent.json
mode: 0644

- name: Manage logrotate crond containers
ansible.builtin.systemd_service:
name: edpm-compute@ovn_metadata_agent
enabled: true
state: started
Loading
Loading