Skip to content

Commit

Permalink
Merge branch 'main' into 2102-fix-zarf-on-windows
Browse files Browse the repository at this point in the history
  • Loading branch information
Racer159 authored Nov 1, 2023
2 parents b297056 + e255baa commit 80b9e35
Show file tree
Hide file tree
Showing 68 changed files with 2,101 additions and 1,225 deletions.
12 changes: 8 additions & 4 deletions .github/workflows/publish-application-packages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,15 @@ jobs:
- name: Build And Publish Application Packages
# Create the dos-games package with the cosign signature, publish to ghcr and copy the tags to allow 'uname -m' to work
run: |
zarf package create -o build -a amd64 examples/dos-games --key=awskms:///${{ secrets.COSIGN_AWS_KMS_KEY }} --confirm
zarf package create -o build -a arm64 examples/dos-games --key=awskms:///${{ secrets.COSIGN_AWS_KMS_KEY }} --confirm
zarf package create -o build -a amd64 examples/dos-games --signing-key=awskms:///${{ secrets.COSIGN_AWS_KMS_KEY }} --confirm
zarf package create -o build -a arm64 examples/dos-games --signing-key=awskms:///${{ secrets.COSIGN_AWS_KMS_KEY }} --confirm
zarf package publish ./build/zarf-package-dos-games-amd64-1.0.0.tar.zst oci://ghcr.io/defenseunicorns/packages
zarf package publish ./build/zarf-package-dos-games-arm64-1.0.0.tar.zst oci://ghcr.io/defenseunicorns/packages
# Publish a the signed dos-games package
zarf package publish ./build/zarf-package-dos-games-amd64-1.0.0.tar.zst oci://ghcr.io/defenseunicorns/packages --key=https://zarf.dev/cosign.pub
zarf package publish ./build/zarf-package-dos-games-arm64-1.0.0.tar.zst oci://ghcr.io/defenseunicorns/packages --key=https://zarf.dev/cosign.pub
# Publish a skeleton of the dos-games package
zarf package publish examples/dos-games oci://ghcr.io/defenseunicorns/packages
zarf tools registry copy ghcr.io/defenseunicorns/packages/dos-games:1.0.0-amd64 ghcr.io/defenseunicorns/packages/dos-games:1.0.0-x86_64
zarf tools registry copy ghcr.io/defenseunicorns/packages/dos-games:1.0.0-arm64 ghcr.io/defenseunicorns/packages/dos-games:1.0.0-aarch64
Expand Down
23 changes: 23 additions & 0 deletions adr/0021-composable-components.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# 21. Composable Components

Date: 2023-10-26

## Status

Accepted

## Context

Zarf has supports composing components together between packages on `zarf package create` since v0.16.0. This has allowed package creators to make more complex packages from smaller reusable bits. As this functionality grew however there were a few problems that developed:

1. Import chains did not handle scaling to larger numbers of layers with test coverage usually only covering the first import.
2. When OCI skeletons were added they were largely bolted on after the fact without rethinking how they would impact composability.
3. Component filtering via the `only` filter was not implemented in a central location leading to bugs with create-time filters.

## Decision

We decided to separate composability into its own package that represents a composability import chain as a doubly linked list. This allows us to represent the whole chain as it exists relative to the "head" Zarf package (the definition that Zarf was asked to build) to more easily handle packages that are in different locations (such as OCI skeletons in one's cache). We also run the compose functions on all components so that the additional filter logic that is needed for these components can be handled more concisely and built upon (as it might for `flavor` https://github.com/defenseunicorns/zarf/issues/2101).

## Consequences

Maintaining the full context within a linked list does use more memory and some operations on it are less efficient than they could be if we one-shotted the compose. This is a decent tradeoff however as most import chains won't be longer than 4 or 5 elements in practice and these structs and operations are relatively small.
5 changes: 3 additions & 2 deletions docs/2-the-zarf-cli/100-cli-commands/zarf_package_create.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,16 @@ zarf package create [ DIRECTORY ] [flags]
```
--confirm Confirm package creation without prompting
--differential string [beta] Build a package that only contains the differential changes from local resources and differing remote resources from the specified previously built package
-f, --flavor string The flavor of components to include in the resulting package (i.e. have a matching or empty "only.flavor" key)
-h, --help help for create
-k, --key string Path to private key file for signing packages
--key-pass string Password to the private key file used for signing packages
-m, --max-package-size int Specify the maximum size of the package in megabytes, packages larger than this will be split into multiple parts to be loaded onto smaller media (i.e. DVDs). Use 0 to disable splitting.
-o, --output string Specify the output (either a directory or an oci:// URL) for the created Zarf package
--registry-override stringToString Specify a map of domains to override on package create when pulling images (e.g. --registry-override docker.io=dockerio-reg.enterprise.intranet) (default [])
-s, --sbom View SBOM contents after creating the package
--sbom-out string Specify an output directory for the SBOMs from the created Zarf package
--set stringToString Specify package variables to set on the command line (KEY=value) (default [])
--signing-key string Path to private key file for signing packages
--signing-key-pass string Password to the private key file used for signing packages
--skip-sbom Skip generating SBOM for this package
```

Expand Down
97 changes: 17 additions & 80 deletions docs/3-create-a-zarf-package/4-zarf-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -491,45 +491,6 @@ must respect the following conditions
</blockquote>
</details>

<details open>
<summary>
<strong> <a name="build_OCIImportedComponents"></a>OCIImportedComponents</strong>
</summary>
&nbsp;
<blockquote>

## build > OCIImportedComponents

**Description:** Map of components that were imported via OCI. The keys are OCI Package URLs and values are the component names

| | |
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| **Type** | `object` |
| **Additional properties** | [![Any type: allowed](https://img.shields.io/badge/Any%20type-allowed-green)](# "Additional Properties of any type are allowed.") |

<details>
<summary>
<strong> <a name="build_OCIImportedComponents_pattern1"></a>Pattern Property .*</strong>
</summary>
&nbsp;
<blockquote>

:::note
All properties whose name matches the regular expression
```.*``` ([Test](https://regex101.com/?regex=.%2A))
must respect the following conditions
:::

| | |
| -------- | -------- |
| **Type** | `string` |

</blockquote>
</details>

</blockquote>
</details>

<details>
<summary>
<strong> <a name="build_lastNonBreakingVersion"></a>lastNonBreakingVersion</strong>
Expand Down Expand Up @@ -757,6 +718,22 @@ Must be one of:
</blockquote>
</details>

<details>
<summary>
<strong> <a name="components_items_only_flavor"></a>flavor</strong>
</summary>
&nbsp;
<blockquote>

**Description:** Only include this component when a matching '--flavor' is specified on 'zarf package create'

| | |
| -------- | -------- |
| **Type** | `string` |

</blockquote>
</details>

</blockquote>
</details>

Expand Down Expand Up @@ -1002,50 +979,10 @@ Must be one of:

| | |
| ------------------------- | -------------------------------------------------------------------------------------------------------- |
| **Type** | `combining` |
| **Type** | `object` |
| **Additional properties** | [![Not allowed](https://img.shields.io/badge/Not%20allowed-red)](# "Additional Properties not allowed.") |
| **Defined in** | #/definitions/ZarfChart |

<blockquote>

| One of(Option) |
| ---------------------------------------------------- |
| [url](#components_items_charts_items_oneOf_i0) |
| [localPath](#components_items_charts_items_oneOf_i1) |

<blockquote>

### <a name="components_items_charts_items_oneOf_i0"></a>Property `url`

**Title:** url

| | |
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| **Type** | `object` |
| **Additional properties** | [![Any type: allowed](https://img.shields.io/badge/Any%20type-allowed-green)](# "Additional Properties of any type are allowed.") |

#### <a name="autogenerated_heading_2"></a>The following properties are required
* url

</blockquote>
<blockquote>

### <a name="components_items_charts_items_oneOf_i1"></a>Property `localPath`

**Title:** localPath

| | |
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| **Type** | `object` |
| **Additional properties** | [![Any type: allowed](https://img.shields.io/badge/Any%20type-allowed-green)](# "Additional Properties of any type are allowed.") |

#### <a name="autogenerated_heading_2"></a>The following properties are required
* localPath

</blockquote>

</blockquote>

<details>
<summary>
<strong> <a name="components_items_charts_items_name"></a>name *</strong>
Expand Down
2 changes: 1 addition & 1 deletion examples/component-choice/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import ExampleYAML from "@site/src/components/ExampleYAML";

:::caution

Component Choice is currently a [Deprecated Feature](../../docs/9-roadmap.md#alpha). This feature will be removed in Zarf v1.0.0. Please migrate any existing packages you may have that utilize it.
Component Choice is currently a [Deprecated Feature](../../docs/9-roadmap.md#alpha). This feature will be removed in Zarf v1.0.0. Please migrate any existing packages you may have that utilize it. In doing so you may want to consider [Package Flavors](../package-flavors/README.md) as an alternative.

:::

Expand Down
23 changes: 3 additions & 20 deletions examples/composable-packages/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@ import ExampleYAML from "@site/src/components/ExampleYAML";

# Composable Packages

This example demonstrates using Zarf to import components from existing Zarf package definitions while merging overrides to add or change functionality. It uses the existing [DOS games](../dos-games/README.md) and [WordPress](../wordpress/README.md) examples by simply adding `import` keys in the new [zarf.yaml](zarf.yaml) file.
This example demonstrates using Zarf to import components from existing Zarf package definitions while merging overrides to add or change functionality. It uses the existing [DOS games](../dos-games/README.md) example by simply adding `import` keys in the new [zarf.yaml](zarf.yaml) file.

The `import` key in Zarf supports two modes to pull in a component:

1. The `path` key allows you to specify a path to a directory that contains the `zarf.yaml` that you wish to import on your local filesystem. This allows you to have a common component that you can reuse across multiple packages *within* a project.
1. The `path` key allows you to specify a path to a directory that contains the `zarf.yaml` that you wish to import on your local filesystem. This allows you to have a common component that you can reuse across multiple packages *within* a project (i.e. within one team/codebase).

2. The `url` key allows you to specify an `oci://` URL to a skeleton package that was published to an OCI registry. Skeleton packages are special package bundles that contain the `zarf.yaml` package definition and any local files referenced by that definition at publish time. This allows you to version a set of components and import them into multiple packages *across* projects.
2. The `url` key allows you to specify an `oci://` URL to a skeleton package that was published to an OCI registry. Skeleton packages are special package bundles that contain the `zarf.yaml` package definition and any local files referenced by that definition at publish time. This allows you to version a set of reusable components and import them into multiple packages *across* projects (i.e. across teams/codebases).

:::tip

Expand Down Expand Up @@ -40,23 +40,6 @@ To view the example in its entirety, select the `Edit this page` link below the

:::

:::note

Creating this example requires a locally hosted container registry that has the `wordpress` skeleton package published and available. You can do this by running the following commands:

```bash
docker run -d -p 555:5000 --restart=always --name registry registry:2
zarf package publish examples/wordpress oci://127.0.0.1:555 --insecure
```

You will also need to pass the `--insecure` flag to `zarf package create` to pull from the `http` registry:

```bash
zarf package create examples/composable-packages/ --insecure
```

:::

<ExampleYAML src={require('./zarf.yaml')} showLink={false} />

:::info
Expand Down
8 changes: 3 additions & 5 deletions examples/composable-packages/zarf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,18 @@ components:
files:
- quake-service.yaml

- name: oci-wordpress-url
- name: oci-games-url
# The component logic keys ('required', 'group', and 'default') always override those of the imported package
# required: false # the initial value overrides the child component
# group: "" # the initial value overrides the child component
# default: false # the initial value overrides the child component
import:
# The URL to the skeleton package containing this component's package definition
url: oci://localhost:555/wordpress:16.0.4-skeleton
url: oci://🦄/dos-games:1.0.0-skeleton
# Example optional custom name to point to in the imported package (default is to use this component's name)
name: wordpress
name: baseline
# Un'name'd Zarf primitives will be appended to the end of the primitive's list for that component.
actions:
onDeploy:
before:
- cmd: ./zarf tools kubectl get -n dos-games deployment -o jsonpath={.items[0].metadata.creationTimestamp}
setVariables:
- name: WORDPRESS_BLOG_NAME
17 changes: 17 additions & 0 deletions examples/package-flavors/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import ExampleYAML from "@site/src/components/ExampleYAML";

# Package Flavors

This example demonstrates how to define variants of packages within the same package definition. This can be combined with [Composable Packages](../composable-packages/README.md) to build up packages and include the necessary [merge overrides](../composable-packages/README.md#merge-strategies) for each variant.

Given package flavors are built by specifying the `--flavor` flag on `zarf package create`. This will include any components that match that flavor or that do not specify a flavor.

## `zarf.yaml` {#zarf.yaml}

:::info

To view the example in its entirety, select the `Edit this page` link below the article and select the parent folder.

:::

<ExampleYAML src={require('./zarf.yaml')} showLink={false} />
18 changes: 18 additions & 0 deletions examples/package-flavors/pod.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
apiVersion: v1
kind: Pod
metadata:
name: enterprise-linux
labels:
app: enterprise-linux
spec:
containers:
- name: enterprise-linux-container
image: "###ZARF_VAR_IMAGE###"
command: [ "sh", "-c", "while true; do ls; sleep 1; done"]
resources:
requests:
memory: "32Mi"
cpu: "50m"
limits:
memory: "128Mi"
cpu: "250m"
70 changes: 70 additions & 0 deletions examples/package-flavors/zarf.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
kind: ZarfPackageConfig
metadata:
name: package-flavors
description: Simple example to show how to use the `only.flavor` key to build package variants.

components:
- name: image
required: true
description: "Sets the Enterprise Linux flavor to Rocky Linux"
only:
flavor: rocky-road
images:
- rockylinux:9-minimal
actions:
onDeploy:
before:
- cmd: echo "rockylinux:9-minimal"
setVariables:
- name: IMAGE

- name: image
required: true
description: "Sets the Enterprise Linux flavor to Oracle Linux"
only:
flavor: oracle-cookie-crunch
images:
- oraclelinux:9-slim
actions:
onDeploy:
before:
- cmd: echo "oraclelinux:9-slim"
setVariables:
- name: IMAGE

- name: image
required: true
description: "Sets the Enterprise Linux flavor to Alma Linux"
only:
flavor: vanilla-alma-nd
images:
- almalinux:9-minimal
actions:
onDeploy:
before:
- cmd: echo "almalinux:9-minimal"
setVariables:
- name: IMAGE

- name: image
required: true
description: "Sets the Enterprise Linux flavor to OpenSUSE"
only:
flavor: strawberry-suse
images:
- opensuse/leap:15
actions:
onDeploy:
before:
- cmd: echo "opensuse/leap:15"
setVariables:
- name: IMAGE

- name: pod
description: "The pod that runs the specified flavor of Enterprise Linux"
required: true
manifests:
- name: enterprise-linux
namespace: enterprise-linux
files:
- pod.yaml
5 changes: 2 additions & 3 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ require (
github.com/anchore/stereoscope v0.0.0-20231027135531-5909e353ee88
github.com/anchore/syft v0.84.1
github.com/derailed/k9s v0.27.4
github.com/distribution/distribution v2.8.3+incompatible
github.com/distribution/reference v0.5.0
github.com/docker/cli v24.0.6+incompatible
github.com/fairwindsops/pluto/v5 v5.18.4
github.com/fatih/color v1.15.0
Expand Down Expand Up @@ -186,9 +186,8 @@ require (
github.com/digitorus/pkcs7 v0.0.0-20230818184609-3a137a874352 // indirect
github.com/digitorus/timestamp v0.0.0-20230821155606-d1ad5ca9624c // indirect
github.com/dimchansky/utfbom v1.1.1 // indirect
github.com/distribution/reference v0.5.0 // indirect
github.com/docker/distribution v2.8.3+incompatible // indirect
github.com/docker/docker v24.0.6+incompatible // indirect
github.com/docker/docker v24.0.7+incompatible // indirect
github.com/docker/docker-credential-helpers v0.7.0 // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect
Expand Down
Loading

0 comments on commit 80b9e35

Please sign in to comment.