Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add wildcard and deselection support to --components #2175

Merged
merged 30 commits into from
Dec 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
5881d97
feat: add wildcard support to --components
Racer159 Dec 3, 2023
8d04a67
add tests and deselection logic
Racer159 Dec 3, 2023
4ecab68
Adjust helm-no-wait back
Racer159 Dec 3, 2023
2c812c9
add docs for the deploy side of --components
Racer159 Dec 3, 2023
6a9063f
Base refactor of remove and mirror
Racer159 Dec 6, 2023
449442a
Refactor mirror and remove to use a common function
Racer159 Dec 6, 2023
24a26cf
Merge branch 'main' into 1794-wildcard-exclude-syntax-components
Racer159 Dec 6, 2023
e6fb42e
Adjust verbiage for components flags
Racer159 Dec 6, 2023
b21288f
Merge branch 'main' into 1794-wildcard-exclude-syntax-components
Racer159 Dec 6, 2023
74a2a34
Merge branch 'main' into 1794-wildcard-exclude-syntax-components
Racer159 Dec 6, 2023
2c52e41
Resovle feedback
Racer159 Dec 7, 2023
a67d304
Refactor component logic to be more readable
Racer159 Dec 11, 2023
9ce4cc2
unexport enum
Racer159 Dec 11, 2023
aa7dcf2
Match the edges of Current Zarf
Racer159 Dec 11, 2023
a15d545
add create validations
Racer159 Dec 11, 2023
f76d92b
Normalize errors
Racer159 Dec 11, 2023
fafa1a9
Merge branch 'main' into 1794-wildcard-exclude-syntax-components
Racer159 Dec 11, 2023
fb9709e
Merge branch 'main' into 1794-wildcard-exclude-syntax-components
Racer159 Dec 11, 2023
c0d3943
Merge branch 'main' into 1794-wildcard-exclude-syntax-components
Racer159 Dec 15, 2023
151a1da
Change '-' from suffix to prefix
Racer159 Dec 15, 2023
7c34881
Switch to typed iota
Racer159 Dec 15, 2023
3e7f476
Switch to typed iota
Racer159 Dec 15, 2023
d2ee022
Address linting issues
Racer159 Dec 15, 2023
fa7442c
Fix dev deploy
Racer159 Dec 15, 2023
c9454e0
fix comments
Racer159 Dec 15, 2023
b0de8c3
Fixup the docs+schema
Racer159 Dec 15, 2023
1c03f88
Respect exclusions first
Racer159 Dec 15, 2023
62ec12f
Merge branch 'main' into 1794-wildcard-exclude-syntax-components
Racer159 Dec 15, 2023
4f28616
Merge branch 'main' into 1794-wildcard-exclude-syntax-components
Racer159 Dec 16, 2023
e8b776b
Merge branch 'main' into 1794-wildcard-exclude-syntax-components
Racer159 Dec 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/2-the-zarf-cli/100-cli-commands/zarf_dev_deploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ zarf dev deploy [flags]
## Options

```
--components string Comma-separated list of components to install. Adding this flag will skip the init prompts for which components to install
--components string Comma-separated list of components to deploy. Adding this flag will skip the prompts for selected components. Globbing component names with '*' and deselecting 'default' components with a leading '-' are also supported.
--create-set stringToString Specify package variables to set on the command line (KEY=value) (default [])
--deploy-set stringToString Specify deployment variables to set on the command line (KEY=value) (default [])
-f, --flavor string The flavor of components to include in the resulting package (i.e. have a matching or empty "only.flavor" key)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ zarf package deploy [ PACKAGE_SOURCE ] [flags]

```
--adopt-existing-resources Adopts any pre-existing K8s resources into the Helm charts managed by Zarf. ONLY use when you have existing deployments you want Zarf to takeover.
--components string Comma-separated list of components to install. Adding this flag will skip the init prompts for which components to install
--components string Comma-separated list of components to deploy. Adding this flag will skip the prompts for selected components. Globbing component names with '*' and deselecting 'default' components with a leading '-' are also supported.
--confirm Confirms package deployment without prompting. ONLY use with packages you trust. Skips prompts to review SBOM, configure variables, select optional components and review potential breaking changes.
-h, --help help for deploy
--set stringToString Specify deployment variables to set on the command line (KEY=value) (default [])
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Mirrors a Zarf package's internal resources to specified image registries and gi

## Synopsis

Unpacks resources and dependencies from a Zarf package archive and mirrors them into the specified
Unpacks resources and dependencies from a Zarf package archive and mirrors them into the specified
image registries and git repositories within the target environment

```
Expand Down Expand Up @@ -39,7 +39,7 @@ $ zarf package mirror-resources <your-package.tar.zst> \
## Options

```
--components string Comma-separated list of components to mirror. This list will be respected regardless of a component's 'required' status.
--components string Comma-separated list of components to mirror. This list will be respected regardless of a component's 'required' or 'default' status. Globbing component names with '*' and deselecting components with a leading '-' are also supported.
--confirm Confirms package deployment without prompting. ONLY use with packages you trust. Skips prompts to review SBOM, configure variables, select optional components and review potential breaking changes.
--git-push-password string Password for the push-user to access the git server
--git-push-username string Username to access to the git server Zarf is configured to use. User must be able to create repositories via 'git push' (default "zarf-git-user")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ zarf package remove { PACKAGE_SOURCE | PACKAGE_NAME } --confirm [flags]
## Options

```
--components string Comma-separated list of components to uninstall
--components string Comma-separated list of components to remove. This list will be respected regardless of a component's 'required' or 'default' status. Globbing component names with '*' and deselecting components with a leading '-' are also supported.
--confirm REQUIRED. Confirm the removal action to prevent accidental deletions
-h, --help help for remove
```
Expand Down
18 changes: 18 additions & 0 deletions docs/3-create-a-zarf-package/2-zarf-components.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,3 +204,21 @@ $ zarf package deploy ./path/to/package.tar.zst --confirm
# deploy optional-component-1 and optional-component-2 components whether they are required or not
$ zarf package deploy ./path/to/package.tar.zst --components=optional-component-1,optional-component-2
```

:::tip

You can deploy components in a package using globbing as well. The following would deploy all components regardless of optional status:

```bash
# deploy optional-component-1 and optional-component-2 components whether they are required or not
$ zarf package deploy ./path/to/package.tar.zst --components=*
```

If you have any `default` components in a package definition you can also exclude those from the CLI with a leading dash (`-`) (similar to how you can exclude search terms in a search engine).

```bash
# deploy optional-component-1 but exclude default-component-1
$ zarf package deploy ./path/to/package.tar.zst --components=optional-component-1,-default-component-1
```

:::
12 changes: 6 additions & 6 deletions docs/3-create-a-zarf-package/4-zarf-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,9 @@ Must be one of:
| -------- | -------- |
| **Type** | `string` |

| Restrictions | |
| --------------------------------- | --------------------------------------------------------------------------------- |
| **Must match regular expression** | ```^[a-z0-9\-]+$``` [Test](https://regex101.com/?regex=%5E%5Ba-z0-9%5C-%5D%2B%24) |
| Restrictions | |
| --------------------------------- | ----------------------------------------------------------------------------------------------------- |
| **Must match regular expression** | ```^[a-z0-9\-]*[a-z0-9]$``` [Test](https://regex101.com/?regex=%5E%5Ba-z0-9%5C-%5D%2A%5Ba-z0-9%5D%24) |

</blockquote>
</details>
Expand Down Expand Up @@ -554,9 +554,9 @@ must respect the following conditions
| -------- | -------- |
| **Type** | `string` |

| Restrictions | |
| --------------------------------- | --------------------------------------------------------------------------------- |
| **Must match regular expression** | ```^[a-z0-9\-]+$``` [Test](https://regex101.com/?regex=%5E%5Ba-z0-9%5C-%5D%2B%24) |
| Restrictions | |
| --------------------------------- | ----------------------------------------------------------------------------------------------------- |
| **Must match regular expression** | ```^[a-z0-9\-]*[a-z0-9]$``` [Test](https://regex101.com/?regex=%5E%5Ba-z0-9%5C-%5D%2A%5Ba-z0-9%5D%24) |

</blockquote>
</details>
Expand Down
4 changes: 0 additions & 4 deletions examples/git-data/zarf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,13 @@ metadata:

components:
- name: full-repo
required: true
repos:
# The following performs a full Git Repo Mirror with `go-git` (internal to Zarf)
- https://github.com/defenseunicorns/zarf-public-test.git
# The following performs a full Git Repo Mirror forcing a fallback to host `git`
- https://dev.azure.com/defenseunicorns/zarf-public-test/_git/zarf-public-test

- name: specific-tag
required: true
repos:
# The following performs a tag Git Repo Mirror with `go-git` (internal to Zarf)
- https://github.com/defenseunicorns/[email protected]
Expand All @@ -24,15 +22,13 @@ components:
- https://dev.azure.com/defenseunicorns/zarf-public-test/_git/[email protected]

- name: specific-branch
required: true
repos:
# The following performs a branch Git Repo Mirror with `go-git` (internal to Zarf)
- https://github.com/defenseunicorns/zarf-public-test.git@refs/heads/dragons
# The following performs a branch Git Repo Mirror forcing a fallback to host `git`
- https://dev.azure.com/defenseunicorns/zarf-public-test/_git/zarf-public-test@refs/heads/dragons

- name: specific-hash
required: true
repos:
# The following performs a SHA Git Repo Mirror with `go-git` (internal to Zarf)
- https://github.com/defenseunicorns/zarf-public-test.git@01a23218923f24194133b5eb11268cf8d73ff1bb
Expand Down
21 changes: 16 additions & 5 deletions src/config/lang/english.go
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,7 @@ $ zarf init --artifact-push-password={PASSWORD} --artifact-push-username={USERNA
"Kubernetes clusters are accessed via credentials in your current kubecontext defined in '~/.kube/config'"

CmdPackageMirrorShort = "Mirrors a Zarf package's internal resources to specified image registries and git repositories"
CmdPackageMirrorLong = "Unpacks resources and dependencies from a Zarf package archive and mirrors them into the specified \n" +
CmdPackageMirrorLong = "Unpacks resources and dependencies from a Zarf package archive and mirrors them into the specified\n" +
"image registries and git repositories within the target environment"
CmdPackageMirrorExample = `
# Mirror resources to internal Zarf resources
Expand Down Expand Up @@ -286,7 +286,7 @@ $ zarf package mirror-resources <your-package.tar.zst> \
CmdPackageDeployFlagConfirm = "Confirms package deployment without prompting. ONLY use with packages you trust. Skips prompts to review SBOM, configure variables, select optional components and review potential breaking changes."
CmdPackageDeployFlagAdoptExistingResources = "Adopts any pre-existing K8s resources into the Helm charts managed by Zarf. ONLY use when you have existing deployments you want Zarf to takeover."
CmdPackageDeployFlagSet = "Specify deployment variables to set on the command line (KEY=value)"
CmdPackageDeployFlagComponents = "Comma-separated list of components to install. Adding this flag will skip the init prompts for which components to install"
CmdPackageDeployFlagComponents = "Comma-separated list of components to deploy. Adding this flag will skip the prompts for selected components. Globbing component names with '*' and deselecting 'default' components with a leading '-' are also supported."
CmdPackageDeployFlagShasum = "Shasum of the package to deploy. Required if deploying a remote package and \"--insecure\" is not provided"
CmdPackageDeployFlagSget = "[Deprecated] Path to public sget key file for remote packages signed via cosign. This flag will be removed in v1.0.0 please use the --key flag instead."
CmdPackageDeployFlagSkipWebhooks = "[alpha] Skip waiting for external webhooks to execute as each package component is deployed"
Expand All @@ -296,7 +296,7 @@ $ zarf package mirror-resources <your-package.tar.zst> \
CmdPackageDeployInvalidCLIVersionWarn = "CLIVersion is set to '%s' which can cause issues with package creation and deployment. To avoid such issues, please set the value to the valid semantic version for this version of Zarf."
CmdPackageDeployErr = "Failed to deploy package: %s"

CmdPackageMirrorFlagComponents = "Comma-separated list of components to mirror. This list will be respected regardless of a component's 'required' status."
CmdPackageMirrorFlagComponents = "Comma-separated list of components to mirror. This list will be respected regardless of a component's 'required' or 'default' status. Globbing component names with '*' and deselecting components with a leading '-' are also supported."
CmdPackageMirrorFlagNoChecksum = "Turns off the addition of a checksum to image tags (as would be used by the Zarf Agent) while mirroring images."

CmdPackageInspectFlagSbom = "View SBOM contents while inspecting the package"
Expand All @@ -305,7 +305,7 @@ $ zarf package mirror-resources <your-package.tar.zst> \

CmdPackageRemoveShort = "Removes a Zarf package that has been deployed already (runs offline)"
CmdPackageRemoveFlagConfirm = "REQUIRED. Confirm the removal action to prevent accidental deletions"
CmdPackageRemoveFlagComponents = "Comma-separated list of components to uninstall"
CmdPackageRemoveFlagComponents = "Comma-separated list of components to remove. This list will be respected regardless of a component's 'required' or 'default' status. Globbing component names with '*' and deselecting components with a leading '-' are also supported."
CmdPackageRemoveTarballErr = "Invalid tarball path provided"
CmdPackageRemoveExtractErr = "Unable to extract the package contents"
CmdPackageRemoveErr = "Unable to remove the package with an error of: %s"
Expand Down Expand Up @@ -609,6 +609,14 @@ const (
PkgCreateErrDifferentialSameVersion = "unable to create a differential package with the same version as the package you are using as a reference; the package version must be incremented"
)

// src/internal/packager/deploy.
const (
PkgDeployErrMultipleComponentsSameGroup = "You cannot specify multiple components (%q, %q) within the same group (%q) when using the --components flag."
PkgDeployErrNoDefaultOrSelection = "You must make a selection from %q with the --components flag as there is no default in their group."
PkgDeployErrNoCompatibleComponentsForSelection = "No compatible components found that matched %q. Please check spelling and try again."
PkgDeployErrComponentSelectionCanceled = "Component selection canceled: %s"
)

// src/internal/packager/validate.
const (
PkgValidateTemplateDeprecation = "Package template %q is using the deprecated syntax ###ZARF_PKG_VAR_%s###. This will be removed in Zarf v1.0.0. Please update to ###ZARF_PKG_TMPL_%s###."
Expand All @@ -624,11 +632,14 @@ const (
PkgValidateErrChartNamespaceMissing = "chart %q must include a namespace"
PkgValidateErrChartURLOrPath = "chart %q must have either a url or localPath"
PkgValidateErrChartVersion = "chart %q must include a chart version"
PkgValidateErrComponentName = "component name %q must be all lowercase and contain no special characters except '-' and cannot start with a '-'"
PkgValidateErrComponentNameNotUnique = "component name %q is not unique"
PkgValidateErrComponent = "invalid component %q: %w"
PkgValidateErrComponentReqDefault = "component %q cannot be both required and default"
PkgValidateErrComponentReqGrouped = "component %q cannot be both required and grouped"
PkgValidateErrComponentYOLO = "component %q incompatible with the online-only package flag (metadata.yolo): %w"
PkgValidateErrGroupMultipleDefaults = "group %q has multiple defaults (%q, %q)"
PkgValidateErrGroupOneComponent = "group %q only has one component (%q)"
PkgValidateErrConstant = "invalid package constant: %w"
PkgValidateErrImportDefinition = "invalid imported definition for %s: %s"
PkgValidateErrInitNoYOLO = "sorry, you can't YOLO an init package"
Expand All @@ -640,7 +651,7 @@ const (
PkgValidateErrName = "invalid package name: %w"
PkgValidateErrPkgConstantName = "constant name %q must be all uppercase and contain no special characters except _"
PkgValidateErrPkgConstantPattern = "provided value for constant %q does not match pattern %q"
PkgValidateErrPkgName = "package name %q must be all lowercase and contain no special characters except -"
PkgValidateErrPkgName = "package name %q must be all lowercase and contain no special characters except '-' and cannot start with a '-'"
PkgValidateErrVariable = "invalid package variable: %w"
PkgValidateErrYOLONoArch = "cluster architecture not allowed"
PkgValidateErrYOLONoDistro = "cluster distros not allowed"
Expand Down
3 changes: 1 addition & 2 deletions src/internal/packager/helm/post-render.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,10 @@ func (h *Helm) newRenderer() (*renderer, error) {

func (r *renderer) Run(renderedManifests *bytes.Buffer) (*bytes.Buffer, error) {
// This is very low cost and consistent for how we replace elsewhere, also good for debugging
tempDir, err := utils.MakeTempDir(config.CommonOptions.TempDirectory)
tempDir, err := utils.MakeTempDir(r.chartPath)
if err != nil {
return nil, fmt.Errorf("unable to create tmpdir: %w", err)
}
defer os.RemoveAll(tempDir)
Noxsios marked this conversation as resolved.
Show resolved Hide resolved
path := filepath.Join(tempDir, "chart.yaml")

// Write the context to a file for processing
Expand Down
31 changes: 27 additions & 4 deletions src/internal/packager/validate/validate.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ import (
)

var (
// IsLowercaseNumberHyphen is a regex for lowercase, numbers and hyphens.
// https://regex101.com/r/FLdG9G/1
IsLowercaseNumberHyphen = regexp.MustCompile(`^[a-z0-9\-]+$`).MatchString
// IsLowercaseNumberHyphenNoStartHyphen is a regex for lowercase, numbers and hyphens that cannot start with a hyphen.
// https://regex101.com/r/FLdG9G/2
IsLowercaseNumberHyphenNoStartHyphen = regexp.MustCompile(`^[a-z0-9][a-z0-9\-]*$`).MatchString
// IsUppercaseNumberUnderscore is a regex for uppercase, numbers and underscores.
// https://regex101.com/r/tfsEuZ/1
IsUppercaseNumberUnderscore = regexp.MustCompile(`^[A-Z0-9_]+$`).MatchString
Expand Down Expand Up @@ -49,6 +49,8 @@ func Run(pkg types.ZarfPackage) error {
}

uniqueComponentNames := make(map[string]bool)
groupDefault := make(map[string]string)
groupedComponents := make(map[string][]string)

for _, component := range pkg.Components {
// ensure component name is unique
Expand All @@ -60,6 +62,23 @@ func Run(pkg types.ZarfPackage) error {
if err := validateComponent(pkg, component); err != nil {
return fmt.Errorf(lang.PkgValidateErrComponent, component.Name, err)
}

// ensure groups don't have multiple defaults or only one component
if component.Group != "" {
if component.Default {
if _, ok := groupDefault[component.Group]; ok {
return fmt.Errorf(lang.PkgValidateErrGroupMultipleDefaults, component.Group, groupDefault[component.Group], component.Name)
}
groupDefault[component.Group] = component.Name
}
groupedComponents[component.Group] = append(groupedComponents[component.Group], component.Name)
}
}

for groupKey, componentNames := range groupedComponents {
if len(componentNames) == 1 {
return fmt.Errorf(lang.PkgValidateErrGroupOneComponent, groupKey, componentNames[0])
}
Racer159 marked this conversation as resolved.
Show resolved Hide resolved
}

return nil
Expand Down Expand Up @@ -111,6 +130,10 @@ func oneIfNotEmpty(testString string) int {
}

func validateComponent(pkg types.ZarfPackage, component types.ZarfComponent) error {
if !IsLowercaseNumberHyphenNoStartHyphen(component.Name) {
return fmt.Errorf(lang.PkgValidateErrComponentName, component.Name)
}

if component.Required {
if component.Default {
return fmt.Errorf(lang.PkgValidateErrComponentReqDefault, component.Name)
Expand Down Expand Up @@ -254,7 +277,7 @@ func validateYOLO(component types.ZarfComponent) error {
}

func validatePackageName(subject string) error {
if !IsLowercaseNumberHyphen(subject) {
if !IsLowercaseNumberHyphenNoStartHyphen(subject) {
return fmt.Errorf(lang.PkgValidateErrPkgName, subject)
}

Expand Down
Loading
Loading