Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add var argument to kedro databricks deploy cli #77

Closed
wants to merge 1 commit into from
Closed

feat: add var argument to kedro databricks deploy cli #77

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion src/kedro_databricks/deploy.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ def build_project(self): # pragma: no cover
result = run_cmd(build_cmd, msg=self._msg)
return result

def deploy_project(self, target: str, debug: bool = False):
def deploy_project(self, target: str, var: str = None, debug: bool = False):
"""Deploy the project to Databricks.

Args:
Expand All @@ -144,6 +144,8 @@ def deploy_project(self, target: str, debug: bool = False):
f"{self._msg}: Running `databricks bundle deploy --target {target}`"
)
deploy_cmd = ["databricks", "bundle", "deploy", "--target", target]
if var:
deploy_cmd.extend(["--var", var])
if debug:
deploy_cmd.append("--debug")
run_cmd(deploy_cmd, msg=self._msg)
Expand Down
4 changes: 3 additions & 1 deletion src/kedro_databricks/plugin.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ def bundle(
@click.option("-c", "--conf", default=DEFAULT_CONF_FOLDER, help=CONF_HELP)
@click.option("-d", "--debug/--no-debug", default=False, help="Enable debug mode")
@click.option("-p", "--pipeline", default=None, help="Bundle a single pipeline")
@click.option("-v", "--var", default=None, help="set values for variables defined in bundle config. Example: --var=\"foo=bar\"")
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So Databricks allows you to pass this multiple times.... should we do the same?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, did not see it was possible.

I would say we should do as they do since we are trying to replicate what is doing databricks cli. What do you think?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should 😊 this was such a minor addition I actually implemented it yesterday when I was doing some or refactoring. See PR #80.

It just needs tests before it can be merged

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw the MR, it s perfect, thank you for that!

@click.pass_obj
def deploy(
metadata: ProjectMetadata,
Expand All @@ -126,6 +127,7 @@ def deploy(
conf: str,
pipeline: str,
debug: bool,
var: str
):
"""Deploy the asset bundle to Databricks"""

Expand All @@ -145,4 +147,4 @@ def deploy(
controller.upload_project_data()
if target is None:
target = env
controller.deploy_project(target=target, debug=debug)
controller.deploy_project(target=target, debug=debug, var=var)