Skip to content

Commit

Permalink
Add new config parameter for file sources temp dir and update the web…
Browse files Browse the repository at this point in the history
…dav file source
  • Loading branch information
sanjaysrikakulam committed Oct 10, 2024
1 parent 3d28b2b commit 22863a1
Show file tree
Hide file tree
Showing 9 changed files with 10 additions and 6 deletions.
Binary file modified doc/source/admin/file_source_templates.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified doc/source/admin/file_source_webdav_configuration.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified doc/source/admin/file_source_webdav_configuration_template.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions lib/galaxy/config/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -741,6 +741,7 @@ class GalaxyAppConfiguration(BaseAppConfiguration, CommonConfigurationMixin):
user_tool_section_filters: List[str]
visualization_plugins_directory: str
workflow_resource_params_mapper: str
file_source_temp_dir: str

def __init__(self, **kwargs):
super().__init__(**kwargs)
Expand Down
5 changes: 5 additions & 0 deletions lib/galaxy/config/sample/galaxy.yml.sample
Original file line number Diff line number Diff line change
Expand Up @@ -697,6 +697,11 @@ galaxy:
# cleaned up in default Celery task configuration.
#short_term_storage_cleanup_interval: 3600

# The file_source_temp_dir is a temporary directory used by file sources
# that require temporary storage for file operations.
# Defaults to new_file_path.
#file_source_temp_dir: null

# Configured FileSource plugins.
# The value of this option will be resolved with respect to
# <config_dir>.
Expand Down
4 changes: 2 additions & 2 deletions lib/galaxy/config/schemas/config_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3746,7 +3746,7 @@ mapping:
Workflows launched with URI/URL inputs that are not marked as 'deferred'
are "materialized" (or undeferred) by the workflow scheduler. This might be
a lengthy process. Setting this to 'True' will place the invocation back in
the queue after materialization before scheduling the workflow so it is less
the queue after materialization before scheduling the workflow so it is less
likely to starve other workflow scheduling. Ideally, Galaxy would allow more
fine grain control of handlers but until then, this provides a way to tip the
balance between "doing more work" and "being more fair". The default here is
Expand Down Expand Up @@ -4110,7 +4110,7 @@ mapping:
required: false
desc: |
Directory to store temporary files for file sources. This defaults to new_file_path if not set.
file_source_webdav_use_temp_files:
type: bool
default: true
Expand Down
3 changes: 2 additions & 1 deletion lib/galaxy/files/sources/webdav.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
except ImportError:
WebDAVFS = None

import tempfile
from typing import (
cast,
Optional,
Expand All @@ -12,6 +11,8 @@

from typing_extensions import NotRequired

from typing_extensions import NotRequired

from . import (
FilesSourceOptions,
FilesSourceProperties,
Expand Down
1 change: 0 additions & 1 deletion lib/galaxy/files/templates/examples/production_webdav.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,3 @@
login: '{{ variables.login }}'
writable: '{{ variables.writable }}'
password: '{{ secrets.password }}'
temp_path: null # Uses /tmp as default, configure it if needed
2 changes: 0 additions & 2 deletions lib/galaxy/files/templates/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,6 @@ class WebdavFileSourceTemplateConfiguration(StrictModel):
login: Union[str, TemplateExpansion]
password: Union[str, TemplateExpansion]
writable: Union[bool, TemplateExpansion] = False
temp_path: Optional[Union[str, TemplateExpansion]] = None
template_start: Optional[str] = None
template_end: Optional[str] = None

Expand All @@ -140,7 +139,6 @@ class WebdavFileSourceConfiguration(StrictModel):
login: str
password: str
writable: bool = False
temp_path: Optional[str] = None


FileSourceTemplateConfiguration = Union[
Expand Down

0 comments on commit 22863a1

Please sign in to comment.