Skip to content

Commit

Permalink
Fixes bug where data is downloaded even if already available (#53)
Browse files Browse the repository at this point in the history
  • Loading branch information
djperrefort authored Jun 25, 2020
1 parent e63b8f8 commit 464e3e2
Show file tree
Hide file tree
Showing 6 changed files with 17 additions and 10 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ build/
dist/

# Light Curve Data
sndata/sweetspot/whircs
sndata/data

# Testing cache files
.pytest_cache/
Expand Down
5 changes: 5 additions & 0 deletions docs/source/change_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ Change Log
This page documents any API changes between different versions of the
``sndata`` package.

V 1.1.1
-------

- Fixes bug where existing data is not skipped during download by default.

V 1.1.0
-------

Expand Down
2 changes: 1 addition & 1 deletion docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
author = 'Daniel Perrefort'

# The short X.Y version
version = '1.0.0'
version = '1.1.1'

# The full version, including alpha/beta/rc tags
release = version
Expand Down
7 changes: 5 additions & 2 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,10 @@ data release may have from the rest of the package. In general you should:
Available Data
--------------

SNData provides data access for the following supernova surveys. If you are
having issues downloading data for a particular survey or data release, please
check the `Server Status Page`_.

+------------------------------------------------------+--------------------------+---------------+
| Survey Name | Data Release | Data Type |
+======================================================+==========================+===============+
Expand All @@ -68,8 +72,6 @@ Available Data
+------------------------------------------------------+--------------------------+---------------+
+ Joint Light-Curve Analysis | `Betoule et al. 2014`_ | Photometric |
+------------------------------------------------------+--------------------------+---------------+
+ Lick Observatory Supernova Search | `Stahl et al. 2019`_ | Photometric |
+------------------------------------------------------+--------------------------+---------------+
| | `Sako et al. 2018`_ | Photometric |
+ Sloan Digital Sky Survey +--------------------------+---------------+
| | `Sako et al. 2018`_ | Spectroscopic |
Expand All @@ -79,6 +81,7 @@ Available Data
+ Sweetspot | `Weyant et al. 2018`_ | Photometric |
+------------------------------------------------------+--------------------------+---------------+

.. _Server Status Page: https://stats.uptimerobot.com/gQ8lkslGWO
.. _DR1: https://csp.obs.carnegiescience.edu/news-items/CSP_spectra_DR1
.. _DR3: https://csp.obs.carnegiescience.edu/news-items/csp-dr3-photometry-released
.. _SN3YR: https://des.ncsa.illinois.edu/releases/sn
Expand Down
2 changes: 1 addition & 1 deletion sndata/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from . import *
from ._combine_data import CombinedDataset, get_zp

__version__ = '1.1.0'
__version__ = '1.1.1'
__author__ = 'Daniel Perrefort'
__license__ = 'GPL 3.0'

Expand Down
9 changes: 4 additions & 5 deletions sndata/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -180,25 +180,24 @@ def download_file(
verbose: Print status to stdout
"""

response = requests.get(url, timeout=timeout)
response.raise_for_status()

if file_obj is None:
if path is None:
raise ValueError('Must specify either ``path`` or ``file_obj``')

# Skip downcload if file already exists or url unavailable
# Skip download if file already exists or url unavailable
path = Path(path)
if not (force or not path.exists()):
return

path.parent.mkdir(parents=True, exist_ok=True)
file_obj = open(path, 'wb')

# Establish remote connection
if verbose:
print(f'Fetching {url}')

# Establish remote connection
response = requests.get(url, timeout=timeout)
response.raise_for_status()
file_obj.write(response.content)

if path:
Expand Down

0 comments on commit 464e3e2

Please sign in to comment.