Skip to content

Commit

Permalink
need to close the multiprocessing pool (#28)
Browse files Browse the repository at this point in the history
Co-authored-by: dustinlo <[email protected]>
  • Loading branch information
DustinKLo and dustinlo authored Apr 28, 2023
1 parent d3ac1d1 commit 09e2e9b
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions purge.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ def purge_products(query, component, operation, delete_from_obj_store=True):
p.map(delete_from_object_store, results) # deleting objects from storage (s3, etc.)
else:
logger.info("skip purging datasets from object store. delete_from_object_store=False")
p.close()

dataset_purge_stats = {}
deleted_docs_count = 0
Expand Down

0 comments on commit 09e2e9b

Please sign in to comment.