Add multi-resolution TS map fitting

[Yong2Sheng]

This PR adds the method to do multi-resolution (also called multi-order coverage, MOC) TS map fitting.

Changes to FastTSMap

1. FastTSMap is the parent class of MOCTSMap

2. I need to make several changes to FastTSMap to make it work with MOCTSMap:

   • Add pixel_idx parameter for fast_ts_fit.
     • This parameter is the index of the pixels fitted by FastTSMap.
     • This is added since we only fit a portion of the multi-resolution map. So I need to record the indices of the fitted pixels to match the likelihood values.
     • Note it's index of the pixels in a multi-resolution map, not the unique numbers of the pixels.
   • Add a new function zip_comp to generate the inputs for multiprocessing.
     • I used itertools.product before. However, it will cause duplicated fittings in MOCTSMap.
     • So I use zip_comp to replace it.

New module MOCTSMap.

1. This is the new module responsible for multi-resolution TS map fitting.
2. It checks the likelihood during each iteration, and only fit the child pixels of the mother pixels that meets the criterion.

Changes to the TS map tutorial

1. I renamed the tutorial notebook file name.
2. I added Example 4, which fits the Crab with the multi-resolution map.

Please let me know if you have any questions and suggestions. Thank you!

[codecov]

Codecov Report

Attention: Patch coverage is 31.70732% with 84 lines in your changes missing coverage. Please review.

Project coverage is 77.44%. Comparing base (c87577d) to head (2f6aeee).

Files with missing lines	Patch %	Lines
cosipy/ts_map/moc_ts_fit.py	22.33%	80 Missing ⚠️
cosipy/ts_map/fast_ts_fit.py	76.47%	4 Missing ⚠️

Files with missing lines	Coverage Δ
cosipy/__init__.py	100.00% <100.00%> (ø)
cosipy/ts_map/__init__.py	100.00% <100.00%> (ø)
cosipy/ts_map/fast_ts_fit.py	55.78% <76.47%> (+1.55%)	⬆️
cosipy/ts_map/moc_ts_fit.py	22.33% <22.33%> (ø)

[ldigesu]

Hello Yong, I looked at this tutorial, including the MOC-TS map. It works fine for me, I have only very minor suggestions from the point of view of the user. Apart from those, I think I can complete the review.

i) MOC-TS map: maybe add a markdown reminding the user of the meaning of the parameters max_moc_order and top_number. I think also it would be useful to print on the screen the delta_likelihood between the top pixels (that will continue to be fitted and higher resolution) and the others. So that the user can have a feeling of how significant is the source the the procedure is finding.

ii) Example 3, when the data are downloaded and binned. Maybe you can add another if statement checking for the existence of the .gz version of the file. The .gz file can also be unzipped on the fly using: import subprocess and subprocess.run(["gunzip", crab_unbinned_zipped_path]). The "shutils" command that you use before works only with .zip files, not with .gz. I would also a note to remind the user that the .ymal files have to be edited by hand (updating the paths) before proceeding.

iii) There are a few typos, for instance, in example 2, 1st cell: excpt>>execEpt

iv) Maybe you could consider splitting the tutorial into two notebooks e.g. one with the Crab and one with the GRB, or one with the Fast TS map and one with the MOC TS map.

[Yong2Sheng]

Hi @ldigesu,

Thank you for the review. I updated the PR based on your comments.

Add a markdown reminding the user of the meaning of the parameters max_moc_order and top_number.

I added comments to the cell to explain max_moc_order, top_number, and energy_channel parameters.

It would be useful to print on the screen the delta_likelihood between the top pixels (that will continue to be fitted and higher resolution) and the others?

I printed the top ts values.

Example 3, when the data are downloaded and binned. Maybe you can add another if statement checking for the existence of the .gz version of the file. The .gz file can also be unzipped on the fly using: import subprocess and subprocess.run(["gunzip", crab_unbinned_zipped_path]). The "shutils" command that you use before works only with .zip files, not with .gz. I would also a note to remind the user that the .ymal files have to be edited by hand (updating the paths) before proceeding.

• The notebook avoids downloading the gz files when they already exist.
• The DataIO can read and bin the gz file directly, so we don't have to unzip them.

There are a few typos, for instance, in example 2, 1st cell: excpt>>execEpt

• I didn't see excpt after searching the jupyter lab. Sorry, could you please remind me with the whole sentence with the typo?

Maybe you could consider splitting the tutorial into two notebooks e.g. one with the Crab and one with the GRB, or one with the Fast TS map and one with the MOC TS map

• I think we can put them together because they are the same process, so we can reuse the codes without explaining them again in another notebook.
• What do you think @israelmcmc ?

---

• I also updated the orientation-related cells so that they are compatible with the orientation with spacecraft attitude information.
• I noticed a warning when fetching the files from Wasabil. I have never seen this before. Not sure if we need to fix it now since it doesn't cause any errors @israelmcmc

Run:

%%capture
crab_unbinned_path = data_dir/"Crab_DC2_3months_unbinned_data.fits.gz"

# download 3-month unbinned Crab data ~619.22 MB
if not crab_unbinned_path.exists():
    fetch_wasabi_file("COSI-SMEX/DC2/Data/Sources/Crab_DC2_3months_unbinned_data.fits.gz", crab_unbinned_path)

Warning output:

DEBUG:awscli.plugin:Importing plugin __builtin__: awscli.handlers
DEBUG:awscli.plugin:Initializing plugin __builtin__: <module 'awscli.handlers' from '/Users/shengyong/conda_envs/moc_ts_map/lib/python3.10/site-packages/awscli/handlers.py'>
DEBUG:awscli.clidriver:OrderedDict([('bucket', <awscli.arguments.CLIArgument object at 0x17a258370>), ('if-match', <awscli.arguments.CLIArgument object at 0x17a258eb0>), ('if-modified-since', <awscli.arguments.CLIArgument object at 0x17a259570>), ('if-none-match', <awscli.arguments.CLIArgument object at 0x17a25b2e0>), ('if-unmodified-since', <awscli.arguments.CLIArgument object at 0x17a259a20>), ('key', <awscli.arguments.CLIArgument object at 0x17a259210>), ('range', <awscli.arguments.CLIArgument object at 0x17a2595a0>), ('response-cache-control', <awscli.arguments.CLIArgument object at 0x17a25b2b0>), ('response-content-disposition', <awscli.arguments.CLIArgument object at 0x17a25b010>), ('response-content-encoding', <awscli.arguments.CLIArgument object at 0x17a25bd60>), ('response-content-language', <awscli.arguments.CLIArgument object at 0x17a259450>), ('response-content-type', <awscli.arguments.CLIArgument object at 0x17a25b190>), ('response-expires', <awscli.arguments.CLIArgument object at 0x17a259690>), ('version-id', <awscli.arguments.CLIArgument object at 0x17a2599f0>), ('sse-customer-algorithm', <awscli.arguments.CLIArgument object at 0x17a25b760>), ('sse-customer-key', <awscli.arguments.CLIArgument object at 0x17a25a1a0>), ('sse-customer-key-md5', <awscli.arguments.CLIArgument object at 0x17a25ae30>), ('request-payer', <awscli.arguments.CLIArgument object at 0x17a25add0>), ('part-number', <awscli.arguments.CLIArgument object at 0x17a25abc0>), ('expected-bucket-owner', <awscli.arguments.CLIArgument object at 0x17a25a800>), ('checksum-mode', <awscli.arguments.CLIArgument object at 0x17a258d90>)])
DEBUG:awscli.arguments:Unpacked value of 'cosi-pipeline-public' for parameter "bucket": 'cosi-pipeline-public'
DEBUG:awscli.arguments:Unpacked value of 'COSI-SMEX/DC2/Data/Sources/Crab_DC2_3months_unbinned_data.fits.gz' for parameter "key": 'COSI-SMEX/DC2/Data/Sources/Crab_DC2_3months_unbinned_data.fits.gz'
DEBUG:awscli.formatter:RequestId: E02D97B6F5DB2710:B