-
Notifications
You must be signed in to change notification settings - Fork 3
/
Copy pathDocumentation
1848 lines (1345 loc) · 73.1 KB
/
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
This is documentation for the suite of programs I am writing.
Jan 8 2019 - first big add ; added onto it from time to time...
Jan 12 2020 - biggest revision to the documentation with the addition/removal of functions
################################## LEVEL 0 ####################################
Base level - calling/processing of data
>>>>>>>>>> Lv0_dirs <<<<<<<<<<
Functions:
##### global_par
-- Defining global variables for the directories
-- has BASE_DIR, NICER_DATADIR, NICERSOFT_DATADIR, and NGC300
>>>>>>>>>> Lv0_fits2dict <<<<<<<<<<
Functions:
##### fits2dict(fits_file,ext,par_list)
-- 'Converts' a FITS file to a Python dictionary, with a list of the original
FITS table's columns, of the user's choosing. Can use this for ANY FITS file,
but is meant for mkf/orb files in $OBSID_pipe folders from NICER, or event files
(be it from NICER-data or NICERsoft_outputs or any other mission which has this format)
fits_file - path to the FITS file
ext - which extension number; 1 for EVENTS, 2 for GTI, 3 for PPS_TREND
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI)
TEST:
eventfile = "/Volumes/Samsung_T5/NICERsoft_outputs/1034070101_pipe/cleanfilt.evt"
print(fits2dict(eventfile,1,['TIME','RAWX']))
print(fits2dict(eventfile,2,['START','STOP']))
>>>>>>>>>> Lv0_gunzip <<<<<<<<<<
Functions:
##### unzip_all(obsdir)
-- Does a recursive scan through the directory and unzips all files which have not yet been unzipped
obsdir - directory of all the observation files
TEST:
unzip_all(Lv0_dirs.NICER_DATADIR+'1034070101')
>>>>>>>>>> Lv0_nicer_housekeeping <<<<<<<<<<
Functions:
##### get_att(eventfile,par_list)
-- Getting data from the .att FITS file! Just provide a path to the event file.
I can also input the NICERsoft-output event files because this script will
just search for the ObsID and trawl through the NICER-data folder.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI)
##### get_cat(eventfile,par_list)
-- Getting data from the .cat FITS file! Just provide a path to the event file.
I can also input the NICERsoft-output event files because this script will
just search for the ObsID and trawl through the NICER-data folder.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI)
##### get_mkf(eventfile,par_list)
-- Getting data from the .mkf FITS file! Just provide a path to the event file.
I can also input the NICERsoft-output event files because this script will
just search for the ObsID and trawl through the NICER-data folder.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI)
##### get_orb(eventfile,par_list)
-- Getting data from the .orb FITS file! Just provide a path to the event file.
I can also input the NICERsoft-output event files because this script will
just search for the ObsID and trawl through the NICER-data folder.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI)
##### get_hk(eventfile,mpu_no,par_list)
-- Getting data from the .hk FITS file! Just provide a path to the event file.
I can also input the NICERsoft-output event files because this script will
just search for the ObsID and trawl through the NICER-data folder.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
mpu_no - MPU number, from 0 to 6 inclusive. For the 7 MPUs.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI)
##### get_uf(eventfile,mpu_no,ext,par_list)
-- Getting data from the .uf FITS file! Just provide a path to the event file.
I can also input the NICERsoft-output event files because this script will
just search for the ObsID and trawl through the NICER-data folder.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
mpu_no - MPU number, from 0 to 6 inclusive. For the 7 MPUs.
ext - which extension number; 1 for EVENTS, 2 for GTI, 3 for PPS_TREND
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI)
##### get_ufa(eventfile,mpu_no,ext,par_list)
-- Getting data from the .ufa FITS file! Just provide a path to the event file.
I can also input the NICERsoft-output event files because this script will
just search for the ObsID and trawl through the NICER-data folder.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
mpu_no - MPU number, from 0 to 6 inclusive. For the 7 MPUs.
MPU number 7 corresponds to the COMBINED file!
ext - which extension number; 1 for EVENTS, 2 for GTI
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI)
TEST:
eventfile = '/Volumes/Samsung_T5/NICER-data/1034070101/xti/event_cl/ni1034070101_0mpu7_cl.evt'
print(get_att(eventfile,['TIME','QPARAM']))
print(get_cat(eventfile,['FILENAME','FORMAT']))
print(get_mkf(eventfile,['TIME','NICER_SAA','ANG_DIST']))
print(get_orb(eventfile,['TIME','Vx','Vy']))
print(get_hk(eventfile,'3',['TIME','GIT_HASH']))
print(get_uf(eventfile,'2',1,['TIME','RAWX','DEADTIME']))
print(get_ufa(eventfile,'7',1,['TIME','MPU_UNDER_COUNT','PI_RATIO']))
>>>>>>>>>> Lv0_nicerl2 <<<<<<<<<<
Functions:
##### nicerl2(obsdir,nicerl2_flags)
-- Running nicerl2 to do initial filtering of the ufa file!
obsdir - NICER data directory containing all the data files (e.g., path_to_NICER_dir/1034070101)
nicerl2_flags - a LIST of input flags for nicerl2
TEST:
obsid = '1034070101'
obsdir = Lv0_dirs.NICER_DATADIR + obsid
nicerl2(obsdir,['clobber=YES'])
>>>>>>>>>> Lv0_psrpipe <<<<<<<<<<
Functions:
##### psrpipe(eventfile,flags)
-- Running psrpipe on the observation, to make more cuts! I decided not to
put in pre-determined options for fully flexibility. Though standard flags
would be ['--emin','0.3','--emax','12.0','--shrinkelvcut'], though there are
others. Check out "psrpipe.py -h"! Also made sure that I moved $OBSID_pipe from
the working directory to where NICERSOFT_DATADIR is, though I need to temporarily
store the output folder in the working directory.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
flags - a LIST of input flags for psrpipe
TEST:
obsid = '1034070101'
eventfile = Lv0_dirs.NICER_DATADIR + obsid + '/xti/event_cl/ni' + obsid+'_0mpu7_cl.evt'
psrpipe(eventfile,['--emin','0.3','--emax','12.0'])
>>>>>>>>>> Lv0_scp <<<<<<<<<<
Functions:
##### scp(obsid)
-- To securely copy the files from ciri onto /Volumes/Samsung_T5/NICER-data/
obsid - Observation ID of the object of interest (10-digit str)
TEST:
for i in range(11,25):
scp('10600601' + str(i)) WORKED.
--------------------------------------------------------------------------------
################################################################################
--------------------------------------------------------------------------------
################################## LEVEL 1 #####################################
>>>>>>>>>> Lv1_barycorr <<<<<<<<<<
Functions:
##### get_ra_dec(eventfile)
-- Obtain the RA_OBJ and DEC_OBJ corresponding to the observation!
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
##### read_par(parfile)
-- Function that reads a par file. In particular, for the purposes of barycorr,
it will return POSEPOCH, RAJ, DECJ, PMRA, and PMDEC.
Step 1: Read par file line by line, where each line is stored as a string in the 'contents' array
Step 2a: For PSRJ, RAJ, DECJ, PMRA, and PMDEC, those lines are teased out
Step 2b: The corresponding strings are split up without whitespace
Step 3: Extract the values accordingly
parfile - path of the .par file
##### barycorr(eventfile,outfile,refframe,orbit_file,parfile,output_folder)
-- General function to perform the barycenter corrections for an event file
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
refframe - reference frame for barycenter corrections (usually ICRS)
parfile - name of the .par file
TEST:
obsid = '1911212213'
eventfile = Lv0_dirs.NICER_DATADIR + 'rxj0209/rxj0209kgfilt.evt'
outfile = Lv0_dirs.NICER_DATADIR + 'rxj0209/rxj0209kgfilt_bary.evt'
orbitfile = Lv0_dirs.NICER_DATADIR + 'rxj0209/rxj0209.orb'
parfile = ''
output_folder = Lv0_dirs.NICER_DATADIR + 'rxj0209/'
refframe = 'ICRS'
barycorr(eventfile,outfile,refframe,orbitfile,parfile,output_folder)
>>>>>>>>>> Lv1_data_bin <<<<<<<<<<
Functions:
##### binning_t(eventfile,par_list,tbin_size,t1,t2)
-- Binning routine for when I truncate the data by JUST time interval.
Got to make sure I have TIME and PI called!
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means bin by 0.05s!
t1 - lower time boundary
t2 - upper time boundary
##### binning_E(eventfile,par_list,tbin_size,Ebin_size,E1,E2)
-- Binning routine for when I truncate the data by JUST energy range.
Got to make sure I have TIME and PI called!
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means bin by 0.05s!
Ebin_size - the size of the energy bins (in keV!)
>> e.g., Ebin_size = 0.1 means bin by 0.1keV
>> e.g., Ebin_size = 0.05 means bin by 0.05keV
E1 - lower energy boundary
E2 - upper energy boundary
##### binning_tE(eventfile,par_list,tbin_size,Ebin_size,t1,t2,E1,E2)
-- Binning routine for when I truncated the data by BOTH time interval AND energy range.
Got to make sure I have TIME and PI called!
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means bin by 0.05s!
Ebin_size - the size of the energy bins (in keV!)
>> e.g., Ebin_size = 0.1 means bin by 0.1keV
>> e.g., Ebin_size = 0.05 means bin by 0.05keV
t1 - lower time boundary
t2 - upper time boundary
E1 - lower energy boundary
E2 - upper energy boundary
TEST:
obsid = '1034070101'
eventfile = Lv0_dirs.NICER_DATADIR + obsid + '/xti/event_cl/ni' + obsid + '_0mpu7_cl_bary.evt'
par_list = ['TIME','PI','PI_RATIO']
t1 = 0
t2 = 300
E1 = 0.3
E2 = 6
tbin_size = 1
Ebin_size = 0.05
tbins,summed_data = binning_t(eventfile,par_list,tbin_size,t1,t2)
#print(len(tbins),len(summed_data))
tbins,summed_t_data,Ebins,summed_E_data = binning_E(eventfile,par_list,tbin_size,Ebin_size,E1,E2)
#print(len(tbins),len(summed_t_data),len(Ebins),len(summed_E_data))
tbins,summed_t_data,Ebins,summed_E_data = binning_tE(eventfile,par_list,tbin_size,Ebin_size,t1,t2,E1,E2)
#print(len(tbins),len(summed_t_data),len(Ebins),len(summed_E_data))
>>>>>>>>>> Lv1_data_filter <<<<<<<<<<
Functions:
##### filter_time(eventfile,par_list,t1,t2)
-- Obtain the time stamps that fall in a desired time interval.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means bin by 0.05s!
t1 - time value for the lower boundary (in s)
t2 - time value for the upper boundary (in s)
##### filter_energy(eventfile,par_list,E1,E2)
-- Obtain the time stamps and the corresponding energy value (of the photon)
in a desired energy range.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means bin by 0.05s!
E1 - energy value for the lower boundary (in keV)
E2 - energy value for the upper boundary (in keV)
##### filter_data(eventfile,par_list,t1,t2,E1,E2)
-- Truncate the data such that you get counts in a given time interval and
energy range.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means bin by 0.05s!
t1 - time value for the lower boundary (in s)
t2 - time value for the upper boundary (in s)
E1 - energy value for the lower boundary (in keV)
E2 - energy value for the upper boundary (in keV)
TEST:
obsid = '1034070101'
eventfile = Lv0_dirs.NICER_DATADIR + obsid + '/xti/event_cl/ni' + obsid + '_0mpu7_cl_bary.evt'
par_list = ['TIME','PI','PI_RATIO']
t1 = 0
t2 = 300
E1 = 0.3
E2 = 6
tcut = filter_time(eventfile,par_list,t1,t2)
#print(len(tcut))
tcut,E_cut = filter_energy(eventfile,par_list,E1,E2)
#print(len(tcut),len(E_cut))
tcut,E_cut = filter_data(eventfile,par_list,t1,t2,E1,E2)
#print(len(tcut),len(E_cut))
>>>>>>>>>> Lv1_ngc300_binning <<<<<<<<<<
Functions:
##### get_binned_data(counts_dict,err_dict)
-- Given the dictionaries where the data (counts and uncertainty) are already
binned, put them into lists!
counts_dict - dictionary where the keys are MJD values, and the entries correspond
to the counts/rate for a given MJD
err_dict - dictionary where the keys are MJD values, and the entries correspond
to the UNCERTAINTY in the counts/rate for a given MJD
##### binned_text()
-- Given the MJDs, binned counts, and associated uncertainties, put them into a text file
No arguments because I'll put all the bands in here
>>>>>>>>>> Lv1_ngc300_mathgrp_pha <<<<<<<<<<
Functions:
##### mathpha(bin_size,filetype)
-- Function that takes in a bin size, and does MATHPHA on the set of pha files.
The file names are already saved in the binned .ffphot files. The function
will output pha files of the format 'MJD_binsize_' + filetype + '_cl50.pha'!
bin_size - bin size in days
filetype - either 'bgsub' or 'bg' or 'cl'!
##### grppha(bin_size,filetype)
-- Function that takes in a bin size, and does GRPPHA on a set of pha files.
The input file names will be "$MJD_$binsize_bgsub_cl50.pha". The function
will output pha files of the format 'grp_$MJD_$binsize_$filetype_cl50.pha'!
bin_size - bin size in days
filetype - either 'bgsub' or 'bg' or 'cl'!
TEST:
#mathpha(bin_size,'bgsub')
grppha(bin_size,'bgsub')
#bg_mathpha(bin_size)
#bg_grppha(bin_size)
>>>>>>>>>> Lv1_spectra_txt <<<<<<<<<<
>>>>>>>>>> Lv1_spectral_renorm <<<<<<<<<<
--------------------------------------------------------------------------------
################################################################################
--------------------------------------------------------------------------------
################################## LEVEL 2 #####################################
>>>>>>>>>> Lv2_average_ps_methods <<<<<<<<<<
Functions:
##### do_demodulate(eventfile,segment_length,mode,par_file)
-- Do orbital demodulation on the original events.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
segment_length - length of the segments
par_file - orbital parameter file for input into binary_psr
mode - "all", "t" or "E" ; basically to tell the function where to access files to run do_demodulate
##### do_nicerfits2presto(eventfile,tbin,segment_length)
-- Using nicerfits2presto.py to bin the data, and to convert into PRESTO-readable format.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
tbin - size of the bins in time
segment_length - length of the individual segments for combining power spectra
##### edit_inf(eventfile,tbin,segment_length)
-- Editing the .inf file, as it seems like accelsearch uses some information from the .inf file!
Mainly need to edit the "Number of bins in the time series".
This is only for when we make segments by time though!
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
tbin - size of the bins in time
segment_length - length of the individual segments
##### edit_binary(eventfile,tbin,segment_length)
-- To pad the binary file so that it will be as long as the desired segment length.
The value to pad with for each time bin, is the average count rate in THAT segment!
Jul 10: Do zero-padding instead... so that number of counts is consistent!
Again, this is only for when we make segments by time!
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
tbin - size of the bins in time
segment_length - length of the individual segments
##### realfft(eventfile,segment_length)
-- Performing PRESTO's realfft on the binned data (.dat)
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
segment_length - length of the individual segments
##### presto_dat(eventfile,segment_length,demod)
-- Obtain the dat files that were generated from PRESTO
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
segment_length - length of the segments
demod - whether we're dealing with demodulated data or not!
##### presto_fft(eventfile,segment_length,demod)
-- Obtain the FFT files that were generated from PRESTO
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
segment_length - length of the segments
demod - whether we're dealing with demodulated data or not!
##### segment_threshold(eventfile,segment_length,demod,tbin_size,threshold)
-- Using the .dat files, rebin them into 1s bins, to weed out the segments below
some desired threshold. Will return a *list* of *indices*! This is so that I
can filter out the *sorted* array of .dat and .fft files that are below threshold!
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
segment_length - length of the segments
demod - whether we're dealing with demodulated data or not!
tbin_size - size of the time bin
threshold - if data is under threshold (in percentage), then don't use the segment!
##### average_ps(eventfile,segment_length,demod,tbin_size,threshold,starting_freq,W)
-- Given the full list of .dat and .fft files, and the indices where the PRESTO-binned
data is beyond some threshold, return the averaged power spectrum!
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
segment_length - length of the segments
demod - whether we're dealing with demodulated data or not!
tbin_size - size of the time bin
threshold - if data is under threshold (in percentage), then don't use the segment!
W - number of consecutive frequency bins to AVERAGE over
##### noise_hist(eventfile,segment_length,demod,tbin_size,threshold,starting_freq,W)
-- Given the average spectrum for an ObsID, return the histogram of powers, such
that you have N(>P). This is for powers corresponding to frequencies larger
than some starting frequency (perhaps to avoid red noise).
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
segment_length - length of the segments
demod - whether we're dealing with demodulated data or not!
tbin_size - size of the time bin
threshold - if data is under threshold (in percentage), then don't use the segment!
starting_freq - frequency to start constructing the histogram of powers from
W - number of consecutive frequency bins to AVERAGE over
TEST:
eventfile = Lv0_dirs.NICERSOFT_DATADIR + '1034070101_pipe/ni1034070101_nicersoft_bary.evt'
mode = 't'
segment_length = 100
par_file = Lv0_dirs.NICERSOFT_DATADIR + 'J1231-1411.par'
do_demodulate(eventfile,segment_length,mode,par_file)
>>>>>>>>>> Lv2_color <<<<<<<<<<
Functions:
##### soft_counts(E_bound,pi_data)
-- Will get an array of PI values from the data, where each entry = 1 count.
So construct an array of ones of equal length, then where E >= E_bound, set to 0.
This will give an array where 0 = harder X-rays, 1 = softer X-rays, so when
doing the binning, will get just soft counts.
E_bound - boundary energy considered (in keV)
pi_data - array of PI values
##### hard_counts(E_bound,pi_data)
-- Will get an array of PI values from the data, where each entry = 1 count.
So construct an array of ones of equal length, then where E < E_bound, set to 0.
This will give an array where 0 = harder X-rays, 1 = softer X-rays, so when
doing the binning, will get just soft counts.
E_bound - boundary energy considered (in keV)
pi_data - array of PI values
##### get_color(eventfile,par_list,E_bound,tbin_size)
-- Calculating the color - hard/soft and (hard-soft)/(hard+soft)
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
E_bound - boundary energy considered (in keV)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means bin by 0.05s!
##### get_color_t(eventfile,par_list,E_bound,tbin_size,t1,t2)
-- Calculating the color - hard/soft and (hard-soft)/(hard+soft)
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
E_bound - boundary energy considered (in keV)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means bin by 0.05s!
t1 - lower time boundary
t2 - upper time boundary
##### plotting(eventfile,par_list,E_bound,tbin_size,mode)
-- Plotting the hardness ratio/color diagrams.
t_bins,color,color_diff = get_color(eventfile,par_list,E_bound,tbin_size)
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
E_bound - boundary energy considered (in keV)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means bin by 0.05s!
mode - whether we want to show or save the plot.
##### plotting_t(eventfile,par_list,E_bound,tbin_size,t1,t2,mode)
-- Plotting the hardness ratio/color diagrams.
t_bins,color,color_diff = get_color_t(eventfile,par_list,E_bound,tbin_size)
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
E_bound - boundary energy considered (in keV)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means bin by 0.05s!
t1 - lower time boundary
t2 - upper time boundary
mode - whether we want to show or save the plot.
TEST:
eventfile = '/Volumes/Samsung_T5/NICERsoft_outputs/1034070101_pipe/cleanfilt.evt'
plotting(eventfile,['TIME','PI','PI_FAST'],3,1,'save')
plotting_t(eventfile,['TIME','PI','PI_FAST'],3,1,200,400,'save')
>>>>>>>>>> Lv2_efsearch <<<<<<<<<<
Functions:
##### efsearch(eventfile,n_segments,dper,nphase,nbint,nper,dres,outfile_root,plot_efsearch)
Performing FTOOLS' efsearch!
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
n_segments - no. of segments to break the epoch folding search into
dper - value for period used in the folding; input represents center of range of trial periods
nphase - no. of phases in the folded light curve(s) ; "INDEF" uses the default value
nbint - the number of newbins per interval used in the analysis
nper - no. of periods over which the search is carried out
dres - period resolution is the spacing between two contiguous periods in the search
"INDEF" uses default value of half the Fourier resolution in the interval
outfile_root - prefix for the end file name
plot_efsearch - 'yes' or 'no' to plot the results from efsearch; do "exit" for the next plot!
TEST:
out_baryfile = Lv0_dirs.NICER_DATADIR + 'rxj0209/rxj0209kgfilt_bary.evt'
n_segments = 10 #number of segments to break the epoch folding search into
dper = 9.3 #Value for the period used in the folding. In 'efsearch' the
#input period represents the centre of the range of the trial periods.
nphase = 32 #Number of phases in the folded light curve(s). Typing 'INDEF'
#forces the task to use the default value (see parameter "nbdf").
nbint = int((T/(dper/nphase))/n_segments) # The number of newbins per interval used in the analysis. The
#"nbint" together with the NEWBIN duration determines the length in time of an interval
#and therefore the total number of intervals within the start and stop time over which the
#analysis will be carried out. Typing 'INDEF' forces the task to use the default value
#(see parameter "nbdf"). NOTE: By pressing return "nbint" is set to the value found in the
#parameter file used in a previous run."
nper = 128 #The number of periods over which the search is carried out
dres = 1E-4 # The period resolution is the spacing between two contiguous periods in the search.
#'INDEF' uses the default value of: half the Fourier resolution in the interval (e.g., P^2/T(i)/2 ; T(i) is interval duration)
outfile_root = "testefsearch"
plot_efsearch = 'no' #to plot the results from efsearch ; do "exit" to see the next plot!
Lv2_efsearch.efsearch(out_baryfile,n_segments,dper,nphase,nbint,nper,dres,outfile_root,plot_efsearch)
>>>>>>>>>> Lv2_lc <<<<<<<<<<
Functions:
##### whole(eventfile,par_list,tbin_size,mode)
-- Plot the entire raw time series without any cuts to the data.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means bin by 0.05s!
mode - whether we want to show or save the plot.
##### partial_t(eventfile,par_list,tbin_size,t1,t2,mode)
-- Plot the time series for a desired time interval.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means bin by 0.05s!
t1 - lower time boundary
t2 - upper time boundary
mode - whether we want to show or save the plot
##### partial_E(eventfile,par_list,tbin_size,E1,E2,mode)
-- Plot the time series for a desired energy range.
[Though I don't think this will be used much. Count/s vs energy is pointless,
since we're not folding in response matrix information here to get the flux.
So we're just doing a count/s vs time with an energy cut to the data.]
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means by in 0.05s
E1 - lower energy boundary
E2 - upper energy boundary
##### partial_tE(eventfile,par_list,tbin_size,t1,t2,E1,E2,mode)
-- Plot the time series for a desired time interval and desired energy range.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means by in 0.05s
t1 - lower time boundary
t2 - upper time boundary
E1 - lower energy boundary
E2 - upper energy boundary
TEST:
eventfile = '/Volumes/Samsung_T5/NICERsoft_outputs/1034070101_pipe/cleanfilt.evt'
whole(eventfile,['TIME','PI','PI_FAST'],1,'save')
partial_t(eventfile,['TIME','PI','PI_FAST'],1,0,400,'save')
partial_E(eventfile,['TIME','PI','PI_FAST'],1,0.3,6,'save')
partial_tE(eventfile,['TIME','PI','PI_FAST'],1,0,200,0.3,6,'save')
>>>>>>>>>> Lv2_merged_pulse_methods <<<<<<<<<<
Functions:
##### niextract_gti_energy(merging,data_id,PI1,PI2)
-- Using niextract-events to get segmented data based on the energy range
merging - True/False - whether to use merged data
data_id - 10-digit ObsID or 6-digit ID for the merged event file
PI1 - lower bound of PI (not energy in keV!) desired for the energy range
PI2 - upper bound of PI (not energy in keV!) desired for the energy range
##### do_demodulate(merging,data_id,par_file,E_trunc,PI1,PI2)
-- Using do_demodulate in binary_psr.py in Scott Ransom's PRESTO Python library to
demodulate the time series for the merged event file!
merging - True/False - whether to use merged data
data_id - 10-digit ObsID or 6-digit ID for the merged event file
par_file - orbital parameter file for input into binary_psr
E_trunc - True/False - whether to do energy truncation
PI1 - lower bound of PI (not energy in keV!) desired for the energy range
PI2 - upper bound of PI (not energy in keV!) desired for the energy range
##### pulse_profile(merging,data_id,E_trunc,PI1,PI2,pulse_pars,no_phase_bins)
-- Extracts the time series from the demodulated merged event file, and creates
the pulse profile from Lv2_phase.py!
merging - True/False - whether to use merged data
data_id - 10-digit ObsID or 6-digit ID for the merged event file
E_trunc - True/False - whether to do energy truncation
PI1 - lower bound of PI (not energy in keV!) desired for the energy range
PI2 - upper bound of PI (not energy in keV!) desired for the energy range
pulse_pars - parameters corresponding to the pulse
no_phase_bins - number of phase bins desired
##### plot_pf(merging,data_id,E_trunc,PI1,PI2,pulse_pars,no_phase_bins)
-- Extracts the time series from the demodulated merged event file, and creates
the pulse profile from Lv2_phase.py!
merging - True/False - whether to use merged data
data_id - 10-digit ObsID or 6-digit ID for the merged event file
E_trunc - True/False - whether to do energy truncation
PI1 - lower bound of PI (not energy in keV!) desired for the energy range
PI2 - upper bound of PI (not energy in keV!) desired for the energy range
pulse_pars - parameters corresponding to the pulse
no_phase_bins - number of phase bins desired
TEST:
merging = False
data_id = '1013010105'
E_trunc = False
PI1 = 30
PI2 = 1200
pulse_pars = [29.639575,-3.77535E-10,1.1147E-20]
no_phase_bins = 50
plot_pf(merging,data_id,E_trunc,PI1,PI2,pulse_pars,no_phase_bins)
>>>>>>>>>> Lv2_merging_events <<<<<<<<<<
Functions:
##### merging(obsids)
-- Given a list of ObsIDs, create a file which merges all the rows in the EVENTS
extension of the FITS files!
obsids - list (or array) of ObsIDs
##### merging_GTIs(obsids,merged_id)
-- Given a list of ObsIDs and the merged_id, create the final event file, which
already has all the rows in the EVENTS extension of the FITS files, BUT also
including the GTI extension of the FITS files!
obsids - list (or array) of ObsIDs
merged_id - 6-digit ID for the merged event file
TEST:
obsids = ['2060060363','2060060364','2060060365']
merged_id = '000013'
merging(obsids)
merging_GTIs(obsids,merged_id)
>>>>>>>>>> Lv2_mkdir <<<<<<<<<<
Functions:
##### makedir(dir)
Creating a folder if it does not exist in the directory.
dir - desired directory (provide FULL path!)
TEST:
makedir('/Volumes/Samsung_T5/hahaha')
>>>>>>>>>> Lv2_ngc300_color <<<<<<<<<<
Functions:
##### get_color(bin_size,band1,band2)
-- Obtain colors and the corresponding uncertainties. Will NOT use values where
either the counts/rate from band1 OR band2 are negative! Allowed band values are
"soft1, soft2, A, B, C, D, and inband."
bin_size - binning size desired (1 day or 10 days, for example)
band1 - energy band 1
band2 - energy band 2
>>>>>>>>>> Lv2_phase <<<<<<<<<<
Functions:
##### pulse_profile(f_pulse,times,counts,shift,no_phase_bins)
-- Calculating the pulse profile for the observation. Goes from 0 to 2!
f_pulse - the frequency of the pulse
times - the array of time values
counts - the array of counts values
shift - how much to shift the pulse by in the phase axis.
It only affects how it is presented.
no_phase_bins - number of phase bins desired
##### pulse_folding(t,T,T0,f,fdot,fdotdot,no_phase_bins)
-- Calculating the pulse profile by also incorporating \dot{f} corrections!
Goes from 0 to 2.
t - array of time values
T - sum of all the GTIs
T0 - reference epoch in MJD
f - pulse/folding Frequency
fdot - frequency derivative
fdotdot - second derivative of frequency
no_phase_bins - number of phase bins desired (recommended 20!)
Returns the pulse profile in counts/s/phase bin vs phase. The number of counts
is divided by the exposure time (calculated through total sum of the GTIs)
Also added a "TIMEZERO" manually in the script since it'd be inconvenient to call the eventfile here.
##### whole(eventfile,par_list,tbin_size,pulse_pars,shift,no_phase_bins,mode)
-- Plot the entire raw pulse profile without any cuts to the data.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means bin by 0.05s!
pulse_pars - parameters corresponding to the pulse
shift - how much to shift the pulse by in the phase axis.
It only affects how the pulse profile is 'displaced'.
no_phase_bins - number of phase bins desired
mode - whether we want to show or save the plot.
pulse_pars will have [f,fdot,fdotdot]
##### partial_t(eventfile,par_list,tbin_size,pulse_pars,shift,no_phase_bins,t1,t2,mode)
-- Plot the pulse profile for a desired time interval.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means bin by 0.05s!
pulse_pars - parameters corresponding to the pulse
shift - how much to shift the pulse by in the phase axis.
It only affects how it is presented.
no_phase_bins - number of phase bins desired
t1 - lower time boundary
t2 - upper time boundary
mode - whether we want to show or save the plot
pulse_pars will have [f,fdot,fdotdot]
##### partial_E(eventfile,par_list,tbin_size,Ebin_size,pulse_pars,shift,no_phase_bins,E1,E2,mode)
-- Plot the pulse profile for a desired energy range.
[Though I don't think this will be used much. Count/s vs energy is pointless,
since we're not folding in response matrix information here to get the flux.
So we're just doing a count/s vs time with an energy cut to the data.]
INTERJECTION: This caveat is for the spectrum, NOT the pulse profile!
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means by in 0.05s
Ebin_size - the size of the energy bins (in keV!)
>> e.g., Ebin_size = 0.1 means bin by 0.1keV
>> e.g., Ebin_size = 0.01 means bin by 0.01keV!
pulse_pars - parameters corresponding to the pulse
shift - how much to shift the pulse by in the phase axis.
It only affects how it is presented.
no_phase_bins - number of phase bins desired
E1 - lower energy boundary
E2 - upper energy boundary
pulse_pars will have [f,fdot,fdotdot]
##### partial_tE(eventfile,par_list,tbin_size,Ebin_size,pulse_pars,shift,no_phase_bins,t1,t2,E1,E2,mode)
-- Plot the pulse profile for a desired time interval and desired energy range.
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means by in 0.05s
Ebin_size - the size of the energy bins (in keV!)
>> e.g., Ebin_size = 0.1 means bin by 0.1keV
>> e.g., Ebin_size = 0.01 means bin by 0.01keV!
pulse_pars - parameters corresponding to the pulse
shift - how much to shift the pulse by in the phase axis.
It only affects how it is presented.
no_phase_bins - number of phase bins desired
t1 - lower time boundary
t2 - upper time boundary
E1 - lower energy boundary
E2 - upper energy boundary
mode - whether we want to show or save the plot
pulse_pars will have [f,fdot,fdotdot]
##### partial_subplots_E(eventfile,par_list,tbin_size,Ebin_size,f_pulse,shift,no_phase_bins,subplot_Es,E1,E2,mode)
-- Plot the pulse profile for a desired energy range.
[Though I don't think this will be used much. Count/s vs energy is pointless,
since we're not folding in response matrix information here to get the flux.
So we're just doing a count/s vs time with an energy cut to the data.]
INTERJECTION: This caveat is for the spectrum, NOT the pulse profile!
eventfile - path to the event file. Will extract ObsID from this for the NICER files.
par_list - A list of parameters we'd like to extract from the FITS file
(e.g., from eventcl, PI_FAST, TIME, PI,)
tbin_size - the size of the time bins (in seconds!)
>> e.g., tbin_size = 2 means bin by 2s
>> e.g., tbin_size = 0.05 means by in 0.05s
Ebin_size - the size of the energy bins (in keV!)
>> e.g., Ebin_size = 0.1 means bin by 0.1keV
>> e.g., Ebin_size = 0.01 means bin by 0.01keV!
f_pulse - the frequency of the pulse
shift - how much to shift the pulse by in the phase axis.
It only affects how it is presented.
no_phase_bins - number of phase bins desired
subplot_Es - list of tuples defining energy boundaries for pulse profiles
E1 - lower energy boundary
E2 - upper energy boundary