Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Read and write EPICS Archiver Appliance Protocol Buffer files #2

Merged
merged 18 commits into from
Jan 7, 2025

Conversation

jacob720
Copy link
Collaborator

Created a PBUtils class to read and write EPICS Archiver Appliance Protocol Buffer (PB) files. These files are structured in a specific way, the details of which can be found here. This class handles the interpretation of PB files according to this structure so that the contents can be manipulated before being written back. It can also convert a PB file to text to make it human-readable.

Jacob Williamson added 11 commits December 12, 2024 11:29
PB files in the Archiver Appliance have one message per line,
and special characters are escaped. This requires functions for
escaping and unescaping, and removing and adding newline symbols
when reading and writing.
@jacob720 jacob720 self-assigned this Dec 12, 2024
Copy link

codecov bot commented Dec 12, 2024

Codecov Report

Attention: Patch coverage is 0% with 99 lines in your changes missing coverage. Please review.

Project coverage is 10.81%. Comparing base (29a54e9) to head (d86c9f5).
Report is 19 commits behind head on main.

Files with missing lines Patch % Lines
src/aa_remove_data/pb_utils.py 0.00% 99 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##              main       #2       +/-   ##
============================================
- Coverage   100.00%   10.81%   -89.19%     
============================================
  Files            2        3        +1     
  Lines           12      111       +99     
============================================
  Hits            12       12               
- Misses           0       99       +99     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.



class PBUtils:
def __init__(self, filepath: PathLike | None = None):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

filepath here is optional, but I don't know how useful this class is without it. I would suggest making this a required input and then assigning it to a member variable (self.filepath). That way in the subsequent member functions you can call self.filepath rather than taking the filepath in as an input.

An additional function called 'update_filepath' or similar can also be added to change the target PB file on the fly. This way we are only using the API provided by PBUtils.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a few different filepaths, i.e one to read from, one to write to .txt, one to write to .pb. Should all of these be a member variable?

sample_class = getattr(EPICSEvent_pb2, sample_type_camel)
return sample_class

def write_to_txt(self, filepath: PathLike):
Copy link
Collaborator

@MichaelStubbings MichaelStubbings Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an example, write_to_txt can now just take self as an input and use the self.filepath property in its functionality.

Copy link
Collaborator

@MichaelStubbings MichaelStubbings left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall the code is clean, reads well and is very pythonic. I've left some comments with the aim of trying to provide an API for this utility. I think it would be useful to also include an entrypoint and argparse functionality so that this can be used as a separate command line tool.

Feel free to come and talk to me if any of the comments are confusing, or you would like to discuss your implementation.

@jacob720 jacob720 force-pushed the HLA-809-read-write-pb branch from 5fa25e8 to f4def8f Compare December 16, 2024 10:13
@jacob720 jacob720 force-pushed the HLA-809-read-write-pb branch from f4def8f to 9c317ff Compare December 16, 2024 10:19
pyproject.toml Outdated Show resolved Hide resolved
@jacob720 jacob720 merged commit 2934e56 into main Jan 7, 2025
15 of 17 checks passed
@jacob720 jacob720 deleted the HLA-809-read-write-pb branch January 7, 2025 13:21
@jacob720 jacob720 restored the HLA-809-read-write-pb branch January 9, 2025 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants