Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aster converts triple-quote comment to single-quote comment if \x00 is in comment. #225

Open
medined opened this issue Jul 3, 2024 · 3 comments

Comments

@medined
Copy link

medined commented Jul 3, 2024

I am using ast to parse a python file. Then using astor to print a specific function. When the function has the following docstring

        """
            \x00
        """

it is converted to

'\n            \x00\n        '

by astor.

This causes the python code to fail pylint because the docstring is no longer correct.

P.S. The \x00 in the python comment is part of a longer comment that discusses binary data.

@medined
Copy link
Author

medined commented Jul 3, 2024

I have verified that if my original comment is \x00 then the triple comment is maintained.

@medined
Copy link
Author

medined commented Jul 7, 2024

The issue lies in the following code in string_expr.py:

    try:
        if eval(fancy) == s and '\r' not in fancy:
            return fancy
    except Exception:
        pass

The "source code string cannot contain null bytes" is swallowed. This took me about 4 hours to find. Please send the exception to an error log.

@medined
Copy link
Author

medined commented Jul 7, 2024

I recommend adding the following immediately before the try statement. It's not ideal to change incoming text, but the alternative is that aster is not reliable. This whole issue arose because the OpenDevin project references binary characters in at least one docstring.

s = s.replace('\x00', '\\x00')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants