Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cloudshell_open incorrectly executes shell scriprt (prebuild hook) with gsutil -m commands #245

Open
evil-shrike opened this issue Nov 13, 2023 · 3 comments

Comments

@evil-shrike
Copy link

When a prebuild hook execute gsutil with -m (multiprocessing) option it exits.
Here's a reproducing demo:
https://github.com/evil-shrike/cloud-run-button-test2.git

the script has the following lines:

  echo "Starting copying of application files to $GCS_BASE_PATH"
  gsutil -m rsync -r -x ".*/__pycache__/.*|[.].*" ./app $GCS_BASE_PATH
  echo "Application files have been copied"
  gsutil -m -h "Content-Type:text/plain" cp ./*.yaml $GCS_BASE_PATH/
  echo "All done"

and when executed by cloudshell_open we won't see the "Application files have been copied" output. Needless to say that normally when execute manually from within Cloud Shell terminal it works fine.

outout:

+ ./prebuild.sh
+ gcloud config set project adsapi-sd
Updated property [core/project].
+ ./setup.sh deploy_files
Starting copying of application files to gs://adsapi-sd/crb-test

WARNING: gsutil rsync uses hashes when modification time is not available at
both the source and destination. Your crcmod installation isn't using the
module's C extension, so checksumming will run very slowly. If this is your
first rsync since updating gsutil, this rsync can take significantly longer than
usual. For help installing the extension, please see "gsutil help crcmod".

Building synchronization state...
Starting synchronization...
Copying file://./app/test.py [Content-Type=text/x-python]...
[ ! ] Attempting to build this application with its Dockerfile...                        
[ ! ] FYI, running the following command:

If I remove -m option from gsutil then it starts working as expected.

The issue make deploying applications via Cloud Run Button very cumbersome.

@jamesward
Copy link
Contributor

When Cloud Run Button runs the script that runs gsutil, does it exit with an error? Or does it work except without correct stdout handling?

@evil-shrike
Copy link
Author

It's hard to tell because both prebuild.sh and setup.sh just exit - I added echoing after calls and they didn't appear in the output:
image

@jamesward
Copy link
Contributor

I wonder if we can get more verbose output from your script. Also, it might be worth a small test to see how Cloud Run Button handles scripts that exit with 0 and non-zero codes, and where stderr goes. As I'm not sure currently how these are handled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants