Bundletool performance, Bundletool as an Installer

[kastiglione]

We use apple.experimental.tree_artifact_outputs, and for our apps it takes a consistent 8-9s to run bundletool_experimental.py to construct the .app. This is something I'd like to optimize for local builds, to speed up the edit-compile-run cycle.

Have there been any discussions about future work for the bundletool, specifically the experimental (tree) version. For what it's worth, I notice that experimental.tree_artifact_outputs isn't documented.

Secondly, what do people think about the option of making rules_apple not run bundletool, instead have some {sh,py}_binary rule would produce an "installer" – which wraps bundletool, that we can run after the bazel build. For local development, after each bazel build the .app or .xctest bundles need to be copied to DerivedData, which adds a little more time. This could be faster if we skip the middle man and instead construct the .app where we want it (DerivedData).

[keith]

cc @DavidGoldman who might be interested in this too

[thomasvl]

I'm not sure I'm fully following what you are after.

We've had some thoughts about splitting the rules so something like ios_application makes the bundle, but skips making a zip/ipa. Then adding specific packaging rules to make ipas. But it doesn't sound like that's what you are talking about.

[kastiglione]

splitting the rules so something like ios_application makes the bundle, but skips making a zip/ipa

This would be good, since the bundle would also be cacheable.

I'm not sure I'm fully following what you are after.

I'm thinking of one of two things:

1. A build output that is a runnable script (like an installer). Kind of like a continuation. We could call this script to create the bundle where we want (DerivedData) instead of in bazel-out.
2. An option to not run the bundletool. We could have our own tool consume the generated control file, and build a bundle where we need.

[thomasvl]

Hm, so my initial thought is that "installer" mode sounds really fragile. Wouldn't you have to ensure you run it without any other activity having happened in the workspace? Building any other target (maybe even just running an aspect) could cause bazel to update some symlinks and thus invalidate the paths baked into the script?

If it is done, how complex would it have to be? Since it is now writing into an external location, it would need to know to prune things that no longer should exist in addition to installing the things that should exist? By moving out of the bazel directories, timestamps likely also start to having meaning, so does it start touching them all or does it have to be smart and do everything on the side to see if the file really changes and then only update the timestamp if it as changed to support incremental installs into the devices?

[kastiglione]

Wouldn't you have to ensure you run it without any other activity having happened in the workspace?

The use case I have in mind is running it immediately after the build.

If it is done, how complex would it have to be? …

We currently use rsync when copying a bundle from bazel-out to DerivedData. This allows us to not copy files that are not updated, which should avoid timestamp issues. We use the --delete flag to tell rsync to delete files from the destination that no longer exist in the source.

[thomasvl]

Right, so the "installer" would have to duplicate all that logic, right?

[kastiglione]

As far as I know, any cp command could be replaced by an rsync command. If true, then I think it could be a low complexity change to use rsync instead of cp.