[RFC] Dealing with dev dependencies in NodeJS when building in source

[torresxb1]

When building certain runtimes (Java, NodeJS, Python, Ruby, provided/custom), SAM CLI copies the source code and builds in another directory. We're working on adding an option to allow building in the source code directory instead. This should enable support for local dependencies (see example issue here) and potentially bring build speed improvements.

For NodeJS (without esbuild), SAM CLI currently copies the source code and installs using npm's --production flag, so that dev dependencies don't end up being uploaded. This doesn't affect the node_modules in the source code, so users can continue to use their dev dependencies after building.

However, if we want to build in the source directory, we need to choose between

1. Not installing dev dependencies (i.e. using the --production flag). This means dev dependencies won't be uploaded, but it also means that in order to use dev dependencies users will need to install them themselves after every sam build.
2. Installing dev dependencies (i.e. not using the --production flag). This means users can keep their dev dependencies but these will also be uploaded unless users prune them themselves before packaging/deploying.

We're looking to get feedback from NodeJS users into which option they'd prefer (or if they have other ideas/suggestions).

[roman-boiko]

What about adding a flag to sam build to give the users possibility to choose?
ie running sam build --production will use this flag on npm as well

[brnkrygs]

First, I'd echo @roman-boiko to ask for a flag or some type of escape hatch to force behavior one way or another as a user. Obviously, the sam build command is used in a lot of places from local dev to CI/CD pipelines, and different contexts will warrant different behavior.

Second, assuming we have an escape hatch, this becomes a question of "what is the correct/best default"?

TLDR: default to assuming you're building for production.

To answer this question more fully, I would lean into the principal of least surprise.

That is, what would someone who doesn't know about this new behavior expect to happen? And maybe more importantly, what would be the impact of the unexpected happening? Especially if they have infrastructure/processes already set up and blindly upgrade.

I think its much less risky to build with the --production flag by default rather than allow dev dependencies to be unexpectedly bundled with production dependencies, and end up making their way to a production environment.

Two major reasons are security and performance (which, in a Serverless world also equates to cost).

1. Security: dev packages tend to assume that they're running in a local, trusted environment which can lead to lax security. Indeed, in many cases they are explicitly providing alternative hooks into running processes that are additional risks for a production environment.

2. Performance: dev packages tend to grow large over the course of an application's life. That means more code to carry around, higher package sizes, longer deployment times, and longer cold start times (for dynamic environments like AWS Lambda). They also wrap/inject/tap business code, which can add additional processing and more complex code paths. And when running in AWS Lambda, slower performance directly results in higher cost, and of course a worse experience for customers.

In my opinion, having a secure and performant production environment greatly outweighs the slight inconvenience of having to add an additional flag, set a local environment variable, or re-run an install command when building locally.

[torresxb1]

Thanks for the feedback @roman-boiko @brnkrygs 🙏 , this is very helpful!

[vieirai]

A better solution is to extend the existing esbuild functionality, so even regular NodeJS functions take advantage of the tree shaking?

Not installing dev dependencies will result in a smaller bundle, but it will still be nowhere near as small as a tree shaked file.

[andyloughran]

Agree the --production flag is probably best as a configurable parameter.

[AlexMayleRdn]

TL;DR building in the CodeUri would be great, I agree that prod-only is a good default, would also like an option to disable rpm install all together.

When working with typescript and esbuild, it would be helpful to have some kind of control over whether an rpm install occurs at all. esbuild only bundles what is needed anyway. At least for my use case I'm already developing so everything is installed already. I don't think it's an anti-pattern to expect others to run an install before they attempt to run or deploy the software.

In any case, building in the source directory would be very helpful. We have a similar pattern as the linked issue regarding local dependencies. We have folders which get imported across several lambdas (they aren't declared as dependencies in any package.json file though, esbuild just brings them in). We have to set the CodeUri to src instead of src/<individual-lambda> to make it work (since the common folders wouldn't be copied otherwise). The problem with this is that sam sync updates a bunch of lambdas it doesn't have to because it is watching src and all the lambdas have that as the CodeURi. We'd prefer to be able to list src/<individual-lambda> as the CodeUri for each lambda function and have esbuild bring in anything each one needs.