"Deploy Remaining Nodes" fails to launch more nodes.

[burtlo]

...
[2015-11-13T02:46:12+00:00] INFO: Running queued delayed notifications before re-raising exception
[2015-11-13T02:46:12+00:00] ERROR: Running exception handlers
[2015-11-13T02:46:12+00:00] ERROR: Exception handlers complete
[2015-11-13T02:46:12+00:00] FATAL: Stacktrace dumped to /root/chef_classroom/local-mode-cache/cache/chef-stacktrace.out
[2015-11-13T02:46:12+00:00] ERROR: machine_batch[default] (chef_classroom::deploy_multi_nodes line 9) had an error: RuntimeError: Chef::Resource::AwsImage[ami-f70cdd9c] does not exist!
[2015-11-13T02:46:14+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

[burtlo]

I believe the Windows AMI ami-f70cdd9c is the culprit.

Where is this AMI? Is it public? In the marketplace?

[cheeseplus]

The AMIs are statically mapped here https://github.com/chef-training/chef_classroom/blob/master/libraries/helpers.rb#L60 and I've only tested in us-west-2 with my own personal account using an AMI that @scarolan published (to my account). Not sure if having this static lookup is a great idea in general and we could easily attribute-ize these values.

[gmiranda23]

These are stock AMIs from the AWS Marketplace -- the only way to get consistency across AWS regions IMHO. I'm open to other methods. Right now what that means is that every so often there's maintenance that needs to occur (the static list should be updated). If you attributize these values, you're suggesting that any AMI can get dropped in. And we all know what a can of worms that is.

Perhaps the 'right hard thing' to do is to write some logic to automatically query the Marketplace API in each region to look these up. Until then, the static update may have to suffice.

Do we see any other possibilities that ensure consistency but don't require a manual update every so often?

[cheeseplus]

So I'm totally fine with keeping the lookup stuff but until we get everyone testing in the same account with consistent AMIs there are a few hacks littered about. We could just bake all the AMIs ourself so we have greater control but having a more intelligent lookup I think is a nice middleground. That said updating would be infrequent (and it's really just the Windows AMIs that are bothersome as they aren't public).

[gmiranda23]

Marketplace AMIs are independent of which account you work in. As long as you accept the EULA in whatever account you're in, it should be the same anywhere. The problem is that occasionally the Marketplace updates the Windows images and we have to update the AMI-id. We probably shouldn't have these types of pre-baked AMI hacks in place since it prevents being able to share this code beyond the walls of Chef proper. Avoiding use of pre-baked private AMIs should be a prime concern.

If we absolutely need to use pre-baked AMIs, remember the plan with chef_workstation is to have it eventually pop out pre-baked AMIs to save on build times. But those artifacts should be public, managed/reaped by a CI pipeline, and updated here whenever they change. If we go down the path of pre-baking node instances, we should probably approach those the same way.

As for the lookup route, kitchen-ec2 is currently trying to skin this same cat. We should probably follow their lead.

[burtlo]

We don't need the Windows AMI at moment. I'm going to change the recipe to spin up three Linux nodes for the purposes of the Chef Essentials.

Lets address this additional feature (windows node) when we have thoroughly tested the current workflow.