Better handling of settings and questions in copier.yml

[stereobutter]

Since #846 is about to be merged I'd like to start a discussion about the remaining copier settings (the PR only affects exclude, skip_if_exists, secret_questions and jinja_extensions). I'd be happy to work another PR (or multiple smaller PRs if prefered).

The remaining settings that can be configured from within copier.yml are:

• answers_file
• envops
• migrations
• min_copier_version
• subdirectory
• tasks
• templates_suffix

Proposal

1. concatenate tasks and migrations from multiple documents in order

   # copier.yml
   _tasks:
      - "echo 1"
   ---
   !include dependency.yml
   ---
   _tasks:
     - "echo 3"
   ---

   # dependency.yml
   _tasks:
     - "echo 2"

   would echo 1, 2, 3

2. take the lowest (see Better handling of settings and questions in copier.yml #850 (comment)) highest version for min_copier_version if multiple documents specify different values regardless of document order

   # copier.yml
   _min_copier_version: "6.0.0"
   ---
   !include dependency.yml
   ---

   # dependency.yml
   _min_copier_version: "5.1.0"

   would result in min_copier_version == "6.0.0". Also warn that there is a major version difference in (part of) the template compared to the currently used copier version.

3. in my opinion setting answers_file, envops, subdirectory or templates_suffix to different values in multiple documents is an error in the template and silently overwriting these settings is bad UX for the template designer. The question is whether to keep the current behavior (later documents can overwrite a previously set value) and just emit a warning or to raise an exception. The latter would be backwards incompatible if there were any legitimate use cases for setting and overwriting these values in multiple documents. On the other hand the trouble with warnings is that the person that will see the warning isn't just the template designer when testing their template but also end users just consuming the template. End users wouldn't know what (if anything) to do about the warning (see below for further discussion about this). If we deem that setting these more than once is an error on the part of the template designer I think we should just raise an exception and make people fix their templates.

Related ideas

• similar to 3. above, overwriting a question of the same name in another document is probably an error in the template most of the time but I think there are legitimate use cases where template designers want to overwrite a question e.g. to make it more specific. We could emit a warning but again face the issue that end users will see this warning too and not know what (if anything they should do about it).
• maybe these warnings could be suppressed by default and only shown when a special command line flag e.g. --debug is used?

[pawamoy]

take the lowest version for min_copier_version

I think it should rather take the highest one? If template A needs Copier 5.1.0 minimum, it will work with 6.0.0, while if template B needs Copier 6.0.0 minimum, it won't work with 5.1.0.

[stereobutter]

I think it should rather take the highest one? If template A needs Copier 5.1.0 minimum, it will work with 6.0.0, while if template B needs Copier 6.0.0 minimum, it won't work with 5.1.0.

You of course correct, the result should be the highest of all versions. What tripped me up here is that copier emits warnings for major version conflicts i.e. 6.0.0>=5.1.0 so I wanted to preserve the lowest major version as to not hide these conflicts. It would be possible to check each version given and emit a warning regardless of what version takes precedence for the final min_copier_version.

The nicest (albeit most complicated) solution would be if copier took a version spec like pip e.g. >=5.1.0,<6.0 (or an additional max_copier_version to not make matters too complicated) and we computed the allowed version spec and raised an exception if the version spec from multiple documents do not overlap.

[yajo]

I agree with most that has already been said here.

In any case, I'd abstain from "inventing problems". Let's just fix the real ones that happen (there are enough! 😆)

• We could emit a warning but again face the issue that end users will see this warning too and not know what (if anything they should do about it).

• maybe these warnings could be suppressed by default and only shown when a special command line flag e.g. --debug is used?

There's #368 where I talk about the problem that due to several refactors, Copier right now has an heterogeneous logging system. I'd like to standardize it somehow (maybe coloredlogs?) and add CLI/API features for easily filtering the log verbosity.

Possibly the most immediate action we can take is to warn in the docs that these kind of inheritances can lead to unexpected behaviors.