Add ability to remove HTTP response headers in the HTTP connection manager after HTTP filters have executed

[henrymwang]

Title: Add ability to remove HTTP response headers in the HTTP connection manager after HTTP filters have executed

Description:
The HTTP connection manager offers early header request mutations which run before HTTP filters:

• envoy/api/envoy/extensions/filters/network/http_connection_manager/v3/http_connection_manager.proto

  Line 678 in 0038ebb

  repeated config.core.v3.TypedExtensionConfig early_header_mutation_extensions = 52;

  .

• envoy/source/common/http/conn_manager_impl.cc

  Line 1372 in 0038ebb

  const auto mutate_result = ConnectionManagerUtility::mutateRequestHeaders(

We have a similar need for stripping response headers after HTTP filters have executed. This would be useful in situations where HTTP filters might add response headers unintentionally that we'd like to ensure are never leaked.

This could be implemented by adding a new field to the HTTP connection manager config and updating encodeHeaders here,

envoy/source/common/http/conn_manager_impl.cc

Line 1788 in 0038ebb

ConnectionManagerUtility::mutateResponseHeaders(

.

We can prepare a patch if this sounds reasonable.

[optional Relevant Links:]

Any extra documentation required to understand the issue.

[zuercher]

I sort of expected response_headers_to_remove to handle this, but I don't think it does (maybe it used to).

cc @alyssawilk

[henrymwang]

I sort of expected response_headers_to_remove to handle this, but I don't think it does (maybe it used to).

For a test: I added a network listener with 2 http filters: health-check filter in no pass through mode, then the http router filter. The http health-check filter adds some headers and uses localReply if it matches the /healthy (responds with a 200). When I curl the health path, it always returns the header x-envoy-upstream-healthchecked-cluster, which is automatically added by the http healthcheck filter.

Because the http router filter is last, it also modifies response headers first so it's too late then. Another option might be adding a filter in front to remove the response headers but it would require extra care about filter ordering. My proposal is to add this ability to unconditionally remove response headers, even ones added by http filters running before the http router.