Allow anonymous OpenMetrics scraping

[lozbrown]

Most applications don't consider metrics to be privileged information (although that could be config controlled)

Problem:
In a typical deployment, scraping open metrics is significantly overcomplicated by the requirement for the user and sometimes password and that coordinator and worker cannot be scraped in the same way.

• • Username and password will be required to scrape metrics from the coordinator, becuase this password exists this must be via https (in many situations https is provided by a separate load balancer so requests must go to the external cluster address rather than the node/pods internal address )
• • Username (but not password) is required to scrape metrics from the workers, this can go to the internal address given that there is usually no ingress to workers

In many kubernetes deployments its typical to configure scraping by adding annotations to the pods

This causes the pods internal address to be scraped anonymously, as described above this wont work for either coordinators or workers. as both will need custom scrapes.

It is possible for an engineer to configure these but its unnecessarily complicated and and is outside the normal standards for scraping metrics.

This complexity is the main reason many are still using the jmx_exporter to the point its been introduces into the helm chart as it allows anonymous scrape.

@mattstep @nineinchnick @mosabua

[mosabua]

Considering that they are currently protected I think it would be reasonable to leave them protected but have a setting that allows you to open the endpoint up. Then we do not introduce a breaking change for existing users yet enable the simpler metrics access for those who want to do that. What do you think @electrum @dain @mattstep ?

And would somebody be interested to send a PR?