bootstrap --offline uses https_proxy

[0-wiz-0]

Please excuse my ignorance about rebar3, I didn't package it myself for pkgsrc, I'm just reporting a build bug I noticed. We have a rebar3 package in pkgsrc for a couple years now, but recently we wanted to ensure that packages do not cause network traffic while building (the network traffic should happen in the 'fetch' phase) and set the https_proxy environment variable by default, to an invalid value.

In a recent build I noticed that rebar3 is now broken.
The build log ends with:

Evaluating config script "/tmp/devel/rebar3/work/rebar3-3.23.0/_build/default/lib/rebar/src/rebar.app.src.script"
escript: exception throw: {error,{bad_option,https_proxy,
                                    {{"downloads-forbidden-except-during-fetch",
                                      []},
                                     []}}}
  in function  httpc:bad_option/2 (httpc.erl, line 1383)
  in call from httpc:validate_options/2 (httpc.erl, line 1217)
  in call from httpc:validate_options/1 (httpc.erl, line 1203)
  in call from httpc:set_options/2 (httpc.erl, line 386)
  in call from rebar_utils:set_httpc_options/2 (apps/rebar/src/rebar_utils.erl, line 1004)
  in call from rebar_utils:set_httpc_options/0 (apps/rebar/src/rebar_utils.erl, line 992)
  in call from rebar3:init_config/0 (apps/rebar/src/rebar3.erl, line 224)
  in call from rebar3:run/1 (apps/rebar/src/rebar3.erl, line 100)
*** Error code 127

The build runs ./bootstrap --offline which I would expect to not evaluate or use any proxies.
Can you please advise how to make the build phase not try to use any proxies?

[ferd]

setting the build to offline mode just avoids starting the HTTP libraries:

rebar3/bootstrap

Lines 15 to 23 in b64d94f

	case os:getenv("REBAR_OFFLINE") of
	"1" ->
	ok;
	_ ->
	ensure_app(ssl),
	inets:start(),
	inets:start(httpc, [{profile, rebar}]),
	set_httpc_options()
	end,

If no proxy values are set, it won't call the API and will be fine.

However, by setting the proxy values, the detection that happens at start time (outside of bootstrapping's script) tries to call it. At this point, the code attempts to configure the client proxy and I believe that because we never started the HTTP client (because of the offline option), the operation fails.

Basically, the error is a very unclear way of saying "you're trying to set a proxy on a client that doesn't exist" and that fails.

[0-wiz-0]

Ok, I'm not sure I can follow the logic here.
I can confirm that if I set http_proxy= https_proxy= in the bootstrap environment, it works.
I haven't verified if it tries to use the network in that case in any way or not.
Is that what you recommend?

IMHO, in offline mode, the proxy variables shouldn't cause any behaviour change at all.

[ferd]

Yeah avoiding the error would require making sure that the code in

rebar3/apps/rebar/src/rebar_utils.erl

Lines 991 to 1028 in b64d94f

	set_httpc_options() ->
	set_httpc_options(https_proxy, get_http_vars(https_proxy)),
	set_httpc_options(proxy, get_http_vars(http_proxy)).
	set_httpc_options(_, []) ->
	ok;
	set_httpc_options(Scheme, Proxy) ->
	URI = normalise_proxy(Scheme, Proxy),
	Parts = rebar_uri:parse(URI),
	Host = maps:get(host, Parts, []),
	Port = maps:get(port, Parts, []),
	UserInfo = maps:get(userinfo, Parts, []),
	httpc:set_options([{Scheme, {{Host, Port}, []}}], rebar),
	proxy_ipfamily(Host, inet:gethostbyname(Host)),
	set_proxy_auth(UserInfo).
	proxy_ipfamily(_Host, {ok, _}) ->
	ok;
	proxy_ipfamily(Host, {error, nxdomain}) ->
	maybe_proxy_family(Host, inet_db:res_option(inet6)).
	maybe_proxy_family(Host, true) ->
	maybe_set_ipfamily(inet:gethostbyname(Host, inet), inet);
	maybe_proxy_family(Host, false) ->
	maybe_set_ipfamily(inet:gethostbyname(Host, inet6), inet6).
	maybe_set_ipfamily({ok, _}, Family) ->
	httpc:set_options([{ipfamily, Family}], rebar);
	maybe_set_ipfamily(_, _Family) ->
	ok.
	normalise_proxy(Scheme, URI) ->
	case re:run(URI, "://", [unicode]) of
	nomatch when Scheme =:= https_proxy -> "https://" ++ URI;
	nomatch when Scheme =:= proxy -> "http://" ++ URI;
	_ -> URI
	end.

does not attempt to configure the proxy when REBAR_OFFLINE is set to "1" to avoid trying to configure an HTTP client that was never started.