Reduce InputFileCatalog memory usage

[makortel]

In #46975 (comment) it was discovered that InputFileCatalog can take nearly 500 MB memory per stream for premixed pileup files. While #47013 reduces the memory usage by nearly 200 MB per stream, there are further memory optimization opportunities.

Presently the InputFileCatalog does the LFN-to-PFN translation upfront for all LFNs and file locators. This is particularly costly (compared to benefits) for pileup files, where only small fraction of the input files are really accessed by the job. Doing the LFN-to-PFN translation only upon request would allow saving some factors of memory, but we should think a bit of the tradeoffs (mostly if there are any error conditions that would get deferred).

[makortel]

assign core

[cmsbuild]

New categories assigned: core

@Dr15Jones,@makortel,@smuzaffar you have been requested to review this Pull request/Issue and eventually sign? Thanks

[cmsbuild]

cms-bot internal usage

[cmsbuild]

A new Issue was created by @makortel.

@Dr15Jones, @antoniovilela, @makortel, @mandrenguyen, @rappoccio, @sextonkennedy, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here