[RFC] Allow Sources to read meta data incrementally

[Dr15Jones]

At the moment, they system relies on the fact that our TTree based format writes all the accumulated job meta data at then end of the job. However, begin able to write and then read back the meta data as we process a file could be beneficial

• We might be able to reduce the amount of memory held by the OutputModule.
• We might be able to successfully read back parts of the file which were written by a failed job.
• Being able to incrementally change our meta data structures is also needed to allow concurrent reading of multiple primary files concurrently.
• This might allow us to fully use non-random access file formats in the future.

[Dr15Jones]

assign core

[cmsbuild]

New categories assigned: core

@Dr15Jones,@makortel,@smuzaffar you have been requested to review this Pull request/Issue and eventually sign? Thanks

[cmsbuild]

cms-bot internal usage

[cmsbuild]

A new Issue was created by @Dr15Jones.

@Dr15Jones, @antoniovilela, @makortel, @mandrenguyen, @rappoccio, @sextonkennedy, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here