1. The Genesis Device: Madgraph

All of the code here is taken from the 2022 CMSDAS example, but it has been consolidated for simplicity. The following instructions detail the use of code contained in the madgraph directory in this repository for making gridpacks.

1. Making a gridpack

1. Create a new set of cards for a process (e.g. VBS WWH, any final state) called PROCESS (e.g. VBSWWH_Inclusive)

cd cards
mkdir PROCESS_4f_LO
cp WpToLNu_4f_LO/WpToLNu_4f_LO_run_card.dat PROCESS_4f_LO/PROCESS_4f_LO_run_card.dat
cp WpToLNu_4f_LO/WpToLNu_4f_LO_proc_card.dat PROCESS_4f_LO/PROCESS_4f_LO_proc_card.dat

2. Edit the cards appropriately:
   • PROCESS_4f_LO_proc_card.dat: contains the most basic Madgraph instructions that can often be read/understood verbatim, like generate p p > w w h j j (includes VBS WWH); see the cards directory for additional examples
   • PROCESS_4f_LO_run_card.dat: contains more detailed parameters for Madgraph, like requirements on the momenta of various physics objects. Adding more stringent requirements can make things faster, but at the cost of a limited phase space. Note that adding such requirements (e.g. ptj > 20) will change the cross section that Madgraph calculates!
   • (Optional) PROCESS_4f_LO_customizecards.dat: contains instructions like setting the default value for a given coupling. Note that couplings in external models are typically given a non-zero or non-unity value, so if you want to generate a Standard Model sample, you may need to set the couplings of interest to the SM value by hand using this card.
   • (Optional) PROCESS_4f_LO_reweight_card.dat: contains the configuration for a scan of a number of coupling values. A series of event weights will be computed. When applied, they will scale the event kinematics distributions to look approximately like a corresponding set of coupling values. The model must be specifically configured to work with the settings in this card.
3. Add any additional models to the models directory
4. Create a gridpack

$ ./bin/make_gridpack --help
usage: ./bin/make_gridpack CARDSDIR [OPTIONAL ARGS]

Make a gridpack

required arguments:
  CARDSDIR                     directory containing Madgraph process/run cards

optional arguments:
  -h, --help                   display this message
  -n, --name NAME              process/card name (default: basename of cards dir)
  -m, --modelsdir MODELSDIR    directory containing Madgraph models
  -q, --queue QUEUE            which queue (default: 'local')
  -j, --jobstep JOBSTEP        processing options (default: 'ALL'; allowed: 'ALL','CODEGEN',
                               'INTEGRATE', 'MADSPIN')
  --scram SCRAM                which SCRAM architecture to use (default: slc7_amd64_gcc700)
  --cmssw CMSSW                which CMSSW version to use (default: CMSSW_10_6_0)

5. Check that the gridpack is good; the following command should produce an LHE file

mkdir temp
cd temp
tar xf ../PROCESS_4f_LO_slc7_amd64_gcc700_CMSSW_10_6_0_tarball.tar.xz # contains runcmsgrid.sh
./runcmsgrid.sh $NEVENTS $RANDOMSEED $NCPU
head cmsgrid_final.lhe # make sure that the file has content
cd ..
rm -rf temp

6. Clean up

rm -rf PROCESS_4f_LO
rm -rf PROCESS_4f_LO.log

7. Move the gridpack to Ceph (if it's large enough)

2. Testing the gridpack

A handy script has been included in this repository for testing gridpacks/making LHE files.

./bin/test_gridpack /full/path/to/gridpack.tar.xz [NEVENTS] [SEED]

where NEVENTS is the number of LHE events to produce and SEED is the random seed to use. The results will be saved to $PWD/tests/PROCESS. Note: because CMS uses an old version of MadGraph, the random seed is only used for a portion of the event generation. This means that if you use the exact same gridpack to generate two separate LHE files using the same seed, they will not contain an identical list of output events.

3. Gridpack generation on Condor

Gridpack generation can be computationally expensive, taking many CPU cores for multiple days if not longer. While most CMS Tier 2 sites do not support these kinds of jobs, some do support large CPU allocations (e.g. T2_US_Purdue). However, if the job needs to run for many days, it still might not be suitable and an alternative should be negotiated. This repository includes a handy script for setting up and running a Condor job that creates a gridpack and copies it back to your are on Ceph. NOTE: if your Ceph user area is not at T2_US_UCSD, you will need to make one minor alteration to the Condor executable.

$ ./bin/submit --help
usage: submit [-h] -c CARDS [-t TAG] [-m MEM_KB] [-n N_CPUS] [--cmssw CMSSW] [--scram SCRAM]
              [T2_SITE1 T2_SITE2 ...]

Submit condor jobs for producing a gridpack

required arguments:
  -c CARDS, --cards CARDS       directory containing MadGraph cards

optional arguments:
  -h, --help                    show this help message and exit
  -t TAG, --tag TAG             unique tag for identifying the output gridpack (default: None)
  -m MEM_KB, --mem_kb MEM_KB    amount of memory to request in kilobytes (KB) (default: 2048)
  -n N_CPUS, --n_cpus N_CPUS    number of CPUs to request for gridpack generation (default: 12)
  --cmssw CMSSW                 CMSSW version to use (default: CMSSW_10_6_0)
  --scram SCRAM                 SCRAM architecture to use (default: slc7_amd64_gcc700)

trailing arguments:
  T2_SITE1 T2_SITE2 ...         space-separated list of sites to submit job to (default: T2_US_Purdue)