Create a static driver

Namelist

The namelist is used to provide all necessary information to create the static driver file. The namelist is split into different sections, which provide some kind of organization to the namelist file. The namlist is a file that ends in .ini, a common ending for configuration files in Windows and other applications. Below, each section and its parameters are described.

Note: All parameters need to be provided in a config file. However, if some parameter is not needed, the part after the equal sign can be left blank. Furthermore, all values are interpreted by the script as strings. So if a string is to be provided, no "" need to be set. If an int or float are required, just provide the number (i.e. 1 or 1.0).

[settings]

This section contains top level information about the simulation.

Parameter	dtype	Description
casename	string	Name of the case. Used to construct the file names for the script output.
origin_time	YYYY-MM-DD HH:mm:SS +TZ	Origin time of the PALM simulation. This is a critical parameter and needs to be set correctly, otherwise the sun position and COSMO forcing may be incorrect.
cutorthoimg	boolean	True if for each domain an orthoimage shall be cut to the correct extents. Can be used in VAPOR visualizations. False if not needed.
extentsonly	boolean	If True, only the config file is read, checks are done and the extents are plotted in a graph. This can be used to fine-tune the choice of extents and probe locations.
mapdialect	string	Keyword, which mapping dictionaries shall be used. Available options are tlm, tutorial, DM01AVZH24,custom. This will decide which python file with which dictionaries is loaded as mpd.* module.
orthores	int	Resolution of the image, on which the extents are plotted. 5 seems to be a good number to provide enough resolution on the satellite image to identify important landmarks. For larger domains this number may need to be increased, as larger domains with the same resolution leads to larger file sizes.
rotationangle	float	Angle by which the domain is rotated. Useful in runs with noncyclic BC as specific wind-tunnel like setups. Not really supported at the moment by PALM.
set_vmag	float	Estimated velocities that are set by you or INIFOR. Used to calculate a runtime score, that gives a hint about the cases complexity and therefore its runtime.
simtime	float	Used to construct the inifor namelist, that is also output when running the make_static.py script.
veg_format	string	Write shp or tif or tiff depending on which format your vegetation is.
veg_height_bot	int	Height of lower border of tree crowns. In Zurich, this is set at three meters.
surf_data_mode	string	You can provide land use and pavement shapefiles separately or together. Write separate or together respectively.
src_luse_type	string	Name of the attribute, where the surface type is defined. Default is "OBJEKTART"
src_pav_type	string	Name of the attribute, where the pavement surface type is defined. Default is "BELAGSART"
discrete_zt	bool	If True, the terrain height will be rounded to dz*round(zt/dz)

[paths]

In this section all paths need to be specified to the required files. The path separator that is used here is \ in windows and / for linux paths. Preferrably use absolute paths. Paths in the .ini File do not require to be in quotes, as the datatype will be set to string.

Parameter	Description
input_crs_epsg	Your source geodata has a certain CRS and an EPSG-code, which is supported by palmpy. Provide the EPSG code as an integer.
modulepath	Path to the palmpy folder. Needs a trailing separator. Example: modulepath = C:\Users\Username\Documents
inputfilepath	Path where all the input files are stored. Needs a trailing separator.
outpath	Path where the output files shall be saved. Needs a trailing separator. Example: tmp = C:\Users\Username\Desktop\case\
orthoimage	Filename of the Orthoimage. Needed if cutorthoimg is set to true and extents shall be plotted.
dhm	Filename of the digital height model (without buildings) raster file (geotiff). Used to create the terrain for the PALM simulation.
bb	Filename of the shapefile with land cover information. This information will be resolved in parametrized in 2D. Used to construct the vegetation_type variable.
resolvedforest	Filename of the shapefile with forest polygons that shall be resolved in 3D (e.g. extract from land cover file). If 3D vegetation shall be resolved, one of resolvedforest, treerows or singletrees shall be provided. Used to construct the LAD variable.
treerows	Filename of the shapefile with tree line polygons that shall be resolved in 3D (e.g. puffered from shapefile with line features). If 3D vegetation shall be resolved, one of resolvedforest, treerows or singletrees shall be provided. Used to construct the LAD variable.
singletrees	Filename of the shapefile with single tree polygons that shall be resolved in 3D (e.g. puffered from shapefile with point features). If 3D vegetation shall be resolved, one of resolvedforest, treerows or singletrees shall be provided. Used to construct the LAD variable.
pavementareas	Filename of the shapefile with information about paved areas, such as streets, parking lots, town squares, everything that is not vegetation and is an artificially sealed surface (e.g. merged shapefiles with street, railways, airport movement areas, parking lots etc.). Will be used for pavement_type variable.
gebaeudefoots	Filename of the shapefile with information about building footprints. Used to create buildings in 2D and 3D (to benefit from the latter, HEIGHT_BOT needs to be provided as an attribute).
crops	Filename of the shapefile with information about crop fields. Crop field information may not be present in land cover data as it changes often.
streetsonly	Filename of a shapefile with street information only (needs to be provided for creating the street_type variable for emission representation).
veg_raster	Filename of a raster file containing tree height information relative to above ground.

[probes]

It is possible to specify probe locations in the namelist. The script will plot their positions on the overview plot and plot probe coordinates respective to each chosen domain.

Variable	dtype	Description
probes_E	float, float, ...	List of Easting Coordinates of a probe, separated by a comma. The first entry corresponds the first probe, and so forth.
probes_N	float, float ...	List of Northing Coordinates of a probe, separated by a comma. The first entry corresponds the first probe, and so forth.
probes_name	str, str, ...	List of name strings of the probes, separated by a comma. This will be used to identify the coordinates in the parameters output file.

[change_npars]

In order to change vegetation, pavement, water, soil or albedo parameters individually, set the flags dovegpars or doalbpars to True for each domain. If a flag is set to True, values must be provided or an error is raised (string index out of range). In the following, filterarr refers to an array which is searched for the filtervalue - where the search matches, the newvalue is put into the x_pars array on the npar'th level (see PALM documentation, PIDS Tables). The four values to be provided can be remembered as: "Set the parameter npar to newvalue where the pixel in filterarr equals filtervalue".

Parameter	Description
vegparchanges	Provide in form npar:newvalue:filterarr:filtervalue, where filterarr = 0 means from source array, filterarr = 1 corresponds to array after mapping (palm classes). For crop types (>= 1000) only 0 is supported. Multiple requested changes are to be separated by commas. Example: vegparchanges = 4:0.5:0:1004, 4:0.2:0:1003
watparchanges	Provide in form npar:newvalue:filterarr:filtervalue, where filterarr = 0 means from source array, filterarr = 1 corresponds to array after mapping (palm classes). Multiple requested changes are to be separated by commas.
pavparchanges	Provide in form npar:newvalue:filterarr:filtervalue, where filterarr = 0 means from source array, filterarr = 1 corresponds to array after mapping (palm classes). Multiple requested changes are to be separated by commas.
soilparchanges	Provide in form npar:newvalue:filterarr:filtervalue, where filterarr = 0 means from source array, filterarr = 1 corresponds to array after mapping (palm classes). Multiple requested changes are to be separated by commas.
albedoparchanges	Provide in form npar:newvalue:filterarr:filtervalue, where filterarr = 0 means from source array, filterarr = 1 corresponds to array after mapping (palm classes). Multiple requested changes are to be separated by commas.
bldgparchanges	Provide in form npar:newvalue:filterarr:filtervalue. Filterarr here corresponds to the following attributes: - 0: building_type - 1, 2, 3 : GROUPING1, GROUPING2, GROUPING3 ... of the building footprints shapefile. This way, either all elements of a certain building type can be changed, or logical groupings can be defined on three levels (e.g. by zip code, by color, by group to be changed or whatever). More levels can be added easily to the script.

[domain_#]

For each domain (domain_0, domain_1, ...), the following parameters shall be given.

Parameter	dtype	Description
xlen	int	X extent of the domain (eastwards) in meter.
ylen	int	Y extent of the domain (northwards) in meter.
xmin	float	X (Easting) coordinate position of the lower left domain corner.
ymin	float	Y (Northing) coordinate position of the lower left domain corner.
zmax	float	Domain vertical extent in meters. Required, since palmpy can create 3D-buildings or resolved vegetation. The z coordinate of these variables may be smaller than the total domain z extent, but not larger. If you have skyscrapers in the simulation, but only simulate the first 80m (for example), the 3D building array needs to be cut at 80m. After creating the static driver, the palm simulation namelist may contain different values than this one, but only bigger ones, not smaller ones than zmax
xres	float	Domain resolution in X direction.
yres	float	Domain resolution in Y direction. Currently, only xres = yres is possible, if chosing differently check your static driver!
zres	float	Domain resolution in Z direction.
doterrain	boolean	Create the topography height variable for this domain.
dolandcover	boolean	Create vegetation_type variable for this domain.
dopavedbb	boolean	Create pavement_type variable for this domain.
docropfields	boolean	Incorporate additional crop field information (set vegetation_type = 2) based on an additional shapefile.
dolad	boolean	Create 3D resolved vegetation for this domain.
dobuildings2d	boolean	Create 2D buildings with height information for this domain.
dobuildings3d	boolean	Create 3D buildings (each grid point = 1 or 0) for this domain.
dovegpars	boolean	Change individual vegetation parameters based on provided vegparchanges information for this domain. It is good practice to set this parameter the same for all domains.
doalbedopars	boolean	Change individual albedo parameters based on provided albedoparchanges information for this domain. It is good practice to set this parameter the same for all domains.
dostreettypes	boolean	Create a street_type variable for this domain.
bulkvegclass	boolean	Assign this value / PALM vegetation class to pixels in vegetation_type that have not been covered by the input shapefiles. Usually 3 for grass or 1 for bare soil.
pave_alltouched	boolean	Method selector during shapefile polygon rasterization with GDAL. Activates the GDAL option "ALL_TOUCHED" . This means that the shapefile value is burnt into every pixel the polygon touches, instead of burning in the value only if the centroid of the pixel is within the polygon. If the domain resolution is chosen too coarse, streets may not be rasterized continuously anymore and potential emission locations become unrealistic. With pave_alltouched = True, streets may become too wide, however they are still continuous.
bulkpavclass	boolean	Assign this value / PALM pavement class to pixels in pavement_type that have not been covered by the input shapefiles. Usually 1 for asphalt/concrete mix.
lai_forest	float	Leaf Area Index for forest polygons. Parameter to construct a Leaf Area Density vertical profile.
lai_breihe	float	Leaf Area Index for tree line polygons. Parameter to construct a Leaf Area Density vertical profile.
lai_ebgebu	float	Leaf Area Index for single tree polygons. Parameter to construct a Leaf Area Density vertical profile.
a_forest	float	alpha Shape parameter for beta-distribution for forest polygons. Parameter to construct a Leaf Area Density vertical profile.
b_forest	float	beta shape parameter for beta-distribution for forest polygons. Parameter to construct a Leaf Area Density vertical profile.
a_breihe	float	alpha shape parameter for beta-distribution for tree line polygons. Parameter to construct a Leaf Area Density vertical profile.
b_breihe	float	beta shape parameter for beta-distribution for tree line polygons. Parameter to construct a Leaf Area Density vertical profile.
a_ebgebu	float	alpha shape parameter for beta-distribution for single tree polygons. Parameter to construct a Leaf Area Density vertical profile.
b_ebgebu	float	beta shape parameter for beta-distribution for single tree polygons. Parameter to construct a Leaf Area Density vertical profile.

Running the Script

There are two modes, in which the script can be executed, which is governed by the parameter extentsonly. If this is set to True, the script does not proceed to process the geodata, but instead only shows an overview plot over the choices of domain extents and probe locations and show the results of some implemented tests, that would indicate towards a potential failure of the simulation in PALM. In this way, one can iteratively fine-tune the domaine extents and resolutions in order to achieve a setup that is 1) fulfilling the needs of the user and 2) compliant with the input data expectation by PALM. When a suitable setup is found, the extentsonly parameter can be set to False and the whole script can be run.

There are two ways, how the make_static script can be executed. The script can either be run from inside a Python IDE or from a command line, such as a Linux Bash environment.

From the Command Line Interface (simplest)

The script can also be run from the command line interface, e.g. in a Linux bash environment. With activate palm environment (source activate palmenv), run the following code:

python make_static.py <relative path to namelist>

This will execute the stript and create the output.

Python IDE

Running the script in a Python IDE such as Spyder allows the have the namelist file to be open in one tab and the script in the other. This really simplifies changes to the namelist file or also the script, should individual changes be required or some routine fails due to errors in either shape files or the script itself. The script can be run in Spyder by pressing the green run run button or pressing F5 (Spyder v4).

Generated Output by the Script

The script will generate the following output:

File	Description
<filename>_static(_N0X)	Static files with increasing nest cycle identifier
domainoverview.png	Overview plot of the domain extents and Probe locations superimposed on an orthoimage
<filename>_parameters.txt	Summary of the static generation process with all vital information to set up a PALM case
<filename>_inifornamelist	Ready-to-use namelist for creating a dynamic driver with INIFOR
tmp	tmp-folder with saved ascii raster data, orthoimage for every domain and other files. These can be kept for debugging or visualization purposes.

Static Driver Files

These static driver files are ready to be used in a palm simulations. However, it is advisable to double check the global attributes that were added to this file. These are critical for a simulation and, among other things, dictate which time of day the simulation starts. An error here may lead to a lot of wasted computational resources. The global attributes can be examined with ncdump -h XX_static.

Domain Overview Plot

This plot shows the defined domain extents and, if defined, the probe locations in the domain. This is great for visually double checking the domain setup and make changes if needed.

Parameter File

This file summarizes the whole domain setup and can be used to verify the chosen settings and also as a reference sheet to create the namelist files for the PALM simulation as well. A parameter file may look as follows:

-----------------------------------------
SIMULATION SETUP SUMMARY
-----------------------------------------

Origin Time:			2019-06-07 04:00:00 +02
Topo shifted down by:		429.00 Meter

Domain 0:
	nx/ny/nz		191/191/128
	dx/dy/dz		32.0/32.0/32.0
	Origin E/N:		533068.0/175458.0

Domain 1:
	nx/ny/nz		255/255/64
	dx/dy/dz		8.0/8.0/8.0
	Origin E/N:		535244.0/177826.0
	ll-Position for &nesting_parameters
	x,y:			2176.0, 2368.0

Domain 2:
	nx/ny/nz		383/383/32
	dx/dy/dz		4.0/4.0/4.0
	Origin E/N:		535508.0/178114.0
	ll-Position for &nesting_parameters
	x,y:			2440.0, 2656.0

----------------------
Total Number of Cells:		1.36e+07
   Domain 0:				4.719e+06	34.62%
   Domain 1:				4.194e+06	30.77%
   Domain 2:				4.719e+06	34.62%

Runtime length score:		6.82


---------------------
Probes (for masked output):

Domain 0:
	Probe 1 x/y		3456.0/3072.0
	Probe 2 x/y		2865.0/2900.0
	Probe 3 x/y		3149.0/3171.0
	Probe 4 x/y		3298.0/3306.0
	Probe 5 x/y		3422.0/3424.0
	Probe 6 x/y		3557.0/3549.0
	Probe 7 x/y		3682.0/3666.0
	Probe 8 x/y		3807.0/3777.0
Domain 1:
	Probe 1 x/y		1280.0/704.0
	Probe 2 x/y		689.0/532.0
	Probe 3 x/y		973.0/803.0
	Probe 4 x/y		1122.0/938.0
	Probe 5 x/y		1246.0/1056.0
	Probe 6 x/y		1381.0/1181.0
	Probe 7 x/y		1506.0/1298.0
	Probe 8 x/y		1631.0/1409.0
Domain 2:
	Probe 1 x/y		1016.0/416.0
	Probe 2 x/y		425.0/244.0
	Probe 3 x/y		709.0/515.0
	Probe 4 x/y		858.0/650.0
	Probe 5 x/y		982.0/768.0
	Probe 6 x/y		1117.0/893.0
	Probe 7 x/y		1242.0/1010.0
	Probe 8 x/y		1367.0/1121.0


----------------------
Changes to vegetation/albedo/water/pavement/soil_pars

Manual overrides for vegetation type:
	For class 1004, set npar 4 to 0.25 based on Source Data classes.
	For class 1003, set npar 4 to 0.47 based on Source Data classes.
	For class 1002, set npar 4 to 0.06 based on Source Data classes.
	For class 1001, set npar 4 to 0.03 based on Source Data classes.
	For class 1000, set npar 4 to 0.005 based on Source Data classes.
	For class 1004, set npar 5 to 0.00025 based on Source Data classes.

Manual overrides for albedo type:
	For class 1004, set npar -1 to 0.15 based on Source Data classes.


Created: 2020-05-27 16:27:04

As can be seen, all relevant data to setup a PALM namelist are included. It starts with a listing of the time origin of the simulation and the downshift of the topography in z direction, which must be incorporated when defining output on defined altitude levels (e.g. if an output is desired on an xy-section on 3000 m of altitude and a topography shift was done by 450 m, the output level needs to be set to 2550 m !). After this, all domains are presented with their corresponding nx, ny and nz values (which can be copied as they are into the PALM namelist), resolutions and, in case of nests, their origin position relative to the parent domain.

A section will also present the number of cells of each domain in the simulation. This will help to decide how many cores will be distributed onto each domain (see also chapter Limitations on nx and ny how the choice of cell numbers may affect core distribution). A runtime length score, which is computed from the number of total cells, the anticipated wind speeds and the lowest cell size in the setup, exploits the definition of the CFL criterion (which governs the size of the variable time step) and may provide some guidance in assessing and comparing the specific runtime of cases (as the total length of the simulation is not present, it is only a rough guidance).

In the following section, each probe location is defined relative to the domain origin. This can be used to define masked output and create high resolution probes. This feature will in the future be better implemented by using the virtual_measurement module, however, at this point in time the author has not been able to make it work on the speedflyer cluster of the ZHAW.

The file ends with a reminder of manual changes to vegetation/water/pavement_types by modifying their values by employing X_pars arrays. A timestamp ends the file for organization purposes.

INIFOR Namelist

The script also outputs a ready-to-use namelist that is required to start INIFOR. It contains the necessary information of the parent domain and an end_time, which is to be specified in the palmpy namelist.

An INIFOR namelist may look like this:

&inipar nx = 191, ny = 191, nz = 128,
        dx = 32.0, dy = 32.0, dz = 32.0,
    /
&d3par    end_time = 86400.0
    /

! Created on 2020-05-27 16:27:04

So to create a dynamic driver file for a PALM simulation, the COSMO source files need to be transformed into a INIFOR-compatible format and the parent static driver file and this namelist need to be present. Then, INIFOR can be started (see section Dynamic Driver Generation with INIFOR for more guidance).

tmp-Folder

The tmp folder contains intermediate files that are created while calling GDAL-based palmpy functions in the make_static script. These can be loaded with the makestatictools function mst.loadascgeodata().

But, the tmp folder also contains (provided cutortho = True ) a correctly clipped and resolved orthoimage for each PALM domain of the simulation. This is especially useful when creating a 3D visualization in VAPOR, where a terrain image can be rendered (including it's heights, if zt is copied into an output file). See section VAPOR Visualization for more hints.