Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Features/#450 add metadata kh #1122

Open
wants to merge 92 commits into
base: features/#450-add-metadata
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
960cb71
add metadata for grid.egon_emob_charging_infrastructure
khelfen Mar 14, 2023
f511030
add metadata for supply.egon_home_batteries (resolve open TODOs later)
khelfen Mar 15, 2023
b25298e
Merge branch 'features/#1095-mastr-status-quo' into features/#450-add…
khelfen Mar 15, 2023
d25091a
Merge branch 'features/#450-add-metadata' into features/#450-add-meta…
khelfen Mar 15, 2023
d133ac0
add metadata for mastr datasets
khelfen Mar 15, 2023
bfdb7c9
get deposit id from datasets.yml
khelfen Mar 15, 2023
edf2f8b
Rename 'license' to 'licenses'
ClaraBuettner Mar 13, 2023
092e825
Fix name
ClaraBuettner Mar 13, 2023
b397d2f
Fill metadata for static tables
ClaraBuettner Mar 13, 2023
7ae5374
Apply black and isort
ClaraBuettner Mar 13, 2023
6cbe3b4
Add gas sector sources and contributors
AmeliaNadal Mar 13, 2023
98f1081
Adjust metadata sources
AmeliaNadal Mar 13, 2023
61d9419
Add technology data and dsm source
ClaraBuettner Mar 14, 2023
0d1a921
Add metadata for generaotr and bus timeseries tables
ClaraBuettner Mar 14, 2023
b2b6bfb
Add time series meta data automised
ClaraBuettner Mar 15, 2023
5722f8a
Delete unused imports
ClaraBuettner Mar 15, 2023
962ba72
Add dsm sources
ClaraBuettner Mar 15, 2023
379be47
Apply black and isort
ClaraBuettner Mar 15, 2023
b0edc9c
Add vg250 to generator sources
ClaraBuettner Mar 15, 2023
3f33ae0
Update default values for meta data
ClaraBuettner Mar 15, 2023
3d9b1d3
Fix licenses list in metadata
ClaraBuettner Mar 15, 2023
fe77d98
Change contributors of egon-data source
ClaraBuettner Mar 15, 2023
314b780
Add era5 sources to line timeseries metadata
ClaraBuettner Mar 15, 2023
54e99b6
Remove unused sources
ClaraBuettner Mar 15, 2023
c7f66d5
add me to contributors function in metadata.py
khelfen Mar 16, 2023
ca8e815
add metadata to supply.egon_power_plants_pv_roof_building
khelfen Mar 16, 2023
8156167
add ev metadata
khelfen Mar 17, 2023
86a53bf
some small fixes in ev metadata
khelfen Mar 17, 2023
9ac915d
add basic structure for dsm metadata
khelfen Mar 17, 2023
725ba9e
Merge branch 'dev' into features/#450-add-metadata-kh
khelfen Mar 17, 2023
737c501
add metadata for dsm time series
khelfen Mar 17, 2023
87b712a
Merge branch 'features/#1095-mastr-status-quo' into features/#450-add…
khelfen Mar 17, 2023
f13f7df
Merge branch 'features/#1095-mastr-status-quo' into features/#450-add…
khelfen Mar 17, 2023
60c0f30
fix single quote error
khelfen Mar 17, 2023
01508a5
fix single quote error
khelfen Mar 17, 2023
e8052cb
minor bug fix
khelfen Mar 17, 2023
9343fbe
small fix
khelfen Mar 17, 2023
77ceae3
bug fix in metadata string
khelfen Mar 20, 2023
67a6496
Merge branch 'features/#1095-mastr-status-quo' into features/#450-add…
khelfen Mar 20, 2023
34a22b1
Merge branch 'features/#1095-mastr-status-quo' into features/#450-add…
khelfen Mar 20, 2023
0a702b7
Merge branch 'features/#1095-mastr-status-quo' into features/#450-add…
khelfen Mar 20, 2023
48d3469
small bug fix single quote in sql string
khelfen Mar 20, 2023
06fc083
correct contributors field
khelfen Mar 24, 2023
c5766ae
added metadata check with omi
khelfen Mar 24, 2023
6e4689f
Merge branch 'dev' of github.com:openego/eGon-data into features/#450…
khelfen Apr 3, 2023
07b2341
fix date string
khelfen Apr 3, 2023
f1ed5f0
minor fix
khelfen Apr 3, 2023
8b758da
Merge branch 'dev' into features/#450-add-metadata-kh
khelfen Apr 6, 2023
990e544
add sources to dsm time series tables metadata
khelfen Apr 13, 2023
99e3557
Added data documentation for gas, methane and H2 grid
pieterhexen Jul 19, 2023
20af1c0
Added gas related dataset documentation
pieterhexen Jul 20, 2023
1a5b783
Added rtd Data documentation for gas sector
pieterhexen Jul 21, 2023
86adabe
Add new subsection in about.rst
IlkaCu Aug 8, 2023
ec86353
Extend docs for eHV/HV grid topology
IlkaCu Aug 8, 2023
99240cf
Add documentation for electricity demand
IlkaCu Aug 16, 2023
5727030
Add docu for hydro and biomass
IlkaCu Aug 16, 2023
a2e53f2
Add documentation for DSM
KathiEsterl Aug 16, 2023
ebc7ae3
Prepare documentation of PV ground mounted
KathiEsterl Aug 16, 2023
9caa505
Merge remote
KathiEsterl Aug 16, 2023
5862733
Corrections concerning DSM
KathiEsterl Aug 16, 2023
ba46843
Corrections concerning DSM
KathiEsterl Aug 17, 2023
0a0f915
Documentation of PV ground mounted
KathiEsterl Aug 17, 2023
ecd51ea
Corrections concerning PV ground mounted
KathiEsterl Aug 17, 2023
972c81e
Corrections concerning DSM
KathiEsterl Aug 17, 2023
13e63ea
Corrections concerning PV ground mounted
KathiEsterl Aug 17, 2023
c9d7cc3
Correct link
KathiEsterl Aug 17, 2023
613c160
Merge branch 'dev' into rtd_gas
pieterhexen Aug 29, 2023
3b4d998
Solve merge conflicts
KathiEsterl Dec 8, 2023
28447c6
Correct leftover from merge conflict
KathiEsterl Dec 8, 2023
2c7e29d
Merge pull request #1147 from openego/documentation/#1141-rtd-docs-in…
KathiEsterl Dec 8, 2023
6f174c7
Merge pull request #1148 from openego/rtd_gas
pieterhexen Dec 11, 2023
f9f1dfe
Update changelog
nesnoj Jan 10, 2024
98f2ae1
Update zenodo URL of data-bundle
nesnoj Jan 10, 2024
07db06e
Merge pull request #1155 from openego/fixes/#1154-update-zenodo-link-…
nesnoj Jan 10, 2024
ee8961f
Update zenodo URLs of MaStR data (trial run - testing on sandbox)
nesnoj Jan 10, 2024
8a46671
Update changelog
nesnoj Jan 10, 2024
0481f11
Bump mastr dataset version to 0.0.2
nesnoj Jan 10, 2024
a735052
Update zenodo URLs of MaStR data
nesnoj Jan 10, 2024
f562094
Update docs
nesnoj Jan 10, 2024
f6ed482
Merge pull request #1158 from openego/fixes/#1153-update-zenodo-link-…
nesnoj Jan 11, 2024
bad66e7
documentation wind on-offshore
CarlosEpia Jan 11, 2024
891554f
Fix CRS in ERA5 transformation
nesnoj Jan 16, 2024
b05fb83
Update changelog
nesnoj Jan 16, 2024
7cf077b
Merge pull request #1160 from openego/fixes/#1159_era5_crs
nesnoj Jan 16, 2024
0d33a93
renamed metadata.py
khelfen Aug 22, 2024
14373af
Merge remote-tracking branch 'origin/features/#450-add-metadata' into…
khelfen Aug 22, 2024
39327ea
replace oep_metadata_version function
khelfen Aug 22, 2024
cfe1b13
remove zenodo sandbox from metadata strings
khelfen Aug 22, 2024
7a03d06
Merge pull request #1169 from openego/dev
khelfen Aug 23, 2024
173b574
add call to metadata function for mastr
khelfen Oct 14, 2024
6122eea
update dataset version
khelfen Oct 14, 2024
224aa7a
correct use of ' token in metadata
khelfen Oct 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -494,12 +494,12 @@ Changed
`#1073 <https://github.com/openego/eGon-data/issues/1073>`_
* Change hgv data source to use database
`#1086 <https://github.com/openego/eGon-data/issues/1086>`_
* Rename eMob MIT carrier names (use underscores)
`#1105 <https://github.com/openego/eGon-data/issues/1105>`_
* Change desposit ID for data_bundle download from zenodo sandbox
`#1110 <https://github.com/openego/eGon-data/issues/1110>`_
* Use MaStR geocoding results for pv rooftop to buildings mapping workflow
`#1095 <https://github.com/openego/eGon-data/issues/1095>`_
* Rename eMob MIT carrier names (use underscores)
`#1105 <https://github.com/openego/eGon-data/issues/1105>`_

.. _#799: https://github.com/openego/eGon-data/issues/799

Expand Down Expand Up @@ -747,7 +747,12 @@ Bug Fixes
* Fix conversion factor for CH4 loads abroad in eGon2035
`#1104 <https://github.com/openego/eGon-data/issues/1104>`_
* Change structure of documentation in rtd
`#11126 <https://github.com/openego/eGon-data/issues/1126>`_
`#1126 <https://github.com/openego/eGon-data/issues/1126>`_
* Fix URL of eGon data-bundle dataset
`#1154 <https://github.com/openego/eGon-data/issues/1154>`_
* Fix URLs of MaStR datasets
* Fix CRS in ERA5 transformation
`#1159 <https://github.com/openego/eGon-data/issues/1159>`_

.. _PR #692: https://github.com/openego/eGon-data/pull/692
.. _#343: https://github.com/openego/eGon-data/issues/343
Expand Down
15 changes: 15 additions & 0 deletions docs/about.rst
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,27 @@ Modeling concept and scenarios
===============================

eGon-data provides a data model suitable for calculations and optimizations with the tools eTraGo, eDisGo and eGo and therefore aims to satisfy all requirements regarding the scope and temporal as well as spatial granularity of the resulting data model.



System boundaries and general assumptions
-----------------------------------------

* Sectors
* Focus on Germany
* Neighbouring countries (which ones and why)
* Spatial resolution / aggregartion levels
* Temporal resolution incl. assumptions on weather year

The following image visualizes the different components considered in scenario ``eGon2035``.

.. image:: images/egon-modell-szenario-egon2035.png
:width: 800
:alt: Components of the data models

Scenarios
---------

eGon-data aims to create different scenarios, which differ in terms of RE penetration or the availability of flexibility options. Currently, the following scenarios are available or in progress.

* ``eGon2035`` Mid-termin scenario based on assumptions from the German network expansion plan 'scenario C2035', version 2021 and TYNDP
Expand Down
2 changes: 1 addition & 1 deletion docs/data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ Different flexibility options are part of the model and can be utilized in the o
flexibility potentials and their distribution are needed. The considered technologies described in the following chapters range from different storage units,
through dynamic line rating to Demand-Side-Management measures.

Demand-Side-Management
Demand-Side Management
----------------------

.. include:: data/DSM.rst
Expand Down
51 changes: 50 additions & 1 deletion docs/data/DSM.rst
Original file line number Diff line number Diff line change
@@ -1 +1,50 @@
How did we implement DSM? Results etc.
Demand-side management (DSM) potentials are calculated in function :func:`dsm_cts_ind_processing<egon.data.datasets.DSM_cts_ind.dsm_cts_ind_processing>`.
Potentials relevant for the high and extra-high voltage grid are identified in the function :func:`dsm_cts_ind<egon.data.datasets.DSM_cts_ind.dsm_cts_ind>`,
potentials within the medium- and low-voltage grids are determined within the function :func:`dsm_cts_ind_individual<egon.data.datasets.DSM_cts_ind.dsm_cts_ind_individual>`
in a higher spatial resolution. All this is part of the dataset :py:class:`DsmPotential <egon.data.datasets.DsmPotential>`.
The implementation is documented in detail within the following student work (in German): [EsterlDentzien]_.

Loads eligible to be shifted are assumed within industrial loads and loads from Commercial, Trade and Service (CTS).
Therefore, load time series from these sectors are used as input data (see section ref:`elec_demand-ref`).
Shiftable shares of loads mainly derive from heating and cooling processes and selected energy-intensive
industrial processes (cement production, wood pulp, paper production, recycling paper). Technical and sociotechnical
constraints are considered using the parametrization elaborated in [Heitkoetter]_. An overview over the
resulting potentials for scenario ``eGon2035`` can be seen in figure :ref:`dsm_potential`. The table below summarizes the
aggregated potential for Germany per scenario. As the annual conventional electrical loads are assumed to be lower in the
scenario ``eGon100RE``, also the DSM potential decreases compared to the scenario ``eGon2035``.

.. figure:: /images/DSM_potential.png
:name: dsm_potential
:width: 600

Aggregated DSM potential in Germany for scenario ``eGon2035``

.. list-table:: Aggregated DSM Potential for Germany
:widths: 20 20 20
:header-rows: 1

* -
- CTS
- Industry

* - eGon2035
- 1.2 GW
- 150 MW

* - eGon100RE
- 900 MW
- 150 MW

DSM is modelled following the approach of [Kleinhans]_. DSM components are created wherever
respective loads are seen. Minimum and maximum shiftable power per time step depict time-dependent
charging and discharging power of a storage-equivalent buffers. Time-dependent capacities
of those buffers account for the time frame of management bounding the period within which
the shifting can be conducted. Figure :ref:`dsm_shifted_p-example` shows the resulting potential at one exemplary bus.

.. figure:: /images/shifted_dsm-example.png
:name: dsm_shifted_p-example
:width: 600

Time-dependent DSM potential at one exemplary bus


92 changes: 91 additions & 1 deletion docs/data/electricity_demand.rst
Original file line number Diff line number Diff line change
@@ -1 +1,91 @@
Information about electricity demands and their spatial and temporal aggregation
.. _elec_demand_ref:
The electricity demand considered includes demand from the residential, commercial and industrial sector.
The target values for scenario *eGon2035* are taken from the German grid development plan from 2021 [NEP2021]_,
whereas the distribution on NUTS3-levels corresponds to the data from the research project *DemandRegio* [demandregio]_.
The following table lists the electricity demands per sector:

.. list-table:: Electricity demand per sector
:widths: 25 50
:header-rows: 1

* - Sector
- Annual electricity demand in TWh
* - residential
- 115.1
* - commercial
- 123.5
* - industrial
- 259.5

A further spatial and temporal distribution of the electricity demand is needed to fullfil all requirements of the
subsequent grid optimization. Therefore different, sector-specific distributions methods were developed and applied.

Residential electricity demand
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The annual electricity demands of households on NUTS3-level from *DemandRegio* are scaled to meet the national target
values for the respective scenario in dataset :py:class:`DemandRegio <egon.data.datasets.demandregio.DemandRegio>`.
A further spatial and temporal distribution of residential electricity demands is performed in
:py:class:`HouseholdElectricityDemand <egon.data.datasets.electricity_demand.HouseholdElectricityDemand>` as described
in [Buettner2022]_.
The result is a consistent dataset across aggregation levels with an hourly resolution.

.. figure:: /images/S27-3.png
:name: spatial_distribution_electricity_demand
:width: 400

Electricity demand on NUTS 3-level (upper left); Exemplary MVGD (upper right); Study region in Flensburg (20 Census cells, bottom) from [Buettner2022]_


.. figure:: /images/S27-4a.png
:name: aggregation_level_electricity_demand
:width: 400

Electricity demand time series on different aggregation levels from [Buettner2022]_



Commercial electricity demand
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The distribution of electricity demand from the commercial, trade and service (CTS) sector is also based on data from
*DemandRegio*, which provides annual electricity demands on NUTS3-level for Germany. In dataset
:py:class:`CtsElectricityDemand <egon.data.datasets.electricity_demand.CtsElectricityDemand>` the annual electricity
demands are further distributed to census cells (100x100m cells from [Census]_) based on the distribution of heat demands,
which is taken from the Pan-European Thermal Altlas version 5.0.1 [Peta]_. For further information refer to section
ref:`heat_demand`.
The applied methods for a futher spatial and temporal distribution to buildings is described in [Buettner2022]_ and
performed in dataset :py:class:`CtsDemandBuildings <egon.data.datasets.electricity_demand_timeseries.CtsDemandBuildings>`

Industrial electricity demand
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To distribute the annual industrial electricity demand OSM landuse data as well as information on industrial sites are
taken into account.
In a first step (:py:class:`CtsElectricityDemand <egon.data.datasets.electricity_demand.CtsElectricityDemand>`)
different sources providing information about specific sites and further information on the industry sector in which
the respective industrial site operates are combined. Here, the three data sources [Hotmaps]_, [sEEnergies]_ and
[Schmidt2018]_ are aligned and joined.
Based on the resulting list of industrial sites in Germany and information on industrial landuse areas from OSM [OSM]_
which where extracted and processed in :py:class:`OsmLanduse <egon.data.datasets.loadarea.OsmLanduse>` the annual demands
were distributed.
The spatial and temporal distribution is performed in
:py:class:`IndustrialDemandCurves <egon.data.datasets.industry.IndustrialDemandCurves>`.
For the spatial distribution of annual electricity demands from *DemandRegio* [demandregio]_ which are available on
NUTS3-level are in a first step evenly split 50/50 between industrial sites and OSM-polygons tagged as industrial areas.
Per NUTS-3 area the respective shares are then distributed linearily based on the area of the corresponding landuse polygons
and evenly to the identified industrial sites.
In a next step the temporal disaggregation of the annual demands is carried out taking information about the industrial
sectors and sector-specific standard load profiles from [demandregio]_ into account.
Based on the resulting time series and their peak loads the corresponding grid level and grid connections point is
identified.

Electricity demand in neighbouring countries
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The neighbouring countries considered in the model are represented in a lower spatial resolution of one or two buses per
country. The national demand timeseries in an hourly resolution of the respective countries is taken from the Ten-Year
Network Development Plan, Version 2020 [TYNDP]_. In case no data for the target year is available the data is is
interpolated linearly.
Refer to the corresponding dataset for detailed information:
:py:class:`ElectricalNeighbours <egon.data.datasets.ElectricalNeighbours>`
32 changes: 23 additions & 9 deletions docs/data/electricity_grids.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,32 @@
High and extra-high voltage grids
++++++++++++++++++++++++++++++++++

The model of the German extra-high (eHV) and high voltage (HV) grid is based
on data retrieved from OpenStreetMap (status January 2021) [OSM]_ and additional
parameters for standard transmission lines from [Brakelmann2004]_. To gather all
required information, such as line topology, voltage level, substation locations,
and electrical parameters, to create a calculable power system model, the `osmTGmod
tool <https://github.com/openego/osmTGmod>`_ was used. The corresponding dataset
:py:class:`Osmtgmod <egon.data.datasets..osmtgmod.Osmtgmod>` executes osmTGmod
The model of the German extra-high (eHV) and high voltage (HV) grid is based
on data retrieved from OpenStreetMap (OSM) (status January 2021) [OSM]_ and additional
parameters for standard transmission lines from [Brakelmann2004]_. To gather all
required information, such as line topology, voltage level, substation locations,
and electrical parameters, to create a calculable power system model, the `*osmTGmod*
tool <https://github.com/openego/osmTGmod>`_ was used. The corresponding dataset
:py:class:`Osmtgmod <egon.data.datasets.osmtgmod.Osmtgmod>` executes osmTGmod
and writes the resulting data to the database.

The resulting grid model includes the voltage levels 380, 220 and 110 kV and
all substations interconnecting these grid levels. For further information on the
generation of the grid topology please refer to [Mueller2018]_.
all substations interconnecting the different grid levels. Information about
border crossing lines are as well extracted from *OSM* data by *osmTGmod*.
For further information on the generation of the grid topology please refer to [Mueller2018]_.
The neighbouring countries are included in the model in a significantly lower
spatial resolution with one or two nodes per country. The border crossing lines
extracted by *osmTGmod* are extended to representative nodes of the respective
country in dataset
:py:class:`ElectricalNeighbours <egon.data.datasets.ElectricalNeighbours>`. The
resulting grid topology is shown in the following figure.

..figure:: images/Stromnetz.png
:scale: 50 %
:name: gridtopology_ehv_hv
:alt: Grid topology extra-high and high voltage grid



.. _ding0-grids:

Expand Down
Loading
Loading