clipping-loss-validation.tex

% !TEX spellcheck = en_US

\documentclass[conference]{IEEEtran}
\usepackage{cite}
\usepackage{amsmath,amssymb,amsfonts}
\usepackage{algorithmic}
\usepackage{graphicx}
\usepackage{textcomp}
\usepackage{xcolor}
% add hyperlinks, delete all .aux files if adding hyperref after previous build
\usepackage{hyperref}
% support for unicode charcters like "é" and "ñ"
\usepackage[T1]{fontenc}
% Provides generic commands \degree, \celsius, \perthousand, \micro and \ohm
\usepackage{gensymb}
% splits a section into multiple columns
\usepackage{multicol}
\def\BibTeX{{\rm B\kern-.05em{\sc i\kern-.025em b}\kern-.08em
    T\kern-.1667em\lower.7ex\hbox{E}\kern-.125emX}}
\begin{document}

\title{Validation of Subhourly Clipping Loss Error Corrections}

\author{\IEEEauthorblockN{Abhishek Parikh\textsuperscript{1}, Kevin Anderson\textsuperscript{2}, Kirsten Perry\textsuperscript{2}, William B. Hobbs\textsuperscript{3}, Rounak Kharait\textsuperscript{1} and Mark A. Mikofski\textsuperscript{1}}
	\IEEEauthorblockA{\textsuperscript{1}DNV, San Diego, CA, 92123, USA }
    \IEEEauthorblockA{\textsuperscript{2}NREL, Golden, CO, 80401, USA }
    \IEEEauthorblockA{\textsuperscript{3}Southern Company, Birmingham, AL, 35203, USA }}

\maketitle

\begin{abstract}
Under-performance of solar PV systems is an important issue that increases risks for stakeholders like developers, investors and operators. Recently some attention has focused on underestimation of inverter clipping losses as a possible source of over-prediction where sub-hourly solar variability is high. Several models and data sets have been analyzed over the past few years, with the aim of quantifying, predicting, and correcting underestimated clipping loss errors for sites with high DC/AC ratio and solar variability. In this report, we compare operational data with a machine learning model developed at NREL to correct energy assessments based on hourly weather. In addition, the model is expanded to include site meta-data like DC/AC ratio as factors, so that the model is flexible enough to be used for estimating clipping loss error in energy assessments for a wide variety of sites. 
\end{abstract}

\begin{IEEEkeywords}
inverter, clipping, solar, irradiance, variability, performance, modeling, TMY
\end{IEEEkeywords}

\section{Introduction}
In their "2020 Solar Risk Assessment" \cite{Matsui2020}, kWh Analytics warned that systematic underproduction across the industry exposes investors to increased risk. DNV found that a sample of 39 projects from 2019 were under-performing compared to their pre-construction energy assessments by up to 3\% on average as shown in Fig.~\ref{fig:project-underperformance}. NextEra reported that, "Sub-hourly solar resource variability impacts actual energy production by approximately 1-4\%." Cormode, \textit{et al.}, explained that energy assessments based on hourly weather, like typical meteorological year (TMY) data-sets, underestimate inverter clipping losses by up to 5\% for projects with intra-hourly solar variability, especially for high DC/AC ratios \cite{Cormode2019}. This is demonstrated in Fig.~\ref{fig:irradiance-and-power}, which shows irradiance and power output at 1-minute and 1-hour resolution in the top and bottom panels respectively. On July 10th, there is little intra-hour variability, a mostly clear day, and hourly and sub-hourly simulations are in close agreement. In contrast, July 13th has much higher solar variability, but no clipping is observed in the 1-hour output. However, the 1-minute resolution shows intermittent clipping all day long, indicating that not all of the available irradiance is used by the system, and the hourly simulation underestimates inverter clipping \cite{Kharait}.

\begin{figure}[htbp]
\centerline{\includegraphics[width=9cm]{fig1.png}}
\caption{Project-average validation results for solar energy assessments. Each project-year was adjusted for interannual variability by scaling production by the ratio of TGY to historical monthly insolation.}
\label{fig:project-underperformance}
\end{figure}


\begin{figure}[htbp]
\centerline{\includegraphics[width=9cm]{hourly_v_1-min_clipping.png}}
\caption{Irradiance input from the NIST test-bed weather station in Gaithersburg, MD, and predicted energy output using SolarFarmer at time-resolution of 1-minute, \textit{top panel}, and 1-hour, \textit{bottom panel}. The black dashed line in both panels shows the inverter-rated power, 260-kW.}
\label{fig:irradiance-and-power}
\end{figure}

Several methods have been proposed to correct clipping loss errors in hourly predictions \cite{Cormode2019,Kharait,Anderson2020,Bradford}, but only NextEra validated their method with operational data \cite{Bradford}. In this report, we adopt the NREL machine learning-based model \cite{Anderson2020} to predict clipping loss corrections and apply them to hourly energy assessments. The adjusted energy assessment is then validated against operational data from the same site, and we report on the average and distribution of validation errors.  We have obtained data from seven operating PV systems in different climates and with a range of DC/AC ratios. In this paper, we will present the first of these systems, outline our method, and present preliminary results. The final paper will present validation results from all seven sites plus the NIST ground array \cite{Boyd2017b} as a benchmark.

\section{Methods}

\subsection{Model}

The machine learning (ML) model developed at NREL has already been described in detail \cite{Anderson2020}. This model is an ensemble regressor model comprised of black-box random forest and XGBoost models. Model inputs are taken from satellite data, and include clearsky POA, clearsky GHI, POA, GHI, and cell temperature, as well as features derived from these fields. The model currently is trained on a fixed layout with a single DC/AC ratio, to match the physical site we are validating against. In the final manuscript, DC/AC ratio will be introduced as a new continuous factor, and the model will be retrained with a range of DC/AC ratios so appropriate clipping loss errors can be calculated for a wide range of common project configurations. As described in \cite{Anderson2020}, the model has been trained using high quality irradiance measurements from SURFRAD \cite{Augustine2000} and 1-minute to 30-minute clipping loss errors predicted using SAM \cite{Freeman2018}. The resulting trained model is then used to make predictions at the operational sites.


\subsection{Validation}
A challenge in validating the clipping loss correction model is the lack of quality high-frequency irradiance measurements that are co-located with operational PV systems. Therefore this model differs from traditional approaches for model training and validation because it relies on SURFRAD measurements and SAM simulated data to train the model, but uses operational data from different sites for validation. So no holdout methods are required in validating this model because the training and validation data sets are already independent. The downside of this method is that any systematic biases in SAM are introduced into the clipping loss correction model. In future work, if a large population of high-frequency irradiance and co-located operational data are obtained, the model can be retrained holding out a subset of the data for validation. This procedure would have the benefit of incorporating any physical effects that are not captured by SAM.

Quality of operational datasets is vital to avoid disruption of validation. Data points in the operational dataset related to plant sub-performance instances are considered to be outliers from the validation's standpoint and need to be removed. For this validation effort, data points where the difference between actual and simulated power is more than 50\% were removed. 


Several metrics are used to measure the reduction in clipping loss errors from the hourly energy assessment, and identify if there are any unknown factors that are systematically affecting the results.

\begin{itemize}
\item \textbf{Bias}: delta between the corrected energy assessment predictions and the measured operational data, so positive bias means over-predicted energy output 
\begin{equation}
\Delta E={E_\text{predicted}} - {E_\text{measured}}\label{eq:bias}
\end{equation}
Where $E$ is the energy output.
\item \textbf{Mean bias error (MBE)}: average of the bias, beware a low MBE might hide seasonal or diurnal bias
\begin{equation}
\mathit{MBE}=\frac{\sum_{n=1}^N{\Delta_n}}{N}\label{eq:mbe}
\end{equation}
Where $n$ is a single timestep, and $N$ is the total period of time. For example, with hourly timesteps, $N=8760\text{[hrs]}$, the number of timesteps in a typical annual energy assessment.
\item \textbf{Root mean squared error}: squares of the bias are positive, removing any cancellation of opposing errors, and therefore gives a measure of the width of the error distribution
\begin{equation}
\mathit{RMSE}=\sqrt{\frac{\sum_{n=1}^N{\Delta_n^2}}{N}}\label{eq:rmse}
\end{equation}
\item \textbf{Mean absolute error (MAE)}: a measure of the average bias that removes any cancellation of opposing errors
\begin{equation}
\mathit{MAE}=\frac{\sum_{n=1}^N{\left|\Delta_n\right|}}{N}\label{eq:mae}
\end{equation}
\item \textbf{Error distribution}: a histogram of the bias should look like a normal distribution, centered at zero
\item \textbf{Temporal correlation}: a plot of the bias aggregated monthly or by hour of the day should have a uniform flat distribution
\item \textbf{Auto-correlation}: a plot of the bias versus dependent parameter, energy output ($E$) should also have a uniform distribution
\item \textbf{Cross-correlation}: plots of the bias versus independent parameters like irradiance, DC/AC ratio, clearness index, \textit{etc.}, should all have a uniform distributions
\end{itemize}


\section{Results}

\subsection{Model Parameters}

For the validation, an operational site in the New England region was chosen. The system parameters of this solar site are provided in Table~\ref{table1}.

\begin{table}[htbp]
\caption{Validation Site System Parameters}
\begin{center}
\begin{tabular}{ |c|c| } 
\hline
Array tilt & 20$^{\circ}$ \\
\hline
Array azimuth & 180$^{\circ}$ \\
\hline
Ground coverage ratio & 0.46 \\
\hline
DC capacity & 6,000 kW \\
\hline
AC capacity & 4,560 kW \\
\hline
\end{tabular}
\end{center}
\label{table1}
\end{table}

30-minute NSRDB data from the site was fed into the ML model to make clipping error predictions. Model input parameters included the following:
\begin{itemize}
\item Min-max normalized POA, GHI, clearsky POA, clearsky GHI, and cell temperature
\item First-order differenced normalized POA
\item Difference between Clearsky POA and POA
\end{itemize}

\subsection{Validation Metrics}
The following figures show the bias before and after adjusting for subhourly clipping underestimation.

\begin{figure}[htbp]
\centerline{\includegraphics[width=9cm]{Error_before_correction_3.png}}
\caption{Bias between model and actual before clipping (mean=104 kW)}
\label{fig:before-clipping-error}
\end{figure}

\begin{figure}[htbp]
\centerline{\includegraphics[width=9cm]{Error_after_correction_3.png}}
\caption{Bias between model and actual after clipping (mean=94 kW)}
\label{fig:after-clipping-error}
\end{figure}


\section{Conclusions}
Over-prediction occurs in typical energy assessments that use hourly data if the sites have sub-hourly solar variability and high DC/AC because clipping losses are underestimated. These errors have been quantified using a machine-learning model trained on high-frequency solar irradiance data and simulated operational data to create clipping loss error corrections. The corrections were applied to a typical hourly energy assessment of an existing operational site and compared to the measured output from the same site. The mean bias error was reduced from 5.07\% to 4.54\%. The final report will report the validation results for an additional 7 operating PV sites, with the goal of providing a standard method for correcting clipping loss errors from typical pre-construction energy assessments, and therefore decreasing investor risk.

\section*{Acknowledgment}

This work was authored in part by Alliance for Sustainable Energy, LLC, the manager and operator of the National Renewable Energy Laboratory for the U.S. Department of Energy (DOE) under Contract No. DE-AC36-08GO28308. Funding provided by the U.S. Department of Energy’s Office of Energy Efficiency and Renewable Energy (EERE) under Solar Energy Technologies Office (SETO) Agreement Numbers 34348. The views expressed in the article do not necessarily represent the views of the DOE or the U.S. Government. The U.S. Government retains and the publisher, by accepting the article for publication, acknowledges that the U.S. Government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this work, or allow others to do so, for U.S. Government purposes.

\bibliographystyle{IEEEtran}
% argument is your BibTeX string definitions and bibliography database(s)
\bibliography{IEEEabrv,bibliography}

\end{document}