EOS/docs/akkudoktoreos/prediction.md

% SPDX-License-Identifier: Apache-2.0
(prediction-page)=

# Predictions

Predictions, along with simulations and measurements, form the foundation upon which energy
optimization is executed. In EOS, a standard set of predictions is managed, including:

- Household Load Prediction
- Electricity Price Prediction
- Feed In Tariff Prediction
- PV Power Prediction
- Weather Prediction

## Storing Predictions

EOS stores predictions in a **key-value store**, where the term `prediction key` refers to the
unique key used to retrieve specific prediction data.

## Prediction Providers

Most predictions can be sourced from various providers. The specific provider to use is configured
in the EOS configuration and can be set by prediction type. For example:

```json
{
  "weather": {
    "provider": "ClearOutside"
  }
}
```

Some providers offer multiple prediction keys. For instance, a weather provider might provide data
to prediction keys like:

- `weather_temp_air` (air temperature)
- `weather_wind_speed` (wind speed)

### Prediction Import Providers

Prediction import providers allow you to import prediction data from:

- A file or a JSON string (primarily for initialization), or
- The **PUT** `/v1/prediction/import/{provider_id}` endpoint (recommended for dynamic updates).

An external entity may update the file or JSON string whenever new prediction data becomes
available. However, for production use or regular updates, you should prefer the **PUT** endpoint
over file or JSON string based imports.

:::{admonition} Warning
:class: warning
Be aware that providing dynamic values via a file and/or JSON string **alongside other value
sources** can lead to unintended data overwrites. Moreover, after a restart, even outdated values
from the configuration may be reloaded. To avoid these issues, use the **PUT** endpoint for live
data and rely on file/JSON imports only for initial setup.
:::

The prediction data must be provided in one of the following formats:

#### 1. DateTimeData

A dictionary with the following structure:

```json
    {
        "start_datetime": "2024-01-01 00:00:00",
        "interval": "1 hour",
        "<prediction key>": [value, value, ...],
        "<prediction key>": [value, value, ...],
        ...
    }
```

If `start_datetime` is not provided EOS defaults to the `start_datetime` of the current energy
management run. If `interval` is not provided EOS defaults to one hour.

#### 2. DateTimeDataFrame

A JSON string created from a [pandas](https://pandas.pydata.org/docs/index.html) dataframe with a
`DatetimeIndex`. Use
[pandas.DataFrame.to_json(orient="index")](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_json.html#pandas.DataFrame.to_json).
The column name of the data must be the same as the names of the `prediction key`s.

#### 3. DateTimeSeries

A JSON string created from a [pandas](https://pandas.pydata.org/docs/index.html) series with a
`DatetimeIndex`. Use
[pandas.Series.to_json(orient="index")](https://pandas.pydata.org/docs/reference/api/pandas.Series.to_json.html#pandas.Series.to_json).

## Adjusted Predictions

Certain prediction keys include an `_adjusted` suffix, such as `load_total_adjusted`. These
predictions are adjusted by real data from your system's measurements if given to enhance accuracy.

For example, the load prediction provider `LoadAkkudoktor` takes generic load data assembled by
Akkudoktor.net, maps that to the yearly energy consumption given in the configuration option
`loadakkudoktor_year_energy`, and finally adjusts the predicted load by the `loads`
of your system.

## Prediction Updates

Predictions are updated at the start of each energy management run, i.e., when EOS performs
optimization. Key considerations for updates include:

- Predictions sourced from online providers are usually rate-limited to one retrieval per hour.
- Only predictions with a configured provider are updated.
- Some providers may not support all generic prediction keys, leading to potential gaps
  in updated predictions even after update.

## Accessing Predictions

Prediction data can be accessed using the EOS **REST API** via the `/v1/prediction/<...>` endpoints.

In a standard configuration, the [**REST API**](http://localhost:8503/docs) of a running EOS instance
is available at [http://localhost:8503/docs](http://localhost:8503/docs). This link provides access to
the API documentation and allows you to explore available endpoints interactively.

To view all available prediction keys, use the **GET** `/v1/prediction/keys` endpoint.

If no keys are displayed, or if the ones you need are missing, it indicates that your configuration
lacks the necessary prediction provider settings. You can configure prediction providers by using
the **PUT** `/v1/config` endpoint. You may save your configuration to the EOS configuration file.

## Electricity Price Prediction

Prediction keys:

- `elecprice_marketprice_wh`: Electricity market price per Wh (€/Wh).
- `elecprice_marketprice_kwh`: Electricity market price per kWh (€/kWh).

Configuration options:

- `elecprice`: Electricity price configuration.

  - `provider`: Electricity price provider id of provider to be used.

    - `ElecPriceAkkudoktor`: Retrieves from Akkudoktor.net.
    - `ElecPriceEnergyCharts`: Retrieves from Energy-Charts.info.
    - `ElecPriceSMARD`: Retrieves German/Luxembourg day-ahead prices directly from SMARD.de.
    - `ElecPriceFixed`: Caluclates from configured time window prices.
    - `ElecPriceImport`: Imports from a file or JSON string or by endpoint data provision.

  - `charges_kwh`: Electricity price charges (€/kWh).
  - `charge_components_kwh`: Named constant net-charge components (€/kWh), summed automatically.
  - `network_fees_kwh.windows`: Recurring time windows with variable net-grid fees (€/kWh).
  - `vat_rate`: VAT rate factor applied to electricity price when charges are used (default: 1.19).
  - `elecpricefixed.time_windows.windows`: The time windows with associated electricity prices.
  - `elecpriceimport.import_file_path`: Path to the file to import electricity price forecast data from.
  - `elecpriceimport.import_json`: JSON string, dictionary of electricity price forecast value lists.
  - `energycharts.bidding_zone`: Bidding zone Energy Charts shall provide price data for.

### ElecPriceAkkudoktor Provider

The `ElecPriceAkkudoktor` provider retrieves electricity prices directly from **Akkudoktor.net**,
which supplies price data for the next 24 hours. For periods beyond 24 hours, the provider generates
prices by extrapolating historical price data combined with the most recent actual prices obtained
from Akkudoktor.net. Electricity price charges given in the `charges_kwh` configuration
option are added.

### ElecPriceEnergyCharts Provider

The `ElecPriceEnergyCharts` provider retrieves day-ahead electricity market prices from
[Energy-Charts.info](https://www.Energy-Charts.info). It supports both short-term and extended
forecasting by combining real-time market data with historical price trends.

- For the next 24 hours, market prices are fetched directly from Energy-Charts.info.
- For periods beyond 24 hours, prices are estimated using extrapolation based on historical data
  and the latest available market values.

Charges and VAT

- If constant charges or a matching `network_fees_kwh` window is greater than 0, the electricity
  price is calculated as: `(market price + charges_kwh + sum(charge components) + network fee)
  * vat_rate` where
  `vat_rate` is configurable (default: 1.19 for 19% VAT).
- If all constant charges and network-fee windows are empty or zero, the electricity price is
  simply `market_price` (no VAT applied, preserving the existing raw-market-price behaviour).

`charges_kwh`, every `charge_components_kwh` value, and `network_fees_kwh` are net values. Time
windows repeat daily unless their optional `date` or `day_of_week` constraints are set. The first
matching window is used. Named components make statutory levies and the supplier markup auditable
instead of hiding them in one aggregate. The supplier-specific markup defaults to zero in the
example and must be taken from the electricity supply contract.

**Note:** For the most accurate forecasts, it is recommended to set the `historic_hours` parameter to 840.

### ElecPriceSMARD Provider

The `ElecPriceSMARD` provider retrieves quarter-hourly German/Luxembourg day-ahead prices directly from
the public SMARD chart-data endpoint. It requests the required weekly chunks only and caches the
combined response for one hour. Missing quarter-hour slots beyond the published day-ahead horizon are generated
with the same daily or weekly seasonal ETS forecast as the Energy-Charts provider. Constant charges,
time-variable network fees, and VAT are applied after forecasting the underlying market price.

The same raw SMARD series is also available as `FeedInTariffSMARD` for direct-marketing feed-in
revenue. Import-only charges, network fees, and VAT are deliberately not applied there.

### ElecPriceFixed Provider

The `ElecPriceFixed` provider calculates the day-ahead electricity market prices from the configuration
of electricity price time windows set up by the user.

### ElecPriceImport Provider

The `ElecPriceImport` provider is designed to import electricity prices from:

- A file or a JSON string (primarily for initialization). The data source can be given in the
  `import_file_path` or `import_json` configuration option.
- The **PUT** `/v1/prediction/import/ElecPriceImport` endpoint (recommended for dynamic updates).

The prediction key for the electricity price forecast data is:

- `elecprice_marketprice_wh`: Electricity market price per Wh (€/Wh).

The electricity price forecast data must be provided in one of the formats described in
<project:#prediction-import-providers>.

An external entity may update the file or JSON string whenever new prediction data becomes
available. However, for production use or regular updates, you should prefer the **PUT** endpoint
over file or JSON string based imports.

:::{admonition} Warning
:class: warning
Be aware that providing dynamic values via a file and/or JSON string **alongside other value
sources** can lead to unintended data overwrites. Moreover, after a restart, even outdated values
from the configuration may be reloaded. To avoid these issues, use the **PUT** endpoint for live
data and rely on file/JSON imports only for initial setup.
:::

## Feed In Tariff Prediction

Prediction keys:

- `feed_in_tariff_wh`: Feed-in tariff per Wh (€/Wh).
- `feed_in_tariff_kwh`: Feed-in tariff per kWh (€/kWh).

Configuration options:

- `feedintariff`: Feed-in tariff configuration.

  - `provider`: Feed in tariff provider id of provider to be used.

    - `FeedInTariffFixed`: Provides fixed feed in tariff values.
    - `FeedInTariffAkkudoktor`: Retrieves raw day-ahead market prices from the public
      Akkudoktor API without import charges or VAT.
    - `FeedInTariffEnergyCharts`: Retrieves Energy-Charts day-ahead market prices and extends
      them to the configured prediction horizon when necessary.
    - `FeedInTariffImport`: Imports from a file or JSON string or by endpoint data provision.
    - `FeedInTariffTibber`: Retrieves Tibber's native quarter-hour energy-price component.

  - `provider_settings.FeedInTariffFixed.feed_in_tariff_kwh`: Fixed feed-in tariff (€/kWh).
  - `provider_settings.FeedInTariffEnergyCharts.bidding_zone`: Energy-Charts bidding zone.
  - `provider_settings.FeedInTariffImport.import_file_path`: Path to the file containing imported
    feed-in tariff prediction data.
  - `provider_settings.FeedInTariffImport.import_json`: JSON string containing feed-in tariff
    prediction data.

### FeedInTariffAkkudoktor Provider

The `FeedInTariffAkkudoktor` provider uses raw day-ahead market prices from
`https://api.akkudoktor.net/prices` as `feed_in_tariff_wh`. It does not add electricity import
charges or VAT. Published prices are extended to the configured prediction horizon with the same
seasonal ETS or median fallback used by the Akkudoktor electricity-price provider.

The Akkudoktor endpoint currently forwards hourly market prices from aWATTar. With a 15-minute
optimization interval, EOS holds each hourly price constant for its four quarter-hour slots. This
keeps the slot grid consistent but does not create genuine quarter-hour market prices.

```json
{
  "feedintariff": {
    "direct_marketing_enabled": true,
    "provider": "FeedInTariffAkkudoktor"
  }
}
```

### FeedInTariffTibber Provider

The `FeedInTariffTibber` provider requests `priceInfo` and `priceInfoRange` with
`resolution: QUARTER_HOURLY` and preserves the native 15-minute timestamps. It uses Tibber's
`energy` spot-price component without the `tax` part or EOS electricity-price charges. The
end-customer `total` component is deliberately ignored.

The provider deliberately rejects hourly API responses instead of silently repeating them. It
reuses `elecprice.tibber.access_token` and `elecprice.tibber.home_id`, so no duplicate credentials
are needed.

```json
{
  "feedintariff": {
    "direct_marketing_enabled": true,
    "provider": "FeedInTariffTibber"
  }
}
```

### FeedInTariffEnergyCharts Provider

The `FeedInTariffEnergyCharts` provider uses the raw Energy-Charts day-ahead market price as the
feed-in tariff. It stores prices in `feed_in_tariff_wh` without adding electricity import charges
or VAT. The data is loaded from the Energy-Charts `/price` endpoint for the configured bidding
zone. The native Energy-Charts resolution, including quarter-hour data, is retained.

Energy-Charts usually supplies prices only for the published day-ahead period. If that data does
not cover the complete configured prediction horizon, the provider extends it as follows:

- With more than 800 hours of history, an ETS (Holt-Winters exponential smoothing) forecast with
  weekly seasonality is used.
- With more than 168 hours of history, an ETS forecast with daily seasonality is used.
- With less history, the median of the available values is used as a constant fallback.

The seasonal periods are adjusted to the source resolution. For example, quarter-hour data uses
four values per hour. Values already supplied by Energy-Charts are kept unchanged; only missing
future slots after the last published price are forecast. Consequently, a 15-minute optimization
uses four forecast values per hour without converting them to hourly averages.

When direct marketing is enabled, this feed-in-tariff series is passed to the optimizer separately
from the electricity purchase-price series. The successful Energy-Charts path therefore does not
derive the feed-in tariff by subtracting taxes, grid fees, or other charges from an end-customer
electricity price.

Example configuration:

```json
{
  "feedintariff": {
    "direct_marketing_enabled": true,
    "provider": "FeedInTariffEnergyCharts",
    "provider_settings": {
      "FeedInTariffEnergyCharts": {
        "bidding_zone": "DE-LU"
      }
    }
  }
}
```

### FeedInTariffImport Provider

The `FeedInTariffImport` provider is designed to import feed in tariff prices from:

- A file or a JSON string (primarily for initialization). The data source can be given in the
  `import_file_path` or `import_json` configuration option.
- The **PUT** `/v1/prediction/import/FeedInTariffImport` endpoint (recommended for dynamic updates).

The prediction key for the feed in tariff price forecast data is:

- `feed_in_tariff_wh`: Feed in tariff price per Wh (€/Wh).

The feed in tariff price forecast data must be provided in one of the formats described in
<project:#prediction-import-providers>.

An external entity may update the file or JSON string whenever new prediction data becomes
available. However, for production use or regular updates, you should prefer the **PUT** endpoint
over file or JSON string based imports.

:::{admonition} Warning
:class: warning
Be aware that providing dynamic values via a file and/or JSON string **alongside other value
sources** can lead to unintended data overwrites. Moreover, after a restart, even outdated values
from the configuration may be reloaded. To avoid these issues, use the **PUT** endpoint for live
data and rely on file/JSON imports only for initial setup.
:::

## Load Prediction

Prediction keys:

- `loadforecast_power_w`: Predicted load mean value (W).

Configuration options:

- `load`: Load configuration.

  - `provider`: Load provider id of provider to be used.

    - `LoadAkkudoktor`: Retrieves from local database.
    - `LoadVrm`: Retrieves data from the VRM API by Victron Energy.
    - `LoadImport`: Imports from a file or JSON string or by endpoint data provision.

  - `provider_settings.LoadAkkudoktor.loadakkudoktor_year_energy_kwh`: Yearly energy consumption (kWh).
  - `provider_settings.LoadVRM.load_vrm_token`: API token.
  - `provider_settings.LoadVRM.load_vrm_idsite`: load_vrm_idsite.
  - `provider_settings.LoadImport.loadimport_file_path`: Path to the file to import load forecast data from.
  - `provider_settings.LoadImport.loadimport_json`: JSON string, dictionary of load forecast value lists.

### LoadAkkudoktor Provider

The `LoadAkkudoktor` provider retrieves generic load data from the local database and scales
it to match the annual energy consumption specified in the
`LoadAkkudoktor.loadakkudoktor_year_energy` configuration option.

Prediction keys:

- `loadforecast_power_w`: Predicted load mean value (W).
- `loadakkudoktor_mean_power_w`: Predicted load mean value (W). Same as `loadforecast_power_w`.
- `loadakkudoktor_std_power_w`: Predicted load standard deviation (W).

### LoadAkkudoktorAdjusted Provider

The `LoadAkkudoktorAdjusted` provider retrieves generic load data from the local database and scales
it to match the annual energy consumption specified in the
`LoadAkkudoktor.loadakkudoktor_year_energy` configuration option. In addition, the provider refines
the forecast by incorporating available measured load data, ensuring a more realistic and
site-specific consumption profile.

Prediction keys:

- `loadforecast_power_w`: Adjusted load mean value (W).
- `loadakkudoktor_mean_power_w`: Predicted load mean value (W).
- `loadakkudoktor_std_power_w`: Predicted load standard deviation (W).

For details on how to supply load measurements, see the [Measurements](measurement-page) section.

### LoadVrm Provider

The `LoadVrm` provider retrieves load forecast data from the VRM API by Victron Energy.
To receive forecasts, the system data must be configured under Dynamic ESS in the VRM portal.
To query the forecasts, an API token is required, which can also be created in the VRM portal under
Preferences. This token must be stored in the EOS configuration along with the VRM-Installations-ID.

```json
    {
        "load": {
            "provider": "LoadVrm",
            "provider_settings": {
                "LoadVRM": {
                    "load_vrm_token": "dummy-token",
                    "load_vrm_idsite": 12345
                }
             }
        }
    }
```

The prediction key for the load forecast data is:

- `loadforecast_power_w`: Predicted load mean value (W).

### LoadImport Provider

The `LoadImport` provider is designed to import load forecast data from:

- A file or a JSON string (primarily for initialization). The data source can be given in the
  `loadimport_file_path` or `loadimport_json` configuration option.
- The **PUT** `/v1/prediction/import/LoadImport` endpoint (recommended for dynamic updates).

The prediction key for the load forecast data is:

- `loadforecast_power_w`: Predicted load mean value (W).

The load forecast data must be provided in one of the formats described in
<project:#prediction-import-providers>.

An external entity may update the file or JSON string whenever new prediction data becomes
available. However, for production use or regular updates, you should prefer the **PUT** endpoint
over file or JSON string based imports.

:::{admonition} Warning
:class: warning
Be aware that providing dynamic values via a file and/or JSON string **alongside other value
sources** can lead to unintended data overwrites. Moreover, after a restart, even outdated values
from the configuration may be reloaded. To avoid these issues, use the **PUT** endpoint for live
data and rely on file/JSON imports only for initial setup.
:::

## PV Power Prediction

Prediction keys:

- `pvforecast_ac_power`: Total DC power (W).
- `pvforecast_dc_power`: Total AC power (W).

Configuration options:

- `general`: General configuration.

  - `latitude`: Latitude in decimal degrees, between -90 and 90, north is positive (ISO 19115) (°)"
  - `longitude`: Longitude in decimal degrees, within -180 to 180 (°)

- `pvforecast`: PV forecast configuration.

  - `provider`: PVForecast provider id of provider to be used.

    - `PVForecastAkkudoktor`: Retrieves from Akkudoktor.net.
    - `PVForecastVrm`: Retrieves data from the VRM API by Victron Energy.
    - `PVForecastPVNode`: Retrieves native 15-minute forecasts from the pvnode.com API.
    - `PVForecastForecastSolar`: Retrieves forecasts from the free Forecast.Solar API.
    - `PVForecastSolcast`: Retrieves forecasts from the Solcast rooftop-site API.
    - `PVForecastImport`: Imports from a file or JSON string or by endpoint data provision.
    - `PVForecastAkkudoktorLocal`: Computes the forecast inside EOS from Open-Meteo weather with pvlib.

  - `planes[].surface_tilt`: Tilt angle from horizontal plane. Ignored for two-axis tracking.
  - `planes[].surface_azimuth`: Orientation (azimuth angle) of the (fixed) plane.
                                Clockwise from north (north=0, east=90, south=180, west=270).
  - `planes[].userhorizon`: Elevation of horizon in degrees, at equally spaced azimuth clockwise from north.
  - `planes[].peakpower`: Nominal power of PV system in kW.
  - `planes[].pvtechchoice`: PV technology. One of 'crystSi', 'CIS', 'CdTe', 'Unknown'.
  - `planes[].mountingplace`: Type of mounting for PV system.
                              Options are 'free' for free-standing and 'building' for building-integrated.
  - `planes[].loss`: Sum of PV system losses in percent
  - `planes[].trackingtype`: Type of suntracking.
                             0=fixed,
                             1=single horizontal axis aligned north-south,
                             2=two-axis tracking,
                             3=vertical axis tracking,
                             4=single horizontal axis aligned east-west,
                             5=single inclined axis aligned north-south.
  - `planes[].optimal_surface_tilt`: Calculate the optimum tilt angle. Ignored for two-axis tracking.
  - `planes[].optimalangles`: Calculate the optimum tilt and azimuth angles. Ignored for two-axis tracking.
  - `planes[].albedo`: Proportion of the light hitting the ground that it reflects back.
  - `planes[].module_model`: Model of the PV modules of this plane.
  - `planes[].inverter_model`: Model of the inverter of this plane.
  - `planes[].inverter_paco`: AC power rating of the inverter. [W]
  - `planes[].modules_per_string`: Number of the PV modules of the strings of this plane.
  - `planes[].strings_per_inverter`: Number of the strings of the inverter of this plane.
  - `provider_settings.import_file_path`: Path to the file to import PV forecast data from.
  - `provider_settings.import_json`: JSON string, dictionary of PV forecast value lists.
  - `provider_settings.PVForecastPVNode.api_key`: pvnode.com API key.
  - `provider_settings.PVForecastPVNode.site_id`: pvnode.com saved-site id. Leave empty for inline mode.
  - `provider_settings.PVForecastPVNode.forecast_days`: Forecast horizon in days (1-7).
  - `provider_settings.PVForecastForecastSolar.api_key`: Forecast.Solar API key (optional).
  - `provider_settings.PVForecastSolcast.api_key`: Solcast API key.
  - `provider_settings.PVForecastSolcast.site_id`: Solcast rooftop resource (site) id.
  - `provider_settings.PVForecastAkkudoktorLocal.resolution_minutes`: 15 or 60.
  - `provider_settings.PVForecastAkkudoktorLocal.forecast_days`: 1-16, empty derives it from `prediction.hours`.
  - `provider_settings.PVForecastAkkudoktorLocal.past_days`: 0-92, empty derives it from `prediction.historic_hours`.
  - `provider_settings.PVForecastAkkudoktorLocal.weather_models`: Open-Meteo models; several are averaged.
  - `provider_settings.PVForecastAkkudoktorLocal.transposition_model`: pvlib sky-diffuse model.
  - `provider_settings.PVForecastAkkudoktorLocal.albedo`: Fallback albedo for planes without their own.
  - `provider_settings.PVForecastAkkudoktorLocal.inverter_efficiency`: Nominal inverter efficiency.
  - `provider_settings.PVForecastAkkudoktorLocal.temperature_coefficient`: Module power coefficient in %/degC.
  - `provider_settings.PVForecastAkkudoktorLocal.apply_iam`: Apply the ASHRAE incidence-angle modifier.
  - `provider_settings.PVForecastAkkudoktorLocal.shift_to_interval_start`: Relabel Open-Meteo interval-end stamps.
  - `provider_settings.PVForecastAkkudoktorLocal.calibration_*`: Self-calibration against measured PV production.

---

Detailed definitions taken from
[PVGIS](https://joint-research-centre.ec.europa.eu/photovoltaic-geographical-information-system-pvgis/getting-started-pvgis/pvgis-user-manual_en).

- `pvtechchoice`

The performance of PV modules depends on the temperature and on the solar irradiance, but the exact
dependence varies between different types of PV modules. At the moment we can estimate the losses
due to temperature and irradiance effects for the following types of modules: crystalline silicon
cells; thin film modules made from CIS or CIGS and thin film modules made from Cadmium Telluride
(CdTe).

For other technologies (especially various amorphous technologies), this correction cannot be
calculated here. If you choose one of the first three options here the calculation of performance
will take into account the temperature dependence of the performance of the chosen technology. If
you choose the other option (other/unknown), the calculation will assume a loss of 8% of power due
to temperature effects (a generic value which has found to be reasonable for temperate climates).

PV power output also depends on the spectrum of the solar radiation. PVGIS can calculate how the
variations of the spectrum of sunlight affects the overall energy production from a PV system. At
the moment this calculation can be done for crystalline silicon and CdTe modules. Note that this
calculation is not yet available when using the NSRDB solar radiation database.

- `peakpower`

This is the power that the manufacturer declares that the PV array can produce under standard test
conditions (STC), which are a constant 1000W of solar irradiation per square meter in the plane of
the array, at an array temperature of 25°C. The peak power should be entered in kilowatt-peak (kWp).
If you do not know the declared peak power of your modules but instead know the area of the modules
and the declared conversion efficiency (in percent), you can calculate the peak power as
power = area \* efficiency / 100.

Bifacial modules: PVGIS doesn't make specific calculations for bifacial modules at present. Users
who wish to explore the possible benefits of this technology can input the power value for Bifacial
Nameplate Irradiance. This can also be can also be estimated from the front side peak power P_STC
value and the bifaciality factor, φ (if reported in the module data sheet) as:
P_BNPI = P_STC \* (1 + φ \* 0.135). NB this bifacial approach is not appropriate for BAPV or BIPV
installations or for modules mounting on a N-S axis i.e. facing E-W.

- `loss`

The estimated system losses are all the losses in the system, which cause the power actually
delivered to the electricity grid to be lower than the power produced by the PV modules. There are
several causes for this loss, such as losses in cables, power inverters, dirt (sometimes snow) on
the modules and so on. Over the years the modules also tend to lose a bit of their power, so the
average yearly output over the lifetime of the system will be a few percent lower than the output
in the first years.

We have given a default value of 14% for the overall losses. If you have a good idea that your value
will be different (maybe due to a really high-efficiency inverter) you may reduce this value a little.

- `mountingplace`

For fixed (non-tracking) systems, the way the modules are mounted will have an influence on the
temperature of the module, which in turn affects the efficiency. Experiments have shown that if the
movement of air behind the modules is restricted, the modules can get considerably hotter
(up to 15°C at 1000W/m2 of sunlight).

In PVGIS there are two possibilities: free-standing, meaning that the modules are mounted on a rack
with air flowing freely behind the modules; and building- integrated, which means that the modules
are completely built into the structure of the wall or roof of a building, with no air movement
behind the modules.

Some types of mounting are in between these two extremes, for instance if the modules are mounted on
a roof with curved roof tiles, allowing air to move behind the modules. In such cases, the
performance will be somewhere between the results of the two calculations that are possible here.

- `userhorizon`

Elevation of horizon in degrees, at equally spaced azimuth clockwise from north. In the user horizon
data each number represents the horizon height in degrees in a certain compass direction around the
point of interest. The horizon heights should be given in a clockwise direction starting at North;
that is, from North, going to East, South, West, and back to North. The values are assumed to
represent equal angular distance around the horizon. For instance, if you have 36 values, the first
point is due north, the next is 10 degrees east of north, and so on, until the last point, 10
degrees west of north.

![Userhorizon PVGIS](../_static/horizon_eyefish_en.png)

Most of the configuration options are in line with the
[PVLib](https://pvlib-python.readthedocs.io/en/stable/_modules/pvlib/iotools/pvgis.html) definition
for PVGIS data.

Detailed definitions from **PVLib** for PVGIS data.

- `surface_tilt`:

Tilt angle from horizontal plane.

![Tilt PVGIS](../_static/slope.gif)

- `surface_azimuth`

Orientation (azimuth angle) of the (fixed) plane. Clockwise from north (north=0, east=90, south=180,
west=270). This is offset 180 degrees from the convention used by PVGIS.

![Azimuth PVGIS](../_static/azimuth.gif)

### PVForecastAkkudoktor Provider

The `PVForecastAkkudoktor` provider retrieves the PV power forecast data directly from
**Akkudoktor.net**.

The following prediction configuration options of the PV system must be set:

- `general.latitude`: Latitude in decimal degrees, between -90 and 90, north is positive (ISO 19115) (°)"
- `general.longitude`: Longitude in decimal degrees, within -180 to 180 (°)

For each plane of the PV system the following configuration options must be set:

- `pvforecast.planes[].surface_tilt`: Tilt angle from horizontal plane. Ignored for two-axis tracking.
- `pvforecast.planes[].surface_azimuth`: Orientation (azimuth angle) of the (fixed) plane.
                                         Clockwise from north (north=0, east=90, south=180, west=270).
- `pvforecast.planes[].userhorizon`: Elevation of horizon in degrees, at equally spaced azimuth clockwise from north.
- `pvforecast.planes[].inverter_paco`: AC power rating of the inverter. [W]
- `pvforecast.planes[].peakpower`: Nominal power of PV system in kW.

Example:

```Python
{
  "general": {
    "latitude": 50.1234,
    "longitude": 9.7654,
  },
  "pvforecast": {
    "provider": "PVForecastAkkudoktor",
    "planes": [
      {
        "peakpower": 5.0,
        "surface_azimuth": -10,
        "surface_tilt": 7,
        "userhorizon": [20, 27, 22, 20],
        "inverter_paco": 10000
      },
      {
        "peakpower": 4.8,
        "surface_azimuth": -90,
        "surface_tilt": 7,
        "userhorizon": [30, 30, 30, 50],
        "inverter_paco": 10000
      },
      {
        "peakpower": 1.4,
        "surface_azimuth": -40,
        "surface_tilt": 60,
        "userhorizon": [60, 30, 0, 30],
        "inverter_paco": 2000
      },
      {
        "peakpower": 1.6,
        "surface_azimuth": 5,
        "surface_tilt": 45,
        "userhorizon": [45, 25, 30, 60],
        "inverter_paco": 1400
      }
    ]
  }
}
```

### PVForecastVrm Provider

The `PVForecastVrm` provider retrieves pv power forecast data from the VRM API by Victron Energy.
To receive forecasts, the system data must be configured under Dynamic ESS in the VRM portal.
To query the forecasts, an API token is required, which can also be created in the VRM portal under Preferences.
This token must be stored in the EOS configuration along with the VRM-Installations-ID.

```python
    {
        "pvforecast": {
            "provider": "PVForecastVrm",
            "provider_settings": {
                "pvforecast_vrm_token": "dummy-token",
                "pvforecast_vrm_idsite": 12345
            }
    }
```

The prediction keys for the PV forecast data are:

- `pvforecast_dc_power`: Total DC power (W).

### PVForecastImport Provider

The `PVForecastImport` provider is designed to import PV forecast data from a file or a JSON
string. An external entity should update the file or JSON string whenever new prediction data
becomes available.

The prediction keys for the PV forecast data are:

- `pvforecast_ac_power`: Total AC power (W).
- `pvforecast_dc_power`: Total DC power (W).

The PV forecast data must be provided in one of the formats described in
<project:#prediction-import-providers>. The data source can be given in the
`import_file_path` or `import_json` configuration option.

The data may additionally or solely be provided by the
**PUT** `/v1/prediction/import/PVForecastImport` endpoint.

### PVForecastPVNode Provider

The `PVForecastPVNode` provider retrieves native 15-minute PV power forecasts from the
[pvnode.com](https://pvnode.com) V2 API. Register a site in the pvnode web app and store the API
key together with the site id in the EOS configuration (saved-site mode). Alternatively, leave the
site id empty to send the configured `planes` geometry inline.

```python
    {
        "pvforecast": {
            "provider": "PVForecastPVNode",
            "provider_settings": {
                "PVForecastPVNode": {
                    "api_key": "your-pvnode-key",
                    "site_id": "your-site-id",
                    "forecast_days": 2
                }
            }
        }
    }
```

The prediction keys for the PV forecast data are:

- `pvforecast_ac_power`: Total AC power (W).
- `pvforecast_dc_power`: Total DC power (W).

### PVForecastAkkudoktorLocal Provider

The `PVForecastAkkudoktorLocal` provider does not call a PV forecast service at all. It fetches raw
irradiance and weather from [Open-Meteo](https://open-meteo.com) and runs the whole modelling
chain locally with `pvlib`:

    solar position -> horizon shading -> transposition to the module plane ->
    incidence-angle modifier -> cell temperature -> PVWatts DC -> inverter AC

Three properties make it the right default for long-horizon optimization:

- **Horizon.** Up to 16 forecast days at 15-minute resolution from a single request. Services
  that wrap Open-Meteo cut the horizon much shorter, which starves
  `optimization.tail_horizon_hours`.
- **Call budget.** One request per hour against a ~10k/day non-commercial budget, instead of
  competing for someone else's upstream quota.
- **Honest parameters.** `albedo`, inverter efficiency and the module temperature coefficient are
  real configuration rather than constants baked into a service URL.

No API key is required. The location comes from `general.latitude`/`longitude` and the geometry
from the configured `planes`, including `userhorizon`, `trackingtype`, `mountingplace` and `loss`.

```python
    {
        "pvforecast": {
            "provider": "PVForecastAkkudoktorLocal",
            "provider_settings": {
                "PVForecastAkkudoktorLocal": {
                    "resolution_minutes": 15,
                    "weather_models": ["icon_seamless", "ecmwf_ifs025", "gfs_seamless"],
                    "calibration_enabled": true
                }
            }
        }
    }
```

#### Improving accuracy

**Model ensemble.** Listing several models in `weather_models` averages them per variable. This
is the cheapest reliable way to cut irradiance forecast error and costs no extra API calls,
because Open-Meteo returns all members in the same response. `icon_seamless` (DWD, strong over
Central Europe), `ecmwf_ifs025` and `gfs_seamless` are a reasonable trio.

**Self-calibration.** With `calibration_enabled` the provider compares its own model against
measured PV production over the past `calibration_days` and fits a correction:

- a **global scale factor**, which absorbs a wrong `peakpower`, soiling, degradation and any
  systematic offset in the loss assumption;
- **per-solar-azimuth factors**, which absorb near-field shading that a coarse `userhorizon`
  cannot express - a chimney, a tree, a neighbouring roof.

Each bin is weighted by its modelled energy and shrunk toward the global factor by
`calibration_prior_kwh`, so a thinly sampled bin cannot swing the forecast on its own, and every
factor is clamped to `[calibration_min_factor, calibration_max_factor]` so a broken meter cannot
either. The fitted factors and the resulting change in mean absolute error are logged at INFO
level on every update.

By default, calibration also rejects probable outage or curtailment days. It estimates the
healthy plant ratio from `calibration_reference_days`, excludes days below
`calibration_outage_threshold` of that reference, and falls back to the most recent
`calibration_min_healthy_days` when the normal calibration window contains an outage. This keeps
a battery or inverter failure that limits PV to local demand from becoming a permanent forecast
loss. Set `calibration_outage_filter_enabled` to false only when measured curtailed production,
rather than available PV potential, is the intended prediction target.

The measurement cadence controls the detail that can be learned. Hourly cumulative meter
readings calibrate hourly energy while the native Open-Meteo/pvlib chain continues to supply the
15-minute shape. If every configured PV meter supplies genuine 15-minute readings, calibration
automatically uses those native slots as well. It never interpolates hourly counters into an
invented quarter-hour profile. Azimuth factors are interpolated smoothly between bin centres so
they do not introduce steps into the EMS input curve. The shape fit uses all healthy days in the
reference window, while the global factor still follows the shorter recent window. Finally, the
shape is normalized per forecast day: it redistributes the calibrated energy across the day's
15-minute slots without changing that day's global kWh correction (unless the physical inverter
limit clips a peak).

Calibration requires `measurement.pv_production_emr_keys` to be configured and fed with
cumulative PV production meter readings in kWh:

```python
    {
        "measurement": {
            "pv_production_emr_keys": ["pv1_emr"]
        }
    }
```

Note what this does and does not correct. The comparison runs on past intervals, where the
Open-Meteo rows are analysed rather than forecast weather, so it isolates the error of the *PV
model* from the error of the *weather forecast*. That is deliberate: only the former is
systematic enough to correct. A cloudy day that the weather model got wrong stays wrong.

**Choosing the window.** `calibration_days` trades responsiveness against stability. A long
window averages more weather and gives a steadier factor, but it also reaches back into the
plant's own history - and calibration cannot tell a modelling error from a *plant* error. A
window that spans a string outage, an inverter derating or a period of heavy soiling will fit
that fault as if it were a permanent property of the installation, and the forecast stays
depressed long after the plant recovered. Before trusting a factor, compare modelled against
measured energy *per day*: a run of days at a markedly different ratio is a plant event, and
the window should start after it.

**Bins need data.** The per-azimuth factors are only worth fitting when the window holds enough
daylight hours to populate the bins - roughly a few hundred, so several weeks at the default
15 degrees. With a short window, set `calibration_azimuth_bin_degrees` to 0 to fit the global
factor alone. One well-determined number beats twenty-four noisy ones.

**Geometry is out of scope.** Calibration is a scale factor on the model's output; it never
touches `userhorizon`, `surface_tilt`, `surface_azimuth` or `peakpower`. That makes it the right
tool for *multiplicative* errors and the wrong one for geometric errors. Horizon shading in
particular gates the beam component as a hard function of both solar azimuth *and* elevation, so
a per-azimuth scale factor cannot move the edge of the shadow to where it belongs, and what it
learns in one season is wrong in the next, when the sun crosses the same azimuth at a different
height. A wrong horizon should be corrected in `userhorizon`, not calibrated away.

`scripts/pvforecast_backtest.py` scores configuration variants against the stored meter readings
without waiting for new forecasts to come true, which is the quickest way to test a geometry
change:

```bash
    python scripts/pvforecast_backtest.py --days 30 --tilt 88 --azimuth 175
```

#### Conventions

Two timing conventions are handled explicitly and are worth knowing when comparing against other
providers:

- Open-Meteo radiation values are the mean over the **preceding** interval, so the representative
  sun position for a value stamped `t` is `t - interval/2`.
- EOS records label an interval by its **start**, so a value stamped `t` by Open-Meteo is stored
  at `t - interval`. Set `shift_to_interval_start` to false to keep the raw stamps.

Note also that Open-Meteo's `direct_radiation` is beam irradiance on the *horizontal* plane; the
DNI this chain needs is `direct_normal_irradiance`.

The prediction keys for the PV forecast data are:

- `pvforecast_ac_power`: Total AC power (W).
- `pvforecast_dc_power`: Total DC power (W).

### PVForecastForecastSolar Provider

The `PVForecastForecastSolar` provider retrieves PV power forecasts from the free
[Forecast.Solar](https://forecast.solar) API. No API key is required for the public endpoint; an
optional key raises the rate limit. The location is taken from `general.latitude`/`longitude` and
the system geometry from the configured `planes` (one request per plane, summed per timestamp).

```python
    {
        "pvforecast": {
            "provider": "PVForecastForecastSolar",
            "provider_settings": {
                "PVForecastForecastSolar": {
                    "api_key": null
                }
            }
        }
    }
```

The prediction keys for the PV forecast data are:

- `pvforecast_ac_power`: Total AC power (W).
- `pvforecast_dc_power`: Total DC power (W).

### PVForecastSolcast Provider

The `PVForecastSolcast` provider retrieves PV power forecasts from the
[Solcast](https://solcast.com) rooftop-site API. Register a rooftop site in the Solcast web app and
store the API key together with the resource (site) id in the EOS configuration. Note that the free
tier limits the number of API calls per day.

```python
    {
        "pvforecast": {
            "provider": "PVForecastSolcast",
            "provider_settings": {
                "PVForecastSolcast": {
                    "api_key": "your-solcast-key",
                    "site_id": "your-resource-id"
                }
            }
        }
    }
```

The prediction keys for the PV forecast data are:

- `pvforecast_ac_power`: Total AC power (W).
- `pvforecast_dc_power`: Total DC power (W).

## Weather Prediction

Prediction keys:

- `weather_dew_point`: Dew Point (°C)
- `weather_dhi`: Diffuse Horizontal Irradiance (W/m2)
- `weather_dni`: Direct Normal Irradiance (W/m2)
- `weather_feels_like`: Feels Like (°C)
- `weather_fog`: Fog (%)
- `weather_frost_chance`: Chance of Frost
- `weather_ghi`: Global Horizontal Irradiance (W/m2)
- `weather_high_clouds`: High Clouds (% Sky Obscured)
- `weather_low_clouds`: Low Clouds (% Sky Obscured)
- `weather_medium_clouds`: Medium Clouds (% Sky Obscured)
- `weather_ozone`: Ozone (du)
- `weather_precip_amt`: Precipitation Amount (mm)
- `weather_precip_prob`: Precipitation Probability (%)
- `weather_preciptable_water`: Precipitable Water (cm)
- `weather_precip_type`: Precipitation Type
- `weather_pressure`: Pressure (mb)
- `weather_relative_humidity`: Relative Humidity (%)
- `weather_temp_air`: Temperature (°C)
- `weather_total_clouds`: Total Clouds (% Sky Obscured)
- `weather_visibility`: Visibility (m)
- `weather_wind_direction`: Wind Direction (°)
- `weather_wind_speed`: Wind Speed (kmph)

Configuration options:

- `weather`: General weather configuration.

  - `provider`: Load provider id of provider to be used.

    - `BrightSky`: Retrieves from [BrightSky](https://api.brightsky.dev).
    - `ClearOutside`: Retrieves from [ClearOutside](https://clearoutside.com/forecast).
    - `OpenMeteo`: Retrieves from [OpenMeteo](https://api.open-meteo.com/v1/forecast).
    - `LoadImport`: Imports from a file or JSON string or by endpoint data provision.

  - `provider_settings.import_file_path`: Path to the file to import weatherforecast data from.
  - `provider_settings.import_json`: JSON string, dictionary of weather forecast value lists.

### BrightSky Provider

The `BrightSky` provider retrieves the weather forecast data directly from
[**BrightSky**](https://api.brightsky.dev).

The provider provides forecast data for the following prediction keys:

- `weather_dew_point`: Dew Point (°C)
- `weather_ghi`: Global Horizontal Irradiance (W/m2)
- `weather_precip_amt`: Precipitation Amount (mm)
- `weather_precip_prob`: Precipitation Probability (%)
- `weather_pressure`: Pressure (mb)
- `weather_relative_humidity`: Relative Humidity (%)
- `weather_temp_air`: Temperature (°C)
- `weather_total_clouds`: Total Clouds (% Sky Obscured)
- `weather_visibility`: Visibility (m)
- `weather_wind_direction`: Wind Direction (°)
- `weather_wind_speed`: Wind Speed (kmph)

### ClearOutside Provider

The `ClearOutside` provider retrieves the weather forecast data directly from
[**ClearOutside**](https://clearoutside.com/forecast).

The provider provides forecast data for the following prediction keys:

- `weather_dew_point`: Dew Point (°C)
- `weather_dhi`: Diffuse Horizontal Irradiance (W/m2)
- `weather_dni`: Direct Normal Irradiance (W/m2)
- `weather_feels_like`: Feels Like (°C)
- `weather_fog`: Fog (%)
- `weather_frost_chance`: Chance of Frost
- `weather_ghi`: Global Horizontal Irradiance (W/m2)
- `weather_high_clouds`: High Clouds (% Sky Obscured)
- `weather_low_clouds`: Low Clouds (% Sky Obscured)
- `weather_medium_clouds`: Medium Clouds (% Sky Obscured)
- `weather_ozone`: Ozone (du)
- `weather_precip_amt`: Precipitation Amount (mm)
- `weather_precip_prob`: Precipitation Probability (%)
- `weather_preciptable_water`: Precipitable Water (cm)
- `weather_precip_type`: Precipitation Type
- `weather_pressure`: Pressure (mb)
- `weather_relative_humidity`: Relative Humidity (%)
- `weather_temp_air`: Temperature (°C)
- `weather_total_clouds`: Total Clouds (% Sky Obscured)
- `weather_visibility`: Visibility (m)
- `weather_wind_direction`: Wind Direction (°)
- `weather_wind_speed`: Wind Speed (kmph)

### OpenMeteo Provider

The `OpenMeteo` provider retrieves the weather forecast data directly from
[**OpenMeteo**](https://api.open-meteo.com/v1/forecast).

The provider provides forecast data for the following prediction keys:

- `weather_dew_point`: Dew Point (°C)
- `weather_dhi`: Diffuse Horizontal Irradiance (W/m2)
- `weather_dni`: Direct Normal Irradiance (W/m2)
- `weather_feels_like`: Feels Like (°C)
- `weather_ghi`: Global Horizontal Irradiance (W/m2)
- `weather_high_clouds`: High Clouds (% Sky Obscured)
- `weather_low_clouds`: Low Clouds (% Sky Obscured)
- `weather_medium_clouds`: Medium Clouds (% Sky Obscured)
- `weather_precip_amt`: Precipitation Amount (mm)
- `weather_precip_prob`: Precipitation Probability (%)
- `weather_pressure`: Pressure (mb)
- `weather_relative_humidity`: Relative Humidity (%)
- `weather_temp_air`: Temperature (°C)
- `weather_total_clouds`: Total Clouds (% Sky Obscured)
- `weather_visibility`: Visibility (m)
- `weather_wind_direction`: Wind Direction (°)
- `weather_wind_speed`: Wind Speed (kmph)

### WeatherImport Provider

The `WeatherImport` provider is designed to import weather forecast data from a file or a JSON
string. An external entity should update the file or JSON string whenever new prediction data
becomes available.

The prediction keys for the weather forecast data are:

- `weather_dew_point`: Dew Point (°C)
- `weather_dhi`: Diffuse Horizontal Irradiance (W/m2)
- `weather_dni`: Direct Normal Irradiance (W/m2)
- `weather_feels_like`: Feels Like (°C)
- `weather_fog`: Fog (%)
- `weather_frost_chance`: Chance of Frost
- `weather_ghi`: Global Horizontal Irradiance (W/m2)
- `weather_high_clouds`: High Clouds (% Sky Obscured)
- `weather_low_clouds`: Low Clouds (% Sky Obscured)
- `weather_medium_clouds`: Medium Clouds (% Sky Obscured)
- `weather_ozone`: Ozone (du)
- `weather_precip_amt`: Precipitation Amount (mm)
- `weather_precip_prob`: Precipitation Probability (%)
- `weather_preciptable_water`: Precipitable Water (cm)
- `weather_precip_type`: Precipitation Type
- `weather_pressure`: Pressure (mb)
- `weather_relative_humidity`: Relative Humidity (%)
- `weather_temp_air`: Temperature (°C)
- `weather_total_clouds`: Total Clouds (% Sky Obscured)
- `weather_visibility`: Visibility (m)
- `weather_wind_direction`: Wind Direction (°)
- `weather_wind_speed`: Wind Speed (kmph)

The PV forecast data must be provided in one of the formats described in
<project:#prediction-import-providers>. The data source can be given in the
`import_file_path` or `import_json` configuration option.

The data may additionally or solely be provided by the
**PUT** `/v1/prediction/import/WeatherImport` endpoint.