Open benchmark

How accurate is Helios-Forecast?

There is no honest way to answer without measuring it on real installations, everywhere, and publishing as it goes.The more contributors, the more the answer is worth: that is why I need you.Nothing changes about the integration: it never needs a key, sends nothing of its own accord and computes everything on your machine. Taking part is voluntary, with no account and no identity recorded, and it stops whenever you like by clearing a box.Local, private, open source, free: that is the rule, and it does not move.

Where this stands

The measurement began before the claim.

A forecast can only be judged against what actually happened, and no forecasting service publishes what it announced in the past. A prediction that was not written down at the moment it was made cannot be recovered afterwards. So the recording has to start long before any conclusion, and run for months.

Nothing is asserted on this page yet. It will never rank anyone: it says what the forecast is worth, and will say it better as the data accumulates.

Reading the collector

Coverage

Where the measurements come from.

Every installation taking part declares its country, and that is the only indication of place shown on this page; the raw file also carries its position, smoothed to one decimal. Spread matters as much as headcount: a model that lands well under a Mediterranean sky proves nothing under northern cloud.

Reading the collector

Over time

You do not judge a forecast on one day.

Helios against its two references

Error hour by hour

Error by horizon

Inside the range

Error by sun height

Error by learning depth

Forecast accuracy

The gap on the day's energy

Error by how far ahead the forecast was made

Method

How this is measured.

1

Written down before the day happens

Every hour, each participating installation records the curve it is predicting for the next seven days, its uncertainty band, and the production it has already measured. The record is stamped at the moment of the prediction.
2

Judged against a real meter

The truth is the energy meter of a real installation. No simulated production is used anywhere: a forecast scored against synthetic data measures nothing but its resemblance to whatever generated that data.
3

Impossible to rewrite

Records are append only, enforced by the database itself, which refuses any modification or deletion. A bad day cannot quietly disappear, including at my own hand. A whole round can start over, though: this one was wiped on 8 September 2026 so that it measures 2026.9.5 and the versions after it. What stays impossible is editing a day inside a round.
4

Scored on what matters

Mean absolute error per lead time, normalised by installed capacity, and the same error read against the height of the sun, which is the axis it really follows. The gain over the bare physics and over persistence, that is, over the dumbest forecast there is. Error on the day's energy, the share of days that land within ten per cent, and the same error by how many days ahead it was announced. And how often reality actually falls inside the announced range. Only the hours the installation produced are scored, the same ones for every source: announcing nothing at night is no achievement for anyone.

Reading

How to read these figures.

There is no comparison with other forecast services, and that is a choice. Under the quotas of their free tiers the sample would have been too thin to represent anything, and an unrepresentative comparison is worse than none. The benchmark measures Helios-Forecast against its own references: the bare physical model, and persistence, that is yesterday's day replayed as it was. The point is to know its strengths and weaknesses, version after version, in order to improve it.
<b>One participant</b> so far. That number does not make the forecasts better, it makes the measurement more representative: the more installations, climates and configurations there are, the less any single case weighs in the averages.
The recorded days also serve to prepare the next versions of Helios-Forecast. Each one can be replayed offline against the meter to check that a change to the model really improves things before it ships. A new version then starts from zero in the measurement: it is only scored on days after its release, never on the ones used to tune it.
Less than a full season. Everything that shifts over the course of the year therefore still escapes the measurement, starting with how the model behaves once the sun climbs higher than it has ever seen it.
So there is no conclusion at this stage, and there will never be a ranking: this page compares Helios-Forecast with itself only, version after version. The counters at the top of the page say how far the measurement has come, and nothing else.

Taking part

One installation is not a benchmark.

This benchmark will be worth exactly what its diversity is worth. Shaded installations, unusual orientations, split arrays, difficult climates and long histories are the interesting cases, far more than another clear south facing one. There is no account to create and no name to give: one checkbox in the Helios Forecast settings, and that is all. The upload then runs on its own, once an hour and anonymously.

The data actually sent

The curve predicted for the next seven days, with its upper and lower range.
What the physical model alone would have announced, so the contribution of the learning can be measured.
The production your meter has already recorded over the last seventy-two hours.
The geometry of your panels: tilt, orientation, peak power, inverter ceiling.
The reliability index and the number of days learned.
Your coordinates rounded to two decimals, roughly a kilometre, and the country of the installation. The public file drops one more decimal still.

What is not in your data

No name, no email address, no account.
No Home Assistant entity name.
No consumption, no other sensor, nothing from the rest of the house.
No precise position: the rounding happens at your home, before sending.
No traceable identifier: your installation is designated by a fingerprint computed locally.

To stop everything, clear the box in the settings. Sending ceases within the second, there is nothing to uninstall and nobody to ask.

Raw data

Check it yourself.

Everything the collector has received, from every installation, as it arrived, down to the position, one JSON object per line. The most recent day to check a figure on this page, or the whole record to recompute all of it. The counters on this page read the same thing, and nothing is computed in your browser.

Download the most recent day Download the whole record Live collector state

Each installation's coordinates are smoothed to a single decimal before publication, out of respect for the contributors' privacy. The reach stays that of the Open-Meteo model grid, so the weather replays identically and the arithmetic remains verifiable.