Open benchmark

How accurate is Helios-Forecast?

There is no honest way to answer without measuring it on real installations, everywhere, and publishing as it goes.The more contributors, the more the answer is worth: that is why I need you.Nothing changes about the integration: it never needs a key, sends nothing of its own accord and computes everything on your machine. Taking part is voluntary, with no account and no identity recorded, and it stops whenever you like by clearing a box.Local, private, open source, free: that is the rule, and it does not move.

Where this stands

The measurement began before the claim.

A forecast can only be judged against what actually happened, and no forecasting service publishes what it announced in the past. A prediction that was not written down at the moment it was made cannot be recovered afterwards. So the recording has to start long before any conclusion, and run for months.

Nothing is asserted on this page yet. It will never rank anyone: it says what the forecast is worth, and will say it better as the data accumulates.

Reading the collector

Coverage

Where the measurements come from.

Every installation taking part declares its country, and that is the only indication of place shown on this page; the raw file also carries its position, smoothed to one decimal. Spread matters as much as headcount: a model that lands well under a Mediterranean sky proves nothing under northern cloud.

Reading the collector

Over time

You do not judge a forecast on one day.

The hours when the forecast goes wrong

Hour by hour, the average gap between what Helios-Forecast had announced and what the meters measured, over every installation. The two other curves are there as references: the physics alone, which learns nothing, and persistence, that is the day before replayed as it was. Everything is scaled to installed capacity so every installation counts the same, and read in local time so sunrise lands in the same place everywhere. It does not say how far the forecast is off, but at what time of day.

The error against the height of the sun

The same error, filed by how high the sun stood rather than by the hour of the day. This is the axis the error actually follows: two hours sharing a clock reading have almost nothing in common between December and June, while two hours sharing a sun height behave alike everywhere and in every season. A low-sun weakness shows here and hides in the hourly chart.

Consistent, or lucky?

The error over a whole day, source by source, across every scored day. The line marks the median day, the box the ordinary days, the ends the best and the worst. Two sources can share the same average error and not be worth the same: one is a little wrong every day, the other rarely but badly. For planning a battery, the first is liveable, the second is not.

The error, day after day

Each bar is the gap between the energy announced in the morning and the energy actually produced, as a median across installations, with the range behind it running from the most optimistic to the most cautious. Above zero the forecast aimed too high. This is where being right parts company with having been lucky.

The error by horizon

Mean error, normalised by installed capacity, against the time between the prediction and the moment predicted. Forecasting the next hour and forecasting the day after tomorrow are two different trades. This is the chart someone who knows the field looks at first.

Does the range keep its promise?

Helios-Forecast announces a range rather than a value, and implicitly promises reality falls inside it eight times out of ten. Here is how often that holds. A range nobody checks is decoration.

Method

How this is measured.

1

Written down before the day happens

Every hour, each participating installation records the curve it is predicting for the next seven days, its uncertainty band, and the production it has already measured. The record is stamped at the moment of the prediction.
2

Judged against a real meter

The truth is the energy meter of a real installation. No simulated production is used anywhere: a forecast scored against synthetic data measures nothing but its resemblance to whatever generated that data.
3

Impossible to rewrite

Records are append only, enforced by the database itself, which refuses any modification or deletion. A bad day cannot quietly disappear, including at my own hand. A whole round can start over, though: this one was wiped on 8 September 2026 so that it measures 2026.9.5 and the versions after it. What stays impossible is editing a day inside a round.
4

Scored on what matters

Mean absolute error per lead time, normalised by installed capacity, and the same error read against the height of the sun, which is the axis it really follows. The gain over the bare physics and over persistence, that is, over the dumbest forecast there is. Error on the day's energy. And how often reality actually falls inside the announced range. Only the hours the installation produced are scored, the same ones for every source: announcing nothing at night is no achievement for anyone.

Reading

How to read these figures.

There is no comparison with other forecast services, and that is a choice. Under the quotas of their free tiers the sample would have been too thin to represent anything, and an unrepresentative comparison is worse than none. The benchmark measures Helios-Forecast against its own references: the bare physical model, and persistence, that is yesterday's day replayed as it was. The point is to know its strengths and weaknesses, version after version, in order to improve it.
<b>One participant</b> so far. That number does not make the forecasts better, it makes the measurement more representative: the more installations, climates and configurations there are, the less any single case weighs in the averages.
The recorded days also serve to prepare the next versions of Helios-Forecast. Each one can be replayed offline against the meter to check that a change to the model really improves things before it ships. A new version then starts from zero in the measurement: it is only scored on days after its release, never on the ones used to tune it.
Less than a full season. Everything that shifts over the course of the year therefore still escapes the measurement, starting with how the model behaves once the sun climbs higher than it has ever seen it.
So there is no conclusion at this stage, and there will never be a ranking: this page compares Helios-Forecast with itself only, version after version. The counters at the top of the page say how far the measurement has come, and nothing else.

Taking part

One installation is not a benchmark.

This benchmark will be worth exactly what its diversity is worth. Shaded installations, unusual orientations, split arrays, difficult climates and long histories are the interesting cases, far more than another clear south facing one. There is no account to create and no name to give: one checkbox in the Helios Forecast settings, and that is all. The upload then runs on its own, once an hour and anonymously.

The data actually sent

The curve predicted for the next seven days, with its upper and lower range.
What the physical model alone would have announced, so the contribution of the learning can be measured.
The production your meter has already recorded over the last seventy-two hours.
The geometry of your panels: tilt, orientation, peak power, inverter ceiling.
The reliability index and the number of days learned.
Your coordinates rounded to two decimals, roughly a kilometre, and the country of the installation. The public file drops one more decimal still.

What is not in your data

No name, no email address, no account.
No Home Assistant entity name.
No consumption, no other sensor, nothing from the rest of the house.
No precise position: the rounding happens at your home, before sending.
No traceable identifier: your installation is designated by a fingerprint computed locally.

To stop everything, clear the box in the settings. Sending ceases within the second, there is nothing to uninstall and nobody to ask.

Raw data

Check it yourself.

Everything the collector received for the most recent day, from every installation, exactly as it arrived apart from the position, one JSON object per line. The counters on this page read the same file, and nothing is computed in your browser.

Download the most recent day Live collector state

Each installation's coordinates are smoothed to a single decimal before publication, out of respect for the contributors' privacy. The reach stays that of the Open-Meteo model grid, so the weather replays identically and the arithmetic remains verifiable.