UKH

Over the last few years I've come to the conclusion that Met Office forecasts are less reliable than other providers in predicting rainfall.

I've had a number of days where the Met Office chance of precipitation has been 5 or 10% but there has either been a heavy shower or steady light rain for hours. I know that forecasting is not perfect and that there will be days where the predicted chance of rain is low and it does in fact rain but my impression is that other providers such as Meteo Blue are getting it right more often.

Obviously they use different models but I was curious to know if others have experienced the same and wondered if anyone knew of any specifics that might explain issues with the Met Office forecast?

I use a range of weather forecasts. None are 100% reliable, especially where scattered showers are concerned. XC Weather is my preferred forecast

I use a range too but I'm at the point of stopping checking the Met Office for rainfall at least.

A “5-10% chance of rain” means that there is a 5-10% chance that rain will definitely fall somewhere within the forecasted area, not that there is a 5-10% chance of rain in your specific location. I don't know how many regions the Met office have in England, but Scotland is 6, so the forecasted area covered is actually quite large.

Edit: The best weather services I've found are an app called ventusky, Yr.no and another app hyperlocalweather

+1 for ventusky

I didn't know that it was based on such large areas. Looking at the forecasts would seem to suggest otherwise as there seems to be quite local variations in the chance of rain. I've often wondered how/if they could achieve the apparent granularity of the forecasts both in terms of locations and time. 

Was yesterday the final straw? Met office said mostly sunny in NW with 95% chance tops clear. MWIS said basically clagged in all day. I gambled on a summit sunrise with a 3.30 am start in Strath Carron. MWIS was right🙁.

I'm not sure the met office is better or worse than any other serious site. Best just to look at several for best informed decision; I'm certainly not giving up on it. It has been much more accurate than yr in the NW this last week (though I've heard it said that yr is not what it used to be).

It might well have been! It was raining quite heavily where we went on a 5-10% chance.

It's entirely possible it's all down to my perceptions/luck/chance etc.

I've used a few providers for the last while but I think I need to broaden that out a bit. I'll look at Ventusky, I was aware of them but hadn't used them.

Really? Any rain at all at any location in a given area. ie 50% could mean very little chance of rain where you are. I have no idea what those percentages mean and have never found a definitive explanation of them. I suspect they are essentially meaningless. For short term I go on the met office rainfall map which usually seems pretty good, though I assume the distribution of showers is more or less random.

The friend I am climbing with today was at the Camel yesterday and got a lot wetter than expected.

I think they are all very unreliable in the mountain areas. The best forecast is looking out the window in the morning.

Dry all day at Moy!  Which was not plan A obviously.

+1 for Yr.no, especially in the highlands, though I still primarily use Met Office.

If the 5-10% bit isn't helpful, have you tried looking at the map which shows cloud and rain coverage? I find that the most useful on unsettled days.

Obviously. The ultimate crag of last resort as far as I am concerned. Went there on Friday after getting rained off at Diabaig. Hated it even more than the only other time I had been eight years ago. In the morning I had been musing about moving to Inverness but by the evening I was thinking that I would miss Ratho too much!

Going there again today. I suppose it is exercise.....

I’ve not come across meteo blue, but I came across an analysis a few years ago that showed commercial outlets are more pessimistic in their forecasts than those not reliant on advertising revenue (don’t know if this is relevant in this instance, but I found it interesting all the same!)

Reason being that one is upset by a surprise sunny day, but having a picnic ruined by rain when the forecast gives only a 5% chance sticks in the mind and annoys people. So the outlets that rely on advertising and viewing numbers have a history of deliberately overestimating low chances of rain because then they are perceived to be more accurate. 

I think the historical data tends to show that short range forecasts are pretty bloody good these days, although different analyses use different measures of accuracy. Looks like the met office might have slid down the rankings a little on some measures, but it looks like they are pretty solid on temperature and wind speeds.

I think of it like a dose of unpleasant but highly effective medicine.

The Signal and the Noise mentions this. Also that lots of forecast sites won't publish 50% risks as the public interpret this as them not having a clue.

I've always found the rain radar more useful than the % chance of precipitation figures.

This thread got me to dive down a rabbit hole of weather research....

For the last few years, I've basically said to anyone that cared to listen - just use Windy, as all the main models are on there* and you can compare. Well. I still mainly hold by this, but I hadn't quite grasped that while the metoffice forecast is *based on* the ECMWF, it really just uses that as an "input" to create the boundary conditions for their model to do its thing. It runs on a 1.5km grid rather than the ECMWF  9km grid and, in theory, should be better at predicting local rainfall amounts. The probability (as far as I understand, it was a while ago I read this) comes from running multiple runs of the forecast as seeing how variable the results are.

Now, with regards to recent accuracy.... The models *were* struggling a lot during COVID, as they rely quite a lot on cross-Atlantic plane traffic to give loads of extra data points, which helps keep our maritime forecasts more accurate. Now, here's my half baked observation and theory: I still feel that the models do seem to change more than in the past, day by day and hour by hour as to what they expect 1-3 days away and I wonder if it's a result of climate change - the models are built on "physics" of the weather, but correlated and calibrated** with the real world data from the last 40 years, so if the whole climate is changing, the outcomes from the measured inputs will be at the fringes of the models capability so likely to be less accurate. Just a hunch/pet theory.

Either way, Windy is super handy as you can hit "compare" and see ECMWF (yr.no, metoffice etc.), GFS (wunderground, XC-weather and all those other free sites), Meteoblue and ICON-EU. I basically use this function as my "probability" - if the models agree, then it's likely, if they're all over the place it's a bit more uncertain. I find if ECMWF predicts light rain (<0.4mm) and GFS etc. show none, ECWMF is worth ignoring, but you know if might happen.  (low probablity of light rain showers)

*yr.no uses ECMWF as its base model, so does Met-Office, but they run their own local model using ECMWF to set up the boundary conditions, so if ECWMF is "off" so is the met office. I have noticed i the past ECMW seems way more pessimistic with rain amounts than Metoffice, but doesn't state any "probability"

**I think....must go an research this a bit more.

Ventusky is broadly similar to Windy in that it has all the models selectable. Not sure if you can do the "compare" thing, as I don't use Ventusky.

I thought that half-decent weather forecasting required supercomputers and access to data from a host of ground and satellite based sensors. The Met Office has three CRAYs. How are the other forecasters of UK weather doing their sums?

I'm pretty sure (as you say) it doesn't mean 5-10% in a huge region....   I've not dived deep into the guts of the metoffice model, as I don't use it (not a fan of the interface) but my general understanding of it was that it would use multiple runs of its 1.5km grid, and the probability would just be a statistical prediction based on the divergence of the results (as I alluded to above).  I guess that means for every 10-20 days you go out, it stays dry and the one time it doesn't, you get wet!

[edit to add link, pretty much what I said:  https://www.metoffice.gov.uk/research/weather/ensemble-forecasting/decision... ]

(note, as I said above - metoffice bases its forecast on the ECMWF model and adds it's own special sauce, so if the base model is out, so is met office. 

Another vote for yr.no - have had remarkably accurate hourly forecasts from Siurana to Mengersta and everywhere in between.

My understanding was that the likes of meteoblue, xc weather, Yr.no aren't crunching many numbers themselves but using existing models like ECMFW

Spot on.

It’s a big difference.my maths daughter pointed this out to me,

Um hopefully someone who is more familiar with it than me will be on, but I've worked a bit with the met office on their ocean model and I'm pretty sure the operational forecast system is entirely separate from the ECMWF IFS. They are a founding member of ECMWF but that is because of all the other stuff they do (research, copernicus etc etc) not because they use their models.

AFAIK the Met Office run the Unified Model for all their published atmospheric forecasts:

https://www.metoffice.gov.uk/research/approach/modelling-systems/unified-mo...

This is system they develop along with some other partners but not ECMWF

https://www.metoffice.gov.uk/research/approach/collaboration/unified-model/...

They run the model on different grids, with the global grids they run providing the boundary conditions to the high resolution regional grids (with 1.1km over the UK being the highest resolution). There is quite a variety of runs some on coarser grids (to allow more different parameters to be varied in an ensemble), some on higher resolution to resolve topographic features, and some including assimilation of observations from a range of met stations to 'nudge' the models towards recent observed history.

https://www.metoffice.gov.uk/research/approach/modelling-systems/unified-mo...

Since the new supercomputer a few years ago the runs are (or at least used to be) timed so that there was a new run of one of the setup being added into the mix about every half hour which might account for some of the bouncing around of the forecast (not sure though thats just my guess).

Personally though I always look at the multimodel forecast on MeteoBlue - it includes a lot of seperate national models, plus the ones run by MeteoBlue (they do run their own models based on the NOAA family of atmospheric models). It is usually clear if one forecast is an outlier and can give some clues as to the source of uncertainty (i.e. is it due to the timing of a front, or different interpretations of convective conditions)

Edit to add a link to the multimodel:

https://www.meteoblue.com/en/weather/forecast/multimodel/liafossen_norway_9...

I have the opposite opinion! I think the MO have improved noticeably in recent years and  - particulalry with the timings of rainfall - mainly based on Yorkshire/Lakes experience 

That's correct, the operational model data uses the Met Office's own global model to supply the boundary conditions for the nested higher resolution local models.

However, the forecasts with the percentages that you get if you type in a place name use a blend of a large variety of different models, including the Met Office's deterministic and ensemble (probabilistic) models, observational data, and later in the forecast period, also some ECMWF data, along with a load of complex post processing.

In reply to a previous poster, the area size for the weather symbols and % chance of precipitation are on a post code scale. How the % values are derived is quite complex, but basically runs lots of models a number of times to give what's called an ensemble spread of forecasts, then through some further post processing, arrives at the given % value for that post code area at that validity time.

Based on my experience yr.no can barely get their own country right at times.

I still find the Met Office pressure maps very useful.

I stopped looking at any weather forecasts years ago, I canceled plans too many times based on the forecast saying it was going to be too hot/cold/wet etc only for it to be perfect weather for the activity at hand.

These days I do a combination of 2 things, if it's local I look out of the window, if it's far away I just go,  I don't have any numbers but it certainly feels like I have been unable to do Farr less stuff than I canceled in advance based on a guess by the weather service. 

Ok, stand corrected!  I was blending old info with some recent reading, but clearly got that bit wrong!  (I think it was the fact Metoffice was part of ECMWF that had me thrown, and some other reading on how local numerical models *usually* run based on boundary conditions supplied by another global model, like ECMWF, but I hadn't dived deep into the Unified Model. Thanks. 

I’m surprised so many are saying Yr.no. It was my default for years and I found it accurate and useful, but that changed sometime since the pandemic started; didn’t notice when really until this year.

Not accurate nowadays for me and I don’t trust it anymore for any details.

I’m aware they used to have a big commercial contract with a large Scottish organisation to provide precise localised forecasts in Scotland. Last I heard though was they lost it; could be a possibility why it’s not accurate anymore though just a passing thought on my part.

A useful explanation 

https://www.metoffice.gov.uk/weather/guides/local-weather

The % figure is simply the probability of precipitation at some point during the forecast period.  It actually says this on the tabular Met Office web forecast (click on the circled "i" next to the "Chance of precipitation" row header.

It carries no information about how heavy the precipitation is likely to be if it does occur.  And it certainly should not be interpreted as meaning that precipitation will occur for between 5% and 10% of the forecast period.

But the Met Office give forecasts for specific mountains and use the % chance of precipitation.  So does that mean there would be a 5-10% chance of rain on the other side of the summit cairn... 

It surely refers to  the chance of rain falling in the forecast area at a particular time. 

Even for deciding where to go? That seems crazy to me. So if, on a Friday evening you were deciding where to go for the weekend, you wouldn't want to know that all the forecasts were saying it would, say, be raining the whole time in the Lakes but sunny the whole time in The Peak?

What he said, if I really want an idea of if it will rain I look at the radar
 

Cheers

Toby

I generally look at BBC and MO and choose the nicest.

Very telling that no one even considers MWIS. I’ve given up on them and resent my taxes going to fund it.

Thanks to all contributors I’ve learnt a few things.

They're still my go to for winter in the mountains. I make much less use of them I the summer though.

Yes, me too.

MWIS is a superb service giving specific local orographic detail not available elsewhere apart from met office (but they seem to lack the application of local knowledge). Yes, I think they consistently err on the pessimistic side, but one can allow for that.

The BBC would be the first answer that springs to mind.

I use their synoptic chart for a broad view then XCWeather and a specialist service called RASP which gives details of the boundary layer, a combination of local sporting websites, weather stations and local inhabitants.

The general forecasts are a bit of an entertainment really; I find it hard to work out how their 90 second spiel applies to my area at a particular time, and the quality of the presenter varies a lot. The best one by far that I've seen is the BBC south west chap David Braine. When the lady co-presenters announce him, they have a big smile, their eyelids flutter and they nearly faint.

The problem is that the met office is hiding the proper information that you can use to make informed decision behind a a system that turns the information into stupid symbols without the aid of a real forecaster.  They do this to "protect" their "valuable" information from harvesting by other people who might want to turn the real info into stupid dumbed down symbols.  Ditto the BBC.  The lack of spatial and temporal information makes making your own informed decisions difficult.

When asked, the reason given is we're too stupid to work out out own opinion and we're only capable of understanding the time averaged smeared out results expressed as ambiguous symbols. 

"We" also complain too much if the temporal/spatial information is given out at the high / base resolution because we're incapable of understanding "it's a forecast" and start to believe the time resolution and we moan if the forecast was for rain at 10:34 but it actually rained at 10:46 instead.

So giving out the information they actually have before putting it through a muppet filter (a) removes an opportunity to monetise their forecasts and (b) leads to more complaints.

The problem with the muppet filters is they are _so_ crap.  Things like  the spatio-temporal smearing failing to adequately differentiate between things like showers and constant drizzle making the forecast less and less useful.

These "systems" should, at least, pass the "umbrella test"* but they don't and it's a crying shame that this is so.

Bah.

* Should I take an umbrella to work today?

It could well mean that rain will definitely fall in 5-10% of the forecast area. It’s just that the forecast can’t determine which 5-10%. AKA scattered showers 

I think modern forecasts are brilliant compared to the much vaguer forecasts we had 20 or more years ago. In particular up to date hour by hour rain wind and temperature etc. this is brilliant in keeping up to date with changes in data almost in real time

That doesn't make sense to me; no forecast actual figure can be definite.

I have just looked at the info on the met office website (not noticed it before!). It says:

"Chance of precipitation represents how likely it is that rain (or other types of precipitation, such as sleet, snow, hail and drizzle) will fall from the sky at a certain time"

I take this to mean that if, say, it says 40% for Ullapool at 3pm, then they estimate that there is a 40% probability it will be raining on you if you are standing outside in Ullapool at precisely 3pm. 

Yes, this is undoubtedly true.

I wonder whether people complain more though because forecasts can, with good probability, be more specific in time, place and so on, giving many more details to get a bit wrong (even though they are usually pretty accurate).

It depends how broad and pedantic you want to be. How big is the forecast area? 5-10% is a pretty wide range. Quantum mechanics says that it can’t be definite that my morning bowl of porridge won’t suddenly disappear and materialise on the surface of  one of Jupiter’s moons

Ok, so it could actually mean something like 95% confidence that it it will rain at some time in the hour between, say, 2.30pm and 3.30pm in between 5% and 10% of the forecast area.

Except it apparently doesn't!

If you look at say met office mountain forecast, regional forecast and individual place forecast you often gwt three completely different forecasts.

Personally i have fiund their synoptic charts the most useful but now they seem to only produce them late morning, no use ti to anyone. 

I have just asked someone who works on the modelling side of things at ECMWF. He says, that, extrapolating from his own ensemble, 30% probability of rain at Tarbert for 2pm ought to mean "30% of the members of the ensemble say it will rain at the nearest grid point to Tarbert in the hour up to 2pm".

I have noticed that the rainfall map for a particular time and place somtimes does not agree with a location rain forecast for the same time. I'm not sure whether these apparent anomalies could be explained by the location forecast perhaps being for an hour long period rather than a precise time.

Ok, but that makes little sense to me so I wouldn't know what to do with that info as it strikes me a being fairly useless - it will 100% rain somewhere in the forecast area but we don't know where... particularly since I use the MO for specific mountain forecasts and not large areas, so 5-10% of the area of e.g. Ben Lomond makes even less sense.

I'm happy to keep thinking that it refers to the chance of rain falling at a particular time [actually I think it is the hour leading up to that time it refers to] over the forecast area. So if there is 95% chance of precipitation on Ben Lomond during the afternoon, I assume it is probably going to be wet and if there is 5% chance of precipitation on Ben Lomond in the afternoon I'll assume it is probably going to be dry. I will take a waterproof anyway.  

This is quite useful. Although I'm sure will probably raise more questions than answers.. 

'How do I know if it's going to rain?' https://www.bbc.co.uk/news/uk-44238426.amp

It means that that percentage of their forecast runs result in the specified outcome. With regards to the amount of rainfall, this information is given in the rain radar. I suspect that the amount of rainfall and probability of precipitation are highly correlated (ie a 10% chance of 32mm of rain seems unlikely to crop up in a forecast).

That's exactly what I would expect it to mean, and if I were doing my own forecasting exactly how I would present that information.

I find the sort of patchy weather that we've had a lot of this autumn tends to make weather forecasting look bad. There can be a massive band of rain coming through, it probably won't hit you but there's a small chance that it will (presumably reflected in a few of the models forecasting rain). So you see "10% chance of rain" and think that means "not much rain" but it actually means "probably totally dry but about a 1 in 10 chance that you'll get utterly lashed on for an hour". Or conversely they can expect it to hit you, give you 80% chance of heavy rain and then you get nothing.

I've found it more useful to see some sort of forecast map (or, on the day, rainfall radar) and if there are a lot of blobs of wet around then don't count on none of them hitting us.

This conundrum was the subject of a piece on "More or Less" about a year ago. Normally I find "More or Less" pretty enlightening but on this occasion they brought in someone from the BBC weather centre who was extremely unclear in her explanation I thought. 

The suggestion made up-thread that "10% chance of rain" means 10% chance that it will rain "somewhere in the area" must, I think, be wrong. On a day of scattered showers, that would mean that if the "area" was big enough (say the Lake District) then the figure quoted would often have to be 100%, despite many parts of the area missing the showers completely. 

The suggestion that "10%" means "if I stand somewhere in the specified area for the whole of the specified time" there's a 10% chance that it will rain on me at some point in that period" makes more sense logically, but I'm not convinced that's how we are expected to interpret it. It's not that helpful, because it equates a few spots of rain during one minute with steady rain for the whole period. But it's also not my experience using, mostly, the BBC forecasts as my guide. If I see a string of "10%" figures across the whole day then I will normally confidently head out to climb. Strictly, that 10% across the whole day should mean that the chance of staying dry in each hour is 90%, so the chance of staying dry all day is 90%^8 (for an 8 hour day) which is just 43%, so I'm more likely to get wet than not. But that isn't my experience. My experience is much closer to 90% chance that I'll stay dry all day.

Martin

So you think it means the probability of it raining at some time in an hour interval rather than the probability of it raining at a particular exact time (as the MO website seems to say)?

Yes, it would be very dependent on the size of an area, so pretty meaningless.

They very rarely give a zero probability (if at all) - possibly to avoid complaints! Yes, if it says 10% all day, that basically means an excellent chance of a dry day in my experience.

I'm just thinking of it from the perspective of running numerical simulations over discrete time intervals. So from that perspective a "10% chance of rain at 13:00" means that I ran the simulation for the 12:00-13:00 time interval 100 times (for example) and for 10 of those simulations there was rain.

Presumably there exist weather models that make Bayesian predictions but it's not really my area of expertise.

I think this may be because the probabilities are not calculated just for a single model grid point, but for some weighted average of surrounding grid points. This is intended to allow for the uncertainties in the spatial distribution of the rainfall, especially in showery conditions. 

I usually find it better to look at the predicted rainfall map rather than probabilities for a given location. The new beta maps on the Met Office site https://www.metoffice.gov.uk/weather/maps-and-charts/uk-weather-map have forecast rainfall maps out to 6 days instead of 36 hours which is an improvement. 

Has "Off-duty" appeared to defend them yet?

Exactly that, if I were planning to go to the lakes I would go to the lakes and if it rains I will do things I can do in the rain.  As I said I have been let down by the forecast too many times in the past that I don't check it anymore.

Last time I checked it was in 2019, we were heading to the lakes for a couple of days then on to Scotland for a couple of days and were going to go on the bike, forecast said it was going to piddle it down so we drove instead, the sun was cracking the flags for the full 4 days. Still at least we got to go, on other occasions we canceled trips based on a forecast that didn't manifest. 

The new beta maps on MET seem pretty good and at least appear more accurate than the old with 15minute intervals.