Chronological lows and highs

[kbloom]

I built one of these weather displays a few weeks ago, and having some experience with it now, I find the way that lows and highs is presented to be confusing.

It appears that when the OpenWeatherMap API returns its low temperature and high temperature for the day, that the low means that it's the lowest temperature expected in the 24 hour period that runs from midnight to midnight. That means that typically the low for Monday is the temperature on Monday at about 6 AM before the sun rises, and this low temperature comes before the high, which happens in the afternoon after the sun has been out all day. However, the e-paper display displays the low after the high.

Solution 1: we can simply reverse the order of the low and the high, so that the low comes first.

However, if tomorrow is forcasted to be much colder than today, and the weather pattern is going to dramatically cool off in the afternoon or overnight, then sometimes the low temperature returned by OpenWeatherMap will actually be at midnight at the end of the day. So then the low and high for that day are actually in order.

Solution 2: It turns out that the OpenWeatherMap response actually returns 4 additional temperatures for the day: "day", "night", "eve", and "morn". We could continue to present the Monday's low after the high, by artifically making Monday's low be the minimum of Monday "evening", Tuesday "night" and Tuesday "morn". (I'd consider doing something similar for the high.)

This introduces a new problem, which is that the actual low temperature (Tuesday's "low") could be lower than Tuesday "night" and Tuesday "morn", (without being the special case of Wednesday being much colder than Tuesday), if Tuesday's low occurs sometime between the Tuesday "night" temperature and the Tuesday "morn" temperature.

Summary: I'd like to know the actual low temperature each night (so that I know if I'm expecting a freeze and I need to bring plants in overnight), and I'd like it to appear chronologically, but can only approximate that data at best from OpenWeatherMap.

I'm happy to implement a solution (and send a pull request if others are interested in this functionality), but I need some additional input on what the right solution looks like.

[lmarzen]

Good points. I think we should improve the hi/lo to be day-time high/overnight low as suggested by solution 2. This seems to align better with other prominent weather reporting services.

ex:
https://www.wunderground.com/weather/us/ny/new-york-city

[lmarzen]

An interesting comment on this issue... https://www.chicagotribune.com/2019/02/02/ask-tom-an-explanation-of-our-highlow-temperatures-format/

[kbloom]

I guess my question is the following: Let's say that we take

mon_night_low = min(Mon_eve, Tue_night, Tue_morn)
mon_day_high = max(Mon_morn, Mon_day, Mon_eve)

Are there circumstances where we could then conclude that the value of mon_day_high should be replaced with Mon_high? (This should almost always be possible.)
Are there circumstances where we could then conclude that the value of mon_night_low should replaced with either Mon_low or Tue_low?

I suppose that for forecasts of the next 48 hours, we could also walk through the hourlies, and set
mon_night_low = min(hourly temp from 3PM Mon to 9AM Tue)
mon_day_high = min(hourly temp from 3AM Mon to 9PM Mon)

[lmarzen]

I agree with your formulation of night_low/day_high.

Are there circumstances where we could then conclude that the value of mon_day_high should be replaced with Mon_high? (This should almost always be possible.)

Hmm, good thought. I'm not sure either way, but I am leaning towards that we don't use the daily max at all.

I suppose that for forecasts of the next 48 hours, we could also walk through the hourlies

I was thinking something similar for walking through the hourlies for the next day. Do you think it would be better to use a fixed time as you suggested, or should we use the sunrise/sunset times maybe even if with an offset?

[kbloom]

So I downloaded a year's worth of weather data for my location, and plotted the difference between the mon_day_high = max(Mon_morn, Mon_day, Mon_eve) formula and the true high for the day. The plot below is a histogram of the difference in degrees Farenheit. It seems that taking the max of the 3 timed data points isn't very accurate. The true high is more than 2 degrees higher over 40% of the time, and more than 5 degrees higher 5% of the time.

[lmarzen]

Impressive work. It is somewhat disappointing the way OpenWeatherMap records day/night temperatures. Not sure what we can do about this. Should we add an option anyways?

[kbloom]

Well, before I did the analysis I was planning to implement both option 1 and option 2 and have a configuration option to choose between them. Option 1 would be the default, so anyone who pulled the latest code, and reflashed without looking at new configuration options would now have the low before the high, with no change in meaning of the numbers. I'd have option 1 would be the default, and I'd have documented it as as:

Show the lows and highs as the minimum and maximum temperatures between midnight and midnight. The lowest temperature is often right around sunrise, so this shows the lows and highs in roughly chronological order. Note that some sometimes the evening temperature can be lower than the temperature around sunrise when there are large temperature swings between days, In this case, the daily low will be sometime in the evening (or midnight), but it will be displayed before the daily high, so it turns out that this is only mostly chronological.

I'd have documented option 2 as:

Guess the daily high temperature and overnight low temperature from the predicted temperatures at midnight, 6am, noon, and 6pm. The the numbers that the display shows for the daily high temperature and the overnight low temperature will always be in chronological order, but if the peak temperature is at a different time of day, the temperatures reported this way may be a couple of degrees less extreme than the 24-hour high and low temperatures that OpenWeatherMap predicts.

So for now, I'll definitely do option 1, but since option 2 is pretty inaccurate, I don't think I want to use it myself, so I don't plan to implement it (and hence no configuration flag.)