Accuracy

From the beginning, one of the authors' primary goals was to solicit comments from the users of our products. One of the most obvious aspects of our job for which the public has a valuable opinion is accuracy of the forecasts. The questions used can be grouped into two main sections: confidence in the forecasts, and perception of accuracy of the forecasts.

The first question asked, "Generally, how much confidence do you have in the weather forecast for the following times?" and listed "today, tonight, tomorrow, and the extended outlook (day 3 through day 5)" each for a separate rating. Responses were favorable for the first 24-hour period (today and tonight), where close to 90% said that they had "some" or "full" confidence. By day two, tomorrow's forecast, only 71% of the people maintained "some" or "full" confidence. Not unexpectedly, confidence fell considerably lower for the extended outlook. Only 38% had "some" confidence in the 3 to 5 day forecast leaving 61% with "little" or "none." The results of this question were broken down in relation to the respondents' primary weather source with some interesting results.

Each source was rated by calculating the percentage of respondents who said they had "some" or "full" confidence in the forecast for the above mentioned times and then took the average for the entire forecast period. The highest confidence ratings came from people who chose local TV (Anchorage) (84%) and NOAA Weather Radio (79%) as their primary weather source, whereas the lowest ratings came from those who got their forecast from the Weather Channel (65%) and who checked "other" (39%). The Weather Channel's average rating was brought down by the score on the extended forecast (only 35% had "some" confidence in it). The Weather Channel "local forecast" segment shows the "Juneau and Vicinity forecast" as provided by the National Weather Service, but the three to five day outlook is a product they prepare themselves and is often much different than the NWS extended forecast written for the same time period.

Another type of question asked people to give a rating (excellent, good, fair, or poor) to a forecast given the actual weather condition that day. For example,

• What would be your opinion of a forecast that stated "chance of rain," and it did not rain at your house?

For the version of this question which predicted "70% chance of rain (snow), and it did not rain (snow) at your house," 34% of the respondents chose the answer "poor," 48% chose " fair," and 15% marked "good." People were slightly less critical of the forecast when the word likely was used in lieu of the 70% chance. When the verbal qualifier was used, only 18% considered the forecast "poor," 52% said "fair," and 25% actually called the forecast "good." These results were very similar to Sink's findings for the same questions. She found that 37% of her respondents felt the forecast with the 70% POP was "poor," but only 11% rated the likely forecast as "poor."

The authors also asked this question using a chance of rain (snow), both with and without a 30% POP. Here we found 67% of the people rated the forecast with the 30% POP as "good" or "excellent," and 61% of the people who received the chance forecast without the POP said the forecast was "good" or "excellent." More respondents marked the forecast without the POP as "fair" (33% vs. 26%). Sink's results for these questions were more definitive. Similarly, 60% of her respondents rated the chance forecast better than "fair," but when the 30% POP was added, her numbers jumped to 81% of the people considering the forecast to be "good" or "excellent."

These results also correlate well with results taken from another question which asked respondents to "Circle the percent probability you associate with the following terms." The average percent probability people associated with the word likely was 62.5%. This may explain why they were more critical of the 70% chance of rain forecast than the rain likely forecast. A similar comparison may be made for the chance of rain version. The average percent probability people associated with the word chance was 41.8%. Again, this is in agreement with the results from the aforementioned question couplet where people were more critical of a forecast using the word chance alone than with a 30% POP. To test this theory, it would be useful to use a 40% and 60% POP for the comparisons to see if the results converge. The authors originally chose to use 30% and 70% so that the results could be compared to Sink's. Realistically, until a proper survey is conducted, altering the question would not produce decisive results.

There are some other problems with this question group that make the results difficult to interpret definitively. For one thing, when given a choice of two extremes (excellent and poor) and two middle-of-the-road options (good and fair), a majority of the people (an average of 76%) went for the middle answers. A possible solution may have been to offer only two choices, "good" and "poor;" however, the idea of rating a POP, with its inherent uncertainty, in such a black and white manner seemed like a gross oversimplification. Another problem is the possibility that people may have marked down that a busted 70% chance of rain forecast is "good," simply because they don't like the rain. We did have some remarks written on the questionnaires indicating this opinion, such as, "Too much rain!"

Also included on the Accuracy questionnaires were questions pertaining to the forecast high and low temperatures. One such question stated,

• The temperature forecast is "A high near 60." The high that day turns out to be 62 degrees. In your opinion, how accurate was the temperature forecast?

These two questions were used together to investigate the possibility that a freezing temperature forecast would be judged more critically than the afternoon high on a mild day. Results seem to indicate that people judged forecast accuracy based on closeness to the actual temperature and not on the relative importance of the forecast.