WEBVTT

1
00:00:00.000 --> 00:00:03.080
Several forecasts can have exactly the same error.

2
00:00:03.080 --> 00:00:05.920
That tells us their point predictions ended up

3
00:00:05.920 --> 00:00:08.270
equally far from the reported outcome.

4
00:00:08.270 --> 00:00:11.100
It does not tell us whether they used the same

5
00:00:11.100 --> 00:00:13.570
information, were made at the same time,

6
00:00:13.570 --> 00:00:15.670
or expressed the same uncertainty.

7
00:00:15.670 --> 00:00:18.260
ForeGlass keeps those differences visible.

8
00:00:18.260 --> 00:00:20.860
A probabilistic forecast includes more than

9
00:00:20.860 --> 00:00:23.510
a single number. It can declare a median and

10
00:00:23.510 --> 00:00:25.560
a range with a stated probability.

11
00:00:25.560 --> 00:00:28.100
In this illustration, three forecasts have

12
00:00:28.100 --> 00:00:30.750
the same median but different eighty-percent

13
00:00:30.750 --> 00:00:32.740
ranges. The observation is three.

14
00:00:32.740 --> 00:00:35.580
A very wide range can cover the outcome without

15
00:00:35.580 --> 00:00:38.590
being very informative. A narrow range that misses

16
00:00:38.590 --> 00:00:39.860
can be overconfident.

17
00:00:39.860 --> 00:00:44.130
These are the same invented forecasts as before. All predict a

18
00:00:44.130 --> 00:00:48.330
central value of two. The observed value is three. Each range

19
00:00:48.330 --> 00:00:52.600
gives a lower and upper bound intended to cover eighty percent

20
00:00:52.600 --> 00:00:56.670
of possible outcomes. A says one to three. B says one point

21
00:00:56.670 --> 00:01:00.870
nine to two point one, so it misses. C says zero to four. The

22
00:01:00.870 --> 00:01:05.070
score combines central error, range width and a penalty for a

23
00:01:05.070 --> 00:01:09.280
miss. Lower is better: A scores best here. Declare the ranges

24
00:01:09.280 --> 00:01:13.550
before the outcome. One case does not establish overall skill.

25
00:01:13.910 --> 00:01:17.730
To compare fairly, match the measure, target period, official

26
00:01:17.730 --> 00:01:21.620
outcome release and forecast deadline. Then choose a track. In

27
00:01:21.620 --> 00:01:25.630
the same-inputs track, all methods receive the same data. In the

28
00:01:25.630 --> 00:01:29.760
own-inputs track, each method uses its permitted information. Keep

29
00:01:29.760 --> 00:01:33.710
these tracks separate. Forecast dates and available information

30
00:01:33.710 --> 00:01:37.470
stay visible. Being earlier does not earn an invented bonus.

31
00:01:37.830 --> 00:01:41.900
Start with a simple baseline chosen before the results are known.

32
00:01:41.900 --> 00:01:45.840
For example, it could repeat the latest observed value. Add the

33
00:01:45.840 --> 00:01:50.160
scores over the same cases, using the same weights. Here the baseline

34
00:01:50.160 --> 00:01:54.230
totals ten and the model totals eight: twenty percent lower loss,

35
00:01:54.230 --> 00:01:58.360
not twenty percent accuracy. If the baseline's total loss is zero,

36
00:01:58.360 --> 00:02:02.360
that percentage is undefined. Show the underlying losses anyway.

37
00:02:02.730 --> 00:02:06.230
Ten forecasts of one outcome still give us just one observed

38
00:02:06.230 --> 00:02:09.620
case. Combining forecasts into an ensemble does not create

39
00:02:09.620 --> 00:02:13.010
another outcome. To learn about forecasting skill, we need

40
00:02:13.010 --> 00:02:16.410
results for more distinct outcomes over time. Record which

41
00:02:16.410 --> 00:02:19.850
forecasts share data or build on each other. Even different

42
00:02:19.850 --> 00:02:23.540
outcomes can move together during the same recession. Show both

43
00:02:23.540 --> 00:02:27.040
the number of forecasts and the number of distinct outcomes.

44
00:02:27.410 --> 00:02:31.160
Keep the data version, forecast, date, outcome and scoring rule

45
00:02:31.160 --> 00:02:35.150
together. The forecasting application calculates the score. Comuvia

46
00:02:35.150 --> 00:02:38.900
SDK helps retain the records and their links, so another person

47
00:02:38.900 --> 00:02:42.830
can inspect the calculation. A matching hash proves that the bytes

48
00:02:42.830 --> 00:02:46.640
match. It does not prove the claim is true, that the information

49
00:02:46.640 --> 00:02:50.330
was available earlier, or that the forecasts were independent.

50
00:02:50.700 --> 00:02:53.710
Our first published comparison covers one shared

51
00:02:53.710 --> 00:02:56.850
outcome. It supports case-level error comparisons,

52
00:02:56.850 --> 00:02:59.360
with the original limitations preserved.

53
00:02:59.360 --> 00:03:02.300
Broader forecasting-skill claims need repeated,

54
00:03:02.300 --> 00:03:04.870
comparable outcomes and properly declared

55
00:03:04.870 --> 00:03:08.010
uncertainty. Explore the ForeGlass Comparison page

56
00:03:08.010 --> 00:03:11.080
to see both the results and what the evidence can

57
00:03:11.080 --> 00:03:11.580
support.