Are USDA Corn Yield Forecasts Getting Better or Worse over Time?

A widespread view in the trade is that the final yield estimate for the U.S. average corn yield in 2014 will be well above the USDA August forecast of 167.4 bushels per acre, perhaps as large as 175 bushels.  Much of the expected yield increase is based on the concept that “big crops get bigger,” with support also coming from relatively high crop condition ratings late in the season.  A recent farmdoc daily article (August 20, 2014), however, revealed that not all big crops have gotten bigger in the past and that weather conditions this summer may not have been consistent with a large increase in the USDA yield forecast from August to January.  An implication of the expectation that the USDA yield forecast will increase substantially in future Crop Production reports is that the August forecast this year will have an unusually large forecast error.  The purpose of this article is to examine trends in the accuracy of USDA corn yield forecasts in recent years in order to provide an additional perspective on the debate about the U.S. corn yield in 2014.  Two previous farmdoc daily articles (August 19, 2011; September 1, 2011) and a research report earlier this year examined the accuracy of USDA corn yield forecasts.  We update those analyses here.

USDA Forecasting Procedures

We begin with a review of the procedures used by the USDA to generate corn yield forecasts and estimates.  A key to understanding the forecasts is the fact that USDA uses two types of surveys to collect data for the monthly USDA production forecasts in August through November.  These are referred to as the Agricultural Yield Survey (or the farmer-reported survey) and the Objective Yield Survey (or the field measurement survey).  The sample of farm operations for the Agricultural Yield Survey (AYS) is drawn from those who responded to the survey of planted acreage in June.  The sampling design to select the operations to be surveyed uses multiple control items, such as number and type of commodities planted and desired sample size for each commodity, to determine the probability of selecting a particular operation.  The same operations are interviewed each month from August through November.  Most of the survey data are collected in electronic form using computer-assisted telephone interviewing.  Each state in the survey is expected to achieve a minimum response rate of 80 percent.

The monthly AYS data are reviewed for consistency with previous surveys for the individual respondents and an across-record review is conducted to identify any extreme values that need to be re-checked.  A summary program which accounts for sampling weights and includes an adjustment for non-respondents is used to generate an indication of expected average yield for Agricultural Statistics Districts (regions within states) and for each state surveyed.  The yield indications from the survey reflect the judgment of respondents (farmers) and historical relationships indicate that respondents tend to be conservative in estimating final yields (under-estimate yield potential) particularly under drought conditions.  This tendency is quantified and factored into the official yield forecasts.

The Objective Yield Survey (OYS) is designed to generate yield forecasts based on actual plant counts and measurements, eliminating some of the biases associated with the farmer-reported yields.  The sample of fields selected for the OYS survey is selected from farms that reported corn planted or to be planted in the June survey of acreage.  Records from the June survey are sorted by state, district, county, segment, tract, crop, and field.  A random sample of fields is drawn with the probability of selection of any particular field being proportional to the size of the tract.  Two counting areas, or plots, are randomly selected in each field.  Objective measurements (such as counts of plants and ears) are made for each plot each month during the survey cycle.  When mature, the plots are harvested and yield is calculated based on actual production minus an allowance for harvest loss.  Enumerators count all fruit and fruiting positions in corn and, if ears have formed, a sample of ears is measured for length and circumference.  Just before the field is harvested, both plots are hand harvested and weighed by the enumerator.  Four ears are sent to the USDA lab for shelling and measurement of moisture.  These data are used to compute gross yield at 15.5 percent moisture.  Harvest loss is measured in separate units near the yield plots.

Prior to maturity and harvest, the OYS corn yield is forecast based on the forecast of the number of ears, the forecast of the weight per ear, and the forecast of harvest loss.  Forecasts are based on conditions as of the survey date and projected assuming normal weather conditions for the remainder of the growing season.  The state average gross yield for the OYS is the simple average of the gross yields for all the sample fields.  In addition, a state yield forecast is also made by first averaging the forecast or actual yield factors (such as stalk counts, ear counts, and ear weight for corn) and then forecasting the state average yield directly from these averages.  This forecast is based on a regression analysis of the historical relationship (15 years) between the yield factors and the state average yield.  Historical relationships indicate that OYS yield indications tend to over-estimate yield potential when estimating final yields.  This tendency is quantified and factored into the official yield forecasts.

The survey and forecasting procedures described here produce a number of indictors of the net yield of corn.  In August these indicators include: i) average field level yields from the OYS, ii) average state level counts from the OYS, and iii) the average yield reported by farm operators in the AYS.  After harvest begins, yields reported by farmers are also included as an indicator of final yield.  Each of the indicators results in a point yield forecast for which forecast errors are computed based on the historical relationships between forecasts and actual yield.  The range of yields is evaluated relative to all of the pieces of available data to assist in the selection of the official yield forecast.  This process is completed independently in each state and at the national level.  A formal Agricultural Statistics Board (ASB) consisting of 7 to 10 statisticians is convened to review regional yield indicators and determine an official yield forecast.  Data for the final estimates released in January are collected in the December Agricultural Survey in which respondents report actual acres harvested and the actual yield or production.  More detailed discussions of USDA crop yield forecasting procedures can be found in this farmdoc daily article (August 28, 2013), a Marketing and Outlook Brief, and in the USDA publications found here and here.

USDA Forecast Accuracy

To evaluate the historic accuracy of the USDA corn yield forecasts, the August, September, October, and November forecasts are compared to the “final” yield estimate released in January (we say “final” because January estimates are sometimes revised based on the Agricultural Census conducted every five years). The differences between the forecasts and the final estimates in percentages over 1990-2013 are presented in Figures 1 through 4.  When interpreting the errors, note that a positive error implies an under-estimate on the part of USDA and a negative error implies an over-estimate.  As is well-known, errors associated with the USDA corn yield forecasts are occasionally very large, such as 1993 and 1995.  These examples of large errors are not surprising due to the unusual weather events that occurred in those years.  It is interesting to note that USDA corn yield forecast errors in 2012 were extremely small, with the August forecast exactly equal to the final estimate, and thus, having a zero forecast error.  This is surprising given the magnitude of drought conditions that prevailed in 2012 (farmdoc daily, April 17, 2013) and the difficulty of forecasting corn yields under these circumstances.  The figures suggest a clear downward trend in USDA corn forecast errors over time. Forecast performance since 2011 has been particularly impressive.

The USDA corn yield forecasts are examined for both bias and changes in accuracy through time.  The average percent errors for each forecast month for the entire sample period and for sub-samples formed from 1990-2001 and 2002-2013 are presented in Table 1.  The average error calculations presented in Table 1 reveal no consistent patterns in the magnitude of the average errors and reveal no bias in the forecasts.  The smallest average error was 0.04 percent for the August forecast in 1990-2001 and the largest average error was 0.63 percent for the August forecast in 2002-2013.  Given the small size of the average errors, it is not surprising that no statistically significant bias estimates were found across months or sub-samples.

Table 2 presents the average absolute percent errors in the corn yield forecasts for each month and sample period.  As expected, the average absolute error declines as the growing season moves from pre-harvest through harvest.  For the entire sample period, for example, the absolute error averaged 4.19 percent for the August forecast and only 0.74 percent for the November forecast.  Examining the two sub-samples, the absolute percent errors were smaller in 2002-2013 for every forecast month except November, where the average errors were very close in both sub-samples.  The decline in the October absolute forecast error is marginally statistically significant with a p-value of 0.10.  Changes through time are also examined by regressing the absolute percent errors against a constant and a linear time trend:

•                                     |NASS Percent Error_m,t|= a + bTrend_t + e_t

where Trend_t is a time trend variable for crop year t that takes a value of 1 in 1990, 2 in 1991, and so on and e_t is a standard, normal error term.  The estimated trend coefficients are negative for the August, September, and October yield forecasts.  The estimated coefficients for both September and October are statistically different from zero at the 5 percent level (the downtrend in absolute errors is statistically significant even if 2012 is dropped from the sample). For example, the absolute percent error associated with the September forecasts declined by an average of 0.18 percent per year over this period. While the trend coefficient for August was not statistically significant, the magnitude was very similar to that estimated for September and October.  Overall, the results confirm the patterns observed in Figures 1 through 4 and provide convincing evidence that USDA corn yield estimates have improved through time, in the sense of absolute forecast accuracy.

Implications

Our analysis of USDA yield forecasts for corn over 1990-2013 did not reveal any evidence of bias in August, September, October, or November.   There is compelling evidence that the accuracy of USDA corn yield forecasts has improved over time, particularly since 2011.  It is especially interesting to note that USDA corn yield forecast errors in 2012 were extremely small, with the August forecast exactly equal to the final estimate.  This performance was exceptional given the severe drought that occurred in the summer of 2012. What, if anything, do these results imply about the ongoing debate about the direction of USDA corn yield forecasts in remaining Crop Production reports during 2014?  While it is, of course, true that longer-term trends in accuracy will not necessarily dominate in any particular year, an unusually large August forecast error this year (5 percent or more) would definitely be counter to the trend towards increasingly accurate USDA corn forecasts over time.