U.S. Census Bureau Conference On Population Estimates:
Meeting User Needs
Bringing It All Together
Today’s papers and discussions confirm that this conference deals with a great topic. Data users care about the Census Bureau’s estimates, and they care a lot. And it is evident that the Census Bureau is aware of how important its estimates are to users. That awareness is reflected in the guiding principles paper, the convening of this conference, and the impressive list of attendees.
We data users expect a lot from the Census Bureau’s estimates. We want them to be released on time, and reflecting recent dates. We also want them to be consistent with other sources and with our expectations. We also want the Census Bureau estimates to be right the first time, and to change smoothly from year to year. We don’t want dramatic revisions, but we want the right to get revisions for estimates that we don’t like. Most of all, we want the Census Bureau estimates to be accurate. And as we have heard, there are real consequences to estimation error such as the misspecification of birth rates and labor force estimates, and the misallocation of federal resources.
The Census Bureau’s estimates have been immensely popular. Although produced for a Title 13 mandate, they are used for many purposes. Some are official, such as funds allocation or denominators for rates computed by federal agencies, but many are unrelated to government functions. Some users focus on small areas and others at the national level. Some are concerned only with total population, while others focus on demographic characteristics such as age, race and ethnicity. Some are focused on the estimates themselves, but most are concerned primarily with an end application for the estimates.
The stakes are getting higher for the Census estimates, as they are now used as weights for the American Community Survey (ACS). As ACS use becomes more widespread, the number of users with at least an indirect stake in the estimates will proliferate. And as issues arise with ACS data, the role of the Census Bureau’s estimates will become more apparent, and it will become more difficult to hide behind the old excuse that “they’re just estimates.”
So the Census Bureau estimates are expected to meet the needs of many users with a wide range of applications. And as we have seen, users are not shy about criticizing the Census Bureau estimates. Again, users care a great deal about the Census Bureau estimates, but this caring often manifests itself in discontent – with specific estimates, specific methods, and sometimes with the estimates program as a whole.
Those of us who produce demographic estimates are highly aware of their limitations, and most of us acknowledge and even emphasize the difficulties encountered in the pursuit of estimation accuracy. Those who simply use estimates are equally comfortable diagnosing their problems, and suggesting solutions. But often such criticisms are narrowly focused on why a specific estimate is believed to be inaccurate.
But that is not what we have heard in today’s discussion. Rather we have heard thoughtful critiques of the Census Bureau’s work, with comments geared to the estimates program as a whole. There is a sense among those gathered that the Census Bureau has taken its current methods about as far as they can go, and that the program needs to be moved forward. But while broad in perspective, the discussion includes specific recommendations – such as for improved housing unit estimates, greater use of the Master Address File (MAF), a possible national housing register, administrative records matching, and improved estimates of persons per household and occupancy rates. The benefits of composite methods and the averaging of alternative estimates were a recurring theme.
What we did not hear were calls for the greater use of advanced statistical modeling. For the attendees at this conference, estimation is all about the data. The sense is that improvements to the Census Bureau’s estimates will have to come through improved data resources more than improved statistical methods. For example, it is realized that without timely and accurate data resources, advanced statistical models would do little to improve estimation accuracy in the wake of events like Hurricane Katrina.
And as far as new data resources are concerned, the ACS may be the most important and promising new resource we have seen in a long time. There are concerns with the ACS residence rules, and other issues, but the ACS has clear potential to improve the Census Bureau estimates of characteristics such as population by age/sex and race/ethnicity – where accuracy has been a concern in recent years.
Many of us who deal in demographic estimates have a tendency to focus on the shortcomings of the data sources that we are currently using. For example, when a problem estimate is encountered, we can recite the limitations of IRS migration data. But for new data sources, we tend to focus on the untapped potential. We have seen some of that in today’s discussion, where we have considered the great potential of the MAF. But it is one thing to recommend a promising new data source, and another to produce quality estimates with it. So while many of us agree that the MAF holds great potential for improving the Census Bureau’s estimates, we can expect that, if put into use, we would soon detect and readily recite the limitations of the MAF. But that’s as much about human nature and demographic estimation. Again, what we have heard today are comments and recommendations from a group that is experienced in producing demographic estimates, and aware of the difficulties involved.
But while the comments have been knowledgeable, let’s keep in mind that the Census Bureau estimates are a rare thing – an annual set of estimates produced for all areas nationwide. Who else produces annual nationwide demographic estimates? A few private data providers and forecasting firms develop national estimates, but not the state and local demographers who work with the Census Bureau on its estimates program.
State and local estimates involve their own challenges, but, for example, Stan Smith and Steve Murdock don’t have to worry about counties in New York State, Joe Salvo and Warren Brown need not be concerned with administrative data in California, Linda Gage does not have to deal with remote Alaska, and Greg Williams (who does deal with remote Alaska) has data not available for other states. Only a few state and local demographers have to estimate the impact of Hurricane Katrina. That’s the strength of the state and local programs – they can focus on specific areas, and use unique local data resources – unencumbered by whatever is going on in other states. But the Census Bureau must estimate all areas, and they must be responsive to the thousands of users who have specific knowledge of the thousands of areas they are estimating.
Of course, this is the idea behind the Federal State Co-operative Program for Population Estimates – to tap into the knowledge and resources of the state demographers. The idea is for the states to provide input for the Census Bureau to use in producing a complete set of integrated nationwide estimates consistent with local input. But in reality, the Census estimates often differ from those produce by the states, and state demographers have expressed frustration that their input is limited to review and comment. There is also frustration with the Census Bureau’s continued use of one-size-fits-all methods – a self-imposed constraint that was called into question throughout the conference.
But again, the Census Bureau faces unique constraints, and simply deferring to state produced estimates is not a solution. There is much to be said for a nationwide set of county population estimates, produced by an impartial party, and controlled to an independent estimate of total U.S. population. In contrast to the state data programs, the Census Bureau faces a zero-sum constraint. Because national population is fixed, the Census estimates can add population to the counties in one state only by subtracting population from the counties in another state – and adhering to this constraint can disrupt consistency with state-produced estimates.
Despite these constraints, there is agreement that the Census Bureau estimates program needs to move forward with greater openness to new methods and data, and increased coordination with state demographers. Zero-sum requirements and imperfect data will continue to be a challenge, but even among the critics, there is great support for the Census Bureau’s estimates – from knowledgeable users who need the Census Bureau’s estimates to be as accurate as possible. Much of that family of expert users is gathered here at the conference. It is a talented group, ready to work with the Census Bureau to find creative approaches to improving estimation accuracy. Hopefully, this conference has been an important early step in that process.