Minutes of the MARCH 4, 2011 COPAFS MEETING

Ed Spar. Executive Director’s Report

Spar remarked that for the first time in 19 years as COPAFS Executive Director, he really does not “have a clue” what’s going on. He was referring to the budget situation, and noted that, when asked what kinds of cuts they might face, agency officials indicate that they don’t know either. Everyone seems to be in the dark, and working with hypothetical scenarios.

Spar did note two cuts that seem to be definite. First, the Federal Financial Statistics Program at the Census Bureau’s Governments Division will be eliminated – saving about $700,000 per year. Second, the Statistical Abstract will be terminated – not just the printed publication, but the compilation itself. The savings is something like $2 million.

Consumer Expenditure Survey Redesign
Jennifer Edgar. Bureau of Labor Statistics

Edgar listed the motivations for redesigning the Consumer Expenditure (CE) Survey – reducing measurement error, maintaining/improving response rates, meeting user needs, responding to changes in society and technology, and the need for flexibility.

Evidence of measurement error comes from inconsistencies with sources such as PCE (Personal Consumption Expenditures), differences in expenditures by mode of interview, and an internal review citing item nonresponse and measurement error as areas of concern. Possible sources of measurement error include the survey’s length and complexity (even interviewers might be rushing through it), and conditioning, as respondents learn they can save time by simply saying they do not purchase specified products. Response rates have been pretty good (70+ percent), but have been declining.

The CE data have a variety of users. In addition to BLS, which uses the data for cost weights for the CPI, the data are used by BEA, IRS, the Department of Defense, the Census Bureau, and others including researchers and the general public. It is a varied group of users with varying needs for the data.

BLS sees a need for flexibility and quick response to changes in society and technology. Such changes include shifts in consumer behavior, such as online bill paying, online shopping, and the impact of big box stores on consumption patterns. Other factors include respondents’ increasing use of technology, and the emergence of new data collection methods.

The “Gemini” project to redesign the CE Survey started in 2009-2010 with the production of a database of CE research, and efforts to refine data requirements and priorities, and define data quality. In 2010-2011, the project will sponsor internal and external events to gather information and better determine user needs. Design alternatives will be assessed in 2012, research will continue through 2013. The target for recommending and implementing a redesigned survey is 2013-2014.

Gemini research is addressing factors including recall, interview frequency, split questionnaire designs, interview length, balance edits (which would identify discrepancies between expenditures and income), and the utility of diaries and other consumer purchase records. A 2010 methods workshop was held to gather existing knowledge on key topics, and while the workshop no absolute answers or definite research projects, it resulted in useful information and promising leads.

Edgar identified three key recommendations for the redesign. The first is the need to identify the CE Survey objectives. What are the minimum requirements, can they be changed, and what is most needed for the future? The second recommendation calls for careful reconsideration of recall, or the ability of respondents to accurately recall expenditure information. The third recommendation is summarized by the phrase “It depends . . .” There are no global solutions when it comes to the CE Survey – the effectiveness and feasibility of design alternatives varies by expenditure categories and other factors.

When asked about private sector surveys of consumer expenditures (and the technologies they use), Edgar noted that they are looking at technologies, such as the Nielsen scanners, and talking with the private companies. And with so many purchase records now in electronic form, some wondered why a household survey is still needed. The answer is that the household survey provides the demographic characteristics of the persons or households making purchases. Edgar commented that the redesign might not be moving them away from a household survey, but could be moving away from a recall survey.

The 112th Congress: A Presentation to COPAFS
Mary Jo Hoeksema. Population Association of America/Association of Population Centers

Hoeksema, who is Director of Government Affairs at the Population Association of America and the Association of Population Centers, noted that when COPAFS met in December, everyone was still analyzing election results, and Congress was in a lame duck session. It was a productive session, but Congress failed to pass appropriations, and we are still dealing with the aftermath. Congress also did not pass legislation that would have established a five year fixed term for the Census Bureau Director, and given the Census Bureau more independence within the Department of Commerce.

Looking to the new Congress, Hoeksema noted that the House is now firmly controlled by the Republicans, with many of the new members never having held public office before their election. The Democrats still hold the Senate, but with a smaller majority. Hoeksema then outlined key changes that data users should be aware of with the 112th Congress.

Key players in the House include Paul Ryan (R-WI), Chairman of the Budget Committee. Harold Rogers (R-KY) chairs the Appropriations Committee, with Dennis Rehberg (R-MT) chairing the Labor, Health and Human Services and Education Subcommittee, and Frank Wolf (R-VA) chairing the Commerce, Justice, and Science Subcommittee. Darrell Issa (R-CA) is Chairman of the Oversight and Government Reform Committee, with Trey Gowdy (R-SC), chairing the Subcommittee on Health Care, District of Columbia, Census and National Archives. Hoeksema noted that this Committee has oversight over the Census Bureau, but is currently much more involved with health care.

On the minority side, Chris Van Hollen (D-MD) is ranking member on the Budget Committee. Norm Dicks (D-WA) is ranking member on Appropriations, with Rosa DeLauro (D-CT) and Chaka Fattah (D-PA) serving as ranking members on the Labor, Health and Human Services and Commerce, Justice, and Science Subcommittees, respectively. Elijah Cummings (D-MD) is ranking member on Oversight and Government Reform, with Danny Davis (D-IL) ranking member on the Subcommittee on Health Care, District of Columbia, Census and National Archives.

Key newcomers in the Senate include Mark Kirk (R-IL), a former Population Resource Center board member, who has been appointed to Appropriations, and Patrick Toomey (R-PA), a former House member who, when in the House, introduced amendments to de-fund peer-reviewed grants. Hoeksema also pointed to Jerry Moran (R-KS) who is appointed to Appropriations, Roy Blunt (R-MO), who is appointed to Commerce, Science, Transportation and Appropriations, and Ron Johnson (R-WI), who is appointed to the Subcommittee on Appropriations.

As Hoeksema described it, the legislative agenda in the House is all about cutting federal spending – in FY 2011 and 2012, and beyond. On February 19, the House passed a continuing resolution (CR) for FY 2011 that reduces federal spending by $84 billion from 2010 levels. The CR would not pass the Senate, but included over 500 amendments that would have (among other things) eliminated funding for the ACS, eliminated all funding for Census Bureau periodic programs, eliminated five peer-reviewed NIH and NSF grants, and transferred $80 million from the Census Bureau to the Economic Development Administration. As of Hoeksema’s presentation, the government was running on a two-week CR that cuts $4 billion from current levels (largely cuts that were expected to be made anyway).

Other legislative priorities include the repeal of the Affordable Health Act, and enhanced oversight, which is likely to include hearings on the outcome of the 2010 census, and evaluations of its performance. Asked why the ACS is so high on the list of programs to attack, Hoeksema pointed to the resolution the Republicans passed in August to eliminate the ACS or make response to it voluntary. The resolution gained little traction on the Hill, but became a popular talking point in the campaign, and as Hoeksema noted, the ACS is receiving the same type of criticism once directed at the census long form.

Upcoming activities include final action on FY 2011 appropriations, and consideration of the FY 2012 budgets. The president’s proposals call for some increases for NIH, NCHS, and the Census Bureau (including the increased ACS sample), but all of this is up in the air given the leadership and priorities of the new Congress.

Hoeksema noted that need for stakeholder engagement is especially critical now. She encouraged stakeholders to inform their representatives of the need for federal data, and to do so in a way that makes it personal. For example, instead of abstract arguments about the need for data, Hoeksema said stakeholders should describe what will happen to them and their work if the proposed cuts are made. What would you as a data user be unable to do?

ACS 5-Year Data: A First Look at the First Release
Ken Hodges. The Nielsen Company

In December, the Census Bureau released its first set of 5-year data from the American Community Survey (ACS). Reflecting data collected from 2005 through 2009, it is a massive release including over 11 billion estimates covering 670,000 geographies. While most 5-year data are available on American FactFinder, the data for block groups (BGs) are only available on the ACS Summary File. Because the 5-year ACS sample is sharply smaller than that of the census long form, the BG data are prone to large errors, and restricting to the Summary File is a way to limit access to sophisticated users, who are encouraged to use the BG data only as building blocks to aggregations.

Hodges noted the excitement over the long-awaited 5-year data release, is mixed with apprehension over whether users will embrace small area data with sometimes large margins of error (MOE). Rather than a comprehensive review reaching a final verdict, Hodges described his presentation as one user’s first look at the 5-year data, with a focus on the estimates for very small areas.

In contrast to the 2000 long form, which covered 15.8 percent of households, the 5-year ACS is based on interviews with only 7.6 percent of households. As a result, over 90 percent of block groups had fewer than 100 ACS interviews. Hodges acknowledged that the MOEs are unsettling, but cautioned against making too much of them. Even where the MOE is larger than a cell value, the estimate is not necessarily inaccurate or unusable – especially for sparsely populated cells. For example, if ACS estimates 20 high income households, with a MOE of +/- 30, in an area with 2,000 households, the implied range of from 0 to 1.5 percent of households is narrow.

Hodges focused on “Households by Type and Size” because of its typical distributions. Very few small areas have many households with 6 or 7 or more people, so departures from the expected pattern are a strong indication of error – independent of the MOE. The ACS shows such departures, with some BGs having no households with 4, 5, and 6 persons, but many (86 in one example) with 7 or more – even though the 2000 census showed no tendency to large households.

These are likely BGs where ACS captured one or two large households, and the numbers were weighted to conspicuously high levels. Because the ACS sample is stretched, many BGs that do have a few large households have none captured by ACS, and therefore have an (under) estimate of zero. The aggressive weighting of the few large households captured by ACS compensates for the many “zero” estimates in BGs that have a few. Only by strongly weighting the few large households that are found, do the BG data sum to a reasonable national total. As Hodges noted, one could improve the accuracy of the individual BG estimates (by reducing the very high estimates), but this would impair estimation accuracy for BG aggregations. Put another way, making the estimates for individual BGs less accurate improves the accuracy of BG aggregations.

To offset the gloom of problematic outliers, Hodges assured that most BG distributions look reasonable, and showed examples where ACS provides significant enhancements. For areas that have grown from few to many households since 2000, ACS distributions are decidedly more reasonable than those based on the handful of households there in 2000 (where all might be in a single category). Nine years after a census in rapid growth areas, we can expect 5-year ACS data to provide a more accurate statistical portrait than nine-year-old long form data.

Hodges then described a “curious” finding with respect to ACS geography. When investigating BGs with large numbers of households but no ACS data, he noticed that ACS tract/BG geographic codes do not always match those of the 2000 census – for which the data are supposedly provided. The discrepancies are not widespread, but the Census Bureau acknowledges that in 18 counties, ACS data were inadvertently produced for 2010 census geography. The issue is still being investigated, and is documented on the Census Bureau’s website.

Next, Hodges contrasted the sampling rates (ratio of unweighted units / households) of BGs with those for places (cities and towns). For all BGs, the mean ratio was 9 percent, while for places it was about 20 percent, and the place advantage is consistent when controlling for number of households. Small government units were given sampling priority in the census long form as well, but with the smaller overall sample, the impact on small statistical geographies is more severe.

Given the large number of places with fewer than 100 households (for which data are available on American FactFinder), Hodges presented examples of ACS data for “Taylor City,” North Dakota – a town with just 65 households in 2000 (82 in the ACS), and 31 ACS interviews. In addition to Household Type and Size, the review included Race and Ethnicity, Year Structure Built, House Heating Fuel, and Occupation. Thanks perhaps to the relatively strong sample, the data for this very small “city” look reasonable when compared against comparable data from the 2000 census.

Hodges cautioned that it is too early to pass final judgment on the ACS 5-year data, but suggested that so far, they appear to be a mixed bag – with small area data suffering the effects of a thin sample, but still providing much useful information. The data appear to be good enough to merit user support, but not so good that they can absorb significant ACS cuts. Perhaps most important, many users would miss the ACS if it were eliminated.

Concerns from COPAFS Constituencies

No concerns were raised, and the meeting was adjourned.