Minutes of the June 8, 2007 COPAFS Meeting
COPAFS chair Ralph Rector started the meeting by introducing Ed Spar for his Executive Director’s report.
Ed Spar. Executive Director’s Report.
Spar remarked on the large attendance at today’s meeting, then noted that there was little news on budgets. Spar then described the Federal Register notice on proposed changes in the criteria for defining statistical geographies (such as block groups) for the 2010 census. Describing the notice as a good summary of the issues, he suggested that the elimination of Census County Divisions and the increase in the minimum population of block groups might be the most controversial proposals. Comments are due to the Census Bureau by July 5. Spar then proclaimed that “SIPP will never die.” His understanding is that DEWS (the Dynamics of Economic Well-Being System), that was to supersede SIPP, will now be folded into a re-authorized SIPP. There is talk of a reduction in sample size, however, so COPAFS members will want to stay tuned for updates.
HUBERT is the Census Bureau’s Housing Unit Based Estimates Research Team, which is testing the option of county population estimates based on housing unit data. This approach was a recurring recommendation at last summer’s COPAFS-hosted conference on the Census Bureau’s estimates program. The Bureau has not committed to the housing unit approach, but Spar described them as looking at it very seriously. It is hoped that we will have a COPAFS presentation on HUBERT by the end of the year.
Spar drew our attention to a new CNSTAT report, Understanding Business Dynamics, describing it as a primer on economic data, and an excellent treatment of confidentiality and other issues. He recommended it highly to COPAFS members, and indicated that copies could be acquired by contacting Connie Citro at email@example.com. Spar also noted that in April, the Census Bureau released special tables of 1, 3, and 5, year average data from the ACS test sites, with the objective of giving users a chance to see what multi-year average data will look like.
Spar completed his report by noting the retirement of Terri Ann Lowenthal, a long time COPAFS colleague and friend, who provided us with so many excellent updates on congressional issues. And a few housekeeping notes. The remaining 2007 meetings are scheduled for September 14 and December 14. And in order to avoid conflicts with FESAC (Federal Economic Statistics Advisory Committee) meetings, the June and December COPAFS meetings will now be held on the first Friday of the month.
Update on Census Bureau Economic Programs.
Harvey Monk. U.S. Census Bureau
Harvey Monk, the Census Bureau’s Assistant Director for Economic Programs, noted that the budgets for the major economic programs fared better than they had hoped. But he said the funding is definitely needed for the major programs, including Current Economic Statistics, the Economic Census, and the Census of Governments.
Monk focused on the 2007 Economic Census, with its broad goals of providing the nation with comprehensive and detailed information about the U.S. economy, and establishing a statistical foundation for economic analysis. Taken every five years, the Economic Census covers 26 million businesses and collects data from five million business locations. Response to the Economic Census is required by law, and data are collected through direct collection (mailed forms from five million business locations), electronic reporting (available to 3.5 million businesses) and extensive use of administrative records (including IRS data for 1.5 million small employers and 19 million businesses with no paid employees).
Monk listed the major milestones for the 2007 Economic Census including:
||Advance mailing to 15,000 largest companies
||Mail 400,000 forms to 1,300 largest companies.
||Mail 3.1 million forms to an additional 500,000 companies.
|Feb 12, 2008:
||Economic Census due date.
||Begin follow up on “delinquents.”
||National Processing Center units complete work.
||First Economic Census data product (Advance Report).
||90 pct. of data products released.
||Final product released.
Improvements and innovations for the 2007 Census include expanded content and coverage – including new information on employer contributions for healthcare and pension plans. Monk also described steps to maintain response rates – including enhanced electronic reporting capabilities, toll-free assistance, and aggressive follow up on delinquents. Dissemination of data products is to be expanded and accelerated, with all data available on American FactFinder (beginning March 2009) with improved functionality. Monk explained that all data will be disseminated through the website or direct requests to the Census Bureau. There will be no paper, CD or DVD products.
In response to a question about evaluations, Monk explained that evaluations of the Economic Census are ongoing in the math/stat function, rather than a separate project. And when asked about the long-term preservation of the electronic-only 2007 data, Monk assured that the data would be archived – through processes that were beyond the scope of his presentation.
A Review of Current Plans for the 2010 Census.
Teresa Angueira. U.S. Census Bureau.
With Jay Waite now Deputy Director at Census, Teresa Angueira is now Acting Associate Director for Decennial Census. Angueira described the extensive research, development and testing that has supported 2010 census planning – addressing issues such as content, questionnaire design, coverage improvement and measurement, and the use of handheld computers for field operations.
With respect to content, Angueira recalled the proposal to drop the “Some other race” category, and the mandate that it be retained, and the “three question” option (involving simplified race and ethnicity questions plus write in ancestry) that has been dropped due to the a resulting loss of detail. With respect to coverage, she described the tests of alternative instructions for communicating residence rules as well as alternative versions of questions for detecting missed persons and erroneous enumerations. Angueira described these “coverage probes” as moderately effective.
The 2000 census revealed the need for improvements to the collection of data in group quarters (GQ), so extensive testing has been done in this area. The Census Bureau has worked to update GQ definitions to reflect new industry terms, is integrating GQ and residential addresses on the MAF, and is testing additional sources of GQ addresses.
Other tests have looked at self-response options, including the effectiveness of replacement questionnaires, the postal tracking of returns (to minimize the number of replacement questionnaires sent), the timing of advance mailings, and the Internet response option—which will not be offered in 2010. In response to an observation that the Canadian census offers an Internet option, Jay Waite observed that the Canadian government has a secure Internet system that their census can plug into, but that the U.S. government has no similar system. The risks associated with the Internet response option are considered too high relative to the minimal impact it would have on response rates.
The Census Bureau has high hopes for the increased use of automation in 2010. As we have often heard, the use of GPS equipped hand held computers is the key to this automation, and the payoff in terms of reduced paper and staff requirements, improved geocoding, and reduced field workloads. Angueira reminded us that this automation is made possible by the short form only 2010 census.
Angueira then noted that research and development for the 2010 census have now concluded, and they are moving to the implementation phase. The 2008 Census Dress Rehearsal in San Joaquin County, CA and Fayetteville, NC is an early step in this implementation, as is the Local Update of Census Addresses (LUCA) program. LUCA notices have been sent to local governments, and invitations to participate will be sent in August. Major contracts for 2010 census work (including printing, communications, DADS development, and regional census centers) are being awarded, and content is set. When asked about reports that the “Foster Child” category (of relationship to householder) would be restored, Angueira explained that that is for the ACS only, and that the category is still being dropped from the 2010 census.
The session concluded with discussion of handheld computers. Jay Waite, who has a long history with this initiative, explained that the computers are custom made, and are good for nothing other than follow up for census non-response—and maybe pounding nails. They do not host games, will be worthless to pawn shops, and anyone who steals one would also have to steal the enumerator’s thumb, as thumbprints are used for authentication, instead of chronically forgotten passwords. We also discussed the new DADS development, where an outreach effort (headed by Nancy Gordon) is expected to begin in September. Outreach will not likely take the form of in-person meetings, but rather a website, where users can provide comments. And in response to a final question on coverage evaluations, Angueira said the 2010 census will include coverage measures, but there are no plans for adjusted counts. The emphasis will be on the components of coverage (missed persons and erroneous enumerations) rather than just net coverage.
CNSTAT Panel Report: Using the American Community Survey: Benefits and Challenges.
Connie Citro. Committee on Naitonal Statistics.
Graham Kalton. Westat and Panel Chair.
Connie Citro explained that the CNSTAT panel was requested by the Census Bureau, and noted that the report (now in pre-publication form) is available online or by contacting her at firstname.lastname@example.org. The publication version is to become available in July.
The report is designed to help users understand the benefits and challenges of the ACS by describing applications, and providing guidelines for ACS data use. The report also recommends further work for the Census Bureau on key aspects of the ACS. As Citro put it, the ACS is still a new survey, with room to improve.
The panel’s bottom line conclusion is that the ACS provides advantages in terms of the timeliness and frequency of data releases, and better data quality resulting from follow up by trained interviewers. However, ACS challenges include larger sampling error, the huge volume of data (which is also a plus), the complexity of period estimates, and the impact of controls based on population and housing estimates. In its recommendations, the report stresses the importance of ACS funding, and calls for a comprehensive program of user education, outreach and feedback – involving intermediaries (such as state data centers, COPAFS and APDU) and advisory groups.
The report makes much of the increased sampling error of ACS data (for small areas and populations) relative to long form data, and points to the relatively small sample size as the reason. In what Citro identified as a “big recommendation,” the panel calls for the re-allocation of the ACS sample, and an increase in overall sample size. Because the ACS follows up on only a sample of non-responding housing units, the number of responses is much smaller than that from a census long form – even with ACS responses combined over five years. To convey the magnitude of the difference, Citro noted that the initial ACS sample would have to increase to 23 to 24 million (or about 4.5 million per year) to yield a response volume equivalent to a long form. Funding such an increase would be a challenge.
With respect to data products, the report notes how useful it would be to provide month of interview in the PUMS product (even if “fuzzed” to protect confidentiality), and calls on the Census Bureau to recognize the confidentiality protection achieved through multi-year averaging. Another “big” recommendation is that the Census Bureau dedicate R&D resources for innovative longer-term ACS projects – the idea being that the ACS should enable applications we have not thought of yet.
Graham Kalton then reviewed some technical and methodological issues related to ACS data. The quick list includes sample allocation, MAF, multi-year period estimates, residence rules, mixed mode data collection, inflation adjustments, and group quarters.
Kalton described how even the 1-year estimates are complicated by monthly samples, period estimation issues, and weighting adjustments based on estimates for 1,951 counties and county combinations. Weighting is itself a 9-step process beginning with base weights, and then accounting for factors such as variation in monthly response rates, non-interview factors, mode bias, and the population and housing unit controls.
Kalton focused on the population and housing control factors. The housing estimates are designed to compensate for MAF over-and under-coverage, but they are subject to error, and some question their ability to improve on the MAF counts. And when changes are made to the estimates, or when new census counts become available, discontinuities in the annual ACS data are inevitable. The MAF has its own limitations, and Kalton cut through the many technical concerns to the conclusion that “we really need a good MAF.” The population controls are designed to compensate for within-household non-coverage, but they are subject to greater error than the census counts used to weight long form data. Of particular concern are the estimates of age and race, where error is believed to be especially high, and uncontrolled ACS data might be more accurate. Kalton pointed out that we do not yet have a way of taking these types of errors into account when assessing the error levels of ACS data.
The issues related to the 3-year and 5-year period estimates are even more complicated, as we have to determine what we want these estimates to reflect – the end year of the period, the mid point of the period, or the average across the period. Weighting is different depending on which of the above options is chosen. For example, applying equal weight to each year’s data minimizes variance, but yields 3-year or 5-year period estimates that place the burden of interpretation on users. This is the approach that the Census Bureau plans to take with the multi-year data.
Kalton concluded by describing some of the issues related to estimating change from ACS data, argued that much research is still needed, and reiterated the panel’s call for ongoing ACS research with extensive user input.
U.S. Census Bureau Review of the CNSTAT Report.
Susan Schechter. U.S. Census Bureau.
Schechter, Chief of the Census Bureau’s American Community Survey Office, started with a brief ACS update, noting that the 2006 ACS data will be released this summer, and will be the first to include data on group quarters. She said the Census Bureau is reaching out to users for feedback on the 2005 data, and is developing guidelines for comparing the 2006 ACS data with Census 2000, and with the 2005 ACS.
Schechter then described methods panel tests of questionnaire design, and content changes emerging from the 2006 content test. Changes are proposed for seven population questions (citizenship, school enrollment, education, migration, disability, employment status, and weeks worked) and five housing questions (year built, number of rooms and bedrooms, plumbing/kitchen/telephone, food stamp benefits, and property value). New questions are planned for health insurance coverage, veterans service-related disabilities, and marital history; and the questions on seasonal residence and veteran years of service are being dropped.
For users preparing to transition from long form to ACS data, Schechter described the multi-year estimates study now available on the Census website. Based on data from the ACS test sites, the product consists of 14 sets of 1, 3, and 5-year estimates providing a simulation of multi-year ACS data for small areas.
Turning to the CNSTAT report, Schechter stressed how much they value the panel’s work, and welcome its recommendations. She noted that most of the recommendations relate to issues the Bureau had already identified, but that the report will help them prioritize on the importance of funding, research, evaluation, and user feedback.
A key priority is to establish annual funding for methods panel research, and to explore options for an expanded ACS sample size. The main R&D challenge is the ability to allocate the time and resources needed to accomplish a broad spectrum of projects. With no staff dedicated to such work, they have to get to these projects as they can. With respect to education and outreach, the Census Bureau wants to be more proactive and less reactive, and is developing a communications strategy to achieve this objective. Schechter remarked that, since joining Census from OMB, she has seen how hard it can be to respond to large volumes of user feedback, but she assured that the Bureau is determined to do this.
Schechter then presented a list of the CNSTAT recommendations with initial Census Bureau responses. She reviewed the list quickly, and the responses were not very specific, with many indicating only general timing objectives. Schechter concluded by reiterating how much the Census Bureau appreciates and agrees with the CNSTAT recommendations.
The follow up Q&A saw more discussion of the pros and cons of using county estimates as ACS controls, more questions on the estimation of seasonal populations, and the need for more user input on ACS data products. Alfredo Navaro (Census Bureau) acknowledged the need for more research on the use of county estimates as controls, and Debbie Griffin (also from Census) explained that, while they have no plans for seasonal estimates, they have considered the possibility of an indicator in the ACS PUMS products. Confidentiality remains a concern with these options.
Concerns of COPAFS Constituents
No concerns were raised, and the meeting was adjourned.