Minutes of the September 23, 2010 COPAFS Meeting

Judie Mopsik. Report from Chair of COPAFS Board of Directors.

Judie announced that the COPAFS Board is looking for new members for 2011 and she invited anyone who is interested in serving on the Board to let her or one of the other officers know.

Ed Spar. Executive Director’s Report

Spar announced that he will be sending an email to everyone who completed the COPAFS Recruitment Questionnaire to see if they have a contact at any of the organizations they suggested for COPAFS membership.

With respect to the FY2011 budget, Spar noted that there will be a Continuing Resolution (CR) because no budget has been passed yet. Spar observed that a CR could have a negative effect because the President’s FY2011 budget had a lot of good things in it, including $30 million to expand the ACS sample size, and a $44 million increase overall for the ACS. The ERS budget request includes $2 million to test SCOP (Statistical Community of Practice). Under SCOP, agencies would feed their datasets into a centralized system. BLS is in the throes of modernizing the CES and the budget includes money for that. The last time there was a major change to the CES was in 1981. The budget request for BEA includes funds for a dashboard to show economic data over time. The budget request for NCHS has funds for the full collection of Vital Statistics data.

Turning to other issues, Spar noted that the RNC passed a resolution on the ACS and he read the last two resolves from that document which call for the elimination of the ACS or making response to the ACS voluntary. If response is made voluntary then users can expect a significant drop in response rate and an increase in item nonresponse as well. The Census Bureau considers this one of their major issues to tackle next year. Spar will send the RNC resolution to all COPAFS members.

Finally, Spar reminded everyone that the OMB Policy Seminar will be held on December 14 and 15 at the Convention Center. Interested persons can register online on the COPAFS website. Spar also announced that the new Director of Statistics of Income is Susan Boehmer from IRS.

The final 2010 COPAFS Meeting will be held on December 3rd.

National Compensation Trends
William Wiatrowski, Bureau of Labor Statistics

Wiatrowski, who is in charge of the National Compensation Survey (NCS), began by noting that there was a great deal of interest in sick leave when pandemic flu happened last year. Most of the NCS data are national and most are for the private sector, but the NCS does provide some local data for major metropolitan areas. Wiatrowski indicated that there is a great deal of compensation data from the NCS and that a sizeable number of new tables had been posted on the BLS website on September 22, 2010. He highlighted the fact that benefits make up 30 percent of employers’ costs.

Wiatrowski then provided a detailed overview of the survey design for the NCS. The NCS provides wage data by occupation and worker characteristics, employer costs for benefits, and details about benefits such as who has access, who is covered, and what the benefit provides to workers. The NCS has been collected for most of the 127 years BLS has existed. In the 1970s, the NCS moved toward more consistent data collection, with use of the NCS name starting in the 1990s. The NCS provides good coverage of the types of information you can get from the employer, but does not have demographic information about the workers. The NCS is an establishment survey that uses a three stage sample design: 1) sample of geographic areas that includes 150 metro and non-metro areas; 2) sample of establishments that covers all private industries of all sizes as well as state and local governments; and 3) within each establishment, a sample of occupations is selected. NCS data are collected from the Human Resource, payroll, or personnel office of each establishment, not from individual workers. The NCS excludes nonprofit organizations and workers who set their own pay. Data are collected for the physical location of the establishment. The source for the samples is the unemployment system. Private industry establishments are in the survey for five years, and one-fifth rotate in and out each year. The initial response rate is about 80 percent, but there is some attrition over the five-year period. Written benefit plan descriptions, and wage and benefit data are collected at the start of the survey, and wages and employer costs are updated each quarter while benefit data are updated once a year. Wiatrowski noted that the NCS does not collect data on stock options because it is too difficult to establish the cost of those options. Electronic collection of data for the NCS has been expanded and some establishments send a copy of their payroll.

The NCS collects information about the industry classification, number of workers, and location of each establishment. It also collects information about the occupation, full-time/part-time status, union/non-union status, and work level of all employees. The work level is defined by a point factor system that is used to establish a grade level for each occupation because wage data are provided to OPM for salary comparisons. The NCS collects data on wages and incentives such as commissions, piece rates, and production bonuses, as well as other cash payments such as premium pay for overtime, shift differentials, and non-production bonuses. The benefit data collected includes paid and unpaid leave, insurances including health, retirement savings, and other legally required benefits. The NCS determines what benefits are offered for each occupation, as these may differ for exempt versus non-exempt workers. The NCS also collects information on how many workers take or use each benefit, what the cost is to the employer for each benefit, and what workers must do to get the benefit, including eligibility requirements and required contributions.

Wiatrowski next turned to highlights of some of the important findings from NCS data. Benefit costs are driven by health costs while the others move with wage changes – like vacation costs with salary. As of March, 2010, wages made up 71 percent of employer costs for employee compensation, while legally required benefits and insurance each made up 8 percent, retirement and savings made up 3 percent, paid leave made up 7 percent, and supplemental pay made up 3 percent of employer costs. NCS data show a movement away from cash compensation since the mid 1960s, and indicate that health costs have been increasing as a share of overall compensation costs. Retirement costs are dominated by Social Security taxes and retirement plan costs have been fairly stable as a share of overall compensation costs since 2000. NCS data also show how the benefit landscape has been changing over the last 30 years. With health care, there has been movement towards a preferred provider organization structure. With all benefits, the onus is now on employees to choose which plans they want to participate in and they bear a larger share of the cost. Since 2003, the NCS has been collecting data on automatic enrollment of employees in savings and thrift plans. The share of workers enrolled in such plans has increased from about 5 percent in 2003 to almost 20 percent in 2009. The NCS data also show a movement away from defined benefit plans towards defined contribution plans. Overall, employees have more choice in retirement plans than they did in the mid 1980s, and fewer investment choices included employer stock by 2005. Wiatrowski also noted that the NCS was adding questions on the coverage of domestic partners and data would be available in the summer of 2011. Finally, the presentation concluded with some highlights of the types of NCS data that are available for local areas, including the first-ever data on employer costs by location.

Facilitating Innovation in the Federal Statistical System
Hermann Habermann, Committee on National Statistics

Habermann began by stating that he would be presenting his own personal views and that these are not endorsed by the Committee on National Statistics (CNSTAT). He noted that he is currently writing a summary report based on a seminar and a subsequent workshop on facilitating innovation in the federal statistical system that were sponsored by CNSTAT and the American Academy of Political and Social Science.

Habermann commented that innovation has been going on for a long time in the federal statistical system – examples include improving surveys, time series analysis, SAIPE, complete mobile health testing, as well as innovation in research. However, he thinks it is important to talk about innovation now because the environment in which agencies operate is changing dramatically. First, the pressures are changing. Respondent’s willingness to participate is declining and response rates are falling, and costs are increasing. Second, there are demands for more information. Most people operate on the basis of their beliefs rather than on the basis of data and information. If these are contradictory they reinforce beliefs – they do not change them. A very intense war is coming – on one side are those who place a high value on information in society and on the other side are those who will argue against collecting that type of information. Budgets are going to be more problematic than we want to acknowledge – for example, in past years the Census Bureau’s budget was cut to provide more funding for law enforcement. It will be harder for agencies to defend this collection of information.

Politicians have a different clock speed than researchers and it is getting worse – longitudinal data to them is data collected over two years. When Obama wanted to know how people were thinking, he held a town hall meeting. This was not a representative sample, but the results were generalized anyway. Today, people will choose an approximate answer to the right question rather than wait for the correct answer. The emerging areas of data visualization and techniques and the impact of communication technologies are also changing the environment in which agencies operate.

Habermann then highlighted the following specific areas for increased innovation:

  • Integrating survey and administrative data at all levels.
  • Improving small area estimates.
  • Developing better ways to understand the relationship between nonresponse rates and bias.
  • Developing better ways to resolve the tension between disclosure rules and the needs of programs and individuals for small area data.
  • Understanding how to capitalize on data visualization and communication technologies.
  • Determining how to move away from hierarchical search models to free-form network models.
  • Determining how to form cooperative arrangements as a system (e.g. SCOP) because individual agencies don’t have the IT resources to deal with this.

He also noted that there is some tension between official and unofficial statistics (e.g. the OECD Wiki site), and that lots of environmental data are not from government agencies.

Habermann then turned to a description of the following barriers to innovation:

  • U.S. federal system is decentralized and this won’t change. Most agencies don’t have the resources to get the required critical mass for innovation.
  • Current operations tend to take precedence over investments for the future. There is a lack of attention to innovation and it has a lack of standing – it is easy to put it off to tomorrow.
  • The inability to hire non-U.S. citizens.
  • It takes a long time to hire people. Smaller agencies do better than larger ones, but federal agencies have bureaucratic rules so they lose good people to organizations like NORC and Westat.
  • There are too many cumbersome procurement/acquisition rules to enter into contracts with academics and places like NORC for innovation.
  • Innovation is risky and can fail – agencies don’t get many points for innovation in the federal system but get lots of demerits if things fail.

Habermann then offered some models to move forward with innovation:

  • SCOP is a bright spot – it is a recognition of need.
  • Census Bureau efforts with formation of a research directorate– but with an election in 2 years, he worries about solutions that are dependent on people.
  • The most important characteristic for innovation is leadership – with it things can be done.
  • Technical and managerial leadership is critical and OMB could help agencies by giving them the opportunity to hire non-U.S. citizens, and by creating procurement rules that would enable them to enter into flexible agreements on innovative projects. But, he is not optimistic about the likelihood of Congress changing the rules on hiring non-citizens.
  • Need to look to one of the larger agencies – Census Bureau or BLS – and ask them to be a center for research. However, the problem is that asking these agencies does not change the procurement rules or hiring non-citizen problems. Habermann also noted that he worries about who will replace the good people who are leaving and that federal agencies don’t have enough good people as it is.

Habermann then outlined a solution that he is proposing: to form a FFRDC – a federally funded research and development center (model is in DOD – they have a number of them). It would be private and non-profit and only do work for the federal government. The FFRDC would not be subject to federal rules for procurement or for hiring non-citizens. He estimates that it would only cost $15 million to fund the FFRDC per year. Habermann commented that, at the seminar, agency heads agreed with creative realignment of the boundaries between agencies and cooperation to engage in innovation. Habermann concluded by noting that more important than a solution is the recognition that times are changing and there could be big changes and we need to do something. In his words, “the statistical system needs to recognize that the water is getting hot and the statistical system needs to jump out.”

Survey Cost Savings for the National Health Interview Survey
Barbara O’Hare, U.S. Census Bureau and Jennifer Madans, National Center for Health Statistics

O’Hare began by explaining that the Director of the Census Bureau, Bob Groves, has established an initiative to identify cost savings for the reimbursable surveys conducted by the Census Bureau. Reimbursable surveys are those that are sponsored by another federal agency. This initiative is in response to the challenges facing federal agencies in data collection – declining respondent cooperation and rising operational costs. She noted that data collection can’t be sustained under the old models, and new approaches need to be developed to contain costs while maintaining data quality. Under this initiative, task forces have been formed to identify potential costs savings for the following surveys: The National Health Interview Survey; The Consumer Expenditure Survey; the National Crime Victimization Survey; the Current Population Survey; The American Housing Survey; and the National Ambulatory Medical Care Survey/National Hospital Ambulatory Medical Care Survey. Task force members include key staff of the sponsoring agency; survey staff from the Census Bureau; an end data user; and a survey methodologist. These task forces are small, short-lived teams designed to foster dialogue between the Census Bureau as the data collection agency and the survey sponsor agency, and to identify the most promising cost-saving opportunities to pursue. A summary report is being prepared for each task force.

O’Hare highlighted the following overarching findings from the task forces:

  • Need for more detailed and new information – especially on costs.
  • Need for research on historical and new data to understand cost drivers and interventions.
  • Census corporate issues need to be addressed.
  • Some survey-specific initiatives

    • Providing wireless internet to field operators.
    • Eliminating redundancy in questionnaires to reduce the time and burden to complete.
    • Hours per case guidelines for field reps.
    • Appending phone numbers to records before they are delivered.

      She then indicated that the Census Bureau is committed to corporate infrastructure investments to increase operational efficiency. These include:

    • Improving sharing of cost information with sponsoring agencies.
    • Collecting more detailed field activity data for analyses and monitoring
    • Incorporating responsive design and expanded measures of survey performance success during collection – making real-time adjustments – e.g. double sampling – sub-sampling among those who do not respond.
    • Pursuing administrative records opportunities – like subsidized housing in the American Housing Survey – people are poor reporters of this but HUD has good data.
    • Committing resources to developing web instruments – Computer Audio-Recorded Interviewing, web communication channels with field reps.
    • Continual collaboration with sponsoring agencies to follow through on cost concerns.

Madans then provided an overview of the most promising opportunities for cost savings that the task force identified for the National Health Interview Survey (NHIS):

  • Move from weekly to monthly field data collection assignments.
  • Use the Post Office’s Delivery Sequence File or other external vendor files to reduce the cost of the address frame.
  • Establish a more detailed cost accounting system, down to the case level.
  • Streamline survey management responsibilities to reduce staff costs.
  • Revise the Field Representative performance appraisal system to balance expectations across performance standards.
  • Provide limited internet access to field reps through Census Bureau laptop computers.

Madans noted that these are not new ideas, but it is useful to formalize and systematize the issues and the formal solutions.

O’Hare concluded by saying that the most important thing about this initiative is the fact that it is being done over six different surveys to identify cross-cutting needs and solutions. Changes are expensive and more worthwhile if they will move the field forward and benefit more than one survey. O’Hare will write a final report summarizing the findings and recommendations and then identifying action plans.

Measuring Group Quarters Population in the American Community Survey
Paul Voss, Committee on National Statistics

Voss described a panel he is chairing for a Committee on National Statistics (CNSTAT) project sponsored by the Census Bureau. The charge for the panel is to conduct an in-depth review of the statistical methodology for measuring the group quarters (GQ) population in the American Community Survey (ACS) and to advise the Census Bureau on measuring GQ in the ACS. The panel’s activities include six meetings, several workshops, and preparation of an interim and a final report.

Voss identified the following statistical and operational issues:

  • Sample frame
  • Sampling
  • Weighting and pop controls

He noted that at the state level, the variances for GQ are not so large, but below the state level they are quite high.

Voss then outlined the following data user issues:

  • Who are the users of ACS GQ data?
  • How are these data used?
  • Who would object if the panel recommended to the Bureau that some or all GQ subpopulations be placed out of scope in the ACS?
  • Why would they object?
  • What is the basis of objection to such a recommendation?

Voss indicated that the CNSTAT panel needs to hear from data users in the next year, and that he is requesting such input formally as chair of the panel.

Voss then shared some observations about the GQ data in the ACS. Early indications suggest that there are few users of the ACS GQ per se because there is not much data out there yet. For example, 40 percent of counties and 75 percent of school districts have not yet seen an ACS number. The eventual ACS data for GQ populations is going to be pretty thin – people did not use the long form data much either. At the state level, ACS data users will get population characteristics for the GQ population, but not by type of GQ. Below the state level, users will only get a total number for the GQ population. At the national level, users will get GQ population characteristics by GQ type. Voss suggested that the 2010 census data showing GQ populations by type and age-sex-race characteristics might be sufficient for most data users. Voss also noted that there are many other special surveys of GQ populations designed to meet federal data and programmatic needs, and asked what the added value is of getting GQ estimates from ACS. He also discussed some of the problems with the ACS GQ data, such as zeros for GQ population for areas that have GQ facilities in the MAF.

Voss then turned to consideration of what would happen if the GQ population were not included in the total population universe. He asked who would object and why? Voss noted that there are seven major types of GQ that the Bureau estimates, and that there are 8 million total people living in Group Quarters, which is about 3 percent of the U.S. population. The Census Bureau has been collecting GQ data since 1850, and it has been part of the decennial sample data collection since 1940. However, the comprehensiveness of GQ tabs has varied from census to census. GQ data has been collected in the ACS since 2006.

Voss next highlighted some of the reasons why collection of GQ data should be retained in the ACS:

  • Because the Bureau promised – it is needed for the ACS to replace the Long Form.
  • Since 1970, the Long Form sample has provided content-rich and statistically strong data for the full US population – for important programmatic needs.
  • Controls for Long-Form estimates were from the Short Form – e.g., counts that we had in the field at the same time.
  • Because ACS GQ data are much improved over the 2000 Long Form GQ data because they have reduced levels of item imputation.
  • Because some users may need these data:
    • Need certain GQ data for particular programs.
    • Many who need it in the ACS universe.

Voss also outlined some reasons why the Bureau should consider dropping some or all ACS GQ data:

  • The need for the data is not well documented.
  • Not much GQ data is going to be published anyway.
  • There are alternative source of GQ data that will be better.

Voss also asked about the implications for the ACS PUMS of dropping GQ data.

Voss summarized by noting again that there are some problems with the data – relating to high variances and zero populations. Also, he indicated that the imputation rates are uncomfortably large (around 10 to 15% - but much higher in prisons and dorms and through proxy – as high as 50% or more on some items). He expressed concern about the fact that the controls are now estimate-based as we have lost the controls from the short form census. Finally, he noted that the ACS has already dropped some small, hard-to-survey GQ populations, such as those in soup kitchens, etc.

Voss concluded by asking if there are there some appropriate cost-benefit issues in light of the fact that collection of GQ data costs $89 per person, and collection of Household data costs $13 per person. The GQ is much more expensive. Voss indicated that the CNSTAT panel will try to put out a survey and will have a data user conference. In the meantime, those with any questions or those who would like to provide input can either contact Voss at: or the CNSTAT study director Krisztina Marton at:

Concerns From COPAFS Constituencies

No concerns were raised, but it was suggested again that this topic should be moved up in the agenda to the time slot right after lunch.