Appendix B

Sources and Qualifications of the Data: 1997 Client/Patient Sample Survey

Survey Design

The survey was conducted during 1997 and included all types of specialty mental health care organizations located in the 50 States, the District of Columbia, and the Territories. The types of organizations included in the survey were State and county mental hospitals, private psychiatric hospitals, multiservice mental health organizations, Department of Veterans Affairs medical centers, non-Federal general hospitals with separate psychiatric services, residential treatment centers for emotionally disturbed children, freestanding outpatient mental health clinics, and freestanding partial care organizations. The survey covered the inpatient, residential, and less than 24-hour care programs operated by these types of organizations during a 1-month period in 1997.

The target population included two groups: (1) all persons newly admitted, readmitted, or transferred into the program during a specified survey month who were not already residents/on the rolls of the program on the first day of the survey month, referred to as the admission population, and (2) all persons who were admitted to the program before the first day of the specified survey month and who received service from the program during the survey month, referred to as the under care population. An over sample of children and youth under age 18 was included in the sample design so that reliable national estimates could be generated for this specific population subgroup. Separate survey questionnaires were designed to collect data from four groups— adult admissions, adults under care, child admissions, and children under care, from within the inpatient, residential, and less than 24-hourcare programs of the mental health organizations identified above.

The survey was conducted by the Survey and Analysis Branch (SAB), Division of State and Community Systems Development (DSCSD), Center for Mental Health Services (CMHS), Substance Abuse and Mental Health Services Administration (SAMHSA), in cooperation with State mental health agencies.

Sampling Frame

The sampling frame for the survey was the 1994 Inventory of Mental Health Organizations and General Hospital Mental Health Services (IMHO/GHMHS). Unique combinations of the eight organization types and three program types, identified earlier, defined the 14 first-stage primary sampling strata (Table B1). The term "organization/program," used henceforth, refers to these combinations. The measure of size used to stratify the programs was the number of persons under care at the beginning of 1994 plus the number of admissions during 1994.

The sample was based on a two-stage cluster design for all primary strata, with the exception of primary strata 2, 9, and 10 (see Table B1). For these strata, the sample design was a single stage design with all programs selected with certainty. Actual sampling was carried out in several steps. First, to ensure geographic representation of the sample, programs were arranged separately by region, by State within region, and by city within State. A systematic sample of programs was then selected for each cell with a random start in the first sampling interval. This sampling procedure was carried out separately for organizations that operated one or two program types and those that operated three program types (i.e., inpatient, residential, and less than 24-hour). This was done to reduce the burden on organizations so that no more than two programs were selected from any given sampled organization.

For all primary strata, except Department of Veterans Affairs medical centers (strata 9 and 10),which are exclusively for adults, most organizations/programs treated both adults and children. A small number of organizations/programs either treated adults only or treated children only.

Sample Design: Second-Stage Selection— Clients/Patients

For client/patient selection, separate listing booklets were used to establish the sampling frame for each of the four groups (adult admission, adult under care, child admission, and child under care) within each type of program (inpatient, residential, and less than 24-hour). Using separate booklets for adults and children under age 18, sample programs were asked to list the case numbers for all persons newly admitted, readmitted, or transferred into the program during the survey month who were not already resident/on the rolls of the program on the first day of the survey month. Sample programs were also asked to list in separate booklets for and children under age 18 the case numbers for all persons who were admitted to the program before the first day of the survey month and who received service from the program during the survey month. Programs were asked to list case numbers only once in the booklets, and to include all geographic locations of the program. Programs had the option of generating computerized client/patient listings in place of manually completing the listing booklets. Once the listings were completed, programs were asked to call a toll-free telephone number to speak with a survey specialist. Using a specially designed computer program to generate random numbers for the survey and using information obtained directly from the program, the specialist selected "online" random numbers that corresponded to completed line numbers in the program's listing booklets (or computer-generated listings). The specialist informed the program as to which line numbers were selected. The case numbers found on these line numbers identified for the program which persons were to be sampled.

To reduce the burden on an organization/program, the total number of questionnaires that were to be completed on persons sampled from all four groups was limited to a predetermined number based on the size of the program. Smaller programs were requested to complete a maximum of 8 questionnaires; larger programs a maximum of 16 questionnaires.

Table B2 presents the number of persons sampled and the number of respondents in each of the four groups by primary stratum.

For strata 1, 3, 4, 6, 7, 8, 11, 12, and 14, children were over sampled at a rate of 3 to 1 compared to adults. For strata 2, 5, and 13,children were sampled at the same rate as adults. For strata 9 and 10, which refer to the Department of Veterans Affairs medical centers, children were not sampled (i. e., not applicable).

Data Collection and Instruments

Data collection was accomplished primarily by mail, with telephone follow up to participating programs. Initial letters were mailed to the administrators of sample organizations in March 1997 to inform them of the survey, its purpose, anticipated levels of effort that would be required, and the program(s) in their organization that had been selected for the survey. A follow up call was made to the administrators to discuss the survey further, answer questions, and request participation. Numerous attempts were made by certified mail and telephone callbacks to elicit survey participation. Prior to the survey month, a packet of survey materials was sent to the designated person for each program that had agreed to participate. The packets included all necessary survey forms (color-coded listing booklets and corresponding questionnaires) and instructional material (detailed instructions for completing the survey forms, procedures for selecting the sample of persons, information on obtaining survey assistance, and instructions on returning the completed survey forms in the postage-paid return envelopes provided in each packet).

Estimation

The sample for this survey was weighted to produce unbiased national estimates about the number and characteristics of persons served in the inpatient, residential, and less than 24-hour care programs of specialty mental health organizations in the United States. Sample counts were inflated to national estimates in accord with each stage of the sample design and nonresponse patterns. Hence, estimates reported for admissions are weighted to 1-year totals; those for the under care population to 1-day totals.

Limitations of the Design

For this survey, none response errors could exist in three ways: (1) failure to obtain participation from some of the programs selected into the sample; (2) failure to obtain data for some of the persons selected into the sample; and (3) failure to obtain complete data for some sampled persons.

To minimize bias that might exist due to nonresponse, the information reported by responding organizations was adjusted to compensate for pro-gram and person nonresponse. The first-stage adjustment factor was the ratio of the number of sampled programs (after removing the out-of-scope programs) to the number of programs that responded. This adjustment factor was calculated and applied separately to each stratum for each organization by program type combination. The second-stage adjustment factor was the ratio of the number of sampled persons admitted or persons under care to the number of corresponding person respondents, calculated and applied separately for each of the four groups in each program respondent.

Missing items on the survey questionnaires were imputed using a sequential hot deck procedure, as follows: Records were sorted on core sets of variables, such as organization and program type, client/patient type, gender, age, diagnosis, and region, to determine the imputation classes. The value of the variable from the previous completed record in this ordered file was substituted for the unknown value. After the sequential hot deck procedure was performed on a given variable, a determination was made on how many times a given donor was used in the process. If any donor was used five or more times during imputation of a particular variable, a within-class random hot deck procedure was performed instead of a sequential hot deck procedure to impute that variable. That is, records were sorted on core sets of variables to determine the imputation classes. Then an observed value of the variable was selected at random within that imputation class to substitute for the unknown value.

Reliability of Estimates

Background

Because estimates presented in this report are based on sample data, they are likely to differ from figures that would have been obtained from a complete enumeration of the universe of specialty mental health organizations using the same instruments. Results are subject to both sampling and nonsampling errors. Nonsampling errors include biases due to inaccurate reporting, processing, and measurement, as well as errors due to nonresponse and incomplete reporting. These types of errors cannot be measured readily. However, to the extent feasible, each error has been minimized through the procedures used for data collection, editing, quality control, and nonresponse adjustment.

The sampling error (standard error) of a statistic is inversely proportional to the square root of the number of observations in the sample. Thus, as the sample size increases, the standard error decreases. The standard error measures the variability that occurs by chance, because only a sample rather than the entire universe is surveyed. The chances are about two out of three that an estimate from the sample differs by less than one standard error from the value that would be obtained from a complete enumeration. The chances are about 95 out of 100 that the difference is less than twice the standard error, and about 99 out of 100 that it is less than three times as large.

In this chapter, statistical inference is based on the construction of 5-percent confidence intervals for estimates (0.05 level of significance). All statements of comparison in the text relating to differences such as "higher than" and "less than" indicate that the differences are statistically significant at the 0.05 level or better. Terms such as "similar to" or "no difference" mean that a statistical difference does not exist between the estimates being compared. Lack of comment on the difference between any two estimates does not imply that a test was completed and there was a finding of no significance.

Calculation of standard errors

Standard errors were calculated on a personal computer for a broad range of totals and subtotals within age, gender, and race subclasses through the use of SUDAAN Survey Data Analysis Software developed at the Research Triangle Institute by B. V. Shah. This procedures computes estimated standard errors through the use of Taylor series approximation. As applied to data from the present survey, variance estimates for totals and subtotals were calculated for each stratum and then summed across strata to derive standard errors for characteristics of interest. The variance estimate for each stratum includes both the between-program and the within-program components of variance, with corrections for finite populations applied at both sampling stages.

Relative Standard Errors of Totals and Subtotal Estimates, Percentages, and Rates

The relative standard error of a total or subtotal estimate, percentage, or rate for a characteristic of interest is obtained by dividing the standard error of the estimate by the estimate itself and is expressed as a percentage of the estimate.

Relative Standard Errors of Differences Between Two Statistics

The standard error of a difference is approximately the square root of the sum of the squares ofeach standard error considered separately. The relative standard error of a difference is the standarderror of a difference divided by the difference.

Relative Standard Errors of Statistical Sums

The standard error of a sum of a number of independent estimates is the square root of the sum ofthe squares of the standard errors of the separate estimates. The relative standard error of the sum is the standard error divided by the sum.

Table B3 presents standard errors and percent relative standard errors for the estimated numbers, percentages, and rates per 100,000 U.S. civilian population of selected major characteristics for per-sonsunder care and admitted to inpatient, residential, and less than 24-hour care programs, for each type of organization surveyed. The statistics presented in table B3 can be used to show the relativesizes of the characteristics detailed in tables 1 through 19 of Chapter 15. The reader is cautioned that if a relative standard error (i.e., the standard error of an estimate, percentage, or rate divided by the estimate, percentage, or rate itself, expressed as a percent) is 50 percent or higher, the estimate, percentage, or rate is not considered reliable and should not be used.

Index | Previous |Next