 |
This Web site is a component of the SAMHSA Health Information Network. |
|
Appendix B
Sources and Qualifications of the Data:
1997 Client/Patient Sample Survey
Survey Design
Scope of the Survey
The survey was conducted during 1997 and included all types of specialty mental health care organizations located in the 50 States, the District of Columbia, and the Territories. The types of organizations included in the survey were State and county mental hospitals, private psychiatric hospitals, multiservice mental health organizations, Department of Veterans Affairs medical centers, non- Federal general hospitals with separate psychiatric services, residential treatment centers for emotionally disturbed children, freestanding outpatient mental health clinics, and freestanding partial care organizations. The survey covered the inpatient, residential, and less than 24-hour care programs operated by these types of organizations during a 1- month period in 1997.
The target population included two groups: (1) all persons newly admitted, readmitted, or transferred into the program during a specified survey month who were not already residents/on the rolls of the program on the first day of the survey month, referred to as the admission population, and (2) all persons who were admitted to the program before the first day of the specified survey month and who received service from the program during the survey month, referred to as the under care population. An oversample of children and youth under age 18 was included in the sample design so that reliable national estimates could be generated for this specific population subgroup. Separate survey questionnaires were designed to collect data from four groups-adult admissions, adults under care, child admissions, and children under care, from within the inpatient, residential, and less than 24-hour care programs of the mental health organizations identified above.
The survey was conducted by the Survey and Analysis Branch (SAB), Division of State and Community Systems Development (DSCSD), Center for Mental Health Services (CMHS), Substance Abuse and Mental Health Services Administration (SAMHSA), in cooperation with State mental health agencies.
Sampling Frame
The sampling frame for the survey was the 1994 Inventory of Mental Health
Organizations and General Hospital Mental Health Services (IMHO/ GHMHS).
Unique combinations of the eight organization types and three program types,
identified earlier, defined the 14 first-stage primary sampling strata (table
B1). The term "organization/program," used henceforth, refers to these combinations.
The measure of size used to stratify the programs was the number of persons
under care at the beginning of 1994 plus the number of admissions during 1994.
Sample Design: First-Stage Selection- Mental Health Organizations and Programs
The sample was based on a two-stage cluster design for all primary strata, with the exception of primary strata 2, 9, and 10 (see Table B1). For these strata, the sample design was a single-stage design with all programs selected with certainty. Actual sampling was carried out in several steps. First, to ensure geographic representation of the sample, programs were arranged separately by region, by State within region, and by city within State. A systematic sample of programs was then selected for each cell with a random start in the first sampling interval. This sampling procedure was carried out separately for organizations that operated one or two program types and those that operated three program types (i.e., inpatient, residential, and less than 24-hour). This was done to reduce the burden on organizations so that no more than two programs were selected from any given sampled organization.
For all primary strata, except Department of Veterans Affairs medical centers (strata 9 and 10), which are exclusively for adults, most organizations/programs treated both adults and children. A small number of organizations/programs either treated adults only or treated children only.
The total number of programs sampled was 2,737, of which 228 were not within the scope of the survey, that is, "out-of-scope" (e.g., program closed). The overall survey response rate was 64 percent of the target sample. The final column in table B1 presents the number of organizations/programs that responded to the survey by primary stratum.
Sample Design: Second-Stage Selection- Clients/Patients
For client/patient selection, separate listing booklets were used to establish the sampling frame for each of the four groups (adult admission, adult under care, child admission, and child under care) within each type of program (inpatient, residential, and less than 24-hour). Using separate booklets for adults and children under age 18, sample programs were asked to list the case numbers for all persons newly admitted, readmitted, or transferred into the program during the survey month who were not already resident/on the rolls of the program on the first day of the survey month. Sample programs were also asked to list in separate booklets for adults and children under age 18 the case numbers for all persons who were admitted to the program before the first day of the survey month and who received service from the program during the survey month. Programs were asked to list case numbers only once in the booklets, and to include all geographic locations of the program. Programs had the option of generating computerized client/patient listings in place of manually completing the listing booklets. Once the listings were completed, programs were asked to call a toll-free telephone number to speak with a survey specialist. Using a specially designed computer program to generate random numbers for the survey and using information obtained directly from the program, the specialist selected "online" random numbers that corresponded to completed line numbers in the program's listing booklets (or computer-generated listings). The specialist informed the program as to which line numbers were selected. The case numbers found on these line numbers identified for the program which persons were to be sampled.
To reduce the burden on an organization/program, the total number of questionnaires that were to be completed on persons sampled from all four groups was limited to a predetermined number based on the size of the program. Smaller programs were requested to complete a maximum of 8 questionnaires; larger programs a maximum of 16 questionnaires.
Table B2 presents the number of persons sampled and the number of respondents in each of the four groups by primary stratum.
For strata 1, 3, 4, 6, 7, 8, 11, 12, and 14, children were oversampled at a rate of three to one compared to adults. For strata 2, 5, and 13, children were sampled at the same rate as adults. For strata 9 and 10, which refer to the Department of Veterans Affairs medical centers, children were not sampled (i.e., not applicable).
Data Collection and Instruments
Data collection was accomplished primarily by mail, with telephone followup to participating programs. Initial letters were mailed to the administrators of sample organizations in March 1997 to inform them of the survey, its purpose, anticipated levels of effort that would be required, and the program(s) in their organization that had been selected for the survey. A followup call was made to the administrators to discuss the survey further, answer questions, and request participation. Numerous attempts were made by certified mail and telephone callbacks to elicit survey participation. Prior to the survey month, a packet of survey materials was sent to the designated person for each program that had agreed to participate. The packets included all necessary survey forms (color-coded listing booklets and corresponding questionnaires) and instructional material (detailed instructions for completing the survey forms, procedures for selecting the sample of persons, information on obtaining survey assistance, and instructions on returning the completed survey forms in the postage-paid return envelopes provided in each packet).
The data collection forms used for the survey focused on the sociodemographic, clinical, and service use characteristics of persons. Inpatient and residential sample programs used the same color-coded listing booklets and questionnaires. These forms were similar in content to the forms used for less than 24-hour care programs with slight variations in vocabulary to conform to different program usage. Different colored forms were used to differentiate among the four groups: adult admissions, adults under care, child admissions, and children under care, and between inpatient/residential and less than 24-hour care programs.
Estimation
The sample for this survey was weighted to produce unbiased national estimates about the number and characteristics of persons served in the inpatient, residential, and less than 24-hour care programs of specialty mental health organizations in the United States. Sample counts were inflated to national estimates in accord with each stage of the sample design and nonresponse patterns. Hence, estimates reported for admissions are weighted to 1- year totals; those for the under care population to 1- day totals.
Limitations of the Design
Nonresponse
For this survey, nonresponse errors could exist in three ways: (1) failure to obtain participation from some of the programs selected into the sample; (2) failure to obtain data for some of the persons selected into the sample; and (3) failure to obtain complete data for some sampled persons.
To minimize bias that might exist due to nonresponse, the information reported by responding organizations was adjusted to compensate for program and person nonresponse. The first-stage adjustment factor was the ratio of the number of sampled programs (after removing the out-of-scope programs) to the number of programs that responded. This adjustment factor was calculated and applied separately to each stratum for each organization by program type combination. The second-stage adjustment factor was the ratio of the number of sampled persons admitted or persons under care to the number of corresponding person respondents, calculated and applied separately for each of the four groups in each program respondent.
Missing items on the survey questionnaires were imputed using a sequential hot deck procedure, as follows: Records were sorted on core sets of variables, such as organization and program type, client/patient type, gender, age, diagnosis, and region, to determine the imputation classes. The value of the variable from the previous completed record in this ordered file was substituted for the unknown value. After the sequential hot deck procedure was performed on a given variable, a determination was made on how many times a given donor was used in the process. If any donor was used five or more times during imputation of a particular variable, a within-class random hot deck procedure was performed instead of a sequential hot deck procedure to impute that variable. That is, records were sorted on core sets of variables to determine the imputation classes. Then an observed value of the variable was selected at random within that imputation class to substitute for the unknown value.
Reliability of Estimates
Background
Because estimates presented in this report are based on sample data, they are likely to differ from figures that would have been obtained from a complete enumeration of the universe of specialty mental health organizations using the same instruments. Results are subject to both sampling and nonsampling errors. Nonsampling errors include biases due to inaccurate reporting, processing, and measurement, as well as errors due to nonresponse and incomplete reporting. These types of errors cannot be measured readily. However, to the extent feasible, each error has been minimized through the procedures used for data collection, editing, quality control, and nonresponse adjustment.
The sampling error (standard error) of a statistic is inversely proportional to the square root of the number of observations in the sample. Thus, as the sample size increases, the standard error decreases. The standard error measures the variability that occurs by chance, because only a sample rather than the entire universe is surveyed. The chances are about two out of three that an estimate from the sample differs by less than one standard error from the value that would be obtained from a complete enumeration. The chances are about 95 out of 100 that the difference is less than twice the standard error, and about 99 out of 100 that it is less than three times as large.
In this chapter, statistical inference is based on the construction of five-percent confidence intervals for estimates (0.05 level of significance). All statements of comparison in the text relating to differences such as "higher than" and "less than" indicate that the differences are statistically significant at the 0.05 level or better. Terms such as "similar to" or "no difference" mean that a statistical difference does not exist between the estimates being compared. Lack of comment on the difference between any two estimates does not imply that a test was completed and there was a finding of no significance.
Calculation of standard errors
Standard errors were calculated on a personal computer for a broad range of totals and subtotals within age, gender, and race subclasses through the use of SUDAAN Survey Data Analysis Software developed at the Research Triangle Institute by B.V. Shah. This procedures computes estimated standard errors through the use of Taylor series approximation. As applied to data from the present survey, variance estimates for totals and subtotals were calculated for each stratum and then summed across strata to derive standard errors for characteristics of interest. The variance estimate for each stratum includes both the between-program and the within- program components of variance, with corrections for finite populations applied at both sampling stages.
Relative Standard Errors of Totals and Subtotal Estimates, Percentages,
and Rates
The relative standard error of a total or subtotal estimate, percentage, or rate for a characteristic of interest is obtained by dividing the standard error of the estimate by the estimate itself and is expressed as a percentage of the estimate.
Relative Standard Errors of Differences Between Two Statistics
The standard error of a difference is approximately the square root of the sum of the squares of each standard error considered separately. The relative standard error of a difference is the standard error of a difference divided by the difference.
Relative Standard Errors of Statistical Sums
The standard error of a sum of a number of independent estimates is the square root of the sum of the squares of the standard errors of the separate estimates. The relative standard error of the sum is the standard error divided by the sum.
Table B3 presents standard errors and percent relative standard errors for the estimated numbers, percentages, and rates per 100,000 U.S. civilian population of selected major characteristics for persons under care and admitted to inpatient, residential, and less than 24-hour care programs, for each type of organization surveyed. The statistics presented in table B3 can be used to show the relative sizes of the characteristics detailed in tables 1 through 19 of Chapter 15. The reader is cautioned that if a relative standard error (i.e., the standard error of an estimate, percentage, or rate divided by the estimate, percentage, or rate itself, expressed as a percent) is 50 percent or higher, the estimate, percentage, or rate is not considered reliable and should not be used.
Index | Previous | Next
|
|
