Community survey

The following information has been taken from the survey review provided by the Queensland Government Office of Economic and Statistical Research, Office of the Government Statistician at the completion of the survey and as part of the analysis provided. The project team  commmissioned OESR to undertake the community survey on their behalf.

A copy of the questionaire can be accessed here.

Survey objective

The objective of the Resilience Profiles Project – Community Survey 2011 was to assess and measure community resilience in three specific Queensland regions.

Survey frame

The scope of the population encompasses residents of three geographical regions: Tablelands, Rockhampton and Chinchilla. Using combined mail-out and online methodologies, OESR collected the opinions and knowledge of a cross-section of the population in the regions of interest, randomly sampling approximately 1,000 households from each region.

In addition, QCOSS and GU invited a non-random sample of residents (i.e., convenience sample) to participate in the survey, by issuing them with a direct link to a separate but similar web survey. This represents a supplementary sample to the household sample selected by OESR.

Of the 946 households completing a survey 324 (34.2 per cent) were surveyed in the Tablelands region; 300 (31.7 per cent) in the Rockhampton region; and 322 (34.0 per cent) in the Chinchilla region.

Data collection method and fieldwork

Survey administration

Respondents were invited by mail to participate in a hard-copy or web-based version of the survey. OESR distributed paper surveys and cover letters to each selected household. The cover letter included a unique web address for the respondent’s online version of the survey. Respondents could choose the most convenient way to complete the survey. The cover letter also included a toll-free phone number for any technical difficulties with the web survey, and a project phone number for any questions about research outcomes.

Paper surveys were printed with a four-digit identification number regional location reference.  

The web-based survey had the specifications:

  • secure login (convenience sample excluded), with the capacity to save a partially completed survey and login at a later point to complete it;
  • fully functional survey interface, with capacity for single-response, multiple-response and open-ended questions;
  • validations to ensure that mandatory fields are completed; and
  • skip logic to filter survey sections according to population and sample characteristics.

The data collection period was 31 October 2011 to 5 December 2011. During this period, the respondents received an invitation to participate, and approximately two weeks after the initial dispatch of cover letter/survey, a reminder letter or text message was sent to those who had not returned their completed survey. In order to boost response, telephone reminders were also initiated.

Data and responses

Trained OESR staff managed the data entry of the paper surveys. With QCOSS and GU requesting a supplementary convenience sample for the web survey, two separate web surveys were implemented with different web links for access. The second sample resulted in two data extractions and two files with the supplementary convenience sample survey data sent directly to QCOSS and GU in Excel format. OESR was not involved with any further processing of this supplementary sample other than to remove identifying data. Sequencing checks were carried out by OESR. 

Operational results

Final status and scope of respondents on sample frame

The final response status of respondents on the Resilience Profiles Project – Community Survey 2011 frame is described in table 1. Almost thirty two per cent of respondents in the frame completed the survey.  

Table 1: Final status of respondents on sample frame

Final status

Frequency

Percentage

Completed (Mail-out)

735

24.5

Completed (Web)

211

7.0

No response

1,711

57.0

Refused

116

3.9

Unable to participate

13

0.4

Undeliverable

197

6.6

Out of scope

19

0.6

Total

3,002

100.0*

*Percentages may not add to exactly 100.0 due to rounding.

Respondents were classified as in-scope responding if the participant completed or partially completed the survey. Two hundred and eleven householders completed the web survey and 735 respondents completed the paper survey.

Respondents were classified as out-of-scope if the participant was in aged care and too frail to participate, a business office, deceased, did not open the survey or no longer resided in Queensland (n = 19).

The remaining respondents (n=2,037) were classified as in-scope non-responding

Table 2 describes the percentage of respondents who completed or partially completed the survey as a function of the total number of in-scope respondents on the frame.  

Table 2: Final status of in-scope respondents

Status

In-scope responding

In-scope non-responding

Total in-scope

Percentage %

Completed (Mail-out)

735

0

735

24.6

Completed (Web)

211

0

211

7.1

No response

0

1,711

1,711

57.3

Refused

0

116

116

3.9

Unable to participate

0

13

13

0.4

Undeliverable

0

197

197

6.6

Total in-scope

946

2,037

2,983

100.0*

*Percentages may not add to exactly 100.0 due to rounding.

Survey response rate

The response rate is a measure of the quality of response achieved in a survey. This is defined as the number of completed web surveys used in the analysis as a percentage of the total number of potential surveys that would have been achieved had every in-scope respondent completed the survey.

Resilience Profiles Project – Community Survey 2011 achieved a response rate of 31.7 per cent.

Response rate       = In-scope responding / (in-scope responding plus in-scope non-responding)

                                   = 946 / 2,983

                                    = 31.7%  

Table 3: Response rate by region

 

Total respondents

In-scope responding

In-scope non-responding

Out of scope

Response rate

(%)

Tablelands

1,000

324

672

5

32.6

Rockhampton

1,001

300

691

9

30.3

Chinchilla

1,001

322

674

5

32.3

All respondents

3,002

946

2,037

19

31.7

Note: Non-respondents includes participants defined as in-scope who did not respond to the survey. Respondents defined as out-of-scope are excluded from response rate calculations.

Weighting

Household questions

This survey makes use of a household sample frame to estimate household characteristics in three demographic regions. Each responding household in the survey represents a certain number of households within the population of the frame. These numbers are referred to as weights and are used as multipliers in calculations.  

Household weights for each region were calculated to reflect the differing probabilities of being selected and responding. A stratified simple random sample number-raised weight was calculated for each unit as follows:  

Weight = Number of households in region / Number of randomly selected households in region that responded to survey 

The survey was designed to maximise the representativeness of the results, however, 100 per cent accuracy is not possible. As a result, estimates of household characteristics have a level of imprecision associated with them (See Section 5).  

Person questions

When collecting survey data, response rates may differ between different groups. A higher rate of non-response in certain groups, may lead to contact bias in the estimates. For example, older people and women are usually easier to contact than younger people and men and, without adjustment in the weighting process, may have a disproportionately large influence on the results. However, information is unavailable as to how this bias may affect estimates and so it was not possible to weight the person-specific responses.  

As a result, person-specific responses are presented in the output tables as sample frequencies only. No inferences about the wider population beyond the respondents themselves may therefore be made from these person-specific questions.  

Reliability of estimates

Summary

Although the survey has been designed to maximise the representativeness of the household-specific results, it is not possible to be perfectly representative. 

Estimates based on a sample survey are subject to two types of error:

  • Sampling error. Estimates based on information obtained from a sample of households may differ from figures that would have been produced if all households had been included in the survey.
  • Non sampling error. Errors may also occur due to non-response to the survey, inadequacies of the sampling frame, inaccuracies in reporting by respondents and processing errors.

 One measure of the sampling error is the standard error (SE). It measures the extent to which an estimate may vary by chance because only a sample of households was included in the survey.  

Given a large enough sample size, there are about two chances in three that an estimate will differ by less than one standard error from the figure that would have been obtained if all households had been included, and about 19 chances in 20 that the difference will be less than two standard errors.  

An alternative measure of the sampling error is the relative standard error (RSE), which expresses the standard error as a percentage of the estimate. The RSE of an estimate is given by the following expression: 

            RSE = (SE/Estimate) x 100  

where SE stands for the standard error of the estimate.  

Calculation of standard errors

The standard errors for each estimate were calculated using a stratified simple random sample number raised standard error estimation methodology.  

Confidence intervals

The standard errors were used to construct confidence intervals on the estimates. These intervals represent the range within which there is a 19/20 (or 95 per cent) chance that the population value falls. For example an estimate of 65 per cent might have an associated confidence interval of (59.5 per cent, 70.5 per cent). Thus, the probability that the actual value of that percentage is between 59.5 per cent and 70.5 per cent is 0.95.  

Confidence intervals can be obtained from the standard error using the general formula:  

            CI = Estimate + Z x SE  

where Z is the appropriate value from the standard normal table. For example, for a 95 per cent confidence interval, Z = 1.96 (often rounded to 2).