Social Impact of COVID-19 Survey August 2020: The Reopening of Schools

Purpose of the Survey and Reference Period

A questionnaire (PDF 103KB)  for the second round of the Social Impact of COVID-19 survey was conducted by the CSO from 13^th-19^th August. Individuals selected received an email from the CSO and were asked to complete the questionnaire online. The questionnaire asked for information on the following topics:

• Measuring the impact enforced school closures had on students’ education
• Measuring attitudes regarding returning to school
• Life satisfaction levels, in general terms as well as in terms of financial satisfaction and satisfaction with personal relationships
• Compliance levels with official COVID-19 advice

Sample Selection

This survey is the third in a series on the Social Impact of COVID-19. The sample was generated from Labour Force Survey (LFS) respondents that agreed to be contacted for further research and provided an email address and phone number. The Labour Force Survey is a 2-stage sample design stratified using Administrative County and the Pobal HP Deprivation Index.

This Social Impact of COVID-19 survey is third such survey conducted by the CSO.  The sample selection methodology resulted in a sample of 2,226 people.

Data Collection

All potential respondents were contacted by email and were asked to complete an online questionnaire. Data collection was closed on Wednesday 19^th August 2020, at which point the achieved sample was 1,333 individuals.

Sample Design

Timeliness was a key priority in this survey and therefore the sample and subsequent weighting process is one of convenience to some extent. Some consideration needs to be given to the potential impact of sample design on response rates and achieved sample:

• The original LFS sample from which this sample was selected was based on Census 2011 data and designed to represent the population then, so new additions to the population may not be fully represented.
• The sampling frame excludes people who do not live in private households, so may not cover some of those more likely to be negatively affected by the spread of the disease.
• The sampling frame was composed of individuals who had responded through 5 waves of the LFS survey, which could have introduced bias into the sample, as their characteristics may differ in some way to people more inclined to drop out of a longitudinal sample over time.
• The sample did not include people that did not provide an email address, which means members of the population less likely to have an email address, such as older people or people without internet access, were likely to be underrepresented.
• There is a mode effect, whereby the method of administration of the questionnaire can impact responses. For example, the presence of an interviewer in CATI can encourage a higher response rate compared to self-administered web questionnaires. On the other hand, sensitive questions may be more honestly answered without an interviewer present.
• The achieved sample distribution could also have been impacted by non-response bias. This is caused by the fact that some respondents might be more inclined to respond than others, and people who respond to these surveys often have different characteristics compared to non-respondents.

The weighting procedure outlined below was designed to adjust for possible bias in the achieved sample as much as possible.

Weighting

The following weighting process was devised to counteract some of the potential bias within the sample, and to make the final weighted sample distribution as representative as possible of the population.

Stage 1: Non-response

In the first stage of the weighting process, each person in the sample was given a weight of 1. We utilised the current LFS non-response adjustment process, in which a stepwise logistic regression was conducted based on census household-level data, to generate response propensities based upon the following characteristics:

• Personal characteristics (of head of household) – Sex, Marital status, Education, Nationality, PES, Age, Social class, Ethnicity
• Household characteristics – Area type, Dwelling type, Tenure, Number of persons in household, Number of cars owned by household, Number of rooms, Household PC, Urban/Rural, County

The sample is then grouped into strata based on propensity score, for which non-response adjustments were calculated and applied to each respondent.

Stage 2: Calibration

In stage 2, Q2 2020 LFS population estimates were used to benchmark the dataset across key characteristics for calibration. The non-response adjustments were inflated match overall the population total and then calibrated using CALMAR^[1], to ensure that weighted sample distributions matched the Q2 2020 benchmark distributions for a number of key characteristics such as gender, age, education level, region, urban/rural location, household composition.

As outlined above, non-response adjustment has been used to address some of the imbalances between the original sample design and the achieved sample distribution as much as possible, and the subsequent calibration adjusts to key population totals to try and match current population distributions. However, given the non-random nature of the final sample selected, it is unlikely that we can fully account fully for bias inherent in the final sample. For this reason, caution should be taken when attempting to make inferences to the entire population from these results.

^[1] CALMAR is the statistical software developed by INSEE. Calmar is a SAS macro program that implements the calibration approach and adjusts weights assigned to individuals using auxiliary variables.

Questionnaire

The questionnaire focused on the impact of school closures and concerns about returning to school, as well as questions on levels of compliance with government guidelines and wellbeing indicators such as life satisfaction, financial satisfaction and satisfaction with personal relationships.

Data analysis

Some key analysis variables within this publication are:

Primary School Cycle

This analysis variable is broken down by:

• Junior Primary (Junior Infants, Senior Infants and First Class)
• Senior Primary (Second to Sixth Class)

Secondary School Cycle

This analysis variable is broken down by:

• Junior secondary (First to third year)
• Transition year
• Senior secondary (Fifth and Sixth year)

Highest level of education attained

This classification is derived from a single question and refers to educational standards that have been attained and can be compared in some measurable way and it is included in the core LFS on an ongoing basis.

The question is phrased as follows:

What is the highest level of education or training you have ever successfully completed?

For the purposes of this publication these have been classified as follows:

• Higher secondary education or lower
• Post-secondary or Short-cycle tertiary
• Third-level bachelor degree or higher

Well-being

In 2018, the Survey on Income and Living Conditions (SILC) carried out an ad-hoc module on “Material deprivation, well-being and housing difficulties”, which itself provides comparisons with the SILC 2013 “Well-being” module. These surveys provided an interesting reference point for the Social Impact of COVID-19 survey conducted in April 2020. The wellbeing questions in this August survey aim to provide some further insight into feelings of wellbeing in Ireland. While the methodologies across all surveys differ and care should be taken in the interpretation of trends over time, nonetheless the findings from these surveys present an important perspective on the impact of COVID19 in Ireland.

Urban/rural location

Areas are classified as Urban or Rural based on the following population densities derived from Census of Population 2016:

• Urban: Population density greater than or equal to 1,500
• Rural: Population density less than 1,500

NUTS2 Regions

Regional analysis is presented in this publication are based on the NUTS2 (Nomenclature of Territorial Units) classification used by Eurostat. The regions are categorised as follows:

• Northern and Western
• Southern
• Eastern and Midland

These regions are comprised as follows:

Acknowledgement

The Central Statistics Office wishes to thank the participants for their co-operation in agreeing to take part in the Social Impact of COVID-19 Survey and for facilitating the collection of the relevant data.