Nigeria - Teacher Development Programme In-Service Training Impact Evaluation 2017, Endline Survey

Primary tabs

The Teacher Development Programme (TDP) is a six-year (2013-2019) programme funded by the UK Department for International Development (DFID), with a total budget of £34 million. It seeks to improve the quality of teaching in primary schools, junior secondary schools, and colleges of education at the state level in northern Nigeria. It works through in-service training for primary teachers, reform of pre-service teacher education, and strengthening evidence-based research on teaching. This evaluation focuses on the first of these three outputs: in-service training and support for primary teachers in the three core curriculum subjects of English, mathematics, and science. The programme initially operated in three states, Jigawa, Katsina, and Zamfara, and was later extended to Kaduna and Kano. This survey covers a group of schools that were randomly assigned to receive the TDP intervention in Jigawa, Katsina, and Zamfara, and a control group of schools in the same states that did not receive the TDP intervention. The impact evaluation has three main purposes: -Formative - to help inform the implementation of TDP in Phase 1 of in-service activities and the design and implementation in its Phase 2; -Summative - to help inform TDP, DFID Nigeria, and other education stakeholders if TDP's in-service teacher training activities have led to improvements in teacher effectiveness and pupil learning levels in English, mathematics, and science and technology; and -Learning - to assess from TDP what might work for improving teacher effectiveness in Nigeria and elsewhere. The impact evaluation is a theory-based, mixed-methods design. The endline research focuses on the efficiency, effectiveness, impact, and sustainability of the TDP (programme relevance was assessed through the baseline survey). The overarching evaluation questions for the endline research are: -Impact: Has TDP caused changes in pupil learning in English, mathematics, and science in TDP schools? (quantitative) -Effectiveness: Has TDP led to changes in teacher effectiveness? (qualitative and quantitative) -Efficiency: Were TDP results achieved on time and to plan? How does TDP's organisational set-up facilitate delivery? (qualitative and quantitative) -Sustainability: Are TDP's impacts on teacher effectiveness sustainable without further DFID support? (qualitative) At the core of this impact evaluation is a constrained randomised design, with half of the sample schools assigned to receive the TDP intervention, while the other half was assigned to the control group. The main source of quantitative data is the sample panel survey of 330 schools in Jigawa, Katsina, and Zamfara conducted in 2014 and 2017. At each sample school, the head teacher and a sample of teachers and pupils were interviewed and tested on their subject knowledge in English, mathematics, and science; lesson observations were conducted; a teacher roster was compiled; and classroom attendance by teachers and pupils was measured. The baseline and endline surveys were timed to allow sufficient time for TDP impact on pupil learning outcomes to be feasible, while ensuring that a panel of pupils could be sampled at baseline and endline within the same school, and producing results in time for them to be of value to the final stages of the TDP. The endline quantitative survey fieldwork was carried out by OPM Nigeria and Education Data, Research and Evaluation in Nigeria (EDOREN). The data from the endline survey available in the World Bank Microdata Catalog are from the TDP IE quantitative endline survey conducted in 2017. For the qualitative research findings and methods see the final endline report under Related Materials.

Acronym: 
TDPITCIE-EL 2017
Type: 
Microdata
Topics: 
Topic not specified
Economy Coverage: 
Economy Coverage not specified
Languages Supported: 
English
Geographical Coverage: 
Nigeria
Reference ID: 
NGA_2017_TDPITCIE-EL_v01_M
Version Production Date: 
August 12, 2018
Release Date: 
February 27, 2019

Harvest Source

Harvest Source: 
Microdata

Harvest Source ID

Harvest Source ID: 
10316

Last Updated

Last Updated: 
February 27, 2019
Data Collector(s) Name: 
Oxford Policy Management Nigeria
Study Type: 

School survey

Data Collector(s) Name: 
Oxford Policy Management Nigeria
Disclaimer: 
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Funding Name, Abbreviation, Role: 
Department for International Development
Primary Investigator Name, Affiliation: 
Oxford Policy Management Ltd
Questionnaires: 
See 'Scope' section.
Response Rates: 
See 'Sampling Procedure' section.
Sampling Procedure: 
The aim of the sampling design was to define a valid counterfactual (control group) from which comparisons could be made with the treatment group that participates in the Teacher Development Programme (TDP). The control group does not participate in the TDP in-service training but has background characteristics that are, on average, similar to those of the treatment group that does participate in TDP in-service training. The sampling design was based on a quasi-experimental 'constrained' randomisation approach. 'Constrained' is used because certain parameters of the impact evaluation were already fixed. For example, the Local Government Authorities (LGAs) where the TDP was to operate had already been selected by the TDP in agreement with the three states covered by the impact evaluation. In addition, pre-determined groups of schools fulfilling certain criteria (described below) constitute the sampling frame -- this is in contrast to a fully randomised design where one would expect the random drawing of groups (or clusters) of schools from a list of all state primary schools in the region under study. Randomisation was conducted only in allocating groups of schools to 'treatment' or 'control' status. ---------------------------------- Sampling frame construction ---------------------------------- The intended size of the sampling frame was 1,008 public primary schools eligible for the TDP (504 treatment and 504 control schools) in the three states. This would constitute the target population of eligible schools from which the sample of schools would be drawn for the survey. The sampling frame was constructed through the steps described below. Step 1: LGAs ---------------- In each of the three states, 14 LGAs where the TDP would operate had been predetermined by the TDP in agreement with each state: -Jigawa: 14 out of 27 LGAs. -Katsina: 14 out of 34 LGAs. -Zamfara: 14 out of 14 LGAs. Step 2: Primary schools ---------------------------- To be eligible for the TDP, a school must have one head teacher and at least three other teachers and at least eight grade 3 pupils. In each of the 14 LGAs in each state, two sets of 12 eligible primary schools were to be selected. Schools within each set were identified according to geographical proximity to facilitate any training and periodic meetings of teachers, and to create a peer network within the locality. The two sets of schools within each LGA were meant to be seleted to be broadly similar. State Education Boards (SUBEBs) were responsible for the selection of the schools and were provided guidelines for how to do this. For example, to take into account the location of schools (rural/urban), school size in terms of pupils enrolled and number of classrooms, condition of school infrastructure, and existence of a school-based management committee (SBMC). Step 3: Teachers --------------------- Before the sampling of schools, the Local Government Education Authority (LGEA) and head teacher from each school in the two sets (see 2. above), were required to identify three teachers in addition to the head teacher, who would potentially receive the TDP support and training. These teachers were identified using the following criteria: -They teach classes at early grade level (grades 1 to 3); and -They teach classes in any of the three subjecs of English, mathematics, and science. Step 4: Random assignment of treatment/control status ------------------------------------------------------------------- After receiving the lists of the sets of eligible schools and teachers from the TDP coordinators, the impact evaluation team randomly assigned one set of schools among every pair of sets in each LGA to TDP treatment, and the other set to control status. This resulted in 42 (14 LGAs x 3 states) sets consisting of 12 schools each, for a total of 504 schools to receive the TDP, and 42 (14 LGAs x 3 states) sets consisting of 12 schools each for a total of 504 schools that would not receive the TDP. The sample treatment and control schools were then selected from these two lists respectively. In the 504 schools that would receive the TDP, all head teachers and the teachers identified in step 3 (see above) would receive TDP support and training, while the head teachers and teachers in the schools on the list of 504 'non-TDP' schools would not. -------------------- Sampling stages -------------------- Stage 1: Selection of schools ----------------------------------- At the first stage, schools were selected using implicit stratification by state, LGA, and treatment/control status. That is, each of the school sets (see step 4 above), was considered a stratum. Four schools were randomly selected from each of these sets. This yielded an intended sample size of 56 (14 LGAs x 4 schools) treatment schools in each state and 56 (14 LGAs x 4 schools) control schools in each state, for an intended 168 treatment and 168 control schools across the three states, and a total intended sample size of 336 schools. Stage 2a: Selection of teachers -------------------------------------- At each selected school: -The head teacher and the three teachers identified during the construction of the sampling frame (see step 3 above) constituted the sample to be interviewed, with a total intended sample size of 336 head teacher interviews and 1,008 (336 schools x 3 teachers) teacher interviews. -The three selected teachers as well as the head teachers who teach any primary classes would also be observed while teaching (classroom observations), with a total intended sample size of up to 336 head teacher classroom observations and 1,008 (336 schools x 3 teachers) teacher classroom observations. -All head teachers and selected teachers would also be adminstered the Teacher Development Needs Assessment (TDNA) in English, mathematics, and science, for a total intended sample size of 1,344 (336 schools x 4 teachers) head teachers and teachers. Stage 2b: Selection of pupils ----------------------------------- At each selected school, eight pupils who started grade 3 in September 2014 and who were being taught English, mathematics, or science by at least one of the selected teachers during that term, would be randomly selected for the pupil learning assessment. The pupils were drawn from a sampling frame consisting of all eligible grade 3 pupils present on the day of the survey. Eligible pupils were those in grade 3 who were being taught by at least one of the selected teachers. The intended pupil sample size was 2,688 (336 schools x 8 pupils). --------------------------------------- Survey longitudinal/panel design --------------------------------------- The survey uses a panel design, that is, it aims to collect longitudinal data. This means the survey was implemented in the same sample of schools at baseline and endline, and that within these schools, it collected data from the same head teachers, teachers, and pupils over time. However, there was high head teacher, teacher, and pupil sample attrition between the baseline and endline surveys (see 'Sample attrition' section below). ------------------ Replacements ------------------ 1. School replacements ---------------------------- During the baseline fieldwork, five selected schools were found to be ineligible and one selected school could not be visited because of security concerns, these six schools were not replaced. During the endline survey there were no school replacements because the survey uses a panel design. 2. Teacher replacements ------------------------------ There were three different cases of unavailability of selected teachers during the baseline fieldwork: -A selected teacher was not present on the day of the survey due to short term absence, and data collectors attempted to re-visit the school at a later date; -A selected school was found to be very small with fewer than four eligible teachers (including the head teacher), all teachers were interviewed when possible; and -A selected teacher was on long leave, had been transferred, had passed away, or was unidentified, and data collectors would ask the head teacher to name a replacement teacher as per the teacher eligibility criteria (see step 3 above). During the endline fieldwork there were no teacher replacements because the survey uses a panel design. 3. Pupil replacements -------------------------- If one of the eight sampled pupils turned out to not be available, that pupil was randomly replaced with another pupil from the list of eligible pupils. If a selected school had eight or fewer grade 3 pupils present on the day of the baseline survey, all those eligible were selected for the pupil learning assessment. During the endline fieldwork there were no pupil replacements because the survey uses a panel design. ---------------- Sample sizes ---------------- The actual sample sizes at baseline were: -330 schools (intended 336); -330 head teacher interviews (intended 336); -2,575 grade 3 pupils (intended 2,688); -908 teacher interviews (intended 1,008); -1,070 classroom observations (intended up to 1,344 depending on how many head teachers teach any primary classes); and -1,158 Teacher Development Needs Assessments (intended 1,344). The actual sample sizes at endline were: -330 schools (intended 330 in actual BL sample); -329 non-panel (new head teachers were interviewed) and 134 panel head teacher interviews (intended 330 in actual BL sample); -1,566 panel grade 6 pupils (intended 2,575 in actual BL sample); -447 panel teacher interviews (intended 908 in actual BL sample); -460 panel and 574 non-panel classroom observations (intended 1,070 in actual BL sample); and -556 panel and 774 non-panel Teacher Development Needs Assessments (intended 1,158 in actual BL sample). ------------------- Sample attrition ------------------- There was substantial head teacher, teacher, and pupil sample attrition between baseline and endline. The main reasons for head teacher and teacher attrition were transfer to another school and for pupil attrition dropping out of school. Sample attrition rates since baseline: -Schools 0%; -Head teacher interviews 0.3% non-panel sample and 59% panel sample; -Pupils 39% (in grade 3 at baseline and in grade 6 at endline); -Teacher interviews 51%; -Classroom observations 57%; and -Teacher Development Needs Assessments 52%. NOTE: Head teachers that were new at a sample school were interviewed at endline yielding a sample of 329 head teachers (some non-panel and some panel), and a panel sample of 134 head teachers (only head teachers who were head teachers at the same school at baseline and endline). To reduce teacher and pupil attrition due to temporary absences on the day of the survey, two main steps were taken: -School revisits: state coordinators and field teams revisited schools in order to conduct missing interviews for teachers and pupils that were unavailable on the original day of visit; and -Calling pupils from home: for the pupils that lived close to the school but were absent from school on the day of the visit, data collectors worked with the head teachers and teachers to ask pupils to come to the school to take the pupil learning assessment if able to do so. 1. Overall pupil and teacher sample attrition ----------------------------------------------------- For the purposes of this impact evaluation, attrition for two units of analysis were of particular relevance: pupils and teachers. To understand attrition dynamics for those units of analysis in more detail, this evaluation assessed which characteristics of pupils and teachers at baseline help explain whether individuals dropped out of the sample or not since baseline. This means that baseline data were used to compare the group of individuals for which data were also collected at endline (the non-attriters) to the group of individuals for which data were not collected at endline (the attriters). The purpose of this analysis is to understand whether estimates of characteristics of pupils or teachers at endline can generally be thought of as being produced on a sample that is comparable to the original group of individuals sampled at baseline. The attrition analysis and statistics are produced taking into account the full sampling structure of the data (weights, clustering, stratification). Test statistics are also corrected fur multiple hypotheses testing given that these tables are comparing many different indicators at the same time. The baseline variables used to examine sample attrition are of three types: -Pupil and teacher background characteristics, for example, gender; -Pupil and teacher outcomes that the TDP seeks to influence, for example, pupil test scores and teacher subject knowledge scores; and -School characteristics that are likely to be correlated with pupil and teacher behaviour and outcomes, for example, school size. The results for pupils indicate that pupils who drop out of the sample are mainly older and poorer than pupils who stay in the sample. The results for teachers indicate that teachers who dropped out of the sample since baseline were significantly more likely to have Nigeria Certificate in Education (NCE) qualification or higher, and performed significantly better on the Teacher Development Needs Assessment (TDNA) than teachers who remained in the sample. These results indicate that the evidence for selective attrition is weak among pupils but slightly stronger for teachers. This means that taking overall attrition into account, the sample of pupils is still comparable to the original sample. While estimates generated using the teacher sample needs to be interpreted taking teacher attrition into account, and cannot necessarily be assumed to be representative of the target population of teachers at baseline. 2. Differential sample attrition for treatment and control pupils and teachers -------------------------------------------------------------------------------------------- This impact evaluation also examined whether there is differential attrition, which refers to situations where the background characteristics of individuals who drop out between survey rounds differ significantly between the treatment and control groups. This would mean that, after the sample attrition, the two groups are not comparable anymore and that the original assumption of the control group being an appropriate counterfactual to the treatment group for impact identification purposes is no longer correct. The analysis examining key characteristics and outcomes for non-attrited pupils and teachers at baseline across the TDP treatment and control groups indicates that differential attrition is generally not problematic. The estimates suggest that there are no significant differences between the control and treatment pupils who remained in the sample. For teachers there is also very limited indication of differential attrition. ------------------------- Representativeness ------------------------- The sample for this impact evaluation was selected to represent the eligible schools in the TDP clusters and corresponding control clusters for the three states. Therefore the sample is not designed to be representative at the state level and should not be treated as such. However, the survey does include schools with a wide range of characteristics, and statistics are likely to be broadly similar to those for the states as a whole.
Series Information: 
The quantitative endline survey for the impact evaluation of Phase 1 (In-Service Training) of the Teacher Development Programme (TDP) is the second of two rounds of data collection. The first round (baseline) was conducted in October 2014 - January 2015 and the second round (endline) was conducted in October - November 2017.
Subtitle: 
Endline Survey
Unit of Analysis: 
The primary sampling units (PSUs) of the survey are TDP-eligible public primary schools in the three states Jigawa, Katsina, and Zamfara. The secondary sampling units are teachers (selected prior to the PSUs and teaching grades 1-3 in any of the three subjects: English, mathematics, or science), pupils (in grade 3 at baseline and taught English, mathematics, or science by at least one of the sample teachers, and in grade 6 at endline) and lessons taught by the selected teachers (not sampled). See the 'Sampling' section for more details.
Universe: 
The target populations are the schools eligible for the Teacher Development Programme in treatment and control groups in the three states Jigawa, Katsina and Zamfara, and the eligible teachers and pupils within these schools. See the 'Sampling' section for details on the elligibility of teachers and pupils.
Version Description: 
Version 2.1: Edited, anonymous dataset for public distribution.
Version Notes: 
Version 2.1 consists of six edited and anonymised datasets (at school, teacher interview, pupil, lesson, teacher roster and classroom level) with the responses to a small number of questions removed (see 'TDP IE List of variables excluded from endline survey datasets' provided under Technical Documents); these were removed for confidentiality purposes or because they were not needed for the analysis. Some of the datasets also contain selected constructed indicators. These constructed indicators are included either to (i) save data users time as they require complex reshaping and extraction of data from multiple sources (but they could be generated by data users if preferred), or (ii) to provide indicators that cannot be generated from the provided datasets due to the need to keep item responses from the pupil learning assessment confidential.
Weighting: 
***NOTE: THIS SECTION SHOULD BE READ IN CONJUNCTION WITH SECTION 3 OF THE 'Teacher Development Programme - Endline Evaluation (Volume II)' AND THE TECHNICAL DOCUMENT 'Quantitative Analysis: Samples, Weights and Survey Settings' BOTH OF WHICH ARE PROVIDED UNDER RELATED MATERIALS*** In order to make inferences from the TDP baseline survey data, appropriate survey weights were calculated for each sample. The weights are equal to the inverse of the overall sampling probabilities taking into account each stage of selection. Given that the endline survey was conducted as a panel survey in 2017 and as a result all of the responding sample schools, pupils and teachers from the TDP Baseline Survey were included in the endline survey, the basic probabilities and weights are the same. However, it is necessary to adjust each set of baseline weights for the sample pupils, teachers and head teachers based on the attrition for the individual tests, interviews and observations in the endline survey. A comprehensive and detailed explanation of the construction of the baseline and endline survey weights can be found in Section 3 of 'Teacher Development Programme - Endline Evaluation (Volume II)' provided under Related Materials. Note that at endline, the sample of schools as well as the sample of pupils and teachers interviewed/observed/tested represents both the cross-sectional and panel samples. That is because all schools, pupils and teachers included in our sample at endline are also included in the baseline sample (the sample of schools is exactly the same at baseline and endline, while the sample of pupils and teachers at endline is a subset of that sample at baseline due to attrition). For head teachers however, there are two different samples: 1) a cross-sectional sample that includes all head teachers who were interviewed/observed/tested at endline (this includes head teachers who were also inlcuded in the baseline sample as well as new head teachers who became head teachers of the school after the baseline survey) and 2) a panel sample that only includes those head teachers who were interviewed/observed/tested at both baseline and endline. For the endline survey 12 sets of weights were calculated for the different samples. These are: 1. Endline school weight (for panel AND cross-sectional analysis of schools at endline; this is the same as the baseline school weight) 2. Endline head teacher interview weight (for cross-sectional analysis of all HTs at endline) 3. Panel head teacher interview weight (for panel analysis of HTs) 4. Panel head teacher motivation weight (for panel analysis of the motivation module for HTs) 5. Endline (panel) teacher interview weight (for panel AND cross-sectional analysis of teachers at endline) 6. Endline (panel) P6 pupil test weight (for panel AND cross-sectional analysis of pupils at endline) 7. Endline (panel) teacher observation weight - all lessons (for panel AND cross-sectional analysis of teacher observations at endline) 8. Endline (panel) teacher observation weight - excluding lessons that lasted < 9 minutes (for panel AND cross-sectional analysis of teacher observations at endline) 9. Panel head teacher observation weight - all lessons (for panel analysis of HT observations) 10. Panel head teacher observation weight - excluding lessons that lasted < 9 minutes (for panel analysis of HT observations) 11. Endline (panel) teacher tests (TDNA) weight (for panel AND cross-sectional analysis of teachers at endline) 12. Panel head teacher tests (TDNA) weight (for panel analysis of HTs) The public use datasets include all these different sets of weights as well as the variables that account for stratification, clustered sampling and finite population corrections. Additionally, the datasets contain constructed variables that help identify the correct samples for each set of weights. For example, the panel head teacher interview weight must only be applied to the sample of panel head teachers. In the school level dataset, the dummy variable 'n_panel_int' identifies the sample of panel head teachers: it takes the value 1 if the head teacher was interviewed at baseline and endline (i.e. is in the panel sample) and 0 otherwise. The technical document 'Quantitative Analysis: Samples, Weights and Survey Settings' provided under Related Materials lists the weight variable, sample identification variable and survey setting that must be used for each level/type of analysis.

No Visualizations Available.

Oxford Policy Management. Nigeria Teacher Development Programme In-Service Training Impact Evaluation Endline Survey (TDPIE-EL) 2017, Version 2.1 of the public use dataset (August 2018).Ref. NGA_2017_TDPITCIE-EL_v01_M. Downloaded from [url] on [date]

The Teacher Development Programme (TDP) is a six-year (2013-2019) programme funded by the UK Department for International Development (DFID), with a total budget of £34 million. It seeks to improve the quality of teaching in primary schools, junior secondary schools, and colleges of education at the state level in northern Nigeria. It works through in-service training for primary teachers, reform of pre-service teacher education, and strengthening evidence-based research on teaching. This evaluation focuses on the first of these three outputs: in-service training and support for primary teachers in the three core curriculum subjects of English, mathematics, and science. The programme initially operated in three states, Jigawa, Katsina, and Zamfara, and was later extended to Kaduna and Kano. This survey covers a group of schools that were randomly assigned to receive the TDP intervention in Jigawa, Katsina, and Zamfara, and a control group of schools in the same states that did not receive the TDP intervention. The impact evaluation has three main purposes: -Formative - to help inform the implementation of TDP in Phase 1 of in-service activities and the design and implementation in its Phase 2; -Summative - to help inform TDP, DFID Nigeria, and other education stakeholders if TDP's in-service teacher training activities have led to improvements in teacher effectiveness and pupil learning levels in English, mathematics, and science and technology; and -Learning - to assess from TDP what might work for improving teacher effectiveness in Nigeria and elsewhere. The impact evaluation is a theory-based, mixed-methods design. The endline research focuses on the efficiency, effectiveness, impact, and sustainability of the TDP (programme relevance was assessed through the baseline survey). The overarching evaluation questions for the endline research are: -Impact: Has TDP caused changes in pupil learning in English, mathematics, and science in TDP schools? (quantitative) -Effectiveness: Has TDP led to changes in teacher effectiveness? (qualitative and quantitative) -Efficiency: Were TDP results achieved on time and to plan? How does TDP's organisational set-up facilitate delivery? (qualitative and quantitative) -Sustainability: Are TDP's impacts on teacher effectiveness sustainable without further DFID support? (qualitative) At the core of this impact evaluation is a constrained randomised design, with half of the sample schools assigned to receive the TDP intervention, while the other half was assigned to the control group. The main source of quantitative data is the sample panel survey of 330 schools in Jigawa, Katsina, and Zamfara conducted in 2014 and 2017. At each sample school, the head teacher and a sample of teachers and pupils were interviewed and tested on their subject knowledge in English, mathematics, and science; lesson observations were conducted; a teacher roster was compiled; and classroom attendance by teachers and pupils was measured. The baseline and endline surveys were timed to allow sufficient time for TDP impact on pupil learning outcomes to be feasible, while ensuring that a panel of pupils could be sampled at baseline and endline within the same school, and producing results in time for them to be of value to the final stages of the TDP. The endline quantitative survey fieldwork was carried out by OPM Nigeria and Education Data, Research and Evaluation in Nigeria (EDOREN). The data from the endline survey available in the World Bank Microdata Catalog are from the TDP IE quantitative endline survey conducted in 2017. For the qualitative research findings and methods see the final endline report under Related Materials.

FieldValue
Modified Date
2019-03-20
Release Date
Identifier
a9a18ffb-62cd-4801-9f35-9de9bc365ffb
License
License Not Specified
Contact Email
Public Access Level
Public
Rating: 
0
No votes yet
Reference ID: 
NGA_2017_TDPITCIE-EL_v01_M
Acronym: 
TDPITCIE-EL 2017
Type: 
Languages Supported: 
Access Authority Name, Affiliation, Email: 
Oxford Policy Management Ltd, [email protected], http://www.opml.co.uk/
Disclaimer: 
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Response Rates: 
See 'Sampling Procedure' section.
Weighting: 
***NOTE: THIS SECTION SHOULD BE READ IN CONJUNCTION WITH SECTION 3 OF THE 'Teacher Development Programme - Endline Evaluation (Volume II)' AND THE TECHNICAL DOCUMENT 'Quantitative Analysis: Samples, Weights and Survey Settings' BOTH OF WHICH ARE PROVIDED UNDER RELATED MATERIALS*** In order to make inferences from the TDP baseline survey data, appropriate survey weights were calculated for each sample. The weights are equal to the inverse of the overall sampling probabilities taking into account each stage of selection. Given that the endline survey was conducted as a panel survey in 2017 and as a result all of the responding sample schools, pupils and teachers from the TDP Baseline Survey were included in the endline survey, the basic probabilities and weights are the same. However, it is necessary to adjust each set of baseline weights for the sample pupils, teachers and head teachers based on the attrition for the individual tests, interviews and observations in the endline survey. A comprehensive and detailed explanation of the construction of the baseline and endline survey weights can be found in Section 3 of 'Teacher Development Programme - Endline Evaluation (Volume II)' provided under Related Materials. Note that at endline, the sample of schools as well as the sample of pupils and teachers interviewed/observed/tested represents both the cross-sectional and panel samples. That is because all schools, pupils and teachers included in our sample at endline are also included in the baseline sample (the sample of schools is exactly the same at baseline and endline, while the sample of pupils and teachers at endline is a subset of that sample at baseline due to attrition). For head teachers however, there are two different samples: 1) a cross-sectional sample that includes all head teachers who were interviewed/observed/tested at endline (this includes head teachers who were also inlcuded in the baseline sample as well as new head teachers who became head teachers of the school after the baseline survey) and 2) a panel sample that only includes those head teachers who were interviewed/observed/tested at both baseline and endline. For the endline survey 12 sets of weights were calculated for the different samples. These are: 1. Endline school weight (for panel AND cross-sectional analysis of schools at endline; this is the same as the baseline school weight) 2. Endline head teacher interview weight (for cross-sectional analysis of all HTs at endline) 3. Panel head teacher interview weight (for panel analysis of HTs) 4. Panel head teacher motivation weight (for panel analysis of the motivation module for HTs) 5. Endline (panel) teacher interview weight (for panel AND cross-sectional analysis of teachers at endline) 6. Endline (panel) P6 pupil test weight (for panel AND cross-sectional analysis of pupils at endline) 7. Endline (panel) teacher observation weight - all lessons (for panel AND cross-sectional analysis of teacher observations at endline) 8. Endline (panel) teacher observation weight - excluding lessons that lasted < 9 minutes (for panel AND cross-sectional analysis of teacher observations at endline) 9. Panel head teacher observation weight - all lessons (for panel analysis of HT observations) 10. Panel head teacher observation weight - excluding lessons that lasted < 9 minutes (for panel analysis of HT observations) 11. Endline (panel) teacher tests (TDNA) weight (for panel AND cross-sectional analysis of teachers at endline) 12. Panel head teacher tests (TDNA) weight (for panel analysis of HTs) The public use datasets include all these different sets of weights as well as the variables that account for stratification, clustered sampling and finite population corrections. Additionally, the datasets contain constructed variables that help identify the correct samples for each set of weights. For example, the panel head teacher interview weight must only be applied to the sample of panel head teachers. In the school level dataset, the dummy variable 'n_panel_int' identifies the sample of panel head teachers: it takes the value 1 if the head teacher was interviewed at baseline and endline (i.e. is in the panel sample) and 0 otherwise. The technical document 'Quantitative Analysis: Samples, Weights and Survey Settings' provided under Related Materials lists the weight variable, sample identification variable and survey setting that must be used for each level/type of analysis.
Data Collector(s) Name: 
Oxford Policy Management Nigeria
Primary Investigator Name, Affiliation: 
Oxford Policy Management Ltd
Version Description: 
Version 2.1: Edited, anonymous dataset for public distribution.
Subtitle: 
Endline Survey
Unit of Analysis: 
The primary sampling units (PSUs) of the survey are TDP-eligible public primary schools in the three states Jigawa, Katsina, and Zamfara. The secondary sampling units are teachers (selected prior to the PSUs and teaching grades 1-3 in any of the three subjects: English, mathematics, or science), pupils (in grade 3 at baseline and taught English, mathematics, or science by at least one of the sample teachers, and in grade 6 at endline) and lessons taught by the selected teachers (not sampled). See the 'Sampling' section for more details.
Universe: 
The target populations are the schools eligible for the Teacher Development Programme in treatment and control groups in the three states Jigawa, Katsina and Zamfara, and the eligible teachers and pupils within these schools. See the 'Sampling' section for details on the elligibility of teachers and pupils.
Geographical Coverage: 
Data Classification of a Dataset: 
Version Production Date: 
Sunday, August 12, 2018
Series Information: 
The quantitative endline survey for the impact evaluation of Phase 1 (In-Service Training) of the Teacher Development Programme (TDP) is the second of two rounds of data collection. The first round (baseline) was conducted in October 2014 - January 2015 and the second round (endline) was conducted in October - November 2017.
Sampling Procedure: 
The aim of the sampling design was to define a valid counterfactual (control group) from which comparisons could be made with the treatment group that participates in the Teacher Development Programme (TDP). The control group does not participate in the TDP in-service training but has background characteristics that are, on average, similar to those of the treatment group that does participate in TDP in-service training. The sampling design was based on a quasi-experimental 'constrained' randomisation approach. 'Constrained' is used because certain parameters of the impact evaluation were already fixed. For example, the Local Government Authorities (LGAs) where the TDP was to operate had already been selected by the TDP in agreement with the three states covered by the impact evaluation. In addition, pre-determined groups of schools fulfilling certain criteria (described below) constitute the sampling frame -- this is in contrast to a fully randomised design where one would expect the random drawing of groups (or clusters) of schools from a list of all state primary schools in the region under study. Randomisation was conducted only in allocating groups of schools to 'treatment' or 'control' status. ---------------------------------- Sampling frame construction ---------------------------------- The intended size of the sampling frame was 1,008 public primary schools eligible for the TDP (504 treatment and 504 control schools) in the three states. This would constitute the target population of eligible schools from which the sample of schools would be drawn for the survey. The sampling frame was constructed through the steps described below. Step 1: LGAs ---------------- In each of the three states, 14 LGAs where the TDP would operate had been predetermined by the TDP in agreement with each state: -Jigawa: 14 out of 27 LGAs. -Katsina: 14 out of 34 LGAs. -Zamfara: 14 out of 14 LGAs. Step 2: Primary schools ---------------------------- To be eligible for the TDP, a school must have one head teacher and at least three other teachers and at least eight grade 3 pupils. In each of the 14 LGAs in each state, two sets of 12 eligible primary schools were to be selected. Schools within each set were identified according to geographical proximity to facilitate any training and periodic meetings of teachers, and to create a peer network within the locality. The two sets of schools within each LGA were meant to be seleted to be broadly similar. State Education Boards (SUBEBs) were responsible for the selection of the schools and were provided guidelines for how to do this. For example, to take into account the location of schools (rural/urban), school size in terms of pupils enrolled and number of classrooms, condition of school infrastructure, and existence of a school-based management committee (SBMC). Step 3: Teachers --------------------- Before the sampling of schools, the Local Government Education Authority (LGEA) and head teacher from each school in the two sets (see 2. above), were required to identify three teachers in addition to the head teacher, who would potentially receive the TDP support and training. These teachers were identified using the following criteria: -They teach classes at early grade level (grades 1 to 3); and -They teach classes in any of the three subjecs of English, mathematics, and science. Step 4: Random assignment of treatment/control status ------------------------------------------------------------------- After receiving the lists of the sets of eligible schools and teachers from the TDP coordinators, the impact evaluation team randomly assigned one set of schools among every pair of sets in each LGA to TDP treatment, and the other set to control status. This resulted in 42 (14 LGAs x 3 states) sets consisting of 12 schools each, for a total of 504 schools to receive the TDP, and 42 (14 LGAs x 3 states) sets consisting of 12 schools each for a total of 504 schools that would not receive the TDP. The sample treatment and control schools were then selected from these two lists respectively. In the 504 schools that would receive the TDP, all head teachers and the teachers identified in step 3 (see above) would receive TDP support and training, while the head teachers and teachers in the schools on the list of 504 'non-TDP' schools would not. -------------------- Sampling stages -------------------- Stage 1: Selection of schools ----------------------------------- At the first stage, schools were selected using implicit stratification by state, LGA, and treatment/control status. That is, each of the school sets (see step 4 above), was considered a stratum. Four schools were randomly selected from each of these sets. This yielded an intended sample size of 56 (14 LGAs x 4 schools) treatment schools in each state and 56 (14 LGAs x 4 schools) control schools in each state, for an intended 168 treatment and 168 control schools across the three states, and a total intended sample size of 336 schools. Stage 2a: Selection of teachers -------------------------------------- At each selected school: -The head teacher and the three teachers identified during the construction of the sampling frame (see step 3 above) constituted the sample to be interviewed, with a total intended sample size of 336 head teacher interviews and 1,008 (336 schools x 3 teachers) teacher interviews. -The three selected teachers as well as the head teachers who teach any primary classes would also be observed while teaching (classroom observations), with a total intended sample size of up to 336 head teacher classroom observations and 1,008 (336 schools x 3 teachers) teacher classroom observations. -All head teachers and selected teachers would also be adminstered the Teacher Development Needs Assessment (TDNA) in English, mathematics, and science, for a total intended sample size of 1,344 (336 schools x 4 teachers) head teachers and teachers. Stage 2b: Selection of pupils ----------------------------------- At each selected school, eight pupils who started grade 3 in September 2014 and who were being taught English, mathematics, or science by at least one of the selected teachers during that term, would be randomly selected for the pupil learning assessment. The pupils were drawn from a sampling frame consisting of all eligible grade 3 pupils present on the day of the survey. Eligible pupils were those in grade 3 who were being taught by at least one of the selected teachers. The intended pupil sample size was 2,688 (336 schools x 8 pupils). --------------------------------------- Survey longitudinal/panel design --------------------------------------- The survey uses a panel design, that is, it aims to collect longitudinal data. This means the survey was implemented in the same sample of schools at baseline and endline, and that within these schools, it collected data from the same head teachers, teachers, and pupils over time. However, there was high head teacher, teacher, and pupil sample attrition between the baseline and endline surveys (see 'Sample attrition' section below). ------------------ Replacements ------------------ 1. School replacements ---------------------------- During the baseline fieldwork, five selected schools were found to be ineligible and one selected school could not be visited because of security concerns, these six schools were not replaced. During the endline survey there were no school replacements because the survey uses a panel design. 2. Teacher replacements ------------------------------ There were three different cases of unavailability of selected teachers during the baseline fieldwork: -A selected teacher was not present on the day of the survey due to short term absence, and data collectors attempted to re-visit the school at a later date; -A selected school was found to be very small with fewer than four eligible teachers (including the head teacher), all teachers were interviewed when possible; and -A selected teacher was on long leave, had been transferred, had passed away, or was unidentified, and data collectors would ask the head teacher to name a replacement teacher as per the teacher eligibility criteria (see step 3 above). During the endline fieldwork there were no teacher replacements because the survey uses a panel design. 3. Pupil replacements -------------------------- If one of the eight sampled pupils turned out to not be available, that pupil was randomly replaced with another pupil from the list of eligible pupils. If a selected school had eight or fewer grade 3 pupils present on the day of the baseline survey, all those eligible were selected for the pupil learning assessment. During the endline fieldwork there were no pupil replacements because the survey uses a panel design. ---------------- Sample sizes ---------------- The actual sample sizes at baseline were: -330 schools (intended 336); -330 head teacher interviews (intended 336); -2,575 grade 3 pupils (intended 2,688); -908 teacher interviews (intended 1,008); -1,070 classroom observations (intended up to 1,344 depending on how many head teachers teach any primary classes); and -1,158 Teacher Development Needs Assessments (intended 1,344). The actual sample sizes at endline were: -330 schools (intended 330 in actual BL sample); -329 non-panel (new head teachers were interviewed) and 134 panel head teacher interviews (intended 330 in actual BL sample); -1,566 panel grade 6 pupils (intended 2,575 in actual BL sample); -447 panel teacher interviews (intended 908 in actual BL sample); -460 panel and 574 non-panel classroom observations (intended 1,070 in actual BL sample); and -556 panel and 774 non-panel Teacher Development Needs Assessments (intended 1,158 in actual BL sample). ------------------- Sample attrition ------------------- There was substantial head teacher, teacher, and pupil sample attrition between baseline and endline. The main reasons for head teacher and teacher attrition were transfer to another school and for pupil attrition dropping out of school. Sample attrition rates since baseline: -Schools 0%; -Head teacher interviews 0.3% non-panel sample and 59% panel sample; -Pupils 39% (in grade 3 at baseline and in grade 6 at endline); -Teacher interviews 51%; -Classroom observations 57%; and -Teacher Development Needs Assessments 52%. NOTE: Head teachers that were new at a sample school were interviewed at endline yielding a sample of 329 head teachers (some non-panel and some panel), and a panel sample of 134 head teachers (only head teachers who were head teachers at the same school at baseline and endline). To reduce teacher and pupil attrition due to temporary absences on the day of the survey, two main steps were taken: -School revisits: state coordinators and field teams revisited schools in order to conduct missing interviews for teachers and pupils that were unavailable on the original day of visit; and -Calling pupils from home: for the pupils that lived close to the school but were absent from school on the day of the visit, data collectors worked with the head teachers and teachers to ask pupils to come to the school to take the pupil learning assessment if able to do so. 1. Overall pupil and teacher sample attrition ----------------------------------------------------- For the purposes of this impact evaluation, attrition for two units of analysis were of particular relevance: pupils and teachers. To understand attrition dynamics for those units of analysis in more detail, this evaluation assessed which characteristics of pupils and teachers at baseline help explain whether individuals dropped out of the sample or not since baseline. This means that baseline data were used to compare the group of individuals for which data were also collected at endline (the non-attriters) to the group of individuals for which data were not collected at endline (the attriters). The purpose of this analysis is to understand whether estimates of characteristics of pupils or teachers at endline can generally be thought of as being produced on a sample that is comparable to the original group of individuals sampled at baseline. The attrition analysis and statistics are produced taking into account the full sampling structure of the data (weights, clustering, stratification). Test statistics are also corrected fur multiple hypotheses testing given that these tables are comparing many different indicators at the same time. The baseline variables used to examine sample attrition are of three types: -Pupil and teacher background characteristics, for example, gender; -Pupil and teacher outcomes that the TDP seeks to influence, for example, pupil test scores and teacher subject knowledge scores; and -School characteristics that are likely to be correlated with pupil and teacher behaviour and outcomes, for example, school size. The results for pupils indicate that pupils who drop out of the sample are mainly older and poorer than pupils who stay in the sample. The results for teachers indicate that teachers who dropped out of the sample since baseline were significantly more likely to have Nigeria Certificate in Education (NCE) qualification or higher, and performed significantly better on the Teacher Development Needs Assessment (TDNA) than teachers who remained in the sample. These results indicate that the evidence for selective attrition is weak among pupils but slightly stronger for teachers. This means that taking overall attrition into account, the sample of pupils is still comparable to the original sample. While estimates generated using the teacher sample needs to be interpreted taking teacher attrition into account, and cannot necessarily be assumed to be representative of the target population of teachers at baseline. 2. Differential sample attrition for treatment and control pupils and teachers -------------------------------------------------------------------------------------------- This impact evaluation also examined whether there is differential attrition, which refers to situations where the background characteristics of individuals who drop out between survey rounds differ significantly between the treatment and control groups. This would mean that, after the sample attrition, the two groups are not comparable anymore and that the original assumption of the control group being an appropriate counterfactual to the treatment group for impact identification purposes is no longer correct. The analysis examining key characteristics and outcomes for non-attrited pupils and teachers at baseline across the TDP treatment and control groups indicates that differential attrition is generally not problematic. The estimates suggest that there are no significant differences between the control and treatment pupils who remained in the sample. For teachers there is also very limited indication of differential attrition. ------------------------- Representativeness ------------------------- The sample for this impact evaluation was selected to represent the eligible schools in the TDP clusters and corresponding control clusters for the three states. Therefore the sample is not designed to be representative at the state level and should not be treated as such. However, the survey does include schools with a wide range of characteristics, and statistics are likely to be broadly similar to those for the states as a whole.
Release Date: 
Wednesday, February 27, 2019
Last Updated Date: 
Wednesday, February 27, 2019
Questionnaires: 
See 'Scope' section.
Harvest Source: 
Harvest Source ID: 
10316
Version Notes: 
Version 2.1 consists of six edited and anonymised datasets (at school, teacher interview, pupil, lesson, teacher roster and classroom level) with the responses to a small number of questions removed (see 'TDP IE List of variables excluded from endline survey datasets' provided under Technical Documents); these were removed for confidentiality purposes or because they were not needed for the analysis. Some of the datasets also contain selected constructed indicators. These constructed indicators are included either to (i) save data users time as they require complex reshaping and extraction of data from multiple sources (but they could be generated by data users if preferred), or (ii) to provide indicators that cannot be generated from the provided datasets due to the need to keep item responses from the pupil learning assessment confidential.
Citation Text: 
Oxford Policy Management. Nigeria Teacher Development Programme In-Service Training Impact Evaluation Endline Survey (TDPIE-EL) 2017, Version 2.1 of the public use dataset (August 2018).Ref. NGA_2017_TDPITCIE-EL_v01_M. Downloaded from [url] on [date]
Modified date: 
17954
Study Type: 
School survey
Primary Dataset: 
Yes

Data Access and Licensing

This dataset is classified as Public under the Access to Information Classification Policy. Users inside and outside the Bank can access this dataset.

This dataset is made available under the World Bank Microdata Research License

Share Metadata

The information on this page (the dataset metadata) is also available in these formats.

PRINT EMAIL JSON RDF