Announcement of Third Party Monitoring reforms welcomed, but ILO statement requires correction

In August 2018 the Corruption and Human Rights Initiative (CHRI) published an evaluation of the methodology, data-analysis and ethical protocol employed by the International Labour Organization (ILO) to monitor incidences of forced labour during Uzbekistan’s 2017 annual cotton harvest. The evaluation was conducted at the invitation of the ILO’s Chief Technical Adviser. It is authored by Professor Kristian Lasslett and doctoral researcher, Vanessa Gstrein.

Six weeks after the report was released, the ILO issued a formal response. It features a series of welcomed proposals that will potentially help reform the third party monitoring methodology and protocol in Uzbekistan.

However, certain assertions made in the ILO statement potentially misconstrue the methodology and process observed by Lasslett and Gstrein. In the interests of accuracy several points of clarification need to be made.

The statement’s opening line asserts: “The International Labour Office has become aware of a report circulating on social media, prepared by Kristian Lasslett and Vanessa Gstrein”.

The report, in fact, was produced transparently and in consultation with the ILO from its initiation. For example:

1) On 19 March 2018 Professor Lasslett was invited by the ILO’s Chief Technical Adviser to analyse and provide feedback on, the methodology/ethical protocol used by its Third Party Monitoring unit in Uzbekistan.

2) On 2 May 2018 Professor Lasslett emailed the ILO’s Chief Technical Adviser and several senior ILO colleagues to relay and discuss the evaluation’s proposed methodology.

3) On 2 May 2018 the ILO’s Chief Technical Adviser responded positively to the proposal and advised Professor Lasslett on how to access the appropriate sources.

4) On 19 June 2018 and 17 July 2018, questions emerging from the evaluation results were sent to the ILO for comment. No response was received.

5) On 2 August 2018 an advanced copy of the report was emailed to the ILO.

The recent ILO public statement also contains an assertion that does not accurately represent the report’s methodology.

“Lasslett and Gstrein claim that their analysis is based on a monitoring manual used by the ILO. Actually, what has been reviewed is the report of the monitoring of the 2017 harvest, together with an outdated document from 2015 from the internet. The ILO has a comprehensive and well documented monitoring methodology which includes standards, sampling formulas, data handling policies, ethical safeguards, templates, forms and instructions. These are documents which are intended for the use of the persons carrying out the monitoring under contract with the ILO”.

Lasslett and Gstrein did not base their analysis on an outdated monitoring manual used by the ILO. The analysis was based on international benchmarks for conducting sensitive research with vulnerable participants, distilled from the scientific literature. The report also provides an abridged summary of certain guiding principles issued by the ILO, which remain of enduring importance in a context such as Uzbekistan.

Throughout the evaluation every effort was made by the report authors to access relevant source documents from the ILO that would permit a fair and thorough evaluation.

This is demonstrated by an email dated 2 May 2018 addressed to senior ILO officers by Professor Lasslett. It summarises the proposed methodology, and requests guidance on appropriate source documents: “We naturally have access to the ILO monitoring report published earlier this year, but if there are any other documents you believe ought to be considered please let me know”.

A welcoming and helpful reply was received from the ILO on the same day written by its Chief Technical Adviser who heads the Third Party Monitoring unit in Uzbekistan. It makes clear, other than one document available via a third party website, “everything else should already be in the documents available from the ILO website but please do not hesitate to ask if you have any questions”.

The report authors also sent the ILO Third Party Monitoring unit, and its Fundamental Principles and Rights Branch, a series of questions (see below) that sort clarification on a range of issues emerging from the draft evaluation.  The initial email was sent on 19 June 2018. A reminder was sent on 17 July 2018. No response was received.

CHRI and the report authors support serious reform of the monitoring process and welcome the ILO’s announcement in response to the Lasslett and Gstrein report. We are supportive of any further opportunity the ILO might offer to be a critical friend in this process.

Appendix A – Questions sent to the ILO on 19 June 2018 and 17 July 2018

Questions for clarification on TMP Report 2018

General

1. Because on-site interviews and phone surveys involved human subjects, some of who would be in categories deemed vulnerable, was the research protocol subject to an ethical approval process before being implemented? If so, who sat on the ethics review panel? Can a copy of the code of ethics used be provided?

Field interviews

2. How were the interpreters recruited and monitored? Were audits or spot checks carried out to ensure there was no editing or softening of remarks? What instructions and/or training was provided to interpreters to ensure they performed their tasks in a neutral manner?
3. In addition to the questionnaire introduction provided in Annex 5, are cotton pickers provided with any further information that sets out the purpose of the interview and how their data will be stored, analysed and applied?
4. Is a breakdown of the interview sample available, including disaggregated data on participants based on roles, gender, location, etc?
5. How many of the 3000 interviews were conducted with cotton pickers?
6. On average how long did interviews with cotton-pickers last for?
7. Were cotton-pickers compensated for participating in the interview?
8. Is there a consolidated summary of the full findings, including frequencies for each question, that were produced on the basis of analysing the 3000 interview result sheets?
9. Was any comparative analysis undertaken of the responses from the World Bank project areas versus non-World Bank project sites?
10. When the ILO TPM team ‘was made aware of individual situations that could have consisted of a breach of law or contractual arrangements (including from meetings, various interviews, independent NGOs or the media), when appropriate, it transmitted these cases to the FTUU and Ministry of Employment feedback mechanisms for follow-up’. Was written permission granted by the individual before a complaint was made on their behalf? Does the data on uptake of the feedback mechanism exclude cases initiated by the ILO TPM team?
11. Were modified questionnaires used for people under the age of 18?
12. Can a copy of the seven other interview questionnaires not included in Annex 5 be provided?

Telephone poll

13. There is evidence of phone tapping and monitoring of communications in Uzbekistan. Was the impact of this considered on people’s willingness to answer sensitive questions freely on the telephone? Was this risk to the validity of the findings addressed?
14. What was the process for selecting the private service provider that conducted the telephone poll and what were the international standards that are referred to in relation to its conduct? What is the name of this independent private service provider?
15. Was the purpose of the interview made clear to participants at the outset (e.g. aims of the study, how the results would be used)? Was consent obtained? How was anonymity assured? At what point in the interview was eligibility established given the target age group of 18-50?
16. The number of participants in the telephone poll is not presented consistently in the report. Could clarification please be provided on the data?  Was data from all three waves of interviewing included in the analysis of the results?
17. How many of those surveyed in the telephone poll had cotton picking experience?
18. The report notes with respect to the phone survey: ‘The actual gender profile of the poll respondents represents 68% men and 32% women’. Given that women represent approx. 2/3 of the labor force involved in the cotton harvest, and are statistically therefore overrepresented as victims of forced labor, were steps taken to ensure this skewed sample did not diminish estimates of the total number of people forced to labor in the 2017 cotton harvest?
19. Can a copy of the phone survey questionnaire be provided?

2018 TPM  

20. What is the ILO criteria for selecting local civil society activists for joining the ILO monitoring units or other monitoring activities? How many local activists will be involved and what activities will they carry out? How will their findings be used? How will they be protected from retaliation during and after the monitoring activities? Who, from the civil society activists will receive the training, if any and what kind of training?
21. Will the ILO methodology and reporting in 2018-19 include findings and reporting from other civil society activists and journalists beyond selected by the ILO local activists?