V.A. received funding for this project from the Canadian Institutes for Health Research’s Frederick Banting and Charles Best Doctoral Award (#121732) and from the Institut de recherche Robert-Sauvé en santé et en sécurité du travail.

1. Background

1In the field of ergonomics, there is a significant gap between interventions conducted by practitioners out in the field versus interventions that are described in scientific research intended to assess effectiveness in preventing or reducing work-related musculoskeletal disorders. This is a major concern because conclusions stemming from scientific studies can be misleading regarding the actual contribution of ergonomists. This gap also hinders the uptake of research findings intended to improve ergonomic interventions in practice (Buckle, 2011). Discrepancies between ergonomic interventions performed in a natural context and those evaluated in recent research papers will be discussed here. This is followed by a proposal for a methodological framework for a process evaluation in which the research process takes into account the complexity of the ergonomic interventions conducted in the field.

1.2 Ergonomic interventions in practice : tailored to each organization

2Guidelines (Wells et al., 2003) and textbooks (Kilbom & Peterson, 2006 ; Guérin et al., 2007 ; St-Vincent et al., 2014) on the practice of ergonomics describe several steps leading to the implementation of changes made in a given organization to improve both production and workers’ health. These steps are summarized in Table 1.

3A participatory approach is adopted throughout the intervention, meaning that an ergonomist works in close collaboration at each step with several members of the organization with various levels of authority, since no single member holds all of the information regarding a problem or a work situation. Furthermore, as shown in Table 1, the ergonomist will regularly present findings and report to a joint committee (JC) formed of key stakeholders (namely decision makers, workers directly affected by the work situations, and various specialists in the organization) to discuss the orientation, progression, and results of the ergonomic intervention.

Table 1. Summary of ergonomic intervention steps according to textbooks and guidelines

Phase	Step	Ergonomist’s goals
Development	Request analysis	Gather information on the different representations (i.e., perspectives or views) of the problem and expectations regarding the intervention in order to assist the joint committee (JC) in identifying specific departments or work situations requiring analysis and improvement.
	Analysis of work situations	Develop a shared, agreed-upon understanding in the JC of the links between the consequences (health and/or production problems), the activity carried out (what the workers have to do in their work, including – but not limited to – the identification of risk factors), and the determinants (i.e., elements of the work situation causing the problems).
	Selection of work modifications and adoption of an action plan	Provide evidence to assist the JC in prioritizing determinants to be modified (work modifications) according to their relevance (potential for improving work situations) and their compatibility with business goals and the organization’s constraints (human resources, time schedule, other ongoing or future projects, etc.). May be done simultaneously with step 2.
Implementation	Development of potential solutions	Assist a working group composed of workers affected by the work situation and various specialists in developing potential solutions and simulating future work activities to understand their possible impacts (positive and/or negative) on the work situation.
	Development of prototypes or new scenarios	With the working group, progressively add specifications to the selected work modification(s) by testing the prototype and scenarios (e.g., work rotation schedules) with different workers.
	Progressive implementation and follow-up	Assist the JC in identifying optimal conditions to progressively implement the specific work modification(s) (production slowdown period, limited number of workers or type of products, etc.), monitoring and readjusting until implementation is completed.
Evaluation	Impact assessment	Assist the JC in documenting short and long term impacts of the specific work modification(s) implemented.

4Given the participatory approach adopted at the outset and the fact that the solutions will be implemented by the members of the organization, not the ergonomist, the first three steps presented in Table 1 – corresponding to the intervention’s development phase (Goldenhar et al., 2001) – are crucial in the selection of relevant and feasible work modifications in a given organization. Indeed, during this initial phase, a widely described ergonomist’s role is to identify the determinants (or “root causes”) of production and health problems that could benefit from potential work modifications (Wells et al., 2003 ; Kilbom & Peterson, 2006 ; Guérin et al., 2007 ; St-Vincent et al., 2014). The analysis of work situations (Step 2 – Table 1) often leads to many, considerably different determinants that are specific to a given organization. Ultimately, however, the choice of the determinants to be modified (Step 3 – Table 1) belongs to the organization’s stakeholders, not the ergonomist. Thus, a lesser-known yet equally important ergonomist’s role is to provide evidence to the stakeholders that will convince them of the need to act on the most relevant determinants based on the ergonomist’s analysis (Guérin et al., 2007 ; Wells et al., 2013 ; St-Vincent et al., 2014). In other words, modifying the stakeholders’ representations – their beliefs and knowledge about the actual work under study, the causes of the problems (or determinants), and the potential solutions – is an essential intermediate outcome in the intervention’s development phase, as it triggers mobilisation within the organization and the allocation of resources for developing and implementing changes (St-Vincent et al., 2014). It also highlights the importance of the first step (Request analysis – Table 1) to understand the stakeholders’ initial representations of the problems to be solved. To illustrate the previous statements, a brief example of an ergonomic intervention provided by a practitioner out in the field follows

1.2 Brief example of an ergonomic intervention out in the field

5In a metal transformation industry, workers had to take off the burr on very large metal pieces following machining. Operations on the central part of these pieces were extremely demanding for the back, neck, and shoulders (Figure 1). A solution put forward by a management team was to design a racking system, holding the piece in a vertical position to decrease musculoskeletal demands, but the deburr workers were drastically opposed to this solution. The ergonomist’s analysis of the work situation gained through several interactions with deburr workers, as well as managers, machining operators, programmers, and top management, revealed important aspects of the actual work to be done in these awkward positions. Indeed, the precision of the deburring cut was critical : a cut that was not at a perfect right angle would lead to the rejection of the whole piece, which was worth thousands of dollars according to the managers. To ensure the precision of their movements, deburr workers had to partially rest their hand on the piece to maintain the tool in a perpendicular position and had to constantly be attentive to what they were doing. Working on the piece in a vertical position was therefore undesirable, as workers would be forced to bend their torso laterally to see their work and to hold their arms upwards against gravity. Consequently, they would lack appropriate support, which would lead to increased biomechanical demands, decreased precision, and potential rejection of the piece.

6The solution put forward initially (the vertical racking system) was thus abandoned, as the managers’ representation of the actual work changed to take into consideration the quality requirements of the task, an important determinant that influenced the way of doing the work and that was initially overlooked. Other determinants that could be modified were, just to name a few, the trestle height, tool characteristics, lighting, and – most importantly – work quality in the previous production stage (machining). The latter determinant was the most relevant work modification according to the ergonomist, as it had the potential to completely eliminate deburring operations on the central part of the metal pieces if a few specific operations were added to the machine tool program. As the stakeholders became aware that both workers’ health and production could be improved by modifying the machine program, this work modification was selected for implementation. This brief example sheds light on merely one facet of an ergonomic intervention provided by a practitioner out in the field which led to the selection of several work modifications and which were further described in other publications (Albert, 2014 ; Albert et al., 2016).

7In summary, four main points should be highlighted regarding such ergonomic interventions. First, the ergonomic intervention is not the “solution” in itself, but rather a process leading to the change. This complex process includes several steps in which the ergonomist conducts actions that are specifically adapted to an organization’s possibilities and constraints. Second, an important trap to avoid is that of restricting the work situation analysis to the biomechanical loading and risk factors, without seeking to understand the actual work, or, in other words, what workers are doing in these awkward work postures in the first place (St-Vincent et al., 2014). Third, changing key stakeholders’ representations is a crucial intermediate outcome as it contributes to the selection of relevant work modifications in the action plan. Fourth, given the singularity of the actual work performed in each situation, it is quite unlikely that any two interventions provided by ergonomists out in the field lead to the implementation of identical solutions.

1.3 Ergonomic interventions in research : oversimplified to fit experimental designs

8In the last few years, the effectiveness of ergonomic interventions has been evaluated mostly through randomized controlled trials (RCTs). The interventions under study mainly consisted of the same piece of equipment or training provided to a large number of workers from different organizations. Based on such studies, some systematic reviews further concluded that ergonomic interventions were usually not effective in preventing or reducing work-related musculoskeletal disorders (Driessen et al., 2010 ; Leyshon et al., 2010 ; Hoe et al., 2012). The most important problem is that the ergonomic interventions evaluated through experimental designs are defined in oversimplified terms. The intervention under study is solely the “solution” in itself. However, for ergonomists, the solution in itself has low replication value if the actual work, its determinants, the organization’s context and the process leading to its selection are unknown. Indeed, reviews have already pointed out that the development phase of ergonomic interventions in evaluation studies is weakly, if not at all described (Denis et al., 2008 ; Yazdani et al., 2015). In other words, we do not know – as it is not specified in these articles – if steps 1 to 3 described in Table 1 were carried out in these studies or if a solution was simply predefined by the research team and later implemented in selected organizations. This might be the case given the RCTs requirements in terms of internal validity and large sample size to detect effects. Some authors furthermore conclude that these requirements, which ensure the quality of the evaluation design, can however severely compromise the quality of those ergonomic interventions that are suitable for analysis (Neumann et al., 2010). Therefore, systematic reviews including only experimental studies are biased in favour of simple interventions that are easier to assess but that are not necessarily more efficient (Berthelette et al., 2008). The lack of proof of the effectiveness of ergonomic interventions may come instead from the absence of an adequate evaluation approach to assess complex interventions (Coutarel & Petit, 2015). Although there is no sharp boundary between simple and complex interventions, complex interventions are usually described as interventions which : 1) contain several interacting components, 2) require a high degree of flexibility or tailoring of the intervention, 3) are provided to several groups of stakeholders from different organizational levels, and 4) generate various outcomes (Brousselle et al., 2011 ; Craig et al., 2013 ; Moore et al., 2015). Interventions provided by ergonomists out in the field as described in sections 1.1 and 1.2 can therefore be considered as complex interventions. There is an urgent need to develop alternative evaluation approaches that are better suited to these types of interventions.

1.4 Developing an evaluation approach to assess the development phase of complex ergonomic interventions

9The goal of effectiveness research in ergonomic interventions (namely through RCTs) is to determine whether specific interventions can prevent health problems among workers, such as work-related musculoskeletal injuries. However, it is also crucial that the development and implementation of interventions be evaluated (Goldenhar et al., 2001). A few studies have conducted evaluations of the implementation phase in complex ergonomic interventions (Bellemare et al., 2002 ; Vézina et al., 2003 ; Montreuil et al., 2004 ; Baril-Gingras et al., 2007). They have provided useful insights on changes in stakeholders’ representations which occurred throughout the implementation phase and insights into the contextual factors facilitating or hindering the implementation of work modifications. More specifically, two studies provided a framework to document the contextual influence of an organization on the ergonomic intervention (Baril-Gingras et al., 2004 ; St-Vincent et al., 2010). However, to our knowledge, there have, as of yet, been no studies that have specifically evaluated the development phase of complex ergonomic interventions. In other words, although the first three steps (see Table 1) leading to the adoption of an action plan are well described in textbooks and guidelines, the specific contextual factors that influence the actions performed by ergonomists during this phase, and the strategies adopted by ergonomists to overcome obstacles throughout this first phase are unknown. Furthermore, since an ergonomic intervention can be made up of more than a hundred actions (Bellemare et al., 2001), it would be useful to identify the “most effective actions,” that is the actions that contribute to changes in key stakeholders’ representations and that lead to the adoption of the action plan. We hypothesize furthermore that the ergonomist’s actions conducted in a given organizational context represent the transferable aspect of the intervention to future interventions carried out in organizations sharing similar contextual factors. Consequently, an evaluation approach should be developed to that effect.

10A model for the process evaluation of organizational-level, occupational-health interventions indicates that the factors that may have an impact on the outcomes of such interventions can be grouped into three themes : 1) the intervention design and implementation (i.e., processes and actions conducted throughout the intervention), 2) participants’ mental models (i.e., representations), and 3) the intervention context (Nielsen & Randall, 2013). More specifically, the intervention context is defined by the social, political, organizational, and cultural characteristics of the organizations in which the interventions take place (Berthelette et al., 2008). As these three themes show intrinsic similarities with the ergonomist’s concerns about stakeholders’ representations during the development phase of the intervention and have the potential to fill gaps identified in the scientific literature regarding context and effective actions, this model was selected to guide the design of the methodological framework for process evaluation described in the following sections.

1.5 Study aim, evaluation questions, and logic model

11The goal of this article is to present the methodological framework of a process evaluation focussing on the development phase of ergonomic interventions. Results stemming from this process evaluation are available elsewhere (Albert, under review ; Albert et al., submitted). This evaluation protocol seeks to answer the following questions regarding the development phase of ergonomic interventions : 1) which contextual factors influenced the action plan and the various actions by the ergonomist that led to its adoption ? ; 2) which specific actions and/or information transmitted to key stakeholders contributed to changes in their representations and to the action plan ? A logic model of the ergonomic interventions under study is presented in Figure 2, adapted from Nielsen and Randall (2013).

2. Methods / design

12The following methodological framework for a longitudinal multiple case study (n =4) was designed according to the above-mentioned model for the process evaluation of organizational-level, occupational-health interventions (Nielsen & Randall, 2013). It drew furthermore from theoretical and empirical bases, namely : 1. evaluation literature (Chen, 2005 ; Stake, 2006 ; Brousselle et al., 2011 ; Ridde & Dagenais, 2012), 2. publications on evaluations in the ergonomics field (Lamonde, 2000 ; Cole et al., 2002 ; Baril-Gingras et al., 2004 ; Vézina et al., 2006 ; Caroly et al., 2008 ; Landry, 2008 ; Coutarel et al., 2009 ; Baril-Gingras et al., 2010 ; St-Vincent et al., 2010), and 3. a pilot case study for pretesting data collection tools (Albert et al., 2016). This study received ethical approval (#FSH-2014-98) and funding from the Canadian Institutes of Health Research (#121732).

2.1 Interventions under study and organizations

13The four ergonomic interventions (cases) discussed here took place in organizations from different sectors of industry (food processing, pharmaceutical, manufacturing, and transportation). Interventions were provided by emerging ergonomists (hereafter referred to as “EEs”) who were on a final, two-days-a-week, year-long internship leading to a master’s degree in ergonomics. This methodological choice had several advantages and was made to overcome specific challenges identified in previous studies. Firstly, EEs received training with the most up-to-date intervention models and principles in the ergonomics field (St-Vincent et al., 2014). They were closely supervised by a committee of experienced ergonomists and university professors to ensure the quality of their intervention. They also used the same data collection tools (see section 2.3 Data collection for additional details), which included a logbook that was specifically designed to monitor and evaluate complex ergonomic interventions (Vézina et al., 2006). A record for each action was thus kept and documented uniformly across sites, which is generally one of the biggest evaluation challenges in the field of ergonomics (Coutarel et al., 2009) as well as when performing an evaluation on multiple sites (Straw & Herrell, 2002). These ergonomic interventions also started in the same time period, progressed at a similar rate, and followed the same steps (presented in Table 1), allowing inter-case comparisons (N.B. : only the first three steps presented in Table 1 were included in the evaluation, as this protocol focused on the development phase of the ergonomic intervention).

2.2 Type of participants and recruitment

14The participants included the 4 EEs and 4 to 6 key stakeholders from each organization. Stakeholders were members of the joint committee for each intervention or other employees closely involved in the intervention. Recruitment was done on a voluntary basis. The stakeholders’ selection was based on various levels of authority and legitimacy. Authority refers to the stakeholder’s ability to influence decisions involving the ergonomic intervention being evaluated, whereas legitimacy refers to the stakeholder’s knowledge of the actual work under study (Mark & Shotland, 1985). For example, workers performing the work under study were considered as having high legitimacy but generally low authority while top management had high decisional authority but usually little or partial knowledge of the actual work to be done. The EEs’ and stakeholders’ views on the intervention processes, context, and intermediate effects (changes in the stakeholders’ representations and nature of the action plan) were triangulated via several methods in order to generate a multi-faceted portrait of the ergonomic intervention in each organization (Stake, 2006).

2.3 Data collection

15The data collection used mixed methods involving multiple sources of information. Methods, sources, and data collection periods were chosen in order to minimize, as much as possible, interferences between intervention and evaluation (Lamonde, 2000 ; Landry, 2008). The data collection process is summarized in Figure 3 and is described in the following sections.

2.3.1 Context description

16During the first step of the intervention (Request Analysis, Figure 3), the EEs collected data as a natural part of their internship on several characteristics of the organization in which the intervention took place. In total, data on 88 context indicators were collected, based on findings from previous studies documenting context in the fields of ergonomics (Baril-Gingras et al., 2004 ; St-Vincent et al., 2010) and rehabilitation (Durand et al., 2014), as well as on expert consultation. These indicators were grouped into 9 categories : 1) organizational characteristics ; 2) worker characteristics ; 3) economic conditions ; 4) union and labour conditions ; 5) type of management and work organization ; 6) ongoing or future projects ; 7) health and safety prevention ; 8) disability and return-to-work management ; 9) records and statistics. For each case, these data were consulted to triangulate the contextual factors (which influenced the action plan and/or the EEs’ actions that led to its adoption) identified in the interviews (see following sections for details).

2.3.2 Logbook compilation and “self-confrontation” interviews with the EE

17A “self-confrontation” interview is a specific type of reflexive or think-aloud interview commonly used in the field of ergonomics that allows participants to see themselves in action and explain the underlying cognitive processes, for instance, by commenting on notes about their work activity (Guérin et al., 2007 ; St-Vincent et al., 2014). This notes was obtained via a quantitative and qualitative compilation of the logbook kept by the EE (Figure 3). Indeed, the logbook offered a rich and in-depth description of every process (i.e., the EE’s actions), including the type of activity, duration and objectives, stakeholder(s) involved, results and decisions taken (Vézina et al., 2006). The logbook also contained information on the key stakeholders’ representations (Vézina et al., 2006). An overview of the data contained in the logbook is available in Appendix 1. The logbook was designed in a free and open database software format (www.openoffice.org/download/ or https://fr.libreoffice.org/​) that allows the EE to make a quantitative compilation of the actions taken and stakeholders met at each step. This file can be consulted upon request to the first author (VA). Based on the quantitative data from the logbook, the researchers generated several charts in order to illustrate the progression in the types of activities performed by the EE, in the categories of stakeholders involved, and in the frequency of contacts with key stakeholders

18These charts were presented during the self-confrontation interview to assist the EE in identifying contextual factors that influenced his/her actions and the strategies put forward to overcome obstacles and to obtain an in-depth description of each step in the intervention. In total, up to three self-confrontation interviews could be performed with the EE following each step of the intervention’s development phase as shown in Figure 3. The interview following Step 3 may not be necessary if the action plan is selected by stakeholders directly after Step 2. The self-confrontation interview guide can be found in Appendix 2. Subsequent questions on specific contextual factors to be used in the self-confrontation interview were also generated following the qualitative logbook and progress report compilation. This combination of methods is based on findings from a previous evaluation study indicating that the logbook in itself is a reflexive tool. Its potential, however, is maximised when an external researcher interrogates the ergonomist regarding strategies put forward and decisions taken based on specific contextual factors, as they are not always readily apparent for an ergonomist in the middle of an intervention (Caroly et al., 2008).

2.3.3 Progress reports and semi-structured interviews with stakeholders

19Following each step of the ergonomic intervention development phase, the EEs present a formal progress report to the joint committee, which consists of a summary of the actions taken and relevant information collected during the step, using a slide presentation as visual support. Slides from progress reports stimulated further questions, as needed, in the semi-structured interviews with the stakeholders (Figure 3). The goal of these interviews was to identify which specific actions and/or information transmitted by the EE contributed to their decisions regarding the action plan and to changes in their representations. Contextual factors influencing the decisions about the selection of the action plan from the stakeholders’ perspectives were also sought. The stakeholder interview guide was adapted from guidelines in representation assessment (Apostolidis, 2003) and is available in Appendix 3.

2.4 Analysis

20Given that data were collected in real time in the logbook, and this over an extended period, a process-centered analysis was performed, first with in-depth case analyses, then with inter-case comparisons (Sylvain & Lamothe, 2013). Through the process-centered analysis, we sought to identify sequences of events over time which explained how and why particular outcomes were reached in each case (Sylvain & Lamothe, 2013). As mentioned earlier, the evaluation questions were the following : 1) which contextual factors influenced the action plan and the various actions by the ergonomist that led to its adoption ? ; 2) which specific actions and/or information transmitted to key stakeholders contributed to changes in their representations and to the action plan ?

2.4.1 Case analysis

21A quantitative compilation of the logbook was first produced to draw up a general portrait of each step of the intervention (total duration in days/hours, type of activities performed, stakeholders met). Each interview (EE and stakeholders) was transcribed verbatim. A thematic analysis was performed using a mixed coding strategy : predetermined themes (contextual factors, effective actions, pivotal information transmitted to key stakeholders which led to changes in representations, action plan selection process, etc.) and complementary themes that emerged empirically (Paillé & Mucchielli, 2003). Triangulation with qualitative data from the logbook and reports was iteratively sought to increase the validity of the results. These analyses were presented by the researchers to the EE on a final interview to validate, modify, or refine the results that were generated (Stake, 2006). Only minor modifications were required following these interviews. An in-depth narrative description of each intervention (Stake, 2006) and a logic model (Brousselle et al., 2011) were produced for each case.

2.4.2 Inter-case comparisons

22For the first question (contextual factors influencing the intervention and action plan), inter-case comparisons were performed using a matrix (Stake, 2006) to compare and contrast the context indicators influencing each case. A context indicator is considered significant when mentioned explicitly by at least one participant (EE or stakeholder). Indeed, even if more than one source was systematically sought out for each indicator (Stake, 2006), we knew that the participants would quite likely have different perspectives due to their roles (decision maker, supervisor, worker, specialist, etc.) and their knowledge of specific issues in the organization. In our approach, triangulation was also used to reveal different perspectives on a same phenomenon (Stake, 1994). The diversity of participants involved in the evaluation contributed to the richness of the results generated. For the second question (changes in stakeholders’ representations), the analysis of similarities and differences between cases led to the proposal of middle-range theories, developed contextually for each case, but containing hypotheses that might be applied to future interventions performed in similar contexts (David, 2005).

3. Discussion

23To our knowledge, this is the first study to propose a methodological framework for a process evaluation that focuses on the development phase of ergonomic interventions. Our evaluation approach aims to describe and explain how complex ergonomic interventions take place and how variation is induced by each organization’s context. However, our approach is quite different from recent recommendations for alternative research designs to RCTs in the field of occupational health (Schelvis et al., 2015), mainly because the designs put forward in this review aim to control the context rather than understanding its influence on the intervention. It is nonetheless worth recalling that our evaluation protocol draws on principles for evaluating complex interventions that have already proven their worth in the field of occupational and public health (Berthelette et al., 2008 ; Brousselle et al., 2011 ; Ridde & Dagenais, 2012). In the field of ergonomics, a few process evaluations have documented the concepts of recruitment, reach, fidelity, satisfaction, and intervention delivery and intervention received (Pehkonen et al., 2009 ; Driessen et al., 2010 ; Baumann et al., 2012 ; Visser et al., 2014 ; Dale et al., 2016). On the other hand, the process evaluation model selected in the present study focused on gaining a more specific understanding of the context, on how the field intervention was perceived by the various levels of stakeholders within the organization, on what their mental models were, and on how they changed (Nielsen & Randall, 2013). We believe that this model, which has been developed for organizational-level interventions, is more closely related to practitioners’ concerns and may provide useful avenues to improve the effectiveness of measures that target changes in stakeholders’ representations, an important intermediate outcome of ergonomic interventions that should systematically be included in evaluation studies in the field of ergonomics.

3.1 Using process evaluations to select relevant variables in subsequent effectiveness evaluations

24A challenge in evaluating complex interventions lies in the description of the intervention and the identification of adequate pre- and post-variables to detect and appraise outcomes. The small group of researchers who have already evaluated complex ergonomic interventions were indeed confronted with these difficulties. For example, an action-research study indicated that researchers identified signs of improvement following an ergonomic intervention in an electronics manufacturing firm, although their study did not set out to measure changes or outcomes in a pre-/post- fashion (Village et al., 2014). Another single-case study evaluation of an ergonomic intervention performed in the footwear industry concluded that before and after comparisons on relevant work variables could not be performed, since the changes that had, in hindsight, the largest impact on working conditions were either not initially recognized as problematic or considered to be modifiable, and hence not documented at the onset of the evaluation (Guimaraes et al., 2014). Performing a process evaluation of the development phase of a complex intervention could help ergonomists to draw a more complete picture of a given organization’s particular situation, such as the context, stakeholders involved, and main problems to be modified. These are useful insights for selecting relevant and tailored pre- and post-variables to be documented in a subsequent implementation or effectiveness evaluation, when assessing complex interventions which vary highly according to the organizational context in which they take place.

3.2 Strengths and limitations

25A major strength of this methodological framework is the data collection tools which were rigorously developed on methodological and empirical bases as well as pretested in a pilot case study (Albert et al., 2016). The logbook and interview guides are presented in this article in their integral format and could be adapted for use in future evaluations of complex ergonomic interventions in other settings. More specifically, our protocol proposes a semi-structured interview guide for the assessment of stakeholder representations (Appendix 3). This is an important intermediate outcome in ergonomic interventions according to textbooks (Guérin et al., 2007 ; St-Vincent et al., 2014) that should be, in our opinion, systematically documented in future evaluation studies in the field of ergonomics.

26The main limitation of our protocol is the study of interventions conducted by EEs, whose actions and strategies may differ from those of experienced ergonomists. Nonetheless, these EEs were closely supervised by their more experienced colleagues. What is more, this particular form of ergonomic intervention fosters useful comparisons due to their common progression rate and data collection tools. The goal was specifically to take advantage and make use of these innovative data collection tools, which are part of the normal internship, and to better define their contribution to the assessment of ergonomic interventions. The results of this evaluation, which are available in other publications (Albert, under review ; Albert et al., submitted), support the descriptive and explanatory potential of these tools. As suggested by the Medical Research Council guidelines (Moore et al., 2015), it would be useful if future process evaluations of the development phase of ergonomic interventions performed in other settings were to use comparable methods to those described in this article in order to make meaningful comparisons across studies. Indeed, the generalization of results should not be based on one single study, but rather on the possibility of eventually comparing various complex ergonomic interventions evaluated in different studies so as to progressively gain a better understanding of the elements that should be improved in these interventions. Such comparisons would involve a detailed and homogenous description of interventions across studies (Coutarel & Petit, 2015). Our methodological framework suggests, along the same lines as Neumann et al. (2010), that the key descriptive elements of ergonomic interventions should be the effective actions performed at each step, the contextual factors that led to them, and the results of these actions (including changes in stakeholders’ representations), seen from the ergonomist’s and stakeholders’ perspectives.

4. Conclusion

27Evaluation studies in the field of ergonomics have primarily assessed the effectiveness of simple technical changes isolated and removed from the context of organizations in which they were implemented, and thereby generally leading to negative results. A closer look at ergonomic field-interventions as described by practitioners suggests that several actions tailored to the organization’s context precede the implementation of specific changes. Evaluation approaches should therefore focus specifically on the ergonomist’s actions and underlying contextual factors, rather than exclusively on the changes, which are always context dependant. The methodological framework for the process evaluation described in this article could be useful to other researchers interested in assessing the development phase of ergonomic interventions in different settings. Findings from multiple studies could thus build on one another to progressively obtain a better understanding of the mechanisms at work in these complex interventions.