Measurement issues when assessing quality of life outcomes for different types of hernia mesh repair

This study was a cross-sectional survey of long-term QoL after mesh repair for inguinal hernia. The primary outcome measure was the patients' report of their QoL as measured on the CCS. Secondary outcome measures included recurrence of hernia, time until return to work and perioperative pain. Background variables were the time since the operation, body mass index (BMI), demographics (age, ethnic group, social class) and free-text handwritten responses from patients.

Until November 2007 Surgipro™ mesh was the standard type of mesh used at Benenden Hospital and it was replaced solely by Progrip™ after this time. The sampling frame consisted of patients who had undergone inguinal hernia repair at Benenden Hospital from January 2007 to August 2008. The exclusion criteria were bilateral hernias, significant comorbidity, patients requiring an additional procedure and surgery performed by non-resident surgeons. Patients were invited by letter to participate in the study and were provided with a stamped, addressed envelope for returning the completed questionnaires. The survey took about ten minutes to complete. Data were anonymised by assigning trial numbers. All participants gave written informed consent. The study was approved by the West Kent Research Ethics Committee (Ref 09/H1103/43).

Heniford et al's own assessment of the CCS4 found good internal reliability (Cronbach's alpha = 0.979), moderate test–retest reliability (with kappa coefficients ranging between 0.42 and 0.75 for all but one item), moderate concurrent validity with the SF-36 health survey (average correlation 0.40) and good discriminant validity compared with the SF-36.

Inguinal hernia repair has traditionally been done using stitches but in the 1980s the use of mesh began.5 Surgipro™ is constructed from polypropylene and needs to be sutured or stapled to well defined structures to keep it in place. Progrip™ is a self-adhesive mesh that has been developed to reduce postoperative pain for inguinal hernia repair; being self-adhesive means that it does not require any sutures to keep it in place and is easier to use.6 Progrip™ has not yet been compared with other meshes using a validated QoL measure.

Normally distributed variables were analysed using t-tests and Pearson's correlation coefficients. Non-normally distributed or categorical outcomes were measured using the Mann–Whitney U test, Spearman's correlation coefficients and the chi-square test. Analyses were performed using SPSS® v16 (SPSS Inc, Chicago, Illinois, US). The significance threshold was 0.05 and all p-values are two-tailed.

Internal consistency of the CCS was assessed using Cronbach's alpha. The structure of the CCS was explored using principal component analysis. Descriptive statistics and the non-parametric Mann–Whitney U test were used to determine the sensitivity of the CCS in distinguishing between mesh types, patients with or without recurrence of hernia, and patients with or without chronic pain. Differences in the frequency of hernia recurrence and chronic pain between mesh types were analysed using Fisher's exact test.

Of the 684 patients identified, 334 met the inclusion criteria. The 334 participants were contacted by post and 215 responded. Overall, 96 of the 151 Surgipro™ patients responded (response rate: 64%) and 119 of the 183 Progrip™ patients responded (response rate: 65%). Forty-two participants (19.6%) did not answer one or more items from the CCS questionnaire. For other outcomes the full dataset was analysed but for CCS total scores only the 173 completed CCS questionnaires were analysed.

As expected, the time since surgery was significantly longer in the Surgipro™ group (p<0.001) but none of the other background variables (BMI, social class, race, gender, etc) showed any significant difference between the groups.

In the present study the CCS showed good internal reliability (Cronbach's alpha = 0.982). The structure of the CCS was investigated using principal component analysis. The analysis found that participant responses clustered into two subscales (or categories): ‘mesh sensation’ and ‘pain+movement limitations’.

Table 1 shows the main outcomes for the two patient groups with different mesh types. In the Progrip™ group the mesh sensation score was slightly higher, approaching statistical significance. ‘Pain+movement limitations’ scores were similar in the two groups.

Recurrence occurred in five cases (2.4% of all patients). The frequency of recurrence was equivalent in the Surgipro™ group (2/96, 2.1%) and the Progrip™ group (3/118, 2.5%) (Fisher's exact test = 1.0). Mann–Whitney U tests found that those who had a recurrence reported significantly more mesh sensation (p<0.016) and pain+movement limitations (p<0.002) than those who did not have a recurrence. The CCS total score was also higher for those who had a recurrence (p<0.003). Chronic pain occurred at similar rates in the Surgipro™ group (11/95, 11.6%) and the Progrip™ group (22/118, 18.6%) (p<0.157). Those who experienced chronic pain reported significantly more mesh sensation (p<0.001) and pain+movement limitations (p<0.001) than those who did not have chronic pain.

Social classes (professional, intermediate, manual) were equally represented in each mesh group (p<0.869). The majority of patients were Caucasian and there were three non-Caucasian participants in each group (two Asian, two Afro-Caribbean, one African and one Turkish). There were three women in each group. Only 14 of the 215 respondents were NHS patients. This was to be expected because Benenden is an independent hospital and sees only a small number of NHS patients. A total of 58% of operations were on the right side and 42% on the left. There was no significant difference for any of the five surgeons on any of the outcomes, nor did any surgeon perform a significantly different proportion of Suripro™ or Progrip™ operations compared with the other surgeons. The mean BMI in the Surgipro™ group was 26.56 (standard deviation [SD]: 3.51) and in the Progrip™ group it was 26.33 (SD: 3.27) (p<0.570). The mean age in the Surgipro™ group was 64.44 (SD: 11.06) and in the Progrip™ group it was 63.29 (SD: 14.28) (p<0.504). The median (± interquartile range) time since the operation in years for the Surgipro™ group was 2.64 + 0.37 (range: 2.16–3.01) and for the Progrip™ group it was 1.86 ± 0.49 (range: 1.07–2.47) (p<0.001).

The structure of the CCS was investigated using principal component analysis (PCA). The rotation method was varimax with Kaiser normalisation. Rotation converged in three iterations and revealed two components with factor loadings >1.0 (Table 2). The resulting subscales show good factorability: Kaiser–Meyer–Olkin test = 0.923; Bartlett's test of sphericity (p<0.001); 27% of non-redundant residuals were above 0.05. Factor 1 explained 42.5% of the variance and factor 2 explained a further 37.4%.

In the CCS, items prefixed with an ‘a’ relate to the sensation of mesh, items prefixed with a ‘b’ relate to pain and items prefixed with a ‘c’ relate to movement limitation. Most of the items in the first factor were ‘b’ and ‘c’ items, thus relating to pain and movement limitations. The second factor consisted only of question ‘a’ responses. Therefore, all items in this factor related to the sensation of mesh. The only two ‘b’ and ‘c’ items that loaded on the second factor (questions 3b and 3c; see Table 2) were included in the first factor because: i) they were nearly as strongly loaded on the first factor as the second; ii) including them in the first factor retained the face validity of the subscales to a greater degree; and iii) further analysis revealed that it made little difference to outcomes whether 3b and 3c were in the first or second factor. Therefore, the two-factor solution appears to define the subscales by items relating to either mesh sensation or pain+movement limitations and all further analyses using the CCS subscales reported in the present study include items 3b and 3c in the first factor with the other ‘b’ and ‘c’ items. As items 3b and 3c loaded almost equally on both of the subscales, future research might seek to replicate the PCA of the present study. Item 3c showed weak test–retest reliability in the original CCS validation paper4 so its continued inclusion in the CCS should be considered carefully.

In the standardisation of the CCS4 little information is given regarding the PCA except to comment that two factors were found, the first of which explained approximately 70% of the variance. This is in contrast to the present study, which found that the first factor (pain+movement limitations) accounted for 42.5% of the variance and the second factor (mesh sensation) for a further 37.4%. The sample in the present study (n=215) is larger than that in the original CCS paper (n=136) and therefore more appropriate to the statistical power needed for PCA.9 The lack of information on the PCA in the original CCS paper4 is unfortunate as this makes comparison with the present findings difficult.

This study assessed QoL outcomes in two hernia patient groups and found that the type of mesh used in their surgical repair made only a minor difference to their QoL outcomes. The recurrence rate was similar in both mesh groups (Surgipro™ 2.1%; Progrip™ 2.5%), which compares well with other studies of open inguinal mesh repair (eg 4.9%).7

The present study found that responses on the CCS questionnaire clustered into two subscales: ‘mesh sensation’ and ‘pain+movement limitations’. This is interesting because it suggests that patients tend to experience the sensation of mesh in a way that is not necessarily related to pain and movement limitations. Furthermore, the mesh sensation subscale was able to discriminate between the two mesh types despite there being no difference in pain and movement limitations in the two mesh groups. These findings suggest that the Progrip™ mesh was more noticeable to patients but that it did not appreciably affect pain and limit movement.

The two subscales were also useful in elucidating the experiences of patients who experienced a recurrence and suggest that recurrence is more characterised by problems with pain and movement limitations (p<0.002) than being able to sense the surgical mesh (p<0.016). In contrast, while the CCS total score was able to discriminate between those who had a recurrence compared with those who had not (p<0.003), the CCS total score did not reveal which issues (sensing mesh versus experiencing pain and movement limitations) were more important to these patients.

The response rate in the present study was reasonably high (64.5% overall). This is in contrast with the original CCS paper,4 which had a response rate of 13% (136 returned of 1,048 contacted). In the present study the response rate was equivalent in each mesh group, indicating that participation was not related to mesh type. The original CCS paper and the present study both had similar rates of missing data: 14.7% of the original CCS sample had at least one missing item compared with 19.6% for the present study.

Because almost all of the participants were Caucasian, the findings might not generalise well to other ethnic groups.

The duration of time since the operation was different in the two groups but this is unlikely to account for differences in outcomes between the groups. The Surgipro™ repairs were done two to three years prior to the survey and the Progrip™ repairs were done one to two-and-a-half years prior to the survey. However, other research has found that postoperative pain is not a problem for most hernia patients three months after surgery.7 Our study found that there was no relationship between the duration since the operation and any of the outcome measures for either group or for the patients as a whole. This means that any group differences were not the result of the effect of time.

The sample in the present study reported their QoL scores between 2.16 and 3.01 years after surgery and more than half of the responses gave the best possible score (zero). Although this is positive in clinical terms, it means that the CCS subscales are likely to be most useful in finding differences between QoL outcomes in mesh types earlier in the post-surgical period when the patients are more likely to be symptomatic. This suggests that the CCS may be sensitive mainly in relatively symptomatic patient groups, such as Hope et al's.8 See Figure 1.

The distribution of responses in the present study was very right-skewed. Roughly 90% of CCS total scores were less than 10 and 50–60% were zero. The present findings require replication using a population that is more symptomatic than that seen in our study, for example patients who have had a period of recovery of less than three months.

A similar distribution might have inflated the value for Cronbach's alpha in the original CCS paper4 and may account for the contrast between Heniford et al's high Cronbach's alpha (0.979) and relatively weaker test–retest coefficients (0.42 to 0.75).

The Progrip™ group experienced borderline significantly more sensation of mesh and non-significantly more pain and movement limitations than the Surgipro™ group. Recurrence of hernia was found to be similar in the Progrip™ and Surgipro™ groups. Principal component analysis suggests that the CCS questionnaire is more sensitive to patients' experiences when the subscales are used. Future research might benefit from using the three different categories of questions (mesh sensation, pain, and movement limitations) as three separate subscales in studies where each of these three outcomes are of interest in their own right.

A wide variety of mesh types are used in different countries around the world and the present authors suggest that using subscales (rather than the CCS total score) is the best way of assessing QoL outcomes related to different mesh types.

Sources of funding: Benenden Hospital Trust.