Edinburgh Napier University

  Background: Growing evidence supports the benefits of eHealth interventions to increase patient engagement and improve outcomes for a range of conditions. However, ineffective program delivery and usage attrition limit exposure to these interventions and may reduce their effectiveness.

Objective: This study aims to evaluate the delivery fidelity of an eHealth intervention, describe use patterns, compare outcomes between low and high users, and identify mediating factors on intervention delivery and receipt.

Methods: This is a mixed methods study of an internet-based intervention being evaluated for effectiveness in a randomized controlled trial (RCT). The intervention comprised medication and cardiovascular disease (CVD) risk data uploaded from the primary care electronic health record (EHR); interactive, personalized CVD risk score estimation; goal setting and self-monitoring; an interactive social forum; and optional receipt of heart health messages. Fidelity was assessed over 12 months. Trial outcomes were compared between low and high users. Data sources included program delivery records, web log data, trial data, and thematic analysis of communication records.

Results: Most participants in the intervention group (451/486, 93%) had an initial training session conducted by telephone (413/447, 92.4% of participants trained), with a mean duration of 44 minutes (range 10-90 minutes). Staff conducted 98.45% (1776/1804) of the expected follow-ups, mostly by telephone or email. Of the 451 participants who commenced log-ins, 46.8% (211) were categorized as low users (defined as at least one log-in in 3 or fewer months of follow-up), 40.4% (182) were categorized as high users (at least one log-in in more than 3 months of follow-up), and 12.8% (58) were nonadopters (no log-ins after their training session). The mean log-in frequency was 3-4 per month in ongoing users. There was no significant difference between the groups in the primary trial outcome of adherence to guideline-recommended medications (P=.44). In unadjusted analyses, high users had significantly greater eHealth literacy scores (P=.003) and were more likely to meet recommended weekly targets for fruit (P=.03) and fish (P=.004) servings; however, the adjusted findings were not significant. Interactive screen use was highest for goal tracking and lowest for the chat forum. Screens with EHR-derived data held only an early interest for most users. Fidelity measures (reach, content, dose delivered, and dose received) were influenced by the facilitation strategies used by staff, invisible qualities of staff-participant communication, and participants’ responsiveness to intervention attributes.

Conclusions: A multifeature internet-based intervention was delivered with high fidelity to the RCT protocol and was regularly used by 40.4% (182/451) of users over 12 months. Higher log-in frequency as an indicator of greater intervention exposure was not associated with statistically significant improvements in eHealth literacy scores, lifestyle changes, or clinical outcomes. Attributes of the intervention and individualized support influenced initial and ongoing use.