Keywords

Internet, Research design, Social media

 

Authors

  1. Stuart, Wilma Powell PhD, RN, NEA-BC
  2. Moore, Bridget DNP, MBA, RN, NEA-BC

Abstract

Traditional methods for research study recruitment such as snail mail lists and posting flyers may fail to reach the tech-savvy participants needed for today's healthcare studies. Word of mouth can be effective for recruiting a few participants but can rarely accomplish the numbers needed for a representative sample. Social media can be a viable avenue to reach increased numbers of sample participants; however, a good understanding of the risks and benefits of using social media is needed before embarking on active recruitment. A recent study developed an evidence-based participant recruitment plan for the use of Facebook. Potential participant misrepresentation was addressed with clear inclusion criteria, no incentives, and open-ended questions. The Facebook ads to recruit study participation targeted licensed nurses who worked in the prior 2-year period living in the United States based on information in Facebook user profiles. A total of 536 participants responded to all questions on the survey at a cost of $1.78 per completed survey. Daily activity and cost for ads were closely monitored and adjusted to maintain cost control. Facebook can be an effective tool for study participant recruitment across all age ranges for completion of online surveys.

 

Article Content

Researchers have dealt with the challenge of recruitment of study participants for decades. Traditional methods can involve costly print ads, slow or no responses to flyers, and ignored invitations to participate. As societal communication methods evolve, social media should be considered for the recruitment of study participants. In fact, Internet users in the United States have increased from 52% in 2000 to 90% in 2019.1 One social media platform that includes members from multi-generational and socio-economic groups is Facebook (Facebook, Menlo Park, CA). Facebook reports an average of over 2 billion people use the social media platform each month, with over 1.5 billion users daily.2 The popularity of Facebook has caught the attention of researchers seeking study participants. Many researchers, however, do not know where to begin when developing a recruitment plan that includes social media. The purpose of this paper is to describe the development and implementation of an evidence-based recruitment plan for the use of Facebook in research participant recruitment.

 

Social media has evolved as an important source of communication across the world. Although there are multiple approaches to defining social media, a widely accepted definition focuses on online social contact in three realms: blogs, social networking sites, and microblogging.3 Social media may be used for one-way communication such as the use of newspapers, TV, and online blogs or two-way communication with interactive online user profiles. A foundational concept of social media is communication. As technology evolves, new platforms for user networking continue to emerge. The challenge of the rapid evolution of communication technology for social communication results in no single definition capturing all of the nuances of the complexity of social media.3 Facebook is a social media networking platform that allows users to join without a fee for communication with individuals and groups across the world. Sixty-nine percent of adults in the United States have a Facebook account, with 73% visiting the site a minimum of once a day.4

 

A systematic review of studies that used Facebook for study participant recruitment identified 110 studies, although only a little over a quarter of the studies included the cost of Facebook ads.5 Two of the 110 studies utilized Facebook for recruitment without the purchase of ads. The average cost of Facebook ads per completing participant was reported to range between $1.36 and $110 (mean [SD], $17.48 [$23.06]), with overall cost averaging $2407 to recruit 368 participants.5 Representativeness of samples using Facebook for recruitment was consistent with studies using traditional methods.

 

There are two ways Facebook might be used for recruitment, passive and active. Gelinas and others6 distinguish between passive and active recruitment using social media. Social media passive recruitment would include placing ads on social media and developing recruitment pages in social media. Social media active recruitment might include a researcher directly contacting a potential participant based on the knowledge that the participant may meet inclusion criteria.

 

Reagan and others7 used an integrative review strategy to compare recruitment effectiveness for Facebook and traditional recruitment methods. Eighteen studies were identified that used both traditional methods and Facebook for recruitment. Interestingly, only 19.2% (range, 2.5%-80%) of study participants in the 18 studies included in the integrative review (N = 23 600) were recruited through Facebook. However, only 11 of the 18 studies used Facebook paid ads. Reported Facebook ad costs ranged from $377.69 (n = 115) to $11 103.25 (n = 1178). The findings report that Facebook can be a costly but effective tool for recruitment. It is noted that Facebook can be particularly effective for recruiting difficult-to-reach populations. However, comparison of the studies identified inconsistencies in reporting recruitment methods and results. Four focus areas are identified as important components of recruitment plans: planning, initiating, conducting, and reviewing.7

 

Developing Facebook Ads

Carter-Harris8 provided a detailed explanation of the meaning of terms related to the development of a Facebook page and steps for developing an ad campaign. An ad budget should be developed before placement of the ad. Facebook ads are developed in Ad Manager found in the Create tab in Facebook. The ad process in Facebook begins by developing a campaign and identifying the marketing objective. A valid payment source must be placed on file. Limits for the cost of the ads can be daily limits or lifetime ad budgets. The placement of the ad will require an understanding if a per-click or per-impression charge is preferred. If a per-click option is selected, there will be a charge each time a viewer clicks on the ad. If an impression rate is charged, there will be a charge each time the ad is placed on a Facebook user's page. The ad will have a maximum daily or life of the campaign spending amount established, and when the ad is ready for placement, the user will bid for placement. Facebook determines where the ad will be shown depending on an algorithm that is developed from the ad copy, the audience, and those competing for similar ad space. The maximum daily budget or life of the campaign budget will not be exceeded but could be less depending on how frequently the ad was displayed.

 

Facebook ads can be targeted to information available in Facebook user profiles. The audience selection allows the preference of age ranges, sex, location, and customized selections such as nurse, RN, or any area of focused recruitment. Audience selection limits where the ad will be displayed. Carter-Harris8 utilized a Facebook ad with a $500 lifetime budget established. In an 18-day period, 1121 Facebook users responded to the ad, resulting in 423 study participants completing 331 surveys.

 

In addition to understanding the importance of carefully structuring the ad, there are other concerns that must be considered early in the study design. Risks associated with participant misrepresentation, avoiding bots (applications that perform automated tasks), and ethical considerations are important when developing a sample recruitment plan.

 

Participant Misrepresentation

An understanding of the value of using Facebook for recruitment is evolving. Traditional researchers often had direct contact with participants, thus ensuring the participants met inclusion criteria for the study. More recent use of the Internet for participant recruitment has revealed challenges in ensuring the participants are qualified to participate in the study. Kramer et al9 reported that participants who misrepresent their qualifications for study participation create a threat to sample validity. Three strategies were identified to reduce the risk of participant misrepresentation: procedural, technical, and data analytic strategies.

 

For procedural strategies, researchers may limit access to Web-based studies by using unique usernames or passwords for access or, in situations when open access is required, by using inclusion questions as a stop before users are allowed to proceed. Requiring participants to meet inclusion criteria and collecting demographic information that may demonstrate insider knowledge are other ways of establishing procedural strategies. Asking participants for links to how they learned about the study may be another procedural safeguard. The risk of misrepresentation may also be reduced by not advertising the financial incentives for participation. Kramer et al9 report even small amounts such as $10 may trigger misrepresentation, resulting in the completion of multiple studies for higher earnings. If anonymity is not a significant concern, later drawings for gift cards can be considered. Technical strategies such as tracking of Internet protocol (IP) addresses, collecting date and time stamps, and limiting access to approved Web links provide a different approach to reduce the risk of participant misrepresentation. Further, data analytic strategies after the data collection allow for the evaluation of patterns suggesting misrepresentation.9

 

Avoiding Bots

The presence of bots may create an exciting moment for researchers as studies begin to rapidly appear followed by the despair of realizing the study has bot participants. Bots are automated computer programs that can complete multiple surveys within minutes.10 Human bots can also wreak havoc on a study by rapidly completing surveys possibly for incentives. The researcher who receives rapid survey responses must explore further to ensure the validity of the data collected. Signs of possible bot participation include the generation of multiple surveys in a short time frame and responses to open-ended questions that do not make sense. Yarrish and others10 recommend methods to avoid bot participation to include using Completely Automated Public Turing Test to tell Computers and Humans Apart (CAPTCHA), attention check questions, trip questions, open-ended questions, passwords, limited incentives, screening, and careful evaluating platforms used for data collection. Ongoing monitoring of data collection is an important step in early identification of compromised data collection.

 

Ethical Considerations for Online Participant Recruitment

Researcher transparency and respect for participant privacy are important considerations when developing a research recruitment plan using social media. Gelinas and others6 developed two checklists for social media recruitment, one for investigators developing a plan and one for institutional review boards (IRBs) reviewing social media recruitment plans. The checklists highlight the importance of research transparency in using social media for recruitment, including accuracy of researcher identities, accurate representation of the study, avoiding searching social media sites to target potential participants, and avoiding participant social media contacts that might create emotional distress. Ethical concerns for participant recruitment might include joining social media discussion groups to solicit participants, misrepresenting the intent of the researcher's presence in online discussion groups, and such behaviors as scanning individuals' posts on social media for possible study inclusion.

 

In summary, a review of the literature identified that important components of using Facebook for the recruitment of study participants include planning, initiating, conducting, and interviewing.7 The placement of a Facebook ad requires planning that includes an understanding of the recruitment budget as well as the goals of the ad placement. Participant misrepresentation can be avoided using procedural, technical, and analytic strategies.9 Automated completion of surveys is a risk associated with Internet recruitment that can be mitigated with entry and progress checks as well as monitoring of data collection.10 Ethical consideration of online participant recruitment is rooted in transparency and disclosure.

 

Developing an Evidence-Based Recruitment Plan

The recruitment plan for the study of the nurse perception of leader empathy was developed with the use of traditional methods and a plan for the use of Facebook. Traditional methods of study recruitment were initially implemented with the sharing of hard copy flyers at multiple meetings, word of mouth, and posting on professional organization sites when free access was available. Recruitment strategies transitioned from traditional methods to the use of Facebook. A passive approach to social media recruitment was utilized for the empathy study with the placement of ads on Facebook. A Facebook page was developed and reviewed by the university IRB with the initial submission. An ad/flyer was reviewed (Figure 1) by the university IRB and the university marketing department. The study was approved by the IRB as an exempt study as there was no more than minimal risk with the collection of the survey data.

  
Figure 1 - Click to enlarge in new windowFIGURE 1. Ad/flyer.

All Facebook ads were targeted to licensed nurses: RNs, licensed practical nurses, and advanced practice RNs. Secondary limits for Facebook ad placement were selected based on age range, sex, and geographical location. Ads were placed with Facebook from February to August 2018. In addition to using Facebook, a traditional ad was placed in one professional nursing association journal. No incentive for participation was offered. The study was developed and administered in Qualtrics, a cloud-based software for survey management (Qualtrics, Provo, UT). A link to the study was provided on flyers and on the Facebook page.

 

RESULTS

A total of 704 participants consented to participate in the study from February 2018 through September 2018. Of the 704 consenting participants, 536 answered all questions on a 24-item survey and completed the final submission process. A total of 168 surveys of the 704 surveys were not used in the evaluation of the results due to incomplete answers or failure to submit the final step, which re-confirmed study participation. The study participants represented a wide age and geographical range. Demographic information for the study was previously described.11 The total cost of Facebook ads for the 7-month period was $952.81, with a cost per completed survey of $1.78 (Figure 2). Facebook ad manager reported the total number of impressions to be 91 650 for the ad period. The cost of the association journal ad was $375.00. A minimal number of surveys were received during the time period the professional association journal ad was active. No snowballing of sample recruitment was identified.

  
Figure 2 - Click to enlarge in new windowFIGURE 2. Completed surveys to ad cost by month.

DISCUSSION

Researchers who need a large sample size find traditional participant recruitment methods to be slow and unreliable. A recent study of nurse perception of leader empathy estimated a needed sample size of 384.11 Study participants were asked to complete an online survey available using a link to Qualtrics. With a concern that an adequate sample size might not be achieved with traditional recruitment methods, the recruitment plan was developed to include the use of Facebook ads. A review of the literature for an evidence-based recruitment plan identified multiple studies that had used Facebook in the past.5,7-9 A Facebook page was developed for the study. The Facebook page introduced the researchers and displayed the ad reviewed by the IRB and the university marketing department. The Facebook page also provided a visible, easily accessible link to the survey in Qualtrics. The Facebook study page was developed but was not published prior to the IRB review, and a link to the page was provided to the IRB as part of the application package. During the 5 months of the study, the page attracted 123 followers and had 120 likes. A like on a Facebook page acknowledges the content, while a follower on Facebook can see new posts from the page on their Facebook homepage.

 

Traditional methods for participant recruitment were activated prior to the Facebook ad placement. Flyers were placed in nursing breakrooms, distributed at nurse meetings, and emailed to nurse leaders who might influence study participation. Sharing flyers and asking individuals to participate in the study were met with a tepid response on the part of the recipients. Survey responses after the distribution of the flyers were minimal. As part of the recruitment plan, an estimated cost quote was prepared for a print ad. Advertising cost for a newsletter with a reach of 84 000 was estimated to be $726. A decision was made to delay the use of the costly print ad and further explore Facebook ad placement. The initial Facebook ad was placed with a maximum lifetime cost of $25 and resulted in 1616 impressions. As familiarity increased with the ad placement process, the maximum spending allowances were increased. A total of 13 Facebook ads were placed over a 5-month period.

 

Approaches to the Facebook ad placement varied between per-click placements and impressions for exposure of the ad. Both per-click and impression approaches were used as the researchers fine-tuned the most effective campaign strategy. The ads were rotated between the approved ad (Figure 3) and the informed consent ad (Figure 4), with multiple changes to the target audiences to achieve a representative sample.

  
Figure 3 - Click to enlarge in new windowFIGURE 3. First Facebook ad.
 
Figure 4 - Click to enlarge in new windowFIGURE 4. Second Facebook ad.

The design of the recruitment plan included safeguards to protect the validity of the data collected based on the use of social media. The most significant safeguard was the absence of an incentive for participation. The decision not to use incentives was intended to increase the anonymity of participants and to reduce the risk of participant misrepresentation. One potential participant presented a concern on the study Facebook page that the decision not to offer an incentive was a failure to value the time of the participant. An explanation for the reason for the absence of incentives to protect anonymity of the participants was posted by the researchers and "liked" by other participants on the site. The concern expressed by the potential participant could potentially affect study participation but did not have an impact on this study.

 

Additional safeguards included using a feature in Qualtrics to allow only one survey to be submitted from an IP address. Self-disclosure of inclusion criteria was rechecked after the consent to participate in the survey. Open-ended questions were used both to seek a better understanding of the research question and to observe for potential computer-generated nonsensical responses. Open-ended questions were reviewed and compared to individual scores to determine whether there was consistency between the scores and comments. For example, a high score on the tool should be reflected with positive comments, while low scores would be expected to describe concerns.

 

Three areas were monitored daily to ensure tight control of sample recruitment and the cost associated with online advertising. First, charges in the ad account of Facebook were checked daily and compared to the number of survey responses received in Qualtrics. Second, surveys received were analyzed daily for demographic balance and any evidence of misrepresentation. Adjustments to demographic ad targets were made based on the ongoing analysis of surveys received. The third area of daily monitoring was the study Facebook page to ensure researchers had a live connection to participants; this was accomplished with timely acknowledgment of questions and posts from participants. To ensure a continued presence, the Facebook page was updated weekly to thank participants and encourage snowballing recruitment.

 

Based on the three-step monitoring process, the ad was revised frequently to change the target audience and parameters of how the ad would be charged, per click or impressions. Although placement of the ads was frequently revised, only documents reviewed by the IRB were presented. Targeted groups were consistent with IRB approval of inclusion of licensed nurses. Two ads were used, the visually appealing ad (Figure 3) and the verbatim consent form without visuals (Figure 4). Complete surveys generated by ads using the consent form compared to ads using the more traditional ad were notably higher. Although differences between the two ads could consistently be replicated, an understanding of why there is such a difference is not clearly understood. Informal discussions with Facebook users suggest caution in accessing links in ads, and the presence of the information in the consent form may have provided a needed comfort level of legitimacy to encourage accessing the study.

 

Thirty-seven individuals (7%) posted comments on the study Facebook page stating that they had completed the survey. Researchers verified each of the 37 posts was completed by individuals self-identifying as licensed nurses on their personal Facebook page. The participants posting comments reached out to the researchers with Facebook through their own initiative and were not solicited to comment. Surveys were collected and stored in the Qualtrics data system and were not linked to the participants responding on Facebook. Although some effort was made to include Instagram in the sample recruitment, minimal responses were observed. Other recruitment tools such as quick response codes and social media placements such as Twitter were not utilized in this study.

 

LIMITATIONS AND FUTURE IMPLICATIONS

Participants recruited using only Facebook may exclude some potential participants. Researchers who use Facebook for recruitment will need to be sensitive to the potential to exclude potential participants who may not have a Facebook account, who may lack the resources for or access to technology, or who may have low computer literacy. The use of traditional methods such as flyers or hard copy ads in parallel to Facebook may be considered. Insight into the effectiveness of each method can be gained by pausing Facebook ads while placing traditional ads and distributing flyers. An alternative to a computer survey was communicated to participants in the consent process. Participants were informed that a hard copy of the survey could be mailed if desired by the participant. No requests for a hard copy were received.

 

During the period of an international COVID-19 pandemic, the use of social media for research participant recruitment may be timely. As researchers consider using Facebook or other types of social media for recruitment, changes from traditional methods warrant careful examination. An awareness that social media is used solely for recruitment and not for data collection may allay the fears of the tentative researcher. There are some studies that may not be feasible for social media recruitment. Studies have unique needs to ensure the validity of the findings. Recruitment using social media can provide a wealth of opportunity for contact with participants when the recruitment plan is carefully designed and the risks and benefits are evaluated in advance by the research team.

 

CONCLUSION

Increased use of social media for research recruitment can be an effective strategy. It is important to review the evidence for developing an effective sampling plan including acknowledging the risk of misrepresentation and robotic completion of surveys. Social media ads require a significant amount of care from the researchers with daily monitoring of results and revision of ads when warranted. Naive placement of social media ads without understanding the cost could result in a significant cost to the study with minimal results. A good plan with daily attention to the cost trends associated with ad placement, monitoring to ensure participants are representative of the targeted sample, and a personal response to Facebook posts in response to the ad aids in improving recruitment effectiveness.

 

References

 

1. Facebook. (2019). Stats. https://newsroom.fb.com/company-info/ Accessed September 1, 2019. [Context Link]

 

2. Anderson M, Perrin A, Jiang J, Kumar M. (2019). 10% of Americans don't use the internet. Who are they? https://www.pewresearch.org/fact-tank/2019/04/22/some-americans-dont-use-the-int. Accessed January 26, 2020. [Context Link]

 

3. Treem J, Dailey S, Pierce C, Biffl D. What are we talking about when we talk about social media: a framework for study. Sociology Compass. 2016;10(9): 768-784. [Context Link]

 

4. Perrin A, Andersen M. (2019) Share of U.S. adults using social media, including Facebook, is mostly unchanged since 2018. https://www.pewresearch.org/fact-tank/2019/04/10/share-of-u-s-adults-using-socia. Accessed February 3, 2020. [Context Link]

 

5. Thornton L, Batterham P, Fassnacht D, Lambkin F, Calear A, Hunt S. Recruiting for health, medical or psychosocial research using Facebook: systematic review. Internet Interventions. 2016;4: 72-81. [Context Link]

 

6. Gelinas L, Pierce R, Winkler S, Cohen IG, Lynch HF, Bierer B. Using social media as a research recruitment tool: ethical issues and recommendations. The American Journal of Bioethics. 2017;17(3): 3-14. [Context Link]

 

7. Reagan L, Nowlin SY, Birdsall SB, et al. Integrative review of recruitment of research participants through Facebook. Nursing Research. 2019;68(6): 423-432. [Context Link]

 

8. Carter-Harris L. Facebook targeted advertisement for research recruitment: a primer for nurse researchers. Applied Nursing Research. 2016;32: 144-147. [Context Link]

 

9. Kramer J, Rubin A, Coster W, et al. Strategies to address participant misrepresentation for eligibility in web-based research. International Journal of Methods in Psychiatric Research. 2014;23(1): 120-129. [Context Link]

 

10. Yarrish C, Groshon L, Mitchell JD, et al. Finding the signal in the noise: minimizing responses from bots and inattentive humans in online research. Behavior Therapy. 2019;42: 235-242. [Context Link]

 

11. Stuart W, Moore B, Sims B. Understanding nurse perception of leader empathy. Nursing Administration Quarterly. 2020;44(1): 78-86. [Context Link]