4 Things You WISH You Thought of BEFORE Releasing That Research Survey

Have you ever starting looking at your survey data and realized: “Whoops, I wish I would have done that differently?”  I have certainly been there and sadly I have worked with many faculty who have experienced that moment.  Together let’s take this moment to identify 4 common things we WISH we thought about BEFORE releasing a survey:

  1. IRB
  2. Identifiers
  3. Consent
  4. Reverse Design

IRB

Institutional Review Board (IRB) approval almost goes without saying, but we all know individuals who decided after the fact, “This would be great to publish.”  We need to learn from those colleagues.  Be proactive.  Our assumption should be that publishing is always an option.  IRB approval takes time, but much less time than creating a brand-new research project so that you can publish.

Identifiers

After getting data it often leads to more questions.  These questions will often take the form of longitudinal studies or looking at dependent variable correlations with independent variables.  These types of comparisons require some sort of identifier.  In direct terms this is often the difference between anonymity and confidentiality.

Anonymous means that all data includes no identifying values that can link the information to the participant; not even the researcher could identify a specific participant.

Confidential means the researcher can identify the participants.  Usually no one besides the researcher(s) will know the participants.  Researchers will often use codes (known only to them).

Generally, an anonymous survey means a researcher will only be able to look at that one group.  Confidential research might allow looking at a pre-test and post-test in a Paired T-test or to look at how a subject’s grade level, major, or gender impacts research findings.

Consent

Consent can be a very uncomplicated process.  Asking a subject for permission is after all a “yes or no” question.  I strongly suggest building your consent form into whatever system you are using to collect your data.  An online survey (Qualtrics) can have a consent form as the very first question.  You can make it so that “Agreeing” to the consent can the only way a participant will be able to complete the survey.  This saves the trouble of matching consents to responses and having separate files for consent and data that have to be maintained.  The IRB office at WCU has informed consent models on their web page.

Reverse Design

I have had multiple projects where I realized the minute I looked at the data that I wish I had asked one more question or included a different type of data point. There is a way to lessen this by using reverse design.  This means identifying the information you want from the data first.  Then thinking about what types of questions will get us that information.

  • Do I want to know if they did something? Yes or No question
  • Do I want to know how many times they have done something? Closed-ended numerical question
  • Do I want to know how satisfied they were with something? Likert scale
  • Do I want to know how one service compared to another? Ranking

All of these could be about the exact same interaction, but different measures.  Maybe I want to know all of the above.  But, one of the worse feelings is realizing that the questions that everyone answered are not answering the research question I wanted to answer.

Another tip:  Having some sample subjects take the survey before it goes live will also increase the effectiveness of your design.

I hope that sharing these 4 tips will help you as you think about and pursue your research.   For more information on these research tips or others, please contact Stephen LeBeau at 227-3456