Computer says "Not Significant"

As someone who hasn't done a Masters between my undergraduate degree and my PhD, the last time I was remotely responsible for conceiving an experiment was during my fourth year dissertation project.  Even then, it was guided by supervisor's interests and expertise.  Given the technological dramas I've had until this point in my PhD, the experiment I've been working on this semester is really my first 'baby'.

This was a project based entirely on my own interests.  I'd constructed the design from the questions I wanted to ask and the literature in the area.  I painstakingly created the materials.  (Twice).  I collected the data (twice).  The whole process took nearly two semesters.  A learning process, granted, so I'll be faster next time, but either way - a long part of this year has been spent on this one study.

So when we ran our first analysis and the computer was throwing out non-significant after non-significant result, I was gutted.  Truly, awfully gutted.  I'd invested so much time and effort and stress and work into this experiment.  How could it not have worked?

Before that experience, I'd never truly appreciated how devastating it can be when a study you've worked so hard on doesn't really come to fruition.  I thought it made a good topic for a post though, because if you've been through it too I thought you might like to commiserate.  Everyone deals with this sort of thing differently, and remember that this is just a reflection on how I felt about these things.  My perspective is quite probably different from yours.  That's okay too.  But if you've felt this way at all, sometimes it's nice to know you're not the only one ;)

So here's how it was for me.....

First of all, no one outside your academic circle will really understand.

This statement comes with a caveat, but it's an important one to remember.  Your family, partner, friends, dog, hamster will all understand that you are upset or stressed but they won't really get why.  Those lucky people who aren't doing (or haven't done) a PhD, don't really understand how pervasive the PhD becomes in your life.  The frustration and disappointment can become all-consuming.  You'll think about it lying awake at night.  In your office.  In your car.  At the cinema.  It'll keep popping into your head.  I mean, I'm doing a PhD and I don't understand how it happens.  I have a life outside my PhD, but there it is, all the time, in the back of my head.

And for that reason, however hard it is, try to be patient.

When you've explained for the fiftieth time how things went wrong, that you do/don't know what caused it, and the implications for your work, you will probably be ready to punch someone in the face.  (Though that might just be me...)  Try to remember these people are asking because they care, not because they're trying to remind you of your perceived failure.  (And that perceived is the key word there, more on that in a minute).

If it's getting really tough, stick to a strong, unequivocal statement: "I really don't want to talk about my research just now.  We need to talk about something else.  How is you [work/love life/dog/sibling/mother....]?"

You will (if you're like me) feel a crushing sense of self-doubt.

A PhD is a high-stress, high-pressure route to take.  When things go wrong, even if they're not your fault, the sense of guilt or failure can be strong.  If you've identified the problem and you really did make a mistake, do your best to address it.  If it can be fixed with some more testing, a rejigging of the analysis, do it.  Working on fixing the problem will make help you keep going.

But whether the mistake lies at your feet or if it has some unidentified source, you need to remember:

This is not the end of the world.

It might feel like it.  Thoughts have the tendency to run away with themselves.  An example from my own head: "If these results aren't significant that means I can't present them at the conference or submit a paper.  That blows a chance of a publication.  If I don't have enough publications I'm not going to be able to compete on the job market.  If I can't compete, I can't get a job.  That would mean this has all been a waste of time!!"

See what I mean?  I feel stressed just reading that.

But it took a gentle reminder from one of the lecturers here that, in science, 98% of experiments won't get significant results and that is okay. You've got to remember that a PhD is an apprenticeship - a chance to learn how to research.  

A good friend and fellow PhD student also reminded me that a PhD is often a series of firsts.  Your first time running a particular paradigm, analysis, using an experiment builder, eye tracker, whatever.  You don't always know what went wrong because you're still just a little science grasshopper.  It's not really until you defend that you have sufficient knowledge and understanding in your area that would allow you to go back and modify the experiment to (hopefully) get the results you'd wanted.

Ultimately, when things like this happen, the hard truth is that you've just got to keep going.  I'm not denying there were a couple of days I called up my aforementioned good friend asking her to talk me off the quitting ledge.  But you keep going.  You keep trying to figure things out.

Eventually, with enough poking and prodding we identified the problem.  The program I used to run the experiment had been showing different stimuli than I though, so once I recoded for that we were able to rerun the analysis and hey presto, we found something.  But that's not really the moral of this story.  The moral is to keep going.  You are always learning something, whether your results are significant or not. 

Keep going, you can do it!


EDIT: So I maybe didn't make this clear enough, but this post was meant to be reflective of my experiences when I didn't get the expected results for my study.  I thought it might be useful if anyone else had felt this way too.  If you haven't, that's great and don't worry about it!  And, to reiterate, I don't think that if your results are non-significant (i.e. not below the arbitrary alpha of .05) your work is meaningless.  I was only saying I felt that way at the time.  With a little perspective and some friendly reminders from friends and colleagues, I realised this was just another way to learn something new about the area I'm interested in.  Hope that clears things up!

Katy

Since Not Just Another PhD is nearly at 10,000 hits - a big milestone - I'm hosting a giveaway over on our Facebook page.  For your chance to win a SIGNED copy of Piled Higher and Deeper (PhD) comics book, head over to the Facebook page for details on how to enter!