We often hear about “evidence-based approaches” in education, health care and mental health treatments, drug and alcohol addiction treatment, conservation, etc.

What counts as evidence? Is all research equally valid and trustworthy?

Maybe not.

Gazzaley and Rosen (2016) note that the “highest level” of evidence comes from positive research findings from a “randomized, placebo-controlled, double-blinded study” (randomized control trials or RCTs) with a large number of participants; with both laboratory and field research; with evaluations of positive and negative effect sizes; and much independent replication.

Much of the research I’ve been citing has relied on their participants’ self-reports. As Gazzaley and Rosen discovered in their research on distraction and “multitasking” (really, task switching), participants dramatically underestimate their levels of distraction and overestimate their own ability to “multi-task”. So self-report studies might point to interesting findings, but on their own are very likely to seriously underestimate the effect of technological distractions on their functioning (ability to pay sustained attention, to remember, and efficiency and productivity)

Even the experimental research I’ve been citing in these blog posts was limited by the fact that they could not be double, or even single blind. (The participants were randomly assigned to the control or experimental condition (texting, or Facebooking in class), true, but of course participants couldn’t help but be aware which group they were in, so placebo effects were possible).

The Lab at the University of Southern California (San Fransisco) is conducting very carefully constructed research studies into brain training through education and even computer games.  Check them out!  Encouraging findings. neuroscape.ucsf.edu/technology

and a research article on their results at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4264379/pdf/nihms643909.pdf