Consumer brain recording technologies make all sorts of claims that are simply not substantiated in the literature at the level of individual predictions. Will FTC fines be coming in the future?
A recent Neuron paper [1] out about a week ago counted 41 direct to consumer wearable neurodevices on the market of which 22 are EEG recording devices and 19 are direct stimulation devices claiming to do various different things from relieving stress, enhancing mood and improving sleep to increasing concentration, improving productivity, enhancing memory and enhancing physical performance. Really?
Where’s the evidence?
The authors attempted to find the research referenced by these companies that supported their claims. Out of the 41 devices they found links to research for 33 of these devices but only 8 referenced studies specific to the device. The rest had references to general scientific papers not specific to the device and in some cases, even entirely irrelevant to the claims being made. So what is the evidence for these broad claims? Are they supported in the scientific literature? Here are some major issues:
Quality of the signal
Before we can even get into the whole question of what can be interpreted from the EEG, there is the first concern of whether they record sufficiently good quality data to make any sense of it at all. Consumer devices such as Muse, Neurosky and pretty much all the others use dry electrodes which have notoriously bad contact quality, and therefore impedance values, fluctuate wildly. With no real dry sensor innovation in them and no studies demonstrating the quality of their signals by comparison to gel based systems, it is pretty safe to assume they are poor quality EEG signals. They may be just good enough to identify population level trends when the numbers are large enough for signals to emerge out of the noise but simply not good enough to make accurate estimates of EEG features in real time for any individual.
Interpreting the Signal
Assuming some of them had good signal quality, what can you really interpret from the EEG? The EEG literature is rife with correlations between various signal features and every brain state and mental condition. Yet there are some general issues that plague the field – thus far for most conditions there is little if any research out there that has sufficient specificity to make claims with any certainty about the state of any individual at any given time. Rather the vast majority of studies compare two groups or conditions showing differences of 20-40% between their means or correlations on the order of 0.4 to 0.6 for the feature of interest [3]. What this means is that you might be able to make a claim at a population level, but when it comes to the predicting anything at the level of the individual you probably won’t do much better than if you just tossed a coin. When it comes to a wearables, of course, population level data is useless if it can only translate to being accurate for you once in a while.
Uniqueness of a Change
Another challenge is that many different conditions deliver the same results, particularly when it comes to spectral bands. Spectral bands, carved up arbitrarily into the famous alpha, beta, theta and delta bands, form the basis for the claims of many of these devices. However, these are a very macro feature of the EEG signal (see The Blue Frog in the EEG) that fluctuate with just about everything. Just because one condition increased or decreased some spectral band relative to another condition in some study, it doesn’t mean it’s the only conditions where it is changed in this way. For any change associated with one kind of state such as sadness or poor focus or relaxation you can find a whole bunch of other states that do the same thing (see this review for examples). So any interpretation of a real time signal is doubly guessing at the cause of the change.
A third challenge is individual variability. People have their own unique profiles that differ from one another and also fluctuate over time. This is one of the reasons population level data is so difficult to translate into predictions at the individual level. It is also one of the reasons there are so many inconsistencies in the literature and contradictory studies.
A look at the literature around specific areas
You can take a look here at what the literature tells us so far on some things like Sleep, Anxiety, and Depression. Altogether there are promising possibilities that are suggestive at the population level, but are in no way conclusive. So if you take poor signal quality added to weak correlations between EEG features and any particular state and many states providing the same result, what you get out of it as a consumer is essentially junk.
However, many of these devices may be banking on the placebo effect – if you believe enough you are changing something, you probably can change it. It’s just that it doesn’t matter if it’s with the device or a rock.
Who’s minding the space?
Apparently, in 2016 the FDA issued a guidance that it would not enforce regulations for low risk devices marketed for general cognitive enhancement or wellness purposes [2]. (Stimulation devices may be a different story however see Zapping Away Maladies of the Mind with tDCS). Nonetheless, the watchdog for these claims around EEG devices is therefore the Federal Trade Commission (FTC), which has the authority to take action for unfair or deceptive business practices. They are the ones who slapped companies like Lumosity with a $2 million fine for deceptive advertising on its brain training program. Will there be similar fines coming for these wearables?
References
[1] Owning Ethical Innovation: Claims about Commercial Wearable Brain Technologies, Iris Coates McCall, Chloe Lau,Nicole Minielly, and Judy Illes, Neuron May 22, 2019
[2]Mind-Reading or Misleading? Assessing Direct-to-Consumer Electroencephalography (EEG) Devices Marketed for Wellness and Their Ethical and Regulatory Implications, Anna Wexler & Robert Thibault J Cognitive Enhancement August 2018
[3] EEG Frequency Bands in Psychiatric Disorders, Jennifer Newson and Tara Thiagarajan Frontiers in Human Neuroscience, December 2018
Dear author,
Nice and interesting.
It would had made the all article more interesting, as well as it sounded easy and feasible, to proof the first preposition (Quality of the signal).
Excellent anyway, building a different opinion from the main trends, as it seems enthusiasm and marketing are playing a bad role here.