Critiquing the LessWrong Mineral Supplementation $5,000 Contest
| December 2, 2011 | Posted by JosephB under Roche, self-experimentation |
This is an initial critique that may form the basis for a submission to the contest here: http://www.medicineispersonal.com/contest/home and http://lesswrong.com/lw/8nx/announcing_the_quantified_health_prize/
It’s a very cool company and idea, and I wish them well. Don’t take my disagreement to indicate otherwise. I have no expectation of winning. I’m only writing this because they kickstarted my brain, and I needed to get it out.
The proposed methodology does not match the question’s scope.
You want to know, based on scientific literature, what the optimal mineral supplementation regimine is, given personal variation, and the shape of the tradeoff curves for various configurations.
That is a vast, vast question. Even a superficial, a priori examination reveals vast numbers of permutations, potential interactions, etc, extending in a multi-dimensional matrix.
Now let’s examine the nature of the evidence with which you propose to answer this question.
Western evidence-based medicine depends on controlled trials to produce statistically significant results. This system suffers from several flaws -
1. Bias – Academic careers and large studies are both expensive. Funding is directed by corporate, government, and academic guild interests. This results in a pervasive pro-agribusiness, pro-pharmaceutical, pro-allopathic medicine bias in the literature. Non-patentable and dietary cures are therefore significantly under-represented.
2. Myopia – One intervention is studied at a time in isolation. This is absurd, since the human body is a complex integrated organism.
3. Brevity – The expense of tracking lengty longitudinal studies is usually prohibitive.
4. Homogenization – Searching for a statistically significant effect by applying an intervention to a sample population inherently assumes that the population is relatively homogenous. This may not be the case.
5. Cartel behavior – The FDA, NIH, and AMA create a highly regulated environment in which competition is stifled, regulatory capture occurs, and cartel behavior is rampant. This impedes responsiveness to market demands, and encourages profit maximizing behavior at the expense of consumer well-being. Research paper output is one part of this larger phenomenon.
6. Signaling – Researchers in the academic guild are largely cut off from market pressures. Instead, they navigate a terrain shaped by corporate and government interests. Researchers signal their guild status by producing highly technical, useless information. They advance by publishing papers that are cited by many other researchers, and by behaving in ways that please their ultimate sponsors (although not too obviously – that would be a crass guild faux pas). The mutual citation and insular cooperative promotion structure ensures that negative trends are self-reinforcing.
For the above reasons, most scientific medical research is blinkered, wrongheaded, and obtuse, if not outright pernicious.
For example, evidence based medicine “discovered” that the synthetic compound Accutane was effective at curing acne. Statistical trials showed that it worked. However, what evidence-based medicine ignored was the logical and historical evidence.
In fact, Accutane was chemically almost identical to Vitamin A, a dangerous substance with a well-known toxicity profile. Cursory examination of Accutane’s side effects would’ve shown remarkable overlap with Vitaminosis-A.
Instead, evidence-based medicine recommended Accutane for a certain subset of young people, and no long-term damage trials were done. Then allopathic medicine plus pharmaceutical mass marketing and regulatory judo pushed Accutane out to a much wider general population. The resulting psychiatric, teratogenic and long-term side effects were catastrophic.
Meanwhile, natural and safe remedies for acne existed, which the researchers of the evidence-based medicine guild ignored. Not least because the FDA actively prohibits doctors from recommending natural substances that have not gone through an expensive FDA screening process as cures for any medical condition. That screening process, incidentally, is half funded by the pharmaceuticals. Technically, it is illegal for a doctor to recommend an orange as a cure for scurvy.
All of this is to say that the entire Western evidence-based, heavily regulated, corrupt, cartel, allopathic medicinal system is hopelessly broken. Therefore, depending on the research output of this system to answer the question of the optimal human mineral supplementation regimine is a bad idea.
Instead, the proper approach is a holistic one, that incorporates logic, historical evidence, longitudinal self-experimentation, and a healthily skeptical approach to the available body of scientific research and mainstream therapeautic practice.
In other words, we should assume that the vast quantities of available scientific research do not in fact mean that we know anything about the optimal human biological inputs. That is a question too large for the myopic and biased data available from officially sanctioned sources.
It is extraordinarily easy to demonstrate this, once we zoom out of the myopic perspective that scientific research employs by default. The world is larger than 1st world populations eating a standard Western diet. We will not find the optimal set of biological inputs by studying this population, because it is demonstrably less healthy than other populations that have existed, and in some cases still exist, around the world. I am referring to hunter gatherer societies, where most diseases of civilization are unknown, or become prevalent only after introduction of modern dietary influences.
One key observation we can make about these populations is that they are not taking any mineral supplements at all. Instead, they get their food from the environment in which they are living, and to which they are genetically adapted. So the first logical question becomes, why should we supplement mineral intake at all? The fact that this question is literally beyond the scope of the prompt demonstrates the pervasive faultiness of Western evidence-based medicine, and is particularly ironic for a contest hosted by proponents of rational Bayesian thinking.
Let us take as our default hypothesis that the optimal set of human biological inputs is some combination of paleolithically available foods, with variation for population and individual genetics, and individual medical conditions and lifestyle objectives. Any deviation from this default hypothesis should be supported by evidence.
The first thing we observe is that paleolithically available foods are not identical to modern available foods. Since we are focused on mineral supplementation, let’s start there. Even if one eats nominally identical ingredients, mineral content is very different for agribusiness produced meat than for paleolithic game. So it is not sufficient to duplicate the list of ingredients; we must also pay attention to the quality of those ingredients.
Now that we are zoomed in on minerals, let’s note several other complicating factors.
Evidence-based medicine typically does not differentiate between plant and animal sources of minerals. However, there can be very significant differences in the bioavailability between the two, rendering daily recommended values and nutrition tables meaningless. Moreover, mineral intake and required dosages are also impacted by the rest of the diet, and by overall biological inputs. And minerals also interact with each other.
Therefore, we are looking at a huge number of interacting factors, which evidence-based medicine is extremely unlikely to disentangle via experimentation. Nor is our understanding of the biochemical mechanisms of action anywhere near complete enough to take on such a challenge. For all we pretend to know, the human body and its dynamic interaction with the environment is still a black box.
In order to simplify our approach, we need paradigms and methodologies that work. I’ve already described one paradigm – the paleolithic hypothesis. This has good logical and historical support. And we have one methodology – self-experimentation. This method is not myopic, because the individual is aware of all his inputs, and it is also longitudinal and non-generalized. The addition of these two would go a long way towards correcting the inherent flaws in the status quo.
So, can duplicating a paleo diet, in both ingredients and quality, result in optimal mineral intake? Unquestionably, yes. But that does not adequately answer this question, because it is insufficiently specific. Nor does it address the difficulty of obtaining sufficient minerals from modern food sources.
Up next – seafood paleo importance, shoreline/aquatic hypothesis, substantial evidence (paleo, scientific, and logical) for shellfish mineral content, my personal experimentation results supporting AAT, scallops daily 300g as the answer (with paleo diet), followed by optional self-experimentation if desired on supplements. Discuss methodology of self-experimentation, quantification of results, e.g. twitter exoselves, wordpress, and vitality tracking.
UPDATE: Seth Roberts has linked here. His take is worth quoting:
The next time someone praises “evidence-based medicine”, ask them: What about Accutane? It illustrates how evidence-based medicine encourages dangerous drugs. You can’t make lots of money from cheap, time-tested things that we know to be safe (such as dietary changes) so the drug industry revolves around things that are not time-tested and therefore dangerous — far more dangerous than dietary changes. Evidence-based medicine, which says that certain tests (expensive) are much better than other tests (cheap), provides cover for this. Because the required tests are so expensive, they are allowed to be short.
[I will endeavor to answer criticisms up to three strikes per commenter.
Firstly, this is my opinion, tossed off in a few minutes, with no attempt to provide citations or research. Were I attempting to say only what I could conclusively prove, I would've said far less.]
Got to say I disagree with a lot of this.
You hold up Accutane as a failure of evidence-based medicine, but it’s hard to see how you could justify this. Obviously Accutane has a lot of horrible side effects including teratogenicity. Obviously it’s similar to Vitamin A and causes similar side effects. But this is a success of evidence-based medicine, which discovered all of these side-effects practically as soon as the drug was created, and was instrumental in spreading knowledge of them to the doctors who would be prescribing it.
[It's not at all hard to justify. Differences of opinion aside, you are simply unaware of the facts. You make multiple ludicrously false statements about Accutane's history. Why would you do this? Please see this link: http://www.aquatic-ape-diet.com/blog/hoffmann-la-roche%e2%80%99s-cover-up-of-accutane/
That's why there are draconian restrictions on its use, including (in the US) the demand that isoretinoin patients sign a pledge promising not to begin pregnant before they can be prescribed the drug. Right now doctors explain all of these side effects to patients before they can be prescribed the drug, and make sure patients still want the drug even though they know they are undergoing these risks.
I don't know what else you want mainstream medicine to do other than demand that the drug be banned entirely
[The FDA has admitted it would not have been approved under later guidelines. So yes, I think it should be banned under the current regulatory system, according to that system's rules.]
- and since you accuse the FDA of being a cartel,
[The FDA is not a cartel, it is a regulatory government body that protects cartels.]
I expect you agree with me that denying patients the right to use an undeniably effective drug because someone else decides it’s too dangerous isn’t usually a good idea.
[I don't make the rules. I criticized the system and its corruption. I did not offer an alternative, for the obvious reason that it is not relevant.]
I’m also a little turned off by your objections to evidence-based medicine on principle – as if you’re not distinguishing between “the current medical establishment” and “the practice of gathering evidence to support conjectures”.
[That is an epic mischaracterization of my argument. Evidence based medicine is a well-defined movement with books about it. It has nothing to do with your second quotation. See here: http://blog.sethroberts.net/2011/11/06/testing-treatments-eight-questions-for-the-authors/
The latter seems valuable, whether or not the former practices it. If you think there's a big difference between plant and animal sources of minerals, this sounds like a hypothesis that could be tested by experiment - and not necessarily just a contrived experiment with a sample size of one and inevitable confounders.
[Yes, it could and should, but such experimentation is underperformed for reasons I stated. You are not reading in context. I am pro-research, but I am not pro- status quo.]
I agree that self-experimentation is a useful technique for when larger studies have not been done, or when you’re worried about your own particular genetic profile, but I don’t accept that there’s anything inherently superior about it.
[Superior depends on the metric used.]
And inevitably, the same factors that make formal studies difficult (hard to control someone’s diet for their entire life and then see if they get diabetes at age 60) aren’t good for self-experimentation either (if you get diabetes at age 60, the knowledge that you probably should have done something different 30 years ago isn’t so useful).
[Not true, as I explained in the article. You speak in far too many absolutes.
That is more than 3 strikes, so I will not be engaging you further.]
“You don’t provide evidence for a lot of your assertions about the problems with the pharmaceutical research and regulation system,”
[Yes.]
“which makes me think that you are relying for your information on sources aligned with “alternative medicine”.”
[It should make you think that I wrote this in one sitting without opening my browser.]
(A give-away is your use of the term “allopathic”, a word with no scientific meaning which was coined by a homeopath and is used only by the alt-med community).
[The term is useful, regardless of its origin. It means mainstream Western medicine.]
This is not to say that there are not real and profound problems with the pharmaceutical research and regulation system, just that you may have been exposed to exaggerated and over-strong criticisms from authors with both ideological and pecuniary interests in talking down evidence-based medicine in favour of quack remedies.
[My reading habits are irrelevant ad hominem.]
You need to bear in mind that absolutely all of the methodological problems you point to with respect to evidence-based medicine apply to alternative medicine as well, except with vastly more strength
[Depending on how you define methodological, most of my criticisms may not fall under methodology. One that certainly does is myopia, which is necessary to generate statistically significant results. Self-tracking corrects this problem throughout holistic qualitative awareness, thus permitting the accidental discovery of effects, such as the difference that taking Vitamin D in the morning produces. By any definition, your "absolutely all" statement is false.]
because alternative medicine and the “holistic” self-experimentation you prescribe do not have the benefit of large sample sizes, blinded observation and statistical significance.
[You fail to comprehend that these methods themselves generate problems.]
Modern humans didn’t exist long enough for us to be absolutely sure that we were a local fitness maxima even for our evolutionary environment.
[That is either trivial or conjecture.]
However even if we were, it would not follow from the premise that we are the ideal organism to exploit a Palaeolithic diet that the Palaeolithic diet is the ideal one for our organism.
[I never argued so. I said it should be the null hypothesis.]
It could perfectly well be that we are an organism that can usefully assimilate far more of a given mineral than our Palaeolithic ancestors generally had access to, for example. Or it might be that far less of a given mineral is actually better for us but that our ancestors did not have the choice of avoiding that mineral.
[Yes.]
It’s also worth saying that the idea that hunter-gatherers are in better health than modern humans seems highly likely to be the product of faulty reasoning. If a population consists entirely of healthy specimens, it could be because they are such that they are all healthy or it could be that something is killing off all the unhealthy ones.
[That is a fine logical hypothesis, but incredible to those familiar with the area. The burden of proof is on you to show that the absence of diseases of civilization is due to early death of the unfit. You are wasting everyone's time by advancing it. Read Weston A. Price or many other available works on the subject.
That is three strikes for you as well, although a better performance than the first one.]
Tyvm for writing about your experiments and observations with a skeptical and critical tone of science and medicine status-quo. I appreciate your candor and assertiveness, although they have a slight ring of defensive superiority (which I don’t entirely understand).
That said, ultimately there are very few who are trained effectively in the scientific method, much less deeply understand it. Secondly, and to your point, you cannot really use those methods effectively when researching specific causes and causal chains in dynamic complex systems (i.e. chaotic systems) such as human biology (which is an attempted “subset” of the much larger organisms picture).
And since people have to make decisions today with limited information (not evidence…most people never get close to evidence…they read other peoples supposed interactions and observations and second hand elevate those via the labels “facts” and “evidence”), they want to do so with confidence and with deferred responsibility. That sets up the general mentality of avoiding self-experimentation and replying on “science” and “medical” experts to give them the confidence while the experts also take on implicitly the responsibility for all negative outcomes.
This simple psychological self-referential loop is what has caused so much of the “wait until they get sick” mentality of most of western medicine. And given the profit motive is so strong in the scientists and medical researchers and practicianers, and given the government force of the major agencies combined with the government encouraged and enforced monopolies of patent, copyright and trademarks, it’s a self sustaining self-deluding self-denial cycle that has no foreseeable resolution.
I feel very grateful that I have been able to see this and step away from it. And then re-enter on my own terms and with my taking complete and total responsibility for my health. My doctor does not prescribe, he consults with me. I make the choice to submit to the tests, to fill and take the prescription, to return if/when I decide. I don’t know of very many people who choose this balance. Most either completely submit to the “expert system” of scientists and physicians, or completely avoid the system all together. For me, both are extreme and result in avoidable undesirable outcomes.
Again, thank you for sharing from your own N=1 experimental perspective. I value your sharing very deeply.
[...] http://www.aquatic-ape-diet.com/blog/2011/12/02/critiquing-the-lesswrong-mineral-supplementation-500… [...]
Thanks Jim. Clearly you’re a very smart guy. Congrats on managing your relationship with your doctor.
As for the other thing, this isn’t my first internet rodeo, nor my first encounter with LessWrong. There was a time I would’ve patiently tolerated a multi-round exchange with the above sort of interlocutor, but no longer. The three strikes rule is to keep me intellectually honest – any critic gets a chance to take his best shot.
[I'm not going to address the new errors you introduce, other than to observe that you continue to fail to read in context.]
Since the author has constructed an “intellectually honest” system under which they don’t have to respond to my criticisms I will keep my response short.
[Yes, I have demonstrated that you are not credible on this subject.]
“Holistic qualitative awareness” means laying oneself wide open to the placebo effect, the observer effect, wishful thinking and confusing correlation with causation. In my view the reason why evidence-based medicine works and why ayurveda and homeopathy and pre-evidence-based Western medicine with leeches and mercury and such are all useless is that the non-evidence-based systems relied solely on “holistic qualitative awareness” while evidence-based medicine did the much, much harder and more laborious job of using the “myopic” scientific method to find out how the universe actually worked.
(Also your accusation of myopia is somewhat unfair, since the more statistically sophisticated scientists are quite capable of doing a multivariate regression analysis to test the effects of multiple factors on an outcome. The idea that scientists can only change one variable at a time is mathematically outdated, although it’s fair to say many scientists haven’t caught up yet).
[This is a defense of one of your original errors. Sophisticated statistical methods increase the number of factors that can be studied, at the expense of additional data collection and analytical ambiguity. This increase does not begin to bridge the gulf between the holistic awareness of self-experimentation and a typical large sample size study. Statistical analysis inherently requires abstraction. Hence your qualification of "somewhat" to unfair. You are now backpedaling.]
Second lastly, you have a funny way of presenting a null hypothesis when you say “So, can duplicating a paleo diet, in both ingredients and quality, result in optimal mineral intake? Unquestionably, yes. ” If that’s a null hypothesis I’m a banana slug. That’s an unsupported conclusion.
[Another continuation of an original error. Failure to read in context. Optimal here meant "healthy and free from diseases of civilization", not "bodyhacker optimal."]
Lastly, you think you’ve found an interesting result with your self-experimentation with Vitamin D. That’s great – lots of good science started with one unusual observation or a well-chosen bit of self-experimentation. Have you considered trying the same experiment with a larger, blinded group, to see if the effect works on anyone besides yourself? Have you even tried blinding yourself, to see whether taking the pill at different times actually makes any difference when you don’t know whether you are taking Vitamin D or a placebo at the time?
In my experience people who are actually interested in science are actually interested in doing these kinds of more rigorous tests. It’s not hard, and it doesn’t take any special training or expertise. It’s just applied common sense. The people who aren’t interested in science but are interested in posing as scientists find some excuse like “I only care what works for me, who cares about larger sample sizes?” or “I only care what works for me, who cares if it’s a placebo?”.
>In other words, we should assume that the vast quantities of available scientific research do not in fact mean that we know anything about the optimal human biological inputs.
If by “know anything” you mean “have any information”, then this is obviously false. If you mean to use a stricter definition of ‘know’, then this is obviously (almost trivially) true.
Regarding your reply to PhilosophyTutor, you seem to have missed the important bit where “Unquestionably” was given as a description of your “null hypothesis” – that makes it sound a bit more conclusiony.
Yes, that part was poorly written – it was a stream of consciousness first draft. Still, reading in context would’ve avoided that interpretation.
Something you are obviously incapable of doing, given your autistic binary approach to “know anything”.
The problem with LessWrong is they think math can replace having a functional right brain / social adjustment. Hell, they think it can IMPROVE on it.