Hi,
In 2002, Yahoo was still the most popular search engine. The first iPhone was five years away from being launched.
2002 was also the first time I read a few keto studies and tried out the diet. At that time, there weren’t a ton of keto trials on topics outside of epilepsy.
It didn’t take me long to realize that keto, like most diets, was neither “bad” nor “good”. Net benefit or harm is determined by a variety of factors that vary by person.
Alas, dichotomous thinking about diets predominated then and continues to do so. I try to avoid those yearly “best and worst diets” lists put out by major media sources. They’re a great way to generate clicks without encouraging critical thinking.
To complicate matters even further, diet trials tend to be much more difficult to interpret compared to supplement trials, due to the greater number of variables at play.
Some wrinkles to be aware of regarding keto trials
-
Keto typically leads to several pounds of water weight loss due to glycogen depletion. If that’s not accounted for, a trial’s body composition results could be misleading, since water loss isn’t fat loss. Most trials from the past few years account for this water loss, yet results are still commonly misinterpreted by media sources and the public.
-
Keto and vegan diets are two of the most common diets that exclude specific foods. They share a key characteristic: each reduces the number of calorie-dense and/or hyperpalatable food options you have. So then you have to figure out what portion of the benefits seen in the trial are due to the specific diet being tested, and what portion is due to weight loss from less availability of hyperpalatable food. For some people the benefits are mostly from the reduced calorie-dense options, and for others it’s mostly the specific food included or excluded in the diet.
-
One of the major downsides of these diets is lower long-term adherence. Trials can show that a diet with specific rules works in a semi-controlled setting for six months, but real life isn’t a trial where you’re under implicit pressure to adhere. Controlled settings measure efficacy, while real-world settings assess effectiveness. Some people will be able to stick to these diets indefinitely, probably because they don’t find the restrictions onerous at all, while others won’t.
-
Diets that exclude specific foods typically lack flexibility. Still, some people do best having a simple non-flexible rule to follow every day because if they incorporate flexibility, they tend to break instead of bend. Other people do better when they can be flexible about their diet.
-
Using a single result from a keto trial hides the important inter-individual variability in results. Here’s a graphic we made for a big low-carb versus low-fat study years ago. Some people lost tons of weight while others didn’t lose any, or gained some weight. Notice that the spread of results was remarkably similar between the low-carb and low-fat diets.
And the most important wrinkle of them all?
There’s no such thing as a placebo diet, so trialists have to pick a comparator diet that keto (or whatever diet of interest) is tested against.
This has a huuuuuuuuuge impact on the study results! If you test keto versus a low-fat diet, vegan diet, DASH diet, or some other diet, the results could easily go from significant to not significant (or vice versa).
Testing more than two diets in a single trial tends to be prohibitively expensive, so it isn’t often done. Instead, the trialist has to carefully pick the comparator diet. Do you pick a “generally healthy” diet because it’s the most broadly applicable at a population level? Do you pick low-fat because it competes with low-carb at a macro level? Vegan because it’s kinda sorta the opposite diet?
Mediterranean diets often do well in trials — although it’s debatable how authentically Mediterranean some of these diets are — so if the trialist is brave, they can pit keto against Mediterranean.
A month or two ago, I saw that some keto dieters and academics on the web were excited about a trial comparing keto to a Mediterranean diet for psoriatic arthritis, with keto winning out. I filed this away in my brain with the mental tag, “exciting result!”
I should have tagged it as “exciting result?” You see, I forgot to check this really cool website called Examine to see if they had covered it yet. Just last week, I ran across it on the site. It’s still a promising study, but I’m not sure if the people who previously shared the study read the full text. Examine researchers noticed:
Some caution may be appropriate when interpreting these findings, due to the small sample size and relatively high dropout rate (39%). Also, outcomes were listed with incorrectly calculated percentages. Rather than calculate percent changes from baseline to the end of the diet period, the investigators calculated these percentages backwards (i.e., as if time went in reverse) and then reversed the direction of effect (e.g., from an increase to a decrease). This resulted in some percentage reductions being more than 100% (which is impossible).
In addition, 2 of the authors were employed by Evivios Med., the company which provided the ketogenic snacks.
The subtle art of keto research interpretation seems to have a prerequisite: you have to read the actual paper. Most people don’t have time or expertise to read tons of full text papers, and artificial intelligence summaries can’t catch things like the above.
So don’t be like me. Before mentally filing a trial away as “exciting!” without adding in some qualifiers (and definitely before telling other people about an amazing study that came out), check to see if Examine covered it. If we didn’t, let us know and we may queue it up.
Sincerely,
Kamal Patel
Co-founder, Examine