Applied Nate Silver – Disaster! Not Believable

Week 19 is the worst ever! I literally could not believe what the scale said, something had to be wrong, I was still asleep. It simply was not possible! I still can’t believe it as it sinks in, but more, I can’t explain it. I want a fact-based world and life, and today the facts have slapped me in the face.


I’ve been doing fairly thorough record keeping and weekly reports. With an armband recording calories burned which I can cross-correlate with what my exercise machines show and correlate with my subjective experience I know fairly accurately my exercise levels. So, for instance, last week I set intensity records on both treadmill and elliptical and I broke monthly record (already) for stationary bike. I exercise all seven days and did vigorous snow removal one day. So no slacking off on exercise, if anything a small increase.

As to consumption, I didn’t go out to dinner at any restaurant, no chowing down on junk food. In fact, I went off to asian food store and got a ton of vegetables and made a “chow mein” type dish. I ate more fruit and vegetables than I have any other week. I had as few carbs as possible. I had zero red meat and consumed the equivalent of 1/6th of a skinless chicken breast per day, so almost no protein. The only snacking is almonds, dry-roasted.

So I should expect a pretty good week. My raw data through the BodyMedia system (calories-burned (measured) – calories-consumed (diary)) predicted 2.4lb loss. I use three different methods for my own prediction system and average them and that predicted at 1.7lbs loss. I had a goal/prediction last week of 1.5lbs loss. I was pessimistic going into weigh-in so I most recently predicted 0.8lbs loss but I expected to do better than this and was just hedging in negative way so I could have a pleasant surprise. And as baseline my historical rate over 18 weeks has been 2.4lbs/week loss and only once during that time (December holidays) did I have any gain which was only 0.2lbs.

So there is absolutely nothing that would predict what I actually saw. 2.3lbs GAIN! Ugh, disaster, impossible, how could this happen?!

This number is so out of line with any reasonable prediction it is impossible to sensibly explain. Let’s look at this graphically and then try to explain it.


(This miss is so bad it wipes out the last two weeks of losses. It’s only the second gain since starting program and it is huge gain in comparison. The linear trendline largely ignores this point as an outlier, so we have the contrast of a 2.3lb gain vs the long-term 2.4lb losses. But the point has enough impact it bends the arc (a favorite political metaphor now) upward in the 3rd order polynomial trendline. Bad news any way you look at it)


(Adding this week’s raw data shifted the trendline up a bit and significantly decreased its slope, but clearly the point still looks like a weird outlier. What happened? Either this is outlier or the trend is now completely broken. Can I explain this?)


My exercise, as measured by machines, was average of 1078 calories/day burned (that, btw, is a lot – see if you do that much). And this is adjusted down (one of the machine’s formula) from my baseline values to adjust for my new lower weight. The BodyMedia measured me at 3155 calories-burned/day, which is 979 calories/day above my sedentary requirements (thus in line with what exercise machines measure). This is down a bit from my 1385/day average over the past five weeks, but is considerably higher than when I was losing 2.7lbs/week in the first eight weeks of my program. So, in short, exercise was just fine, a little low, but at worst only about 20cals/day below my longer-term average.

So, lack of exercise was not the issue.

Now let’s look at this gain. It represents 8050 calories surplus. And my expected deficit was 8723 (calories-burned – calories-consumed). That is a total discrepancy of 16773 calories! Or, 2396 cals/day!, every day! Now 2396 calories is a lot, I mean pig-out level lot! Worst case fast food meals don’t go to that level. Worst case meals anywhere (some Cheesecake Factory shrimp pasta concoction) just barely hit that level. Now I think I’d know if I added consumption that massive, esp. every single day. My recorded calorie consumption per day is 1909 so this represents 126% of my recorded calories! Now tell me I just screwed up reporting calories by that much, esp. after 7 weeks of getting within 33% (worst case miss). Impossible!

The only thing I may not accurately record is some snacking on almonds. I do weigh (almost) every sample I get and record, but sometimes I may miss a few. But missing by 2396 would require eating 4+ cups of almonds, every day!, and not recording it. Impossible, I might miss 10 grams (1/10th cup) every now and then, but 4 cups – no way. And worse, not just once, but every single day!

So what can possibly explain this:

  1. the religious explanation: I have committed blasphemy against the nutrition gods by all the ranting I’ve done in posts this weeks and I’m being punished for that.
  2. the mechanical explanation: Something has happened to my scale and/or the floor it rests on during weigh-ins and so the data is wrong.
  3. the mystery explanation: Something I ate caused some anomalous gain (claimed to be just “water”) by my wife and this anomalous weight will quickly disappear.

I included #1 just to contrast with what many might conclude, that some external agency is responsible for this. I sinned somehow and am being punished. This explanation would actually be the most believable for some people, but I include it here just to show how silly supernatural explanations look when used to explain something like this.

#2 is possible but not very plausible. Nothing happened to the scale during the week, no jarring, no replacement of batteries, no damage of any kind. It’s possible the batteries are now discharging and it’s failing, but I took a large than normal set of samples and they all fall within the standard spread I’m used to seeing, so this seems unlikely.

So when you discard all the other explanations, whatever is left, however improbable has to explain it (or my lack of imagination is missing the real explanation).

So we have the mysterious “water” weight. Now this is real. It’s the reason that usually when you start a weight loss program you have some very rapid losses at the beginning that taper off. Back when I rowed crew and had to “make weight” I’d sometimes spend an hour in the shower with sweats and raingear just to sweat off a couple of pounds that I would instantly regain after the weigh-in. So despite not wanting to believe this (sounds like a rationalization) it seems to be the only explanation.

Or perhaps bok choy or carrots or snow peas or mung bean sprouts are more loaded with calories than fettuccine alfredo.

So my only candidate villain is soy sauce (plus some extra salt). Soy sauce is very high in sodium and I did use a lot in creating the sauce for the vegetable dish I ate 5 out of 7 days this week (my only meal each day). I did also have some popcorn, including some yesterday and I do put salt on that. And I’ve been craving salt lately and even adding more (my BP is lowest ever so I actually assume with my much reduced eating I could actually be low on salt).

So is that it? An increase in sodium that then triggered uptake (or holding) of water. I view my discrepancy as about 5lbs (my measured gain vs my expected loss). But that is 10 cups of water and that seems like a lot (although my wife’s explanation is all the vegetables I ate were mostly water and maybe due to the soy I retained their water).

This seems to be a testable hypothesis. If I now severely restrict salt intake (and don’t eat ANY of my leftover vegetable dish) plus maximize my commitment to minimize consumption the excise water should fairly quickly exit my system. My wife actually predicts I’d lose this tomorrow, but that seems unlikely (or at least I’d notice eliminating 10 cups) so it probably will take until mid-week to see any change. If, OTOH, I don’t see a fairly rapid decrease during the week, that probably blows this explanation.

But however I look at it, this is a disaster. Based on my traditional longer-term weekly weight loss it may take until the next report just to be back to where I was last week, so potentially I’ve lost two full weeks on my plan here.

And that is going to pose a challenge how to deal with my data, esp. the graphs. If I just continue by long-term trend (let’s say 2.1lbs/week) here’s what the graph would look like:


So even though the r^2 of the trendline (for all the data) isn’t that much different than the previous long-term trendline (red marks) the new trendline clearly poorly represents what is going on. In short I either need two line segments or I just start over and ignore all the previous data. So when there is an anomalous point that has a future (cumulative) effect (rather than being washed out with the next data point) how do I represent this with a meaningful statistical analysis (like, perhaps, adjust future data points by the anomalous amount?)

need some help here, Nate!

(I guess Nate is really busy because he hasn’t gotten back to me yet on my previous questions, but I know he’s out there so I’ll keep appealing)

But far more important than all this statistical work would be getting back on track. If my gain was just water + my projected loss is just being obscured by water gain then potentially my next week’s value should be (204.8-2.1 = 202.7) which would be 6.1lbs drop from today (seems rather unlikely), so let’s just assume I get the gain back (easily, a bold assumption) plus my worst-case drop I expected this week, IOW, 205.7, which would be 3.1lbs drop which is at least reasonable. So let’s assume it won’t be quite that easy, so I’m going to come in with 2.8lbs drop (my early long-term average + a smidge), which then results in:


(which is 1lb more than I predicted last week for this week’s result, meaning I am more than two weeks behind the plan, which is pretty awful).

Now, on the psychological side I almost lost it this morning when I saw the scale. I literally couldn’t believe it and even now, after all this data fiddling, it’s still a shock. This is the kind of event that destroys a diet plan. I am so discouraged I’m tempted to say to hell with it all and go buy half the sweets in the Starbucks counter and at least enjoy this gain (I was hungry all of last week, staggering to think I went through all that not just for “nothing”, but to actually do so badly (I would have actually really enjoyed ice cream and cake and pasta and steak and whatever it would have taken to consume 2396 more calories per day than I recorded). But the good news is that this point is so incredibly out of whack I just can’t believe it and therefore assume something is seriously wrong here and that magically it will be quickly erased (as my wife’s “optimism” suggests).

But if this week’s result is hard to reverse it really speaks to how bad April is going to be where I’m planning a vacation where dining out will be a big part of the vacation and I would expect to gain (plus have little exercise). So what is that going to be, a 10lb gain in one week?! It scares me that my losses, previously rated at 49lbs, are an illusion (even though I and everyone else can see a difference). That 18 weeks of severe dieting and extreme exercise can be so easily reversed. Or that my goal (another 20, now 24 pounds, more) is a illusion and utterly impossible (that I’ve hit the point where losses are hard and gains trivial, so up-and-down, I’m be lucky to stay on this plateau long-term).

So I’d better get some good news soon or I’m going to really be bummed and no idea how I’ll cope with that.


About dmill96

old fat (but now getting trim and fit) guy, who used to create software in Silicon Valley (almost before it was called that), who used to go backpacking and bicycling and cross-country skiing and now geodashes, drives AWD in Wyoming, takes pictures, and writes long blog posts and does xizquvjyk.
This entry was posted in comment, musing, prediction and tagged , , . Bookmark the permalink.

3 Responses to Applied Nate Silver – Disaster! Not Believable

  1. Nona says:

    I actually don’t know how much you lose nor what your goal is. Bear has lost 30 lbs since Oct 22nd with very little exercise as his company has been on mandatory 10 hours a week overtime since Jan 2. Nate Silver stuff really interests you, but I don’t always finish reading it.

    • dmill96 says:

      I know the posts are really long and full of trivial detail so it doesn’t surprise that no one can make it all the way to the end (I’m using the blog as a kind of journal for myself, not just to communicate to others where I would manage to be more concise).

      I’ve lost about 49 pounds and was expecting to make it past 50 today so I was really shocked to see an increase again. I’m slowing down on the loss but still potentially want to go another 20 (now 24) and then start working on maintenance. I’ve been ignoring a lot of the advice, especially about fundamentally changing patterns and instead focusing on fastest possible decrease and then worry about nutrition (which I’m really trying to research as deep as I can and find what is really true) and maintenance.

      The reason I mess with all the statistics is both the method I’ve learned to motivate myself but also searching for how I’ll manage maintenance. Like most people I’ve lost weight before only to gain it back, so this time I want it to be different so I’m really looking to change my habits, but also to add some easy-to-do monitoring technique so I can spot any regain and halt it quickly before I just slide into adding it back. I think once the weight loss is over, any uptick must be addressed as quickly as possible or it becomes an excuse for the next one and the next one and gradually you just drift back.

  2. dmill96 says:

    At least with one day’s data the most recent point does seem to be an anomaly but it’s going to take at least another day’s data to have a clearer picture. It actually looks like daily weigh-ins (despite the issues with scale variability) may make sense to spot unexpected effects like this, which is looking like the consumption of soy like week.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s