## Applied Nate Silver – Disaster! Not Believable – 5

Hey, Nate – how do I predict anything when my data looks like this?

Is any of this real and is it just noise? These data are my average (of seven samples) weight for each day, starting with my “official” weight last Sunday. The question is what prediction can I possibly make for tomorrow.

So let’s look at my results on exercise and calorie intake:

Note there is a deficit every day, averaging 1262 calories (equivalent to 1/3lb per day) yet thus far for these seven days there is a net 0.1lb decrease from a total calorie deficit of 7572. Obviously the measured weight doesn’t track exercise/consumption over the entire six days, IOW, not just a one or two day variation.

But most significantly look at the negative correlation. On the fifth day the calorie deficit was lower than plan, only 898, yet the weight loss measured that day was 1.8lbs, the biggest all week. Then for the sixth day the calorie deficit significantly exceeds plan, 1535, yet the measurement shows a weight gain of 2.5lbs! So weight and calorie deficit are strongly negatively correlated which, of course, violates all the principles of biochemistry.

So everyone is going to say, big deal, there are daily fluctuations, don’t worry about it. BUT, how can you possibly monitor the situation with these kinds of fluctuations. AND even worse how do you stay motivated to be hungry (sometimes a lot) every day plus then be in some pain doing all that exercise (5936 calories burned, over 10 hours) yet the consequence is essentially no change in weight. It’s just too discouraging.

So let’s pull back and look at this in the longer-term:

Here we have a couple weeks of data with the red dots being the daily average and the blue dots being the raw data from the scale (note the large variability just in that). This clearly shows that yesterday was anomalously low and today is anomalously high (the rightmost two points) relative to the trendline.

So the short-term (6 days) trendline would predict about 197.3 for tomorrow and the longer-term (3 weeks) trendline would predict about 196.6. Or, IOW, the average of those two predictions would be another downswing of 1.7lbs from today. But swings of that magnitude are relatively rare so that seems unlikely.

And if we look at a bit further (10 weeks, 8 shown on the graph below):

We’d also predict about 197.7, which actually seems like the more likely value. So even though long-term my weekly loss is 2.2lbs (about plan) because last week was anomalously low IF I actually come in at this predicted level that would only be a 1.1lb change (with > 2lbs predicted by the BodyMedia measurements and food log). In short a whole week of a ton of exercise and a lot of hunger and at best I lose a single pound. And that’s not even a sure thing!

So what do I predict for tomorrow, Nate? I’ve got all this data and lots of analysis but I’m at a total loss to even begin to guess.

BUT, I know this – I can influence the outcome. If I eat “normal” (my reduced intake) and exercise normal (high but not extreme) I will go up tomorrow. And if I try for severe reduction today (1500 calories, meaning no meals at all today) and I push for extreme exercise I might get that “water” fluctuation burned off and thus actually achieve the whopping 1lb loss. So an unpleasant day today to accomplish almost nothing – oh joy. Makes me want to give up.

Now what surprises me is how other people deal with this. I doubt very many people have gone through all the analytics I have, which allows me to actually see much of this data as noise. Two weeks ago I was only doing once-a-week weighin and when that was anomalously high it freaked me. I went into overdrive with more data collecting which has only resulted in more noise and less signal. I went into a stronger commitment for the loss leading to several days of almost total fasting and < 1000/day consumption. So, in essence I “caught up” to my long-term trend. But now it’s back to slowing down again so while overall I’m down it’s like it requires extreme vigilance and extreme deprivation to even have modest losses. And it’s so easy to bounce back up. My “maintenance” regimen is going to have to be way more strict than I hoped. And to stick with that for another 20 years! It’s discouraging.

So. On the day where I had the big drop we were doing a little celebrating (first warm day in months) and I had no meals but instead add popcorn and drank bourbon and I had great weight loss. The next day, I ate “healthy” and exercised a lot and gained a bunch. One day was fun, the other day not so much. The “fun” day was more successful. So what’s my motivation here – unpleasant days and failure?