Applied Nate Silver – Disaster! Not Believable – 1

Subsequent data has proven that last Sunday’s weight was an outlier, but now the question is what should I do about all my analysis?

So first let’s look at the proof it was such an outlier. I’ve now got three additional days of data, weights taken at the same time and roughly same conditions as Sunday:


The blue points are the individual weigh-ins on Sunday-Wednesday and the red points are the average of those. Note the continuing issue with variability (inaccuracy) of scale readings that make this problem harder.

It now seems likely that the Sunday anomaly was caused by water retention (of the various causes I researched contributing to day-to-day weight fluctuations this did, even though I initially dismissed it, seem the most likely). I consumed massive amounts of vegetables (magic food according to the police police) but to have some taste my dish also contained a lot of soy and also oyster sauce, so my sodium intake was elevated. I also added salted popcorn several of the days.

So mystery solved. Monday showed a big drop (Sunday I did my “regular” diet, Starbucks goodie in morning, Burger King lowest-cal junk food in the evening – all evil, of course, according the food police). Tuesday then had a small adjustment and Tuesday was my lowest calorie consumption day since I’ve been recorded, as close as I could come to a fast, but certainly purging the sodium. Now Tuesday, where the weight stabilized, as you can see from the graph, was out of plan. I had to get geodashing done or else miss this month, so on the way back through Lincoln the planned late lunch stop at a terrific but generally “healthy” restaurant was replaced with a stop of good Mexican place since the healthy place was closed. Not quite a pigout but I recorded ~1400 calories for that single meal and that might have been understated. But worse Wednesday was close to a no exercise day, so it’s not surprise, despite the 938 calorie deficit I recorded in BodyMedia that Wednesday is a no-change day.

So mystery solved and now things are stable and I can try to get back on my plan and I’ve learned my lesson – massive amounts of vegetables cause weight gain. So another myth debunked, just as previously I demonstrated whole grains cause weight gain.

Now it’s not fair to claim either of those oh-so-highly-recommended foods cause gains. What is fair is that they’re not a miracle and that consuming a lot of them is just as bad as consuming a lot of fries at Burger King. In short, calories are calories. Had I eaten much smaller portions of the wonder foods it would have been the same as eating smaller portions of Burger King, less is better, what food is mostly irrelevant.

But now back to the Nate Silver part of the question. It’s surprising that week 19 was the first time I showed an anomalous blip in my otherwise nice statistics and graphs. I firmly believe it is a blip as we can see below with more data:


Here the blue dots are again the raw values from the scale and the red dots (a few removed to generate reasonable trendline) are the averages. I did the trendline using the stable Tuesday data instead of the anomalous Sunday data (or Monday where it was still stablizing).

So what value should I use for Sunday?

need some help here, Nate!

The raw value is so far off the trendline it messes up all projections. And I can’t just let “big data” (law of large numbers) solve the problem of filtering out outlier points because I just don’t have enough data.

So here’s my possible solutions:

  1. use my “predicted” weight for Sunday. I predicted 205.0 which was based on multiple prediction techniques and seemed the most reasonable, but that is purely a prediction and mixing it in with measured values seems really statistically dishonest.
  2. use the average from Tuesday, 204.8, or potentially the average from Tuesday and Wednesday, 204.6 on the logic that this is the stablized weight. However, based on BodyMedia recording by Wednesday my weight should have dropped another 1.0, so backward extrapolating 205.6 would be a predicted guess for Sunday.
  3. use the trendline (from chart above) to interpolate for Sunday yielding 204.4, which was below any prediction I had made and thus seems too optimist. But also look at that trendline – it looks shifted downward a bit due to the noisy data.
  4. use a average of all these: #1 – 205.0, #2 – 205.6, #3 – 204.4, so thus 205.0, which seems reasonable but nonetheless is still a cheat, since it is an imputed rather than measured value.
  5. use nothing (just drop the week 19 value). This is looking the most attractive. Saying decisively that value is “wrong” and also recognizing I have too little data to wash out its statistical effect is reasonable, not just wishful thinking. But picking any replacement value is totally arbitrary and inconsistent with all the other data. So I think that’s what I’ll do which means I have nothing to look out until I pick this coming Sunday’s value, which hopefully will be back on plan and the trendline.

So this “disaster” that had me quite upset on Sunday seems to be fading but what are my lessons learned here:

  1. massive amounts of vegetables are bad (of course the food bullies will deny this and claim I wasn’t a true believer and of course should have eaten my massive quantities raw and piled on a much of quinoa and lentils and oats as well)
  2. that despite all the normal notions in diet plans of weekly weigh-ins as the appropriate method of monitoring progress it does look like I should try to get daily (at least some) data as well. That would be a lot easier if I could just take one weight but that continuing issue of scale variability (a spread of about 1% which is also about the same value as any signal) implies the somewhat tedious process of getting multiple weights per day BUT maybe here I can let big data help by still using Sunday weights (with sufficient samples) as the value to use in long-term trendline and daily (and less accurate) weights as an early warning indicator of some drift off plan.

So hopefully I can rest easy after this scare, plus I can keep up my mental energy and commitment to use this week to “catch up” for the lost time by very strong fasting and exercise.

I’ve been ranting about all the nonsense nutrition “experts” are putting out because I still need solutions and all this bad “advice” isn’t helping. Weight loss is the relatively easy part (as all my posts on this show) and I’m relatively confident I’ll make my goals, even with some temporary blips off plan. But maintenance is the big challenge. Like most people I’ve lost weight before and since I’m doing this I’ve obviously gained it back (or more). Now I must succeed at staying low since that’s why actual valid nutritional information is so important to me and why I’m so frustrated with all the terrible information available.

I know that long-term I need more balanced food intake. I know long-term I can not sustain my level of exercise. I know I’ve ignored “nutrition” and only focused on weight loss and long-term I need to pay attention to proper nutrition (if I could ever find out what that is). And I want to live, not starving all the time. Before being a weight loss fanatic I was a foodie, loving to try new food, esp. having gotten into bread baking. I don’t want to give that up. That meal yesterday at the restaurant was a real treat, way more enjoyable than merely the quality of the restaurant.

I want to be able to eat food that tastes good and is fun to eat. I don’t want food to be original sin and I have to eat awful stuff just as penance and I don’t want to be doing self-mortification like so many of the food nuts are. Eating repulsive food as part of some life-denying health kick or trendy fad is not what I want. This doesn’t mean I want junk food ro that I won’t “improve” what I was doing, but I just don’t want any of these unscientific, unappealing self-flagellating awful diets that so many advocate.

If I’m going to worry about getting healthy (healthier since I actually don’t have verifiable health problems) and therefore live longer I want to LIVE not just have more empty and unhappy days on this planet.

About dmill96

old fat (but now getting trim and fit) guy, who used to create software in Silicon Valley (almost before it was called that), who used to go backpacking and bicycling and cross-country skiing and now geodashes, drives AWD in Wyoming, takes pictures, and writes long blog posts and does xizquvjyk.
This entry was posted in comment, musing, prediction and tagged , , . Bookmark the permalink.

One Response to Applied Nate Silver – Disaster! Not Believable – 1

  1. Nona says:

    not being scientific at all – vegatables have a lot of water – that seems to be the mysterious change to me.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s