Applied Nate Silver – 5

I have a bunch of new data and some disparate but related developments I’ll combine in this post. I’ll start with my “money graph” which is the focus of all this attention. [Note: my “money graph” is the “bottom line” of all these other statistics and analysis. It represents the consensus of all the data relative to the longer-term of this plan.]

The red squares represent a very regular dataset before the anomaly that occurred in week 11 (green point) and represent a weekly change (weight loss) of 2,.7267lbs. Immediately after the anomaly I did “catchup” in the following week (green point in week 12) and now I have a somewhat more long-term “sustainable” point for week 13 and the slope (weekly weight loss) but it still represents an unsustainable value of 3.8lbs (I know it’s “unsustainable” not from the data analysis but from first principles since I know my plan and capabilities). OTOH, the new long-term average may be a bit too low (2.5995/week) but the difference between the full duration average and the earlier data is probably not statistically significant.

need some help here, Nate!

(How would “statistically significant” be computed when comparing the slope of two regression lines, one with more data and the rest of the data in common? No idea how to compute this, so I’m just using the term “statistically significant” inaccurately, but with a different between the two slopes of 0.1272, or 4.9% I think that difference is somehow within the error of my measurements, but, OTOH, Nate, how do I compute that. Hey, Nate, if you’re pushing for accuracy in the world of prediction, you could help me out here since this is beyond the stat classes I had).

Now the prediction part:

1. 214.9 (new long-term), 2.3lb loss
2. 213.6 (old long-term), 2.6lb loss
3. 214.5 (current weight, old weekly weight loss), 2.7lbs loss
4. 213.1 (unsustainable short-turn “catchup), 4.1lbs loss
5. 214.2 (current weight, most recent weekly loss) 3.0lbs

Now using  a crude version of how Nate used “average” of all the polls, here’s the “consensus” (just simple average) prediction:

214.1 (3.1lbs loss)

which I believe, based on more than statistical knowledge is too optimistic. But due to my new fanaticism of record-keeping (more later) I’m going to say the old rate of loss is a little too low and this week was a little too high, so I’ll actually predict:

214.4 (2.8lbs loss)

So stay tuned for next week’s report to see how good I did on prediction as well as sticking to my actual regimen.

Now I have some new tools and data to add to all this process. My wife decided I needed more help (plus she loves to buy things at Costco) so last week she got me a BodyMedia armband sensor and 12 month subscription to their Activity Manager website. This takes up the statistics gathering a notch, plus for people who weren’t already doing monitoring by the means I’d already been using, a method of achieving more discipline in their fitness plans.  BodyMedia claims a 3X improvement, which I suspect could be true if: a) before using BodyMedia you were 100% guessing about everything and probably not staying on plan (like the typical person with a New Year’s resolution to achieve weight loss), and, b) if you fanatically use the BodyMedia system, especially all the calorie tracking. So let’s look at how some data I have would have predicted today’s outcome.

#1. I used three difference exercise machines, each of which records calories burned, which with semi-fanatical usage added up to 7966 calories (over six days). This figure doesn’t include any activity (above sedentary) so potentially understates my burn. OTOH, since it doesn’t match the BodyMedia very well I could also believe BodyMedia’s claims of 5% accuracy and then conclude the machines are wrong. Anyway, using my recorded calories (at BodyMedia site, sometimes using their nutritional data, sometimes using other nutritional data, sometimes guessing) I get 9788 calories ingested. Now according to an online calculator for my current weight and “sedentary” activity I need 2228 calories/day, or 15596 for the week. So I ran a calorie deficit of 5808 (the effect of “dieting” alone) which combined with exercise means 13,774 stored (flab) calories burned or 3.8lbs of weight loss. Now my weight loss was only 3.0 lbs, so that means about a 3000 calories discrepancy (425/day). So either exercise, as reported by machines, is overstated, or, calories consumed (manually record at BodyMedia site) is understated, or some of both. I feel relatively confident that my error in recording calories consumed is less than 450/day so therefore I have some reason to believe that calories burned, as reported by machines is overstated.

#2. Now the BodyMedia site claims their sensor is a much more accurate measure of calories burned. But is it? The sensor contains: a) accelerometer (which seems to record treadmill fairly accurately; elliptical, close; recumbent bike, seems to miss), and, b) body sensors – sweating, heat flow, skin temperature. Presumably these body measurements detect exercise even when it doesn’t trigger the accelerometer. [BodyMedia is notably silent about how they generate their results and I find it a challenging concept to convert what their sensor measures to calories burned, but with no more explanation (I assume they think it is proprietary, but I’d like more disclosure) I have to take what I get.) In my crude/anecdotal correlation (this will be subject of future experiments) I think BodyMedia sensor (and data analysis algorithm) understates my exercise (IOW, I believe machines more than it) but it also overstates calories burned from ordinary activity, like washing dishes. So perhaps these offset each other and law of large numbers applies and thus the overall result is not so bad, so let’s just look at it (esp. given last Sunday’s calories consumed data was correct, but I didn’t wear the meter until nearly the end of the day so calories burned is its guess). Anyway it generates a net burn of 10,365 which translates to 2.9lbs loss. And that, as we now know is closer to the result the scale measured so I guess BodyMedia wins.

So, until I do more correlation (which will be hard as I’ve already got some experience and the data is really noisy, so I’ll need lots of Nate Silver) between the machines and the BodyMedia. For instance, look at Calories Burned as it computes:

Some of that is exercise (treadmill and bike, treadmill sometimes “easy”, other times, “vigorous”) and some just ordinary activity (cooking, washing dishes). The only good correlation is that my known couch potato times do show up as fairly minimal actitivity, but, OTOH, washing dishes shows up as more intense than even a deliberately maximum intensity (although brief) workout on treadmill.

Now in terms of my “perceived level of effort” I already have a fairly good idea on intensity ranking of my machines:

1. elliptical at almost any workout level, but especially when I do some “sprints”
2. bike, when I do maximum level (like the old days on a real bike climbing up to Skyline) but since bike is fairly new I can’t sustain high levels very long (even like the climb up Old Woodside Road)
3. treadmill, when I do maximum sustainable effort. In this case ‘maximum’ is based on some restraint due to injury I’ve had and stress treadmill type workout puts on my old joints. I can actually do a higher level (and probably sustain longer than either bike or treadmill) at comparable perceived level of effort but I don’t to avoid injury.
4. bike, routine pedaling (more or less like my old conditioning rides on Foothill Expressway and Junipero Serra)
5. household chores (actually some exercise upper body more since 1-4 are mostly lower body) but nothing I’ve done in the week with BodyMedia montoring me has been as high perceived level of effort as my exercise.

So the BodyMedia system seems to be working (it certainly triggers me to go do more exercise every time I eat something so my “Calorie Balance” stays where it should be (my 2lb/week loss plan = -1000 balance/day, but I’ve managed to average 1728/day, which, btw, would then be about the 2.9lbs loss BodyMedia data predicts. So if I want to achieve my “predicted” weight for next week this will mean actually doing 1440 “balance” average for the seven days of this week, so next week I should have another number to look at. I’m a slave to this thing (for a while at least) so having a better target number will keep me going. For instance, as of 1.5 hours ago (when I uploaded the sensor’s data) I’m only at 831 calorie deficit so that has induced me to at least walk around a bit outside (a thrill, given it is about 18F and I’m not wearing my jacket) so now I want to upload again to see if I’ve burned off the Starbucks snack now.

[After a crash and reboot while attempting to sync BodyMedia I’m now safely ahead with 1150 calorie balance (and thus my fanatical need to achieve these goals achieved, for the moment) and a prediction of 3003 calories for the day, safely below my average of 3359 (during six full days of recording and heavy exercise), but this shows the problem with BodyMedia. Today is effectively a “sedentary” day and thus closer to 2228 calories the calculators predict so BodyMedia is way too high today, since I was averaging over 1300 calories/day exercise (as machines report) vs only 356 (measured with activity over predicted with no activity). So there is still a lot of work to do getting my own predictability with the BodyMedia system]

So until I do more detailed tests on attempting to calibrate exercise and BodyMedia this is about the maximum I can glean out of this week’s data, so see you next week, Nate.