I’ve become a real fan of Nate Silver. I was aware of him back in 2008 when he first received much notice for 538 and then again in this election cycle when his projections eased my fears of impending doom of a Romney victory. But it is his book, The Signal and the Noise: Why Most Predictions Fail – But Some Don’t. New York: Penguin, 2012. ISBN-13 978-1-59-420411-1, that has really grabbed my attention (wow, in looking this up Amazon seems to have declared this “Amazon’s #1 Best Book of the Year for 2012” so I guess my view is shared by millions.
In the times I’ve seen Nate on TV, frankly, I was unimpressed. But in his writing he is entirely different, way more articulate, witty, amusing, and definitely engaging in the extreme (at least to a geek). I often read books where it is clear the author has done some quick background research, probably just Wikipedia, and then includes concepts that are outside the author’s expertise (such as seriously misusing the “efficient market” concept). But Nate is scary smart. In his book he comments on fields that are clearly far outside his background but in those where I have some expertise I find not only no flaws or mistakes, but also serious insight into sometime subtle concepts. This adds tremendous credibility, for me, to whatever he says, and of course an actual track record of being right when other “experts” were disastrously wrong certainly helps. Amusingly in the book he warns both the reader and himself of over-confidence and seems to really live the lesson he’s trying to teach us.
Now the basic subject of the book is actually not new to me. I’ve had multiple courses in both undergrad and graduate level in statistics and prediction and modeling. My post-education work has often involved using these skills so the amount of “technical” material I’m getting from Nate is minimal, BUT, the practical wisdom I’m getting is huge, esp. in seeing how he approaches a variety of prediction environments and the observations about flaws and limits of prediction, which, of course, is exactly the subject of his book. Fascinating and way way more insightful than something like Freakonomics or Gladwell’s various books.
OK, so what. How does any of this relate to my title? Well, I have a practical problem where applying a little of Nate can really help. I have a choice in my life, lose 60+ pounds and keep it off or start stabbing myself with needles. Since I have a huge aversion to needles, but also the side-effects of high glucose levels, weight loss and lifestyle changes are no longer optional. And I need some help pulling this off.
In various things I tried in my life I have found measurement, analysis and predictions are very handy tools for my own motivation. I really got into this when I decided to do a triathlon. I was so far from being able to do it that it was going to take a couple of years of training (even learning how to swim). And to stay on that program I was going to need some way to measure progress and set intermediate goals to keep myself motivated. Of course today most athletes use all sorts of numerical analysis in training, but it was a bit less common when I started this 26 years ago and there weren’t any very useless PCs and certainly none of the training program or data analysis software. So I developed my own metrics I could easily measure and combined them into several formula to determine my progress. Back then I wrote my own software and that was hard enough I did NOT add any degree of sophistication on the statistical side, although I often did use regression and a few other types of simple statistics.
So now I can apply more of what Nate has to tell me to my current goals. So far I’ve done reasonably well without many aids (how hard is it just to drastically reduce food intake, esp. some of the bad ones, and increase exercise) and measure a single simple value, my weight. But what about Nate’s basic discussion, signal vs noise. Is my single one-dimensional measurement correct at projecting my success in this program?
Now since I’d been planning a weight loss program for a long-time I knew, from past programs, that exercise is critical to me. I’m better at burning calories than I am at avoiding ingesting them so exercise is the behavior I can more effectively use. I also know myself: I must be truly committed before starting something or I will fail and until some scary A1C readings I didn’t have the commitment. But at least I’d been “preparing” (aka “stalling”) for about 25 months and thus had extensive data already about my exercise, conveniently available since it is on machines in my basement and thus easy to measure. So a spreadsheet already loaded with exercise data is my baseline. And so it’s easy for me to state that: a) I increased by about 50% my daily calorie burn since retiring, and, b) I increased by about 100% my daily calorie burn since my scare. So with some degree of certainty I can state I’ve increased my daily calorie burn by around 500 calories, but frankly this “estimate” has too much fudging in it that would get me some scathing criticism from Nate. But I won’t have to do to much in this area to improve my data and analysis.
But now we get to the part where Nate is going to help me, actually measuring weight and projecting future results, like when I’ll achieve the 188lb target that is the common upper-bound of “healthy” weight, although for me a bit too low to be practical (when I was in great shape, young, and rowing crew, 176 was my “natural” weight and I’m actually a bit larger (in a good way) now so 188 would be “skinny” for me and I’ll settle for merely “healthy” which I think will be about 195, esp. given the exercise I’m doing where muscle mass (at least lower body) is reasonably substantial. So am I making progress and how soon will I get to my goal (whereupon my new goal will be how to maintain that weight for the rest of my life, way harder than merely losing weight).
So we start with the question of how to measure weight. Now I was already doing a couple of things: 1) a good quality digital scale that reads to 0.1lb (but is that really its “accuracy” or “resolution”?), 2) weighing at the same time of day and conditions (whether just eaten or not), and, 3) without any non-organic (i.e. clothes) parts to throw off scale. So, fine, that should be enough, take one measurement every (day?, week?, hour?) and record. BUT? Based on my history I decided daily weighing tended to screw me up (it was “noisy” in the past, both real changes on 24 hour basis and the highly inaccurate mechanical scale I was using (plus my error in reading an analog dial). So I’d measure weekly, esp. since I have this outing at Starbucks (my Sunday exile) on a regular basis. Weekly should be low-enough resolution that I should hit any discouraging upticks (either signal or noise) and have nice steady gratifying practice (a decrease on every weigh-in). Well, thus far that has worked. BUT!
My first issue is that my nice digital scale gives a range of readings. First, I have to store it in a different place than where I actually have space to do the weighing (i.e. a confined bathroom where scale has to be moved). I have to make this choice since: a) scale needs to be on hard surface (not many of those in a house with rugs), b) I’m naked and not keen on doing weigh-ins in the kitchen. But the “hard surface” I use is actually a composite floor with some texture so I’d already noticed some variations in weight. Plus just over the span of a few minutes, plus moving scale around, plus how I manage to squeeze my relatively large (compared to scale area) feet onto the scale, plus how I stand. In short, a fair amount of variation. So, this morning, as a quick Nate Silver experiment here’s a little data. I moved the scale multiple times, weighed myself several times, then moved both the scale and me and waiting a bit, so these data are over about 15 minutes.
Where is the noise and signal in this?! What is my weight? Now applying “first principles” / “intuition” / “underlying model” it is unlikely I gained a pound a 15 minutes as the regression line shows. And I doubt the scale suddenly changed. And I’d be really surprised in G0 changed. IOW, this is all noise by any reasonable analysis and the trend line is meaningless, which its r^2 of 0.2 reasonably indicates. Fine, lots of noise, but where is the signal. Another quick measurement indicates the mean is 235.1 with a standard deviation of 0.7325, so probably my weight is somewhere between 234.4 and 235.8,
Now one of the things that simple statistical thinking and Nate’s advice would indicate is that I should get more data, esp. under a broader range of experiment conditions, but I also should do something about my scale calibration to know the “normal” distribution of values for a dead weight (am I moving around too much, positioning my feet different each time, having different posture, and thus my readings are out of the error bounds of the scale itself measuring some dead weight; how much of the variation is the different locations on the textured floor). And why do I care? Well, the problem is that based on the “long-term” data I currently have (and it’s crude) my weekly change has been about 2.7lbs (pretty good) and so the “noise” in this measurement is a substantial portion of the “signal”.
So how do I get more accuracy? And why should I care? The second question is easier: a) steady weight loss is positive reinforcement for me and encourages me to continue to be hungry and tired all the time, and, b) failure to lose, (or horrors!) any gains, would be huge disincentive, esp. in a weekly weigh-in. (Starving and sweating all week for nothing!)
So more accuracy has to be in the cards and so thank you Nate for some help.
Here’s the “money” graph that really matters.
This is my six weight “guesses” over the last five weeks with the very gratifying 2.7lb/week weight loss. Despite the high r^2 though clearly there is some variation in the points from the trend line. But of course there has to be some “real” (signal) variation in my weight loss (after all, last week for Thanksgiving and that meant: a) pigging out, and, b) no exercise (other than the work to feed 15 people)) and there is definitely some noise in my measurements. Now back on my first measure I knew some of the factors in weighing that were causing variation so I took more than one value, but unlike doing what Nate would insist on, I made a “biased” guess (undoubtedly somewhat influenced by my desire to get results, a la the Repug “forecasters” who are now enjoying Romney’s landslide). But does the bias in my guess matter?
So let’s look at my “prediction” now. For the moment I’ll assume I can maintain a linear decrease in weight and therefore I hit my target in 15 weeks. (I’ll make a guessestimate now that the probably of that is about 0.05 as there are all sorts of reasons why I’ll miss but I’ll put guessed probability at about 0.02 it will be less than 15 weeks and about 0.95 it will be more; in 15 weeks I can come back and test this prediction).
But that’s also the point. I don’t want to wait 15 weeks to test my prediction since my choices actually cause these values, or, IOW, if I start falling behind I should do something to correct my failure rather than just passively measure it.
So what about today’s weight value which I estimated (my first graph was after I had “measured” and part of an experiment) was 235.7, which is in range of my later experiment and potentially a little high.
So let’s plug in the mean of my experiment and repeat my projection.
Not too bad, only reduces my “prediction” to about 14.3 weeks and raises my weekly loss to 2.8 lbs. So, let’s throw in upper bound on the measurement (no, waste of time since that would be only 235.8 vs the 235.7 I originally used, so let’s use worst case actual measure, 236.2). Well, not much change (less than 1/2 week to goal, 0.05 lb/week drop). And “best case” is, 2 weeks sooner to goal and 0.3 lbs/week.
So, today’s number does matter, although it appears my original “guess” was fairly conservative vs the statistical analysis I tried.
So what am I going to do, at least for a while. I’m going to increase the simple experiment I did today and that is to take as many measurements as possible per day (but only one at any given time because I was “biased” in the way I took readings, even in my experiment) and then use the regression line over multiple days to make my longer range projection (since daily “real” weight is bound to fluctuate with some noise, not just the measurement noise).
And then my real challenge is to decide when to drop an “linear” projection. I know, from first principles, that my loss will not be linear, even with the feedback of measurement altering my behavior to attempt to maintain linear.
First of all, all the stuff I can find on the Net indicates that my daily calorie requirements will change, at minimum, with my weight. And “change” means drop, i.e. as I lose fat I will need less calories to maintain that fat. So if somehow I managed to exactly manage my intake and exercise exactly the same (at least over a week) my weight loss would slow down steadily as I lose weight.
Second, my “determination” is going to wane, and especially I expect it to fluctuate more than taper off (that is, when I find myself flagging in resolve I can probably renew my commitment and get back on the program). But nonetheless I know my longer term (probably more than my current projection of 15 weeks to goal) will flag, but of course, once I reach my goal the challenge will be to do all this statistical stuff to now control weight rather than just lose it (for example, a gain of 1.5 in a week is no big deal, if random, but if the beginning of a trend, I should correct immediately, so Nate, help me out, with spotting deviations in short-term that are predictors of long-term, esp. of bad trends (a flaw I saw in Moneyball in that it’s obvious players do change, over time (getting old sucks and is bound to reduce ability) and in short-term mood fluctuations)).
So I’ll try to use my Sunday exile and this blog to keep making predictions (honestly, not cooking like Repugs do or that I might be tempted to do) and when my prediction begins to deviate take some corrective action.
But is my past five weeks “enough” to really set a 2.7lb/week?
Nominally that means a net reduction of 1390 calories a day (reduced intake, increased burn). Since my burn (increase over baseline) is about 500/day, that means reduced intake of 900/day. Now the online calculator indicates I could consume about 2760 calories/day (light exercise, no weight change), so I’m estimating about 1900 to lose 2.7lbs/week and the calculator says, 2108, which I think is high by a little, so I’ll say my target is about 2000 calories/day and keep up exercise.
Now here is where Nate is so right. I “feel” I’m way below 2000 already, but the data clearly indicates I’m not (I believe the 5 weeks is enough to “prove” that). If you’d asked me to estimate I would have probably said I was around 1500/day, maybe even 1000, but obviously I’m not.
And so I’ve learned something. I’m lying to myself about food intake. Right now that may not matter too much since weight loss is good, but when it comes time for “maintenance” I have a lot of work to do. This loss, especially with exercise, again has been the easy part (despite being tempted to get another snack here at Starbucks) so the food intake is going to be the real work and where I need to apply Nate.