#data research diy boomers millenials

[thinkitmeans]

“Millennial dads have pathetic DIY skills compared to baby boomers“

That’s the headline in this New York Post article.

You’d think, by reading that title, that they had pitted two groups of ‘do-it-yourself’ builders against one another. A room full of millennials with nothing but the fanciest new tools - yet no clue how to use them, and a room full of boomers fixing everything in sight with nothing but a butter knife and some know-how.

You’d be wrong.

Every day, I read articles like this one that interpret data in really ass-backwards ways - either out of sheer ignorance, a need to fit data to the story the writer wants to tell...or probably a bit both. If you’re here, you’re probably just as annoyed as I am, or you’re just here to learn to think a bit more critically about these types of headlines.

I’m not an expert  - I’m more of a policy analyst these days. I will be the first to admit that there are actual statisticians and math wizards that could wipe the floor with me in a heated battle of data wits. However, I have an M.A. in applied social research which included a bunch of methodology and statistics classes which should probably help me in this new task I’ve given myself.

I’m glad you’re along for the ride. Now, back to our butter knife wielding boomers.

Here are the article’s “findings” :

- Millennial dads fall short in most categories of “handiness” scenarios. - New technology being harder to fix is the top reason for the “decline” - Millennial dads are also more likely than their own fathers to prioritize family time over DIY. 

How did they find these things? Turns out a company called “alarm.com” commissioned OnePoll to do a survey of 2,000 dads. However, that’s where our knowledge about the dataset stops, and it brings us to our first problem.

We don’t know who was surveyed. Neither alarm.com, nor the NY Post, nor OnePoll tell us who these people are, just that they are part of OnePoll’s research panel and that they are dads. What age are they - how did they define millennial and boomer? Where are these people living? What is their income? At the very base we should be able to know what the variables were if we want to know if what we are being told has been accurately measured. This is really important because of confounding variables.

Confounding variables are constantly ignored in these types of surveys, yet they can completely change the context of survey results. Take for instance this example : say you did research and found that there is a causal relation between activity level and weight gain. That’s not too hard to believe. However, you can’t immediately say that you’ll gain weight if you don’t exercise. That also depends on your age or how much you eat. These are hidden variables that have an effect that we can’t ignore.

So let’s go back to our variables for this example. One of the questions that was asked was whether or not individuals owned certain tools. So the variables would be age/owning tools. The article tells us that fewer millennials own tools compared to boomers so they are less likely to do their own home repairs.

Without knowing who was surveyed we can’t know if the data was controlled for location - were these rural boomers and urban millenials? Did they ask home-owners or renters? That would be important as renters have landlords.

Also tool ownership doesn’t actually tell you if someone does or does not know how to use them - when I moved out I didn’t own a car, it didn’t mean I didn’t know how to drive - I just couldn’t afford one.

Many of the variables that are ignored could give us a much fuller understanding of what is happening. Though that doesn’t make for fancy headlines.

Alright, let’s put our sample aside and talk about each finding one at a time :

Millenial dads fall short on a variety of DIY topics.

My first question was - how did they find that out? To go back to where we started - were they all put in a room and their handiness was measured? No. They simply asked them if they could “handle” a variety of tasks - and for the most part, older fathers answered they could more often than the younger fathers.

That doesn’t actually tell you that younger fathers are worse at those tasks than the older fathers. It simply tells you that younger fathers THINK they are less likely to succeed at these tasks than older fathers THINK they are. It’s a self-assessment. You could just be finding that boomer fathers overestimate their abilities. That as we age, we become more confident in our abilities, or that we gather more information and expertise. A more accurate statement would be “younger fathers on average 20% less confident in their ability to perform various DIY tasks than older fathers” - not quite as sexy. 

New technology being harder to fix is the top reason for the “decline”

They share very little data about this finding however, you can be sure that they did not find a “decline”. A decline means that you have measured the same thing over a long span of time - a longitudinal study for instance would be able to track a “decline”. “Kids have been doing progressively worse on this standardized test over the past 20 years - there is a decline in their scores.” This poll? Not so much. You could maybe change the question to something like “at the age of 22, when you first became a father, were you able to accomplish these tasks?” but even then you’re assuming that they remember what they knew at that age.

This is an issue with another of their findings “Millennial dads are also more likely than their own fathers to prioritize family time over DIY.” They didn’t ask if they were more likely to prioritize family time when they were young fathers. How many boomer fathers do you know are spending more time with their kids than young fathers?

This brings me to my last point : the source of the research is important to keep in mind.

Who is alarm.com? A company pushing high tech security for your home. Seems convenient that the research came to the conclusion that new tech was difficult to install - best left to experts.

Also, who conducted the survey? A firm that specializes in “PR Surveys”, they say on their website : “originally a tool to secure column inches in national media, today OnePoll’s research and insight is used to add depth to brand storytelling across traditional and digital media. “ Sounds like real thorough research. That being said - they were effective. Their haphazard research has been picked up by several media outlets, and used to feed the narrative that alarm.com sought to create : are you installing a high tech security system? Probably should leave that to the experts, even your tech savvy kids are calling us.

All of this isn’t to completely discount the survey - there may be some interesting findings that could be teased from the questions they asked - just probably not the same conclusions that were spread across newspaper headlines.