The Data is Shit

After many months of working frenetically to get the data in front of the client, that was their response.

“The data is shit”. – (unhappy) (I mean, *really unhappy*) client

Or so the story goes. Fortunately, I was not invited to that particular client meeting. I don’t know if the client actually used those words, but I wouldn’t be surprised if they did because it was a pretty concise description of the data we had been working with for months.


We had politely been referring to the herculean effort as one of (unprecedented) data “harmonization,” so forgive me if I chuckle every time I hear the term “data harmonization”. Seeing as how I played a leading role in this data harmonization effort, you might be surprised that a) I find the client response amusing and b) I would publicly fess up to coordinating this effort.

The Client

I was recruited to work on this project because of my expertise in data management *and* I had experience working with a (successful) data archiving effort for this particular client. I hope I don’t get in trouble for this, but in the age of “government 2.0” and transparency in government, here goes. The client was SAMHSA, the Substance Abuse and Mental Health Services Administration at the US Department of Health and Human Services.

I had previously been working at the official SAMHSA data archive (SAMHDA) at the University of Michigan’s Institute for Social Research (ICPSR), so I was quickly snatched up by an HIT (health information technology) ‘beltway bandit’ company at the end of 2006 when they realized they were in over their heads on a 5-year $25M project for SAMHSA. The project involved standardizing data from disparate sources for their Data Coordination and Consolidation Center (DCCC).

The Data

There was nothing inherently wrong with the data. Really, it was a matter of expectations. Not many people understand data and what it can and cannot do for you. We were given the challenge of consolidating data from significantly different data streams. This is not impossible, but is a delicate effort. Arguably, the more you work with data in this way, the less it will do for you. Even ‘good data’ has its limits.

It’s All about Context

Part of the reason I didn’t take this whole abysmal failure thing too seriously is because I could see it coming for months ahead of time and it was kind of a relief to get it over with. Beyond the ordinary challenges of working with data and managing client expectations of said data is the larger context of the project. It was one of those huge government contracts that didn’t, um, how should I say this? It didn’t really make sense.

The original contract called for this whole data harmonization effort as well as the creation of a data analysis tool to view and analyze the data. Sounds simple enough, right? Simple, yes, but also expensive and unnecessary given that such tools already exist. Apparently the contract was set up in such a way that incentivized creating a whole new data analysis system from scratch. And so the HIT company did exactly that. It spent two years and a few million bucks reinventing the wheel. While this expensive and unnecessary toy was being developed, I worked with a team of SAS programmers to harmonize the data to put into the shiny new object.

Should anyone have been surprised when the client, at least $7M later, looked at the data and said it was shit? They took the luxury car (that they had asked for in the original contract) for a test drive and when it only drove 3 MPH, they were livid. Was the problem that the engine maxed out at 3 MPH or that they accidentally ordered a luxury car when a lawn mower would have been more appropriate?

How to Choose a PCP

I am an expert in PCP (primary care provider) selection. Why, you may ask? Because I have a lot of experience selecting PCPs.


I am a member of the generation of workers that changes jobs at least every two years or so. This means I get a new health insurer every two years or so. And I have yet to find a new health insurer who will cover expenses if I stay with my current PCP.

At this point, I don't even bother trying to remember my doctors' names or build much of a relationship with them. I don't go through the effort of transferring my health data either. Whatever energy I put into that relationship gets flushed down the toilet every two years or so.

Anyway, back to the point of this post: to share my personal wisdom on how to select a PCP. Here are the basic options:

a) Let the health insurer automatically (and presumably, randomly) assign you a PCP.

b) Choose a name randomly from the list they provide (which, by the way, may be out of date and require you making endless phone calls to find a PCP that *actually* accepts that insurance *and* is accepting new patients).

c) Choose someone who has a name that sounds appealing to you. If you are 'white bread' and proud of it, choose a PCP with a last name like Smith or Jones. If you are Latino/Latina, choose a PCP with a common Latin name. You get the picture, right?

d) Choose a PCP with a last name in the second half of the alphabet. If you are into fighting alphabet discrimination, this is a nice route to take. Those PCPs with last names that start with A, B, or C? They get way too much attention based on their arguably unfair placement in directories.

I've learned to take a slightly different route. Warning: this may not work for you.

I examine the entire directory. I assess the available information. This typically includes name and sex. You can guess at information about physician ethnicity from here. Sometimes directories include languages spoken. Again, this gives you a clue as to the ethnic background of the physician, if that makes a difference to you. Whether you are afraid of ethnic differences or embrace them, you can make decisions based on this limited information.

Oh, yeah, and if they list medical school? Bingo! That rounds out all the available information I've ever seen and brings me to sharing three of my most recent experiences in selecting a PCP.

A few years ago, I selected a PCP for my then pre-adolescent son based on one of these simple directories. I scanned through with ease and expertise (given I have plenty of experience analyzing such lists). And voila, I found my son's PCP! He fit my only preference (being that he was male!). I thought my son might like a male PCP. This doctor had an Indian-sounding last name, went to Howard University medical school (a predominantly African-American school in case you live under a rock or in the suburbs), and spoke several languages, including Spanish!

I hit the physician jackpot! I don't speak Spanish, but I thought it was cool that he did, especially considering he was likely Indian (really, how many people of Indian descent speak Spanish?) and he went to Howard?! This dude had to be an interesting person. (And turns out, he was a great doctor. But guess what? I changed jobs and ended up with a new health insurer that considered him 'out of network'.)

The next PCP I picked for my son was within the Kaiser Permanente (KP) system. I had a fairly short list to work with (trust me, this is *not* a complaint) since I wanted to go the facility closest to home. I ended up picking a female PCP based on her last name; it looked like an Arabic name. I figured she probably gets discriminated against by white people who grew up in the suburbs and watch too much network news. And sure enough, she was a lovely PCP for my son...until I lost my job. (Turns out I wasn't eligible for COBRA, so keeping KP was not an option.)

I picked my own PCP at KP based on where he went to medical school. Normally, I prefer female PCPs, but this guy went to medical school at the University of Michigan. I went to school at the University of Michigan (both undergraduate and graduate program) and relied on the University of Michigan Medical System for health care for many years. After having had countless negative experiences with the health care system in and near Flint, Michigan, I was happy with my care through the University of Michigan (ok, maybe there was one or two docs I didn't particularly care for, but that can be chalked up to personality issues).

Seeing as how neither my son nor I currently have health insurance, I have been spared the chore - for now - of selecting new PCPs for either of us. Hopefully I will have health insurance soon. Then, I will go through this process *yet again*.

Entertaining Sanity

I publicly proclaimed (via twitter) the following 2010 New Year's Resolution:

"I will no longer entertain people's delusions,
even if they are willing to pay me to do it."

It's only the last day of January and I've lost business over this resolution this year. Some people might consider this a failure (losing business), but I consider it a success.

Psychology in the Workplace

I had a great phone conversation with a friend in Washington state yesterday. We talked about a lot of things, including this. I almost jumped out of my skin when she referred to "family of origin" issues in the work place. Someone else knows what I'm talking about! (I guess it's not just us social work types that notice these patterns.)

Consulting is One Way to Find the Exit Sign

Forgive me for putting a big picture of myself right here, but there's something I love about this picture (hint: it's a word in the background).

People bring an incredible amount of psychological baggage with them to work every day. If you can avoid the worst offenders, great. But what if your boss is one of these people? As I like to say, "lawd ha' mercy."

Consulting is not an easy answer. I still need to negotiate endless unwritten (and often unreasonable) expectations (more about that in future posts). However, if you're willing to give up health insurance (yes, that is a big 'ask') and the delusion of 'job security' (um, if you haven't figured this out by now, it simply doesn't exist), you are in a position to regain a level of self-respect you may not realize you've somehow lost through your experiences in the traditional 'workplace.'

Retired

I am thirty-something, a few years shy of being 40 years old. I decided to go ahead and officially retire. Now. Actually, I retired a few months ago.

I never really wanted to have a job. I never wanted to get married. I couldn't see myself having children. From my vantage point in the suburbs of Flint, Michigan, I saw absolutely nothing I wanted to replicate.

My parents struggled to send me to college. With a good education, I could get a good job. I, on the other hand, was incredibly relieved to find that college wasn't anywhere near as brain-numbing as high school.

I briefly considered getting a degree in computer science. In high school, I programmed better and faster than my peers. On an Algebra final exam, I proved I could solve algebraic equations better and faster than any of my peers at the University of Michigan-Flint. But, alas, I found my true love: sociology. Introduction to Sociology, Social Stratification, Urban Sociology, Race and Ethnicity...

I was in heaven.

My parents were mortified. (What can you DO with a degree in sociology?! )

I responded by saying, "I can get a PhD and be a college professor". I'll never forget the response I got to that statement. I felt like I was 5 years old and had just announced I was going to be a fireman when I grew up. (Neither of my parents ever completed a BA or BS degree. Apparently me attending graduate school seemed about as realistic a possibility as a trip to Pluto. Now might be a good time to mention that I have always been the "black sheep" of the family, breaking all of the rules.)

With all the talk of web 2.0 and health 2.0, I'm surprised we don't hear more about "work 2.0." What is work 2.0, anyway? I'm not completely sure, but I do know that it's probably similar to "unwork" (think "unconference") in the sense that hierarchy is considered an antiquated concept. Think cooperation. Mutual respect. Not what can you do for me, but what can we do together?

On that note, I am pleased to announce my retirement from "work 1.0."

Slow Start

As with many blogs or other internet advenetures, they often get off to a slow start. You might create a blog, then be too petrified or busy to write the first (definitely the hardest) post. Then couple months later, you write a post. The guilt inspires you to write another one. Next thing you know, you forgot you ever started a blog and feel kind of pathetic as you contemplate resurrecting that pitiful blog you started a year ago...

Cheers to new beginnings! Keep plugging along. Don't quit quitting (if you're a smoker). And keep starting, even if you feel like a slacker.

Marketing Creep

I am really disturbed by how much commercials have crept into our lives. Commercials used to be between television shows, but now they are absolutely everywhere on the internet. It has somehow become the burden of the individual to figure out what is being written spontaneously and what is being subsidized.

My story about 'marketing creep':

Back in 2002, I went to see a movie with a friend. It was the first time I had been to a movie theater in at least 6 years (no joke). I was a little worried about getting there a few minutes late, but then remembered they usually show a few minutes worth of previews for upcoming movies.

We arrive and the previews began...I thought. Wait a minute...what?! There were showing actual commercials - paid advertisements for random things that had nothing to do with anything, as far as I was concerned. I turned to my friend in shock. He looked at me, a little puzzled. I literally turned around and scanned the faces of the people in the movie theater. I was the only person who was trying to figure out what was happening and why no one was responding with disgust (or at least noticeable irritation).

I still feel like that sometimes. Exactly when did it become okay to sell us stuff after we paid a fair amount of money to see a movie? Or with mobile phones...when did it become okay to show me commercials when I check the weather forecast? I paid for my phone. I pay for monthly service. I pay for all the extra data capabilities. And I have to endure advertisements every time I want to see how warm it will be that day?

I'm not sure what's worse: that this has happened or that people are not more upset about it.

Where Am I?

I published a couple of posts on The Health Care Blog recently:

How to Waste a Boatload of ARRA Money - May 22, 2009

Nothing Personal - June 02, 2009

Other than that, I've been on twitter...and also working with Claudio Luis Vera to launch a new web site. Information coming very soon!