Proposed “Consumer Survey”, along with more “supporting” info, now available at OMB website

As you will recall from our post last Friday, an official FCC public notice (in the Federal Register, for crying out loud – how much more official can you get?) said that the proposed “Consumer Survey” would be available for review on November 5. That turned out to be not entirely true, since (apparently because of some delay induced by the OMB computer system), the survey and related supporting documents didn’t show up on the OMB site until November 7. But it’s there now, so we can all breathe a little easier. (Note: A tip of the CommLawBlog hat to the truly dedicated FCC staffperson who emailed us at 9:15 on Sunday morning, November 7, to pass the word along that the materials had shown up on the OMB site. That was service far above and beyond the call of duty, and we are sincerely thankful for her help.)

Here’s the scoop. To view the proposed Survey, go to this page at the OMB site. To make life easier for you, here’s what you should see when you get to that page:


Once there, click on the link to “Consumer Survey (2010).pdf”, which appears in the “Instrument File” column of the table on that page. (We’ve helpfully marked that in the illustration above.) The Survey should pop up on your screen as a PDF file. As we suggested in our last post, it makes for interesting reading.

But wait, there’s more. If you go to this other page at the OMB site, you’ll find links to three additional documents. One (identified as “Supporting Statement A”) is the supporting statement we provided a link to in our earlier post about the Survey. No real news there.

But the other two are pretty interesting in their own right.

“Supporting Statement B” sheds some light on how the Survey will be administered. Turns out the perfessors who designed the Survey have subbed out the actual conduct of the operation to a company (Knowledge Networks, or “KN”). KN will send the Survey to “a random sample of the U.S. population, potentially obtaining information from an estimated 5,000 households.” Statement B contains a lot of impressive stuff about statistical this and estimation that – stuff that, frankly, this blogger doesn’t fully grasp (“Dammit, Jim, I’m a communications lawyer, not a probabilities and statistics expert”). 

It’s all apparently intended to provide assurance that we should have confidence in the accuracy of the Survey results. For example, Supporting Statement B advises that, because the Survey will be conducted online, KN will be able to determine how long it takes any particular respondent to answer any particular question. According to KN, “[t]his gives the contractor a natural measure of the certainty the respondent feels in his or her answer. This information is useful in ameliorating the effects of hypothetical bias.” Perhaps that’s valid, but might the response time be affected not only by the respondent’s “certainty” in his/her answer, but also the respondent’s desire to get on with his/her life sooner rather than later, or possibly the respondent’s need to go to the bathroom, answer the phone, get a bite to eat, etc., while plowing through the 25-page Survey?

Another example: KN has developed a number of strategies to assure accurate response from “low-education and cognitively challenged respondents”. One such strategy involves “re-asking questions through a confirmation exercise if it is apparent from inconsistencies in the survey answers that the respondents do not understand the survey questions”. But if the respondent didn’t understand the survey questions the first time through, will “re-asking” those questions really help?

The bottom line here is that Supporting Statement B strives mightily to soothe us all as to the likely accuracy of the Survey’s results. The problem is that there remains a “black box” sense to the whole thing. And, to some of us not intimately familiar with probs and stats, that sense is not helped by the fact that devices supposedly designed to improve accuracy might also be seen as opportunities to put one’s thumb on the scale.

The third document (file name: “Emergency Letter (Consumer Survey). pdf”) linked on the OMB page is the FCC’s explanation about why the Survey needs to be approved by OMB on an “emergency” basis. And the answer is . . . well, you should read it yourself, because maybe we’re missing something.

The Commission notes that the Survey would be conducted in connection with the statutorily-mandated Quadrennial Broadcast Ownership inquiry. That’s undoubtedly true, but why does that require “emergency” treatment? The FCC doesn’t say.

The FCC then advises that it was unable to award the bid for the Survey until September 30 “[d]ue to a significant delay in obtaining funding”. We all know how that goes, but that explains only why it’s taken so long to get to this point – it does not explain why getting the Survey approved by November 22 is important. The Commission does say that the contractor “is required to submit a draft of the study” to the FCC by January 5, 2011, with the final study due by January 31, 2011. Where those deadlines came from, though, and why they are apparently etched deeply in stone is not disclosed.

So the “emergency” here is apparently that the FCC has established a deadline – based on what, the Commission doesn’t say – and in order to meet that deadline, it would like the Survey to get moving pronto. We strongly suspect that if a broadcaster seeking “emergency” FCC action relied on this kind of showing, the Commission would routinely reject it out of hand – but that just goes to show what happens when the shoe’s on the other foot.

One aspect of the Commission’s “emergency” claim is particularly troubling. It seems clear that the (presumably) FCC-established deadline for completion of the final report is driving the bus here, which seems backwards. If the Survey is intended to provide useful insight into the issues on the table, shouldn’t the researchers be given the time they need to complete their survey thoroughly and analyze the resulting data completely first, before the Commission can even start to set deadlines for wrapping things up? This apparent Deadline-Über-Alles approach does not inspire confidence here.

And let’s take a look at the Survey schedule, if the FCC does get OMB approval on November 22. According to Statement B, KN plans to run the Survey past not one, but two focus groups. We’re guessing that that’s likely to take at least a couple of days. If OMB gives the Survey its blessing on November 22 (the Monday of Thanksgiving week), it seems unlikely that the focus groups will have been conducted, the results compiled, and any adjustments to the Survey made before, say, December 1. So let’s say the Survey gets deployed to the random universe of 5,000 folks on 12/1. Supporting Statement B says that respondents will have “a two-to-four week period to reply”. That means that the responses won’t all be in until after Christmas. But the draft report is due to the Commission on January 4. Is that schedule really consistent with an interest in achieving the most accurate and reliable report which might meaningfully contribute to the resolution of important questions?

If you have any thoughts about any of this, you have until November 22, 2010, to pass them along to the FCC and/or OMB. And as we suggested in our earlier post, since the FCC seems to expect OMB approval on November 22, it would probably be a good idea to get comments in sooner rather than later.