What we want to know -- on engineering communication

Once a year the president speaks to congress about the state of the union and the rest of us get to listen in. That we can listen has been true for only seven or eight decades as first radio and then television expanded the reach of the president's voice. As the president progressed to the podium this year there was lots of smiling, handshaking, hugging, and a general atmosphere of conviviality. We can watch. We can speculate that some of these people are often less convivial than on this occasion. And we also can 'speak;' today we have voice. New media make it possible for individuals to speak to the world. The White House decided to take advantage of our communication prowess and asked that people send messages, labelled with #WHchat, asking questions that would be answered by White House staff and eventually by the vice president. By the time the address started there were just under 1,700 Twitter messages including #WHchat. Most were questions.

What do we want to know? What were the questions?

I am going to look at the questions asked before the address, during the address, and after the address to determine what the questions were and how they did or did not change through the evening. The number of questions including #Wchat on January 24, the day of the address, was 7052. That is not an outporing comparable to the 130,988 tweets using SOTU, state of the union. #WHchat was an effort by the White House to produce a system for interacting with people watching the state of the union address. The questions summarized are the result.

Questions: if you believe "when are you going to stop beating your wife, husband, child, dog, etc.?" is a question then you would probably believe that "When will the president investigate Wall Street bank fraud?" is a question. There is a question mark, but this was an opportunity to say what was important to the writers -- even if posed as questions.

And that was the question. "Bank fraud" was contained in 815 of the 1684 questions posted before the address. If you do a word cloud, after taking out references to Obama and references to the occasion such as SOTU, this is what you get.

They were not all from a single person, but it must have been a campaign because you have exactly the same phrasing without retweeting. Each person had to produce the message. It is possible to copy, but that is more work than retweeting. But it is clear that 816 messages phrased exactly alike cannot be understood as a spontaneous outpouring of sentiment.

During the speech 3088 Twitter messages were posted including #WHchat. Most of the 'questions' were retweets. Retweets were 2662 out of the 3088 tweets posted.

During the speech "bank fraud" went away. It was in only 18 out of the 3088 messages posted.

Many of the messages were critical. A total of 1372 began with

RT @dickmorristweet: #SOTU #WHchat #jobs #hannity #tcot #gop

followed by one or another criticisms. "RT @Moveon" was the second most retweeted 'person' with 166 messages.

The first word cloud is not particularly helpful. The only words that show are those above -- RT @dickmorristweet, etc. However once those words are gone the word cloud takes on a very different character.

It looks like every topic addressed by the president. I have not counted to be sure, but this is 'questions' spread through the topics of the address. The ones that do stand out are: tax [234], pay [177], offshore [148], teachers [139], school [203] , and immigration [117]. But none was mentioned very often. "Tax" mentioned in 234 messages was only 7.5% of the tweets.

The answers were to come after the president's speech. So I collected #WHchat Twitter messages for the two hours after his speech. There were 2288 messages posted to Twitter. "Bank fraud," which was the major tweet before the speech was mentioned in only 16 of these Twitter messages. "RT @dickmorristweet," which was included in 1372 during the speech, was included in only 65 Twitter messages. @moveon, which was mentioned only 166 times during the speech, was mentioned in almost as many messages, 125, after the speech.

What was new?

New? Marijuana stands out, but it appeared in only 315 Twitter messages. It is a favorite question to address to president Obama, but it did not show up on this day until after the address. The question had been posed 14 times before the address and 3 times during the address. Nothing 'stands out' in a big way.

The White House wanted to 'prime the pump' with suggestions for subjects for questions. #education, #energy, #fairness, #jobs and #manufacturing were their themes.

They did not get many questions about the five themes. The only exception is #jobs. #jobs was one of the words in the 1300 messages beginning with RT @dickmorristweet. And looking at the Twitter messages found by searching for each of the five themes on the 24th indicates that many had nothing to do with the state of the union address. They were often like the high school student writing that she needed more #energy. She did not mean natural gas nor shale oil, etc. The search for #jobs illustrates how little the use of these hashtags was connected to the address. Between 2:47 p.m. and midnight on the 24th the search for #jobs found 33,428 Twitter messages. The same search for the same hours on the 25th found 78,933 Twitter messages. On the 24th 5,254 of the #jobs tweets contained SOTU and 2,355 contained Obama. On the 25th only 694 contained SOTU and only 510 contained Obama. One might interpret these numbers as -- the state of the union surpressed Twitter messages about jobs. With only two points in time the evidence is not good enough to be at all confident about that claim. But it is the case that most of the Twitter messages containing #jobs were not about the state of the union address. The same is true for the other hashtags searched for.

What should one make of all this? The questions, as questions, are not very interesting. However, there were four attempts to engineer communication that are worth noting.

The White House decided to take advantage of a standard practice in communicating via Twitter, the hashtag, to set up communication between citizens and the administration. You ask questions and we will answer. It turned out the citizens did not want to take on the status of supplicants. They did not want to ask questions. They wanted voice. They wanted to have their say. The numbers on this are quite persuasive.

First about the numbers -- I can capture only 1,500 messages every five minutes. When the flow of messages is faster than that I get a sample that is not too far from randon in time. Twitter reported that the number of messages per minute containing SOTU during the speech was 8,000 or 40,000 per five minutes. [Twitter, blog post] So my numbers are a 3% sample. The same is undoubtedly true for Obama. It is less likely that barackobama was flowing much faster than 1,500 per five minutes. And it is clear that 7,052 messages in a 24 hour period was not more than 1,500 per five minutes. If I 'round' up SOTU would have occurred more than 700,000 times on the 24th and it did. We know it did because Twitter has given the number. That I know, and the rounding up for the others seems equally plausible.

Whether one uses the sample numbers or their roughly adjusted totals it is clear that asking questions using #WHchat did not take off. It was used in an exceedingly small proportion of the Twitter messages about the state of the union address. The obvious conclusion is 'voice.' People wanted to have their own say. We can now speak to the world, and people were exercising that capability at a pretty high rate on the day of and during the state of the union address. The White House attempt to set up a procedure for interacting, #WHchat, can only be understood as a failure. They wanted people to ask questions. But the people said we want to have our say. It was an engineering effort that produced little.

The most frequently tweeted messages before and during the speech is another version of engineering communication. Half of the messages in these streams of messages were critical of the Obama administration. "Bank fraud' was a critique of the administration's action on the bank frauds that brought down the economy. And the "RT @dickmorristweet" were equally critical though a bit more spread with many criticisms. This is engineering failure in two senses. First, it is failure of Obama administration in their engineering effort. There is no monopoly on hashtags. (Boynton, 2011) An earlier effort by the administration set up a hashtag, #AttackWatch, that was intended to give followers information to answer criticisms of the administration. The problem became the messages critical of the administration overwhelmed the ones responding to the original thrust of the hashtag. Just as with #AttackWatch the opponents took over the #WHchat and used it for criticism. Everyone who did a search on #WHchat found Twitter messages critical of the administration in high volume.

Second, it was an engineering failure of the people driving these critical messages in the failure to take off; 815 and 1372 seem a remarkably small numbers of critical messages to have generated.

Much of this critique is based on the meme lovingly known as 'going viral.' There is much more to say about going viral than I can say here; I have written a series on going viral: You Can't Stomp on Us -- after a Rand Paul organizer did stomp; Going Viral -- in the small; and Now that's going viral: #ScariestWordsEver "President Palin". In #ScariestWordsEver I point out the assumptions undergirding standard notions of going viral and how and which ones are not likely to be met in the Twitter universe.

However, I want to look at 'engineering' for going viral here. Dave Karpf summarizes an account of what Moveon.org does to inspire widespread flow of there points of view. (Karpf, 1/23/2012)

I would call that a third perspective on viral content – the organizational perspective.  Daniel Mintz, a MoveOn staffer and Rutgers alum, kindly agreed to speak to my students last month.  At one point in the discussion, he explained that MoveOn has a simple equation that they use to determine what goes viral.  He drew it on the chalkboard: Virality = (see) x (share) x (come back).  With any piece of content, the organization monitors how many people are clicking on the item to begin with, how many then share it with others, and how many of those others then click as well.  This is data that they can gather, manipulate, test, and act upon.  It informs their decisions, which in turn affect what goes viral, which in turn impacts common wisdom.

The organization monitors the flow of the communications they initiate. They track the number of people who click on the 'message' and the number who then share what they have clicked on, and how many of those others click as well. It is a two level tracking process. Of those who click how many share. Of those shared with how many click. I do not know what software they use, but this is a general form of assessment that is being developed by a number of information technology companies to sell their services to groups or corporations that want to know what is happening to their messages.

This is intelligent assessment of the flow of the messages they initiate. But it pretty clearly does not guarantee going viral. For #WHchat messages from @moveon were retweeted 297 times. That is not as good as either "bank fraud" or "RT @dickmorristweet." And if you look at the messages that were retweeted you find that it was 9 different messages. So the mean number of retweets was only about 30, which is certainly a long way from going viral.

What I want to conclude is that engineering communication is somewhere between almost never and never successful. That does not mean that there are no 'hits.' It does mean that the 'hits' cannot be accounted for based on engineering the communication. Humans now have voice and it appears very difficult to organize that voice. They just go about their communications however they want.

References

Boynton, G.R. (2011) #AttackWatch -- There is no monopoly control when it comes to hashtags

Karpf, Dave (1/23/2012) Three Perspectives on Online Virality, Shouting Loudly

Twitter (1/24/2012) Twitter Blog: Follow the State of the Union on Twitter.

© G. R. Boynton, January 27, 2012