As the producers of the Future of Web Design, Future of Mobile, and Future of Web Apps events, we rely on Twitter for quite a lot of stuff: It keeps us in touch with speakers and attendees, gives us a platform to share cool tips and tricks with the web community, and provides a real-time communications channel for conference attendees.
Here's what it's not very good for: feedback on our conferences.
Don't get me wrong. We love hearing positive things about our speakers, sponsors, and staff, and try to react quickly to issues raised via twitter, but as a barometer of quality, I don't think Twitter is very reliable at all. Here's why:
1. Twitter is public: Market researchers have long understood that anonymity is crucial to good data collection. People are more willing to be honest when they know their opinions will be kept private. That goes for positive as well as negative feedback. Sharing your thoughts on Twitter is akin to standing in a crowded room and shouting your opinion. It may be one of many, but it's still very public.
2. Twitter is personal: Twitter is more closely linked to your actual personality than any other mass communication device. So criticizing someone on Twitter -- even when it's warranted -- just feels like a personal attack. And few of us are willing to humiliate others in person.
3. Twitter is limited: The best data is specific, and it's hard to be specific when slapping together thoughts in 140 character bursts. At our events we see a lot of tweets like "@globalmoxie is just KILLING IT at #FOWD" or "@jontangerine is my hero." But what do those tweets actually tell us, besides that you have a generally positive feeling about the speaker?
We tried something a little different at Future Insights Live this year. Instead of just relying on Twitter, we created a simple online feedback form (with help from our friends at Formstack) and asked attendees to rate each session based on specific questions. The results were pretty surprising. In some cases, speakers who tend to get a lot of love and support on Twitter fared worse in these evaluation forms. In other cases, speakers with nearly no Twitter buzz at all were hailed as superstars by their attendees.
By and large, we were able to collect far more actionable, accurate, and compelling data by going old-school on feedback. We now have the data we need to identify weak spots and bring rising stars out of the shadows and on to the main stage. We'll be relying on those results (and those collected at FOWD London) as we plan our upcoming events and continue to make them better.
Davin Wilfrid is a big fan of Twitter. You can find him at @dwilfrid.