Many people have asked me why we pulled Holler so abruptly last week. The reason wasn’t because we don’t believe there’s a space for a product like Holler, but because we didn’t see the performance we were hoping for. The best way to explain the rationale behind halting the app is through a short story about what we did right.

A 20 Percent Response Rate

For those who aren’t aware, Holler was an iPhone application that we developed for facilitating ad hoc meetups. People could broadcast out to their friends that they were interested in getting together (e.g. grab lunch, have drinks, etc) and their friends could reply directly to them.

When we first launched Holler there were all types of problems with the product. Aside from glaring bugs that needed to be fixed, there were some fundamental issues with how the product was developed. As we started contemplating what we were missing we began adding features that would smooth the on-boarding process (the process through which new users become active) and bolster our invite response rates. What do I mean by invite response rates?

Every time a user registered for Holler, we prompted them with a screen that suggested they “Holler at your friends” at which point they would select their contacts from their phone’s contact list. Once they selected their friends and typed in an activity that they wanted to participate in (e.g. grab lunch) it would literally text message their friend and say “Nick O’Neill wants to grab lunch. Click here to join them: http://holler.com/somelink”

Our response rate on our initial invite system was approximately 20 percent. For email marketers in the audience, you may be thinking “that’s an awesome response rate!”, however only a fraction of the people who actually clicked converted into users.

Achieving An 80 Percent Response Rate

We thought about this process for a couple days and quickly came to the conclusion that the flow was too broken. I looked through our Twilio logs and noticed messages like “Who the hell is this?” and “New number?” and one of my personal favorites “Who dis?” It was obvious: the foreign phone number was a serious problem. Yet something was interesting about this phenomenon: the users weren’t ignoring the message. Typically when you spam someone, you’d expect them to ignore the message, but because this came from a friend, they replied.

So how could we enable a conversation to occur in which the user of Holler explained to their friend that they were testing a new application? We came up with a solution: add private messaging so the invitee could reply directly to their friend and make the message more personal. The message would now look something like this: “Hey John, it’s Nick O’Neill. I’m going to grab lunch. Interested?” Immediately our response rates jumped 400 percent. Success!

Things Aren’t Perfect

Despite the huge surge, things weren’t incredible. Despite the improved communication, people weren’t getting together. By analyzing an anonymized log of communication among users, we were able to figure out that many people didn’t get together. For those who did, their subsequent hollers weren’t receiving responses. Additionally, users who didn’t broadcast invites to their friends and chose to broadcast invites to public Holler users were subject to far lower response rates.

Unfortunately, a non-response in Holler is a powerful motivator to stop using the service. Out of 6,000 people who registered for Holler (this is without any substantial marketing), we had less than 50 relatively active users. There was an additional 100 to 200 people who were opening the app following a push notification from a public group. The bottom line is that the numbers were pretty abysmal. For social applications, you want to get much closer to Facebook-like engagement levels where 50 percent of your monthly active users are returning daily.

While we had close to 25 percent of monthly active users returning regularly, the vast majority of users (well over 90 percent) were not returning to the app. As a result of these numbers I figured it was a good time to pull the app because in the events space, generating a highly active user base is something that is insanely challenging. If you have any feedback or questions, don’t hesitate to post them in the comments!

If you enjoyed this post, please consider subscribing to get these posts in your inbox.