Many new and eager startup enthusiasts working to better a product and idea know that the key to success is understanding and finding that sweet spot for what customers say the want and what they truly need. If you’re following the basic principals for Lean Startups you know that the key concepts are keeping a lean production, continuous deployment with rapid learning through the rigorous feedback loop of ‘Building, Measuring & Learning’. However, it’s easy to get quickly consumed with the process and mentality of mining data just for the sake of more data. There is a such thing as too much, and although gathering the right type of quantitative metrics doesn’t lie often, the fact remains that there is still a disconnect from what all users say they want and what they actually need — and more importantly, what they will actually pay for.
The practice behind UX Design (IxD, IA, UI Design, HCI, etc) is knowing how to process measure, quantify and interpret user feedback. While this has helped define the practice and gives value, it can force some into a corner. By continuing systemic iterative processes without context of a long-term product vision, this will give a less cohesive user experience if you’re testing every little aspect of your product. By being overly driven by data you can run into key issues and lose sight of the fact that emphasizing too much on quantitative metrics are just a small piece of the decision process. Knowing how to properly interpret the information, and align them with customer interviews, user testing, customer support requests and your own gut intuition among other things from your toolbox will help you make smarter decisions. Andrew Chen writes about five of the key problems that you’ll want to avoid while being too over-metrics driven, noting that they:
– Force you into becoming risk-averse
– Have a less cohesive product experience
– Pivot to early
– Cause customer hitchhiking
– and Coerces metrics to replace product vision
Rather than spending a ton of effort in potentially creating something bad, one should learn to be inspired by data instead of purely driven by it. Conventional A/B testing and many other quantitative metrics concepts don’t tell you anything about actual user engagement, user pain points or how they’re feeling. Gathering metric information is meant to help refine your product and is not supposed to be the steward. In stating this, I’m not saying that data is bad for you, as I champion the idea of gathering proper data. Just know that you have to combine all the information and look at the whole picture. Iterating rapidly with continuous deployment can quickly get out of hand and stray away from an original idea without a long-term vision and a proper strategy.