Creating is hard. Innovation is like trying to find the top of a mountain surrounded by thick fog—we stumble around blindly, with no proof we’re nearing the summit. And this fog—it’s a messy, noisy fog filled with uncertainty and risks and customer complaints and competitor one-pagers and three-letter acronyms and conflicting advice and self-doubt and envy-inducing TechCrunch articles featuring our childhood nemesis.
It’s very hard to measure how well we are innovating—how effectively we’re climbing the mountain. We fill the measurement void with our ancestral vice: gut feelings. That leads to unjustified, indefensible, and anxiety-riddled decision making. It also puts our well-being at risk.
Solution: we need a method to measure how much we’re learning, iteratively. And we have to keep that rate of learning high. Let me prove it to you.
Incremental progress helps us deal with uncertainty
As our world evolves and becomes more technology-driven, there is increasing uncertainty and lower “signal-to-noise ratio.” The volume and velocity of data is exploding, yet we’re less certain about our decisions than ever before. VUCA, which stands for volatility, uncertainty, complexity, and ambiguity, is an accepted part of the modern workplace. Uncertainty, change, disruption are the drivers keeping small and mid-sized business (SMB) leaders up at night. Understanding customers is a struggle for 66% of businesses, according to Salesforce surveys. Yet businesses are still expected to accelerate the pace of innovation.
How can we plan and accelerate when uncertainty makes measurement and planning challenging? We must measure and plan incrementally. This is a proven, successful approach to decision-making in uncertain environments: Agile.
“By now, most business leaders are familiar with agile innovation teams. These small, entrepreneurial groups are designed to stay close to customers and adapt quickly to changing conditions. When implemented correctly, they almost always result in higher team productivity and morale, faster time to market, better quality, and lower risk than traditional approaches can achieve” (Rigby, Sutherland, & Noble, 2018).
Agile innovation teams are so predictably successful because they take incremental steps to improve and increase innovation at companies of all sizes.
The tactic of taking incremental steps towards achievement is one well-backed by mathematics, specifically iterative methods for optimization. For instance, coordinate descent algorithms iteratively solve for one variable after another in small, fast steps. Gradient descent algorithms try to climb the hill where the hill seems steepest. And in machine learning, increments of progress are termed the learning rate.
Just like in mathematical optimization, iterative approaches in innovation are meaningless without iterative measurement. It’s necessary to check in on your progress; to assess and adapt. Machines can check on their progress programmatically, but for us lowly humans, habits can be the key.
“With the help of habits, people know where and how they should be heading, without stopping on their way to think through every routine” (Filev, 2013).
Making a habit of frequent check-ins and progress-monitoring is a core tenet of Agile development:
“Iterations are the basic building blocks...each iteration is a standard, fixed-length timebox...[with] the recommended duration of the timebox [being] two weeks” (Scaled Agile, 2019).
The success of incremental approaches in uncertain environments suggests we should make a habit of measuring progress in small, iterative chunks.
Not all incremental measurements are created equal
We’ve agreed that we need to measure progress incrementally. But what is the “progress” we are measuring?
We have a wealth of data to choose from. In recent years, there’s been a push in every industry to focus on data, because data means “success” in common parlance. Yet if we dig deeper, we find that for a lot of companies, discussing—and even gathering—data is purely lip service. Despite knowing that data is not the same as insights, and despite less than half of businesses turning data into any actions, we indiscriminately collect more and more data.
So we must choose our measurement for “progress” carefully, not necessarily selecting something we have lurking in our analytics dashboard. We must also be wary of vanity metrics, such as fundraised dollars: the quickest way to build a billion dollar business is for someone to write you a billion dollar check. The metrics of daily/monthly active users (DAU/MAU) and monthly/annual recurring revenue (MRR/ARR) may also be misleading, depending on our business model—we might miss the ultimately unprofitable acquisition of customers.
Rate of learning as a goal
If we accept that our fundamental challenge is uncertainty, then we accept that the reduction of uncertainty would be our goal. (If you had perfect clairvoyance, the only thing standing between you and world domination would be laziness. Let’s assume we’re not solving for laziness at the moment!)
What are ways to reduce uncertainty?
- Influence your environment to be less volatile and uncertain.
- Learn.
Way #1 above is better restated as “bend the market to your will”, and may be applicable if our business model is organized crime. The rest of us have to reduce uncertainty via way #2: learning.
The concept of learning is pervasive in mathematical optimization. In machine learning, we call it “learning”; in control theory, we call it “feedback control.” And if decades of customer experience have taught us nothing, it’s that customer feedback is a core driver of business value. We need feedback to correct our hypotheses. Learning is a pure form of feedback, feedback that is stripped down to its “insights.”
Elevating learning in an organization is of vital importance. Historically, the “experience curve” has demonstrated that companies that learn through scale and experience can reduce their costs dramatically. The explosion of change and data is creating new competitive forces around learning in particular, prioritizing learning at an organizational level. Indeed, the winners of tomorrow will be those that can accelerate their rate of learning today.
Putting it together: we need to measure and maximize the rate of learning, incrementally and continually.
Is this so startling? In many ways, an innovating team is just a market research firm, an R&D shop, or an explorer cutting through the fog of uncertainty—whether looking for product-market fit, building a repeatable sales model, expanding into international markets, or discovering cost-cutting methods.
How to measure rate of learning
But how do we measure the rate of learning? Here are two methods we’ve found to drive significant results among our portfolio and clients:
Method 1: Ask yourself “what have I learned this week that was meaningful?”
There’s no need to complicate this. Ask yourself on a Friday what you’ve learned in the past week, and share this with your peers and your manager in your scrum or team meeting. Include only learnings that were meaningful to your customer or your unit’s business objective, and see if others agree. Reflecting on the week, it becomes painfully clear whether you’ve been moving up the mountain—or dancing in a muddy hole.
For extra credit, try to find some themes in what you’re learning. (A tool like Sapeum can help if you’re trying to find themes from all your notes or documents.)
Some of our most successful portfolio companies and advisees set dedicated meetings to discuss learnings. They may, for example, institute a Weekly Learnings meeting to summarize recent low-level data and notes, and a Monthly Themes meeting to pull out 2-3 themes across an entire department or even the whole company. (Learn more about this in the Product Loop.) Remember: learning applies to everyone, not just the researchers.
Method 2: Count the number of experiments
We’re scientists, on a journey of discovery, incrementally shifting our hypotheses based on new learnings—so all of our activities are implicitly experiments. Let’s make them explicit experiments. Instead of just emailing some investors, email 30 investors and measure the response rate. Instead of just announcing a feature, announce that feature and measure the click rate.
Where does the rate of learning come in? It’s simply the number of experiments you’re running each week. (We’ll talk more about this in an upcoming article on experiment-driven operations.)
~~~
At Sapeum and our parent, Gaussian, we’ve used these two simple methods to make sure our portfolio companies, advisees, and clients are all laser-focused on building entrenched, market-making businesses. Our advice: measure learning frequently, and hold yourself accountable to keeping your rate of learning high. Once you’re comfortable measuring learning frequently, you can find ways to boost your rate of learning.
Measurement and mastery of the learning rate will increasingly be a critical competitive advantage of all companies, “disruptive” or not. Drop us a line if you ever want to talk more—we’re always keen to talk to smart leaders about their experiences and challenges around innovation and discovery.
This is a cross-post with our parent company, Gaussian.
