Posts tagged metrics

Effectively Communicate Mission-Critical and Relevant Data

Whether you are a CEO, VP Product or a Web Developer, you are constantly evaluating your progress through a series of metrics or analytics, and generally against pre-defined goals. But, are you using the right dashboard to track your metrics day-to-day, and does your dashboard contain rich enough data to allow you to act proactively to ensure you reach your goals?

I recently read the book called Information Dashboard Design: The Effective Visual Communication of Data by Stephen Few. Although it took some digging to find real value, there were great nuggets of wisdom scattered throughout.

Getting back to the sole premise of a dashboard – its purpose is to act as a visual display of the most important information (note: not ALL information) needed to track progress toward your stated goals. In designing your dashboard, it is extremely important that the information is consolidated onto a single screen so that each metric being tracked can be viewed together – at a glance – so that the user can pick up on any trends and relationships between metrics and overall goals.

The book cites two great quotes that drives this concept home and explores a new facet of this equation, our visual perception:

“Dashboards and visualization are cognitive tools that improve your ‘span of control’ over a lot of business data. These tools help people visually identify trends, patterns and anomalies, reason about what they see and help guide them toward effective decisions. As such, these tools need to leverage people’s visual capabilities.” (Source: Richard Brath and Michael Peters, Dashboard Design: Why Design is Important)

“[We should be interested in visualization because the human visual system in a pattern seeker of enormous power and subtlety. The eye and the visual cortex of the brain form a massively parallel processor that provides the highest-bandwidth channel into human cognitive centers. At higher levels of processing, perception and cognition are closely interrelated. However, the visual system has its own rules. We can easily see patterns presented in certain ways, but if they are presented in other ways, they become invisible… If we can understand how perception works, our knowledge can be translated into rules for displaying information. Following perception-based rules, we can present our data in such a way that the important and informative patterns stand out. If we disobey the rules, our data will be incomprehensible or misleading.” (Source: Colin Ware, Information Visualization: Perception for Design, Second Edition)

Limitations of Man

Unfortunately, even the best of us are limited by our biological construct. Humans have limited short-term memory capacities and this is the main reason why information that belongs together shouldn’t be fragmented across multiple pages or dashboards. Once one set of information is no longer visible, it loses its position in short-term memory (unless you have a photographic memory, I suppose); on the flip-side, if all information is within one field of view, your brain can rapidly process information to and from your short-term memory. [HR Moment: If you’re hiring, consider those with photographic memories for data analyst positions!]

Effective Design

To design a most-effective dashboard, ensure to only include information that you absolutely need, in its most summarized form (without the loss of crucial value). This is where the KISS principle (”Keep It Simple, Stupid”) shines bright. The information should be well organized, condensed, specific to its audience/objectives and displayed concisely and clearly with emphasis on the data itself and not on background graphics or fancy designs (which tend to lose meaning and value quickly).

In reading the book, there were a few graphs that I found used space in a very effective way, whilst also allowing the dashboard reader to effectively take in a number of data points in a glance. The first of those graphs is called the “Bullet Graph” (details of how it works in diagram):

The second graph is called a “Pareto Chart” and below is demonstrated in a sales capacity, which quickly shows off what proportion of overall sales is contributed to by each product. Think about measuring the sales quota of each sales employee as a percentage of total sales (this will help sales managers know who to keep and who to let go using a clear and measurable decision-making tool:

My favourite graph, however, is the Sparkline. Its simplicity is unmatched and its span can be adjusted according to the scale you which to present. It is also great for summarizing large datasets into a small graphics that can be displayed alongside each other in dashboards. Sparklines are particularly good for analyzing trending data. In fact, this is used by Google in its Google Analytics product (below, Sparklines on the left of data and represent 3-month trending curves).

At the end of the book, Few shows a few sample designs of dashboards. I think these are among the best dashboards I have ever seen from a design, usability, trending and notifications standpoint. I have included images of dashboards for both sales managers and CIOs below, as designed by Stephen Few and presented in the book.

Sales Dashboard

(Source: Information Dashboard Design, by Stephen Few)

CIO Dashboard

(Source: Information Dashboard Design, by Stephen Few)

  • Share/Bookmark

Understanding Social Game Player Dynamics

Understanding the behavior of players of social games has been an expensive lesson to learn by many companies, often picking up bite-sided pieces of insight through extensive A/B testing and internal metrics over time. Many companies have also tried to better understand the viral invitation process and successful virality of social games both on and off the Facebook platform. An academic paper entitled “Diffusion Dynamics of Games on Online Social Networks” was recently written by Xiao Wei and Jiang Yang from the University of Michigan, Ricardo Matsumura de Araújo from the Federal University of Pelotas, Brazil and Manu Rekhi, VP of strategy, marketing, business and corporate development for Lolapps.

The paper analyses the viral spread of an application and how/why are these processes occurring. SocialTimes.com did a great post that summarizes the academic paper. Alternatively, you can view the entire paper here.

Some of the key findings are summarized below:

  • On average, each inviter has invited 26 friends while the median number is 10
  • Just 10% of users account for 50% of successful invites
  • Around 90% of users share their locale information
  • Around 40% of users share their friend list
  • Only 1% of users share their relationship status
  • Invited users remain in the game longer: over 50% kept on playing for more than a day and 20% of all invited users were still playing 80 days later.
  • Around 80% of non-invited players leave the game within the first day
  • Overall, they find that invitation strategy is more important than demographics in determining invitation success rate

To determine how to create a profitable social game, please explore my previous blog post on the importance of Customer Acquisition Costs for startups.

  • Share/Bookmark

The Importance of Customer Acquisition Costs for Startups

I recently came across the blog of David Skok of Matrix Partners and was inspired to write this post by an article on customer acquisition costs. If you have not yet read through his blog’s vast resources for entrepreneurs, I suggest you do so – particularly if you plan to pitch your startup to VCs anytime soon.

After being pitched countless times by startups, as a VC I’d like to identify a common misconception that web-based startups often have about their own growth potential and the costs associated with their plans. Management of web services companies, SaaS companies and mobile (web-based) applications commonly believe that because they are situated online, customers will come across their service, submit a purchase order (or subscribe) and notify friends or other companies to use the service as well. Although this may happen from time to time, it is very rare for any company to experience sustained viral growth.

Many companies don’t understand the difference between viral marketing and viral growth. Viral marketing is essentially “word of mouth” or “person-to-person distribution” and is the latest buzzword. Viral growth implies a K-factor greater than 1 (i.e. for each new person who tries a product/service, they will each invite more than 1 registered user of the product on average). Since true viral growth is so hard to achieve in practice, many companies miscalculate the actual costs it will incur to acquire customers. As David points out in his article, the majority of startup pitches lack detail/emphasis on how much it will cost to acquire customers. I second this statement entirely.

Business Model Viability
For a business to be profitable on each new customer, startups must balance two variables: (1) Cost to Acquire Customers (CAC); and (2) Lifetime Value of a Customer (LTV).

CAC can be calculated by taking the business’s entire cost of sales and marketing over a given period (including salaries and other employee expenses) and divide it by the number of customers that the business acquired in that period.

LTV can be calculated by looking at the Average Revenue Per User/Customer (ARPU) over the lifetime of a business’s relationship with a customer.

As Steve Blank mentioned in his recent post, an early indication that a business has found the right business model is when the cost of acquiring customers becomes less than the revenues generated from the customer. “For web startups, this is when the cost of customer acquisition is less than the lifetime value of that customer. For biotech startups, it’s when the cost of the R&D required to find and clinically test a drug is less than the market demand for that drug.”


Credit: David Skok.

Zynga is a great example of a company that has managed to decipher the business model of online social gaming. After thousands of A/B tests and experiments, Zynga finally found a business model where CAC was less than LTV. Once they cracked the nut, the company spent so much on customer acquisition that it was rumored that they accounted for upwards of 30% of Facebook’s revenue in 2009 though its aggressive social ad buying strategies. Similar business models and opportunities exist in virtual worlds, massively multiplayer online games (MMOGs) and many other online businesses. Many social games, such as those created by Zynga, leverage virtual currency, micro-transactions, emotional response mechanisms and social influence to promote the sale of decorative and functional virtual goods.

Before investing in a web-centric startup, good VCs will look deep into a company’s business model and know to look for CAC and LTV metrics. In fact, Trident Capital recently held a meeting with their online advertising and ecommerce companies to help exchange best practices for customer acquisition and improving LTV. My advice to startups: prove out your business model and you will have a much better shot at raising VC dollars. Skok suggests that two key equations be followed by web startups:

  • CAC < LTV (3x appears to be a rough minimum for SaaS businesses)
  • CAC should be recovered in < 12 months (for subscription businesses)

Startups, if you’ve already figured out your business model and how to make CAC < LTV, stay very quiet and add as much fuel to the fire as you can afford. Your competitors will likely try to hone-in on your tactics and fight back for their share of the market.


Credit: Steve Blank.

Leverage Startup Metrics
Startups are different from larger companies and therefore need different metrics than larger companies. Metrics will give startups a lens into how well the search for the business model is going and help to identify when to scale the company. Besides CAC and LTV, some essential metrics that startups should be familiar with include Viral Coefficient (K-factor)  and Customer Lifecycle. Dave McClure from Founders Fund recently updated his Startup Metrics for Pirates presentation for web sales pipelines. Take a look!

Questions to my Readers
Please consider the following questions and share your perspectives with my other readers and the tech community at large.

  1. What metrics do you consider the most valuable?
  2. Do you use any tools to help measure specific metrics for your business?
  3. What mistakes have you made (and corrected) that can help others succeed?
  • Share/Bookmark

Larry Cheng Updates Global VC Blog List

Larry Cheng updated his Global VC Blog list today (originally posted in May 2009 as top-100 in Google Reader subscriptions) and has re-ranked the global top VC blogs by average monthly unique visitors on compete.com for Q4 2009 (oct+nov+dec)/3.

As per the latest global VC blog listing, Fred Wilson from Union Square Ventures (Blog: A VC) took the top spot, shifting Guy Kawasaki from Garage Technology Ventures (Blog: How To Change The World) into second place.

Here’s the top 1012 for Q4 2009:

  1. Fred Wilson, Union Square Ventures, A VC (100,279)
  2. Guy Kawasaki, Garage Technology Ventures, How To Change The World (82,838)
  3. Paul Graham, YCombinator, Essays (71,924)
  4. Brad Feld, Foundry Group, Feld Thoughts (45,633)
  5. Mark Suster, GRP Partners, Both Sides of the Table (39,389)
  6. Bill Gurley, Benchmark Capital, Above The Crowd (23,084)
  7. Dave McClure, Founders Fund, Master of 500 Hats (21,462)
  8. Josh Kopelman, First Round Capital, Redeye VC (12,972)
  9. Bijan Sabet, Spark Capital, Bijan Sabet (12,451)
  10. Jeremy Liew, Lightspeed Ventures Partners, LSVP (12,097)
  11. Mark Peter Davis, DFJ Gotham Ventures, Venture Made Transparent (12,010)
  12. Larry Cheng, Volition Capital, Thinking About Thinking (11,851)

I kept 12 for obvious reasons. Check out the full list.

Larry, thanks for keeping tabs on all these metrics — it’s a great service to everyone looking to find knowledge in the VC and startup domains. The only problem with this methodology is that compete.com tracks only US traffic, while the blog listing is global in scope. Perhaps your next update in April 2010 can use Alexa rankings or some other novel solution.

  • Share/Bookmark