Thursday, November 15, 2018

Measuring Information Resource Value, Part 4: Users

We saw in previous entries in the MIRV series that you need two types of metrics: quantitative and qualitative.  Users are applicable to both sets of metrics: you need users to measure usage, and you need to know who to survey to determine the business impact of a service.

But before you can run your quantitative and qualitative analyses, you need to define what actually constitutes a user.

Wait - is this even controversial? You have no idea. (Actually, you probably do). Let's take a look at the problem.

Broadly, there are two different categories of agreements - Per Seat and Group.

The Per Seat User
Among the dimensions that are in constant tension between you and your vendors is how you define a user.

Back in the day, you would buy a subscription to Horizon Research that had 15 seats. Fifteen seats = 15 users. Right?

Nominally, that's true. And you can bet the vendor would love the discussion to end there. But you don't want it to -- at least in the context of determining value. You need more detail about how a service is being used.

How you define a user can, and should, be scoped by each user's usage.  (I will discuss usage metrics in Part 5 of the MIRV series at length). 

In the above example: of the 15 users, only 10 logged in more than once per month. And of those ten, only 8 used viewed on average more than a page per login. In fact, the usage distribution for Horizon in your company is Pareto distributed: 80% of usage is from just 3 users.

So you've got 3 users who've never logged in. You've got another 2 who logged in only 4 times all year. So you could argue only 10 users really count for your ROI analysis. But worth noting: 3 of these ten account for almost 80% of usage!

So how do you define User here: 3, 10, 12, or 15? My money is on 10. Why not 3? Because you don't know the business impact of the usage from your active-but-limited 7 users. You need to consider the qualitative metrics I discuss in Part 3 of the MIRV series.

In other words: The definition of user is inextricably linked to usage. You may have seats that are extraneous to your value analysis. Be ruthless in disqualifying your casual or inactive users in order to come up with a meaningful footprint for the resource. And then you can proceed to assess the business impact (i.e. qualitative metric analysis).

The Group User
Nominally defining the number of users for a per seat license is pretty simple. Even netting out casual or inactive users, while subject to debate, is easy to define.

But what about when you have a group license? What if you're licensing a web app or even a data feed that doesn't come with a fixed number of subscribers, seats, users, etc.?  Typically, a vendor will try to define your group as broadly as possible, including as many potential users as possible.

You will want the opposite: you'll want to define the user base as narrowly as possible because you want the license (and the cost) to reflect the actual value of the service to the firm.

For example: Horizon is offering you enterprise access to their syndicated research for $120,000 a year. Awesome! Especially when you consider they were charging $42,000 for just 6 licenses. As the vendor explained to you: we're giving you access for all 60 people in your firm for just $2,000 per year instead of $7,000 for the 6-seat license.

But hold on - the vendor is arguing that everyone in your company is a user. This doesn't reflect the reality of how the service will be used, and by whom. Consider the many facets you should apply to your user base to come up with a more realistic denominator to define User.

- 40 of 60 people in your company are revenue generating (P&L)
- 30 of the 40 revenue-generating employees are in the division that needs the service (Division);
- 10 of the 30 users in your division are senior staff who won't be doing research (Rank);
- 15 of the twenty junior staffers work in consulting, and 5 are in sales, who don't need the service (Role)

That takes your Actual User base down to 15 people within a firm of 60. That's a nominal per-seat cost of $8,000 per Actual User.

(I talk at length about enterprise versus per seat decision-making here.)

Whatever number you decide to use, your next step in defining users in a group license is to apply the Per Seat user instructions I offer above: you now need to look at usage. Usage stats almost always whittle the number down further - thus increasing the per seat cost. In this example, of the 15 Actual Users - 5 of those 15 may have never logged in, leaving 10 users and increasing the per seat cost to $12K!

Whether you're dealing with a per-seat license or a group license, you need to define user thoughtfully. Critical to that process is Usage, the subject of Part 5.

- Kevan Huston

No comments:

Post a Comment