Law Firm Performance Metrics: Broadening Our Discussion

Last month's medley of views on law firm ranking metrics, while diverse, by no means exhausted what A&FP Board members and other contributors have to say about this important subject. The following two mini articles continue to address ranking-related problems, but will also help us broaden the scope of our discussion.

Ranking metrics are but the highly visible tip of a veritable iceberg of law firm performance measurements, most of which are used for internal financial monitoring and decision support. While each performance metric generally has some theoretical value as an indicator, each is no doubt also subject to misuse.

Jim Davidson describes in greater detail an issue touched on briefly by some previous authors: the fundamental challenge of metric standardization. Jim's analysis provides a clear-eyed view of just how daunting this problem is.

Ed Wesemann acknowledges the familiar criticism that snapshot metrics shouldn't be used to draw conclusions about long-term performance. He then takes the discussion to a new level by proposing a revealing new indicator for internal assessment purposes. As last month's authors (notably Michael Mooney) have observed, many firms have difficulty deciding how to react to their public rankings. Ed's engagingly simple Jaws measure provides one way to determine whether the drink served at a firm's first post-survey meeting should be champagne or Pepto Bismol.

' Joe Danowsky, Editor-in-Chief

Ranking Requires Metric Standardization

By Jim Davidson

Data discrepancies and manipulations make it difficult to come up with meaningful rankings among law firms. While some discrepancies are due to organization type and other structural factors, manipulations are often driven purely by the motivation of survey respondents to make their firms look good. Some of the latter “gaming” may not really be intentional, however. As described below, it arises from confusion as to what to include or exclude in which data element.

Whatever the source, such distortions in the ranking survey databases distort more than the results for the individual firm; all comparisons to that flawed data are skewed as well, albeit to a lesser extent. If, say, 10 firms in a database of 100 are gaming some statistic ' not necessarily the same statistic ' then the integrity of the database as a whole is in question.

All of this is true in the corporate world as well, but I suspect to a much lesser extent. The corporate world is constrained by its own trade associations, as well as the SEC and other government watchdogs; so, apart from cases of outright deception, corporate statistics are subject to less manipulation than those used in our generally secretive industry.

PriceWaterhouse, Altman Weil, Citi and others who gather statistics from law firms do make varying attempts to standardize the input they request. But even some of these major surveys invite self-serving responses by giving skimpy definitions of data elements they request, or invite discrepancies by asking for the same data in multiple formats. (This is not to suggest that law firm surveys should emulate the IRS by publishing 30 pages of instructions for a one-page form!)

Equally important, most surveys also fail to audit the data input for accuracy and consistency. Alas, even the most elaborate methodology among them is subject to data manipulation that would not become obvious in the verification process.

In short, I take all the statistical data currently published with a whole shaker-full of salt. This is not to say that the statistics are worthless, but they are useful only in an overall macro-type context.

I question particularly the data by partner. There are about as many definitions of an equity partner as there are law firms. Some of them don't even require an equity partner to have equity! For that matter, what constitutes equity? Some firms have lawyers who lack an ownership interest in the form of cash but are partners in all other respects.

Moreover, many firms have tiers of partners, and where do you draw the line? Some of the firms reporting astronomic profits per partner report only their top tier as “true” partners, although those in lower tiers may have equity and be the equivalent of equity partners in other firms.

Data by lawyer is less subject to this kind of manipulation, but not entirely exempt. Partners (shareholders) and associates in this context are pretty much a given, but other lawyers are “of counsel.” Do you count them or not? What about those who are paid, sometimes handsomely, for doing little or no work ' who are there as business development tools or for the prestige their name may provide? And what about those who are really retired but collecting pay for work they did in the past?

Finally, some of the published rankings are awfully simplistic in their data gathering process. While I'm sure that's a plus from the standpoint of getting firms to participate, it opens up many avenues for creative interpretation by individual firms, which can materially skew the results.

I wish I had an easy answer to this, or even a complicated answer. But until firms can come to some agreement on definitions, and until auditing of survey data input is improved, I fear an answer will elude us.

Jaws: A Cumulative Performance Metric

By Ed Wesemann

[Editor's Note: This intriguing suggestion for a new profitability measure is adapted from Ed's March 2004 e-mail advisory to the client base of Edge International and other interested individuals.]

Law firms can be incredibly short-term oriented. In the U.S. we can, in part, blame the tax code, which causes firms to view everything on a cash in/cash out basis. But that can't be the whole problem, because law firms in Canada and Europe tend to be just as short sighted ' and most of them are taxed on the accrual basis. Also in part, a short-term viewpoint may be a function of the revolving partnership door at many firms. The people participating in this year's profits may be a different group than will participate in next year's, so current-year profits are the primary focus of everyone's attention. Perhaps it is systemic to businesses in general, brought on by the way that corporate shareholders seem to focus on current dividends and stock prices rather than long-term positioning.

Whatever the motivation, short-term viewpoints make it tough for law firm managing partners who are charged with delivering profitable performance in the immediate term but are also expected to have a vision that builds the firm in the long run.

So how does a law firm's partnership judge the firm's financial performance? Let me suggest a statistic for law firms to look at in judging their long-term success: Jaws.

The Jaws graph, below, depicts the cumulative percentage increase in revenues compared to the cumulative percentage increase in expenses over a 3- to 5-year period. If the jaws are opening ' revenue growth exceeds expenditure growth ' then the firm is growing successfully in a manner that enhances profitability. On the other hand, if the jaws are closing ' expenditure growth exceeds revenue growth ' then you've got a problem.

[IMGCAP(1)]

It's important that this be looked at in nothing less than a 3-year basis. An investment that causes an abnormal growth in expenditures in one year may not show up on the revenue side in the current year.

Jim Davidson recently retired from Denver's Holland & Hart, where he served for 25 years as Director of Finance.
Ed Wesemann has been writing and consulting on law firm cultural and strategic issues for over 20 years. Reach Ed at the Savannah office of Edge International, at 877-922-2040 or [email protected]. Copyright (c) 2004 by H. Edward Wesemann. Reprinted by permission.

Last month's medley of views on law firm ranking metrics, while diverse, by no means exhausted what A&FP Board members and other contributors have to say about this important subject. The following two mini articles continue to address ranking-related problems, but will also help us broaden the scope of our discussion.

Ranking metrics are but the highly visible tip of a veritable iceberg of law firm performance measurements, most of which are used for internal financial monitoring and decision support. While each performance metric generally has some theoretical value as an indicator, each is no doubt also subject to misuse.

Jim Davidson describes in greater detail an issue touched on briefly by some previous authors: the fundamental challenge of metric standardization. Jim's analysis provides a clear-eyed view of just how daunting this problem is.

Ed Wesemann acknowledges the familiar criticism that snapshot metrics shouldn't be used to draw conclusions about long-term performance. He then takes the discussion to a new level by proposing a revealing new indicator for internal assessment purposes. As last month's authors (notably Michael Mooney) have observed, many firms have difficulty deciding how to react to their public rankings. Ed's engagingly simple Jaws measure provides one way to determine whether the drink served at a firm's first post-survey meeting should be champagne or Pepto Bismol.

' Joe Danowsky, Editor-in-Chief

Ranking Requires Metric Standardization

By Jim Davidson

Data discrepancies and manipulations make it difficult to come up with meaningful rankings among law firms. While some discrepancies are due to organization type and other structural factors, manipulations are often driven purely by the motivation of survey respondents to make their firms look good. Some of the latter “gaming” may not really be intentional, however. As described below, it arises from confusion as to what to include or exclude in which data element.

Whatever the source, such distortions in the ranking survey databases distort more than the results for the individual firm; all comparisons to that flawed data are skewed as well, albeit to a lesser extent. If, say, 10 firms in a database of 100 are gaming some statistic ' not necessarily the same statistic ' then the integrity of the database as a whole is in question.

All of this is true in the corporate world as well, but I suspect to a much lesser extent. The corporate world is constrained by its own trade associations, as well as the SEC and other government watchdogs; so, apart from cases of outright deception, corporate statistics are subject to less manipulation than those used in our generally secretive industry.

PriceWaterhouse, Altman Weil, Citi and others who gather statistics from law firms do make varying attempts to standardize the input they request. But even some of these major surveys invite self-serving responses by giving skimpy definitions of data elements they request, or invite discrepancies by asking for the same data in multiple formats. (This is not to suggest that law firm surveys should emulate the IRS by publishing 30 pages of instructions for a one-page form!)

Equally important, most surveys also fail to audit the data input for accuracy and consistency. Alas, even the most elaborate methodology among them is subject to data manipulation that would not become obvious in the verification process.

In short, I take all the statistical data currently published with a whole shaker-full of salt. This is not to say that the statistics are worthless, but they are useful only in an overall macro-type context.

I question particularly the data by partner. There are about as many definitions of an equity partner as there are law firms. Some of them don't even require an equity partner to have equity! For that matter, what constitutes equity? Some firms have lawyers who lack an ownership interest in the form of cash but are partners in all other respects.

Moreover, many firms have tiers of partners, and where do you draw the line? Some of the firms reporting astronomic profits per partner report only their top tier as “true” partners, although those in lower tiers may have equity and be the equivalent of equity partners in other firms.

Data by lawyer is less subject to this kind of manipulation, but not entirely exempt. Partners (shareholders) and associates in this context are pretty much a given, but other lawyers are “of counsel.” Do you count them or not? What about those who are paid, sometimes handsomely, for doing little or no work ' who are there as business development tools or for the prestige their name may provide? And what about those who are really retired but collecting pay for work they did in the past?

Finally, some of the published rankings are awfully simplistic in their data gathering process. While I'm sure that's a plus from the standpoint of getting firms to participate, it opens up many avenues for creative interpretation by individual firms, which can materially skew the results.

I wish I had an easy answer to this, or even a complicated answer. But until firms can come to some agreement on definitions, and until auditing of survey data input is improved, I fear an answer will elude us.

Jaws: A Cumulative Performance Metric

By Ed Wesemann

[Editor's Note: This intriguing suggestion for a new profitability measure is adapted from Ed's March 2004 e-mail advisory to the client base of Edge International and other interested individuals.]

Law firms can be incredibly short-term oriented. In the U.S. we can, in part, blame the tax code, which causes firms to view everything on a cash in/cash out basis. But that can't be the whole problem, because law firms in Canada and Europe tend to be just as short sighted ' and most of them are taxed on the accrual basis. Also in part, a short-term viewpoint may be a function of the revolving partnership door at many firms. The people participating in this year's profits may be a different group than will participate in next year's, so current-year profits are the primary focus of everyone's attention. Perhaps it is systemic to businesses in general, brought on by the way that corporate shareholders seem to focus on current dividends and stock prices rather than long-term positioning.

Whatever the motivation, short-term viewpoints make it tough for law firm managing partners who are charged with delivering profitable performance in the immediate term but are also expected to have a vision that builds the firm in the long run.

So how does a law firm's partnership judge the firm's financial performance? Let me suggest a statistic for law firms to look at in judging their long-term success: Jaws.

The Jaws graph, below, depicts the cumulative percentage increase in revenues compared to the cumulative percentage increase in expenses over a 3- to 5-year period. If the jaws are opening ' revenue growth exceeds expenditure growth ' then the firm is growing successfully in a manner that enhances profitability. On the other hand, if the jaws are closing ' expenditure growth exceeds revenue growth ' then you've got a problem.

[IMGCAP(1)]

It's important that this be looked at in nothing less than a 3-year basis. An investment that causes an abnormal growth in expenditures in one year may not show up on the revenue side in the current year.

Jim Davidson recently retired from Denver's Holland & Hart, where he served for 25 years as Director of Finance.
Ed Wesemann has been writing and consulting on law firm cultural and strategic issues for over 20 years. Reach Ed at the Savannah office of Edge International, at 877-922-2040 or [email protected]. Copyright (c) 2004 by H. Edward Wesemann. Reprinted by permission.