[linux-dvb] How to measure API "goodness"?

Andy Walls awalls at radix.net
Wed Sep 10 02:42:08 CEST 2008

On Tue, 2008-09-09 at 23:14 +0200, Markus Rechberger wrote:
> On Tue, Sep 9, 2008 at 10:59 PM, Simon Kenyon <simon at koala.ie> wrote:
> > On Tue, 2008-09-09 at 17:33 +0200, Markus Rechberger wrote:

>  There are multiple ways which can lead to success, the beauty
> of a patch or framework won't matter too much (nevermind
> if Steven's or Manu's work seems to be more prettier to someone).

This leads into something I've been thinking about the past few days
that's probably worth discussion out loud: 

What are the attributes to measure for comparing APIs or API proposals?
How can each attribute be measure objectively (if possible)?
What are the units for each measurement attribute?
What weight should be given to each attribute?

I've seen several suggestions in the threads already for attributes that
could be considered in a comparison:

1. Complexity (internal to the kernel)
2. Complexity (visible to the application)
3. Extensibility/Future adaptability
4. Implementation maturity (if one exists already)
5. Number of currently supported devices
6. Number of applications already using an implementation
7. Status of an implementation in the kernel (already there, leverages
or consistent with another API, etc.)
8. Ease of use for applications
9. Elegance/Beauty 

I'm sure I've missed some that were discussed, but it doesn't seem that
everything in the list above all are relevant to an API comparison, and
there could very well be things missing from the list.

I was going to look for some CS journal article which may provide
insight into metrics for performing such a comparison, but I haven't
found the time.

But I was thinking it reasonable that metrics, that get the most weight
in an evaluation, be in line with the purpose of an API: 

   Provide a well defined interface, that is consistent over time, which
   applications can call and whose source code can remain insulated from
   differences and changes in the underlying service, for some
   (unspecified) period of time into the future.

(I made that up.)  

That leads me to think that maybe the most important measures should be:

1. Projected invariance of the application facing side over time.

2. The amount of application code that would be forced to change given
forseeable changes or growth in the API due to change or growth in the
underlying service.

3. The transparency of differences in the underlying service (e.g.
capture devices from different manufacturers or using different
chipsets) to the applications calling the API.

4. The functionality provided to applications to deal with differences
that cannot be made transparent to the application.

5. The feasibility of maintaining the desirable properties of an API
while kernel software maintenance move forward.

Beauty, complexity, existing implementations (out of kernel), and ease
of use don't really rank, given my made up definition of an API.
(libX11 isn't an easy to use API, but it has stood the test of time.)

Given the back and forth on the list, I thought some discussion on how
one might perform a technical evaluation of an API may be productive.
The list conversations on certain point aspects of API proposals, would
benefit from rough concensus on how API "goodness" should be measured in
the first place, instead of arguing over perceptions/measurements that
may not be that important to a "good" API.


More information about the linux-dvb mailing list