BPAI Judicial Diversity and Appeal Win Rate

I recently created a database of 10,000 recently released ex parte decisions by the Board of Patent Appeals and Interferences (BPAI) with dates ranging from November 2009 to June 2011. These ex pare BPAI decisions are easy to work with because they always involve a patent applicant appealing an examiner’s rejection of one or more claims. As I discussed in previous work, vast majority of ex parte BPAI decisions involve questions of obviousness and the remainder most often involve anticipation.

General Results: Of BPAI opinions in my sample decided on the merits, about 52% fully affirmed the examiner rejections, 34% reversed, and 14% affirmed-in-part. (About 8% of the decisions were not included in this study because they involved other outcomes such as remands and decisions on motions for rehearing). 

Judicial Diversity: These averages hide what appears to be major differences within the ranks of BPAI Administrative Patent Judges.  To reach this conclusion, I identified the judge that authored each opinion as well as the other judges associated with the decision.  For the 76 most active judges, I then calculated for each judge the percentage of cases where he or she at least partially sided with the patent applicant — i.e., either reversed or affirmed-in-part.  The chart below shows a histogram of the outcomes — showing that there is a wide diversity in expected outcomes depending upon the judges on the panel.

PatentLawImage136

At the low end, about 14 of the 76 judges at least partially side with the patent applicant in less than 40% of cases. At the high end, 10 of the 76 judges at least partially side with the patent applicant in more than 60% of cases.

The statistics here are fairly powerful because each judge in the sample is involved with at least 100 decisions and the median judge was involved with 370 decisions.  Thus, based upon our sample size we can easily reject the null hypothesis that the judges all have the same likelihood of at least partially siding with the patent applicant. And, while there are some differences between appeals arising from the various technology centers, those differences do not explain the judicial diversity shown above.  An additional potential confounding factor is the fact that the assignment of judges is not entirely random — however, in these ex parte cases judges appear to be chosen based on technical expertise rather than the likelihood that the patentee should win.

BPAI Backlog: As an important aside, the BPAI’s backlog of pending appeals continues to grow. Thus far in FY2011, the backlog has grown more than 25% and now rests at over 22,000 pending appeals.

61 thoughts on “BPAI Judicial Diversity and Appeal Win Rate

  1. 61

    “I wonder what a reversal rate of 2/3 indicates.”

    Let me guess, “outstanding” ratings for 60+% of the examining corps, SPE’s, QAS’s, TC Directors, etc?

    Lulz

  2. 60

    My question was rhetorical.

    Your question was wrong – rhetorical or not. You do not get to take the question out of context – that is dishonest.

  3. 59

    Truth in Stats is not me.

    But I did almost the same study about three years ago. I got a lot of statistics by Freedom of Information Act request, etc. etc.

    My result was strikingly similar — examiners affirmed 15-20% of all appeals for almost every year 2000-07.

    My sample ended just before KSR. For the last couple years, examiner win rates have been higher, because the appeal commenced under pre-KSR law, and was decided post-KSR. We’ll see where it settles in a few years.

    The upshot is pretty clear — most appeals arise out of examiner error, not the nefarious applicant motives attributed above. I won’t say they don’t exist, but they’re pretty small.

    At least one court noted that a reversal rate of 1/3 suggests that an agency’s procedures may be so flawed as to violate Constitutional due process. Mattern v. Weinberger, 519 F.2d 150, 161 (3d Cir. 1975). I wonder what a reversal rate of 2/3 indicates.

  4. 58

    My question was rhetorical. My point is that the “pool of data” being discussed in Dennis’ post or in the quoted comment is woefully incomplete for statistically addressing the question: “how often is an examiner “completely right in what he does?”

  5. 57

    “affirmed-in-part” says nothing about what happened to the claims that were NOT affirmed (not always reversed), so it is incomplete.

  6. 56

    In your cut and paste of the comment, you evidently missed:

    after the Appeals process is started

    That rather answers your question.

  7. 55

    Let’s ask the question, “How often is an examiner held to be completely right in what he does?”

    The full pool of data includes not just the “end data” in Dennis’ study of final Appeals decisions, but the 75% that is kicked back after the Appeals process is started (hence the term used here of “cherry-picking” is indeed apt). That 75% completely works against the examiner.

    Doesn’t the full pool of data also include cases that are granted or rejected without appeals?

  8. 54

    It’s pretty hard to get an SPE in an interview when the examiner’s a primary. I’ve never requested a QAS be present, but I guess one could ask that they take time out of their busy day of doing absolutely nothing and attend an interview. Not sure what the point would be. They’re useless in the conferences, so what use would they have in an interview?

    You’re being disingenuous and you know it. Arguments that are made at interviews and written responses are routinely deemed “not persuasive,” but when a pre-appeal request or appeal brief actually gets filed, magically those same arguments are suddenly persuasive and prosecution is re-opened. The only solution to these shenanigans is to require the Director (i.e. Mr. Kappos, currently) or the Commissioner (i.e. Mr. Stoll, currently) to approve any re-opening after an appeal brief is filed. And the examiner would be required to explain, in writing, why prosecution needs to be re-opened at that stage in the prosecution and applicant must be allowed to respond in writing why it should not.

    According to some anon examiner upthread, as there’s no re-opening going on, at least not in his little world, there shouldn’t be any objection to such a system.

  9. 53

    US Appeals Court decisions tend to say both ‘affirmed-in-part’ and ‘reversed-in-part’.

  10. 51

    I’ve stated elsewhere that I’m down with that. However, you know, you can already just have an interview and request the SPE and the examiner. You already have, in that room, enough people to make the decision as usually it is pretty much a 3 person vote as to whether or not the case goes on up, usually the SPE required as being on board, but, apparently not always. And of course, if you persuade the examiner/SPE then the case wouldn’t go to appeal anyway because appeals guys don’t send cases up.

    Bottom line, learn to be persuasive and you already have an “open” appeal conf. for all practical purposes.

  11. 49

    No I wouldn’t. Mr. Kappos needs to end the charades of these “conferences” and let applicants and their representatives in.

    Let us in, Mr. Kappos!

  12. 48

    “That’s looking at the glass as being half-empty. Shouldn’t that be “14% reversed-in-part.” if you view the glass as being half full?”

    I think that is just the legal terminology prevalent in the decisions.

    “Seriously, doesn’t that show bias on the BPAI’s part that it is ‘affirmed-in-part’ rather than ‘reversed-in-part’?”

    Um no?

  13. 47

    “Don’t want to go through the appeal process? What, it’s too hard to pull a QAS away from their lunch two hour to sign off on a notice of panel decision that’s already been filled out “proceed to BPAI”?”

    You apparently would be surprised.

  14. 46

    My clients have 1000’s of patents in prosecution with 1-3 in house counsels. Those guys are also responsible for securing protection on NEW inventions and picking what to file apps on.

    Which do you think they focus their efforts on: pruning the existing portfolio or securing protection on new product lines? Obviously it’s the one that would be a disaster if not kept up, not the one that adds a bit of ongoing cost.

  15. 45

    “14% affirmed-in-part.”

    That’s looking at the glass as being half-empty. Shouldn’t that be “14% reversed-in-part.” if you view the glass as being half full?
    Seriously, doesn’t that show bias on the BPAI’s part that it is ‘affirmed-in-part’ rather than ‘reversed-in-part’?

  16. 44

    Don’t forget to include in the statistics the scandalous practice of crappy final rejection, followed by panel review resulting in re-opening of prosecution on the merits. Even if this practice results in the Examiner issuing the final rejection that the appplicant SHOULD have received in the first place, e.g. better developed arguments as should properly be the case for a final rejection, it involves cost, delay and unnecessary administrative gymnastics.

  17. 43

    Don’t want to go through the appeal process? What, it’s too hard to pull a QAS away from their lunch two hour to sign off on a notice of panel decision that’s already been filled out “proceed to BPAI”?

    Lulz

  18. 42

    Why of course, 6, regardless that the reference is anticipating, they may argue, as others here seem to think proper, that the reference is in a non analogous art and it would not be obvious for one of ordinary skill to look there.

  19. 41

    I mean seriously if they will fight an explicit disclosure of their invention in a statutory bar, while ADMITTING that it is disclosed, they will appeal anything.

  20. 40

    Yeah IBM will appeal just about anything. I’m not even sure how the numbers stack up, but they’ve appealed some whoppers from me. And they were going to appeal some other ones but I finally learned how best to deal with them if you’ve indicated some claims as allowable already. Just suggest to them that they take the claims you’re offering and go after the others in a con and then let them fight to the death on a question like if a reference discloses a lot of different species including the one their claiming, does the reference anticipate? Of course there is some caselaw saying that such a situation might not be anticipation in some instances but in others it surely is.

  21. 39

    “and the erstwhile creater of value.”

    Lulz, like what, one in every 1k applications are for something “of value”?

  22. 38

    “You can take a look at the numbers that Ron Katznelson put together”

    Are they as accurate as his overall numbers were? Off only by a couple of double digits? Or are his numbers off a little more in this instance?

    Ron is a joke and he’s also apparently not very good at math or developing stat methodologies.

  23. 37

    “That, clueless examiner, is a stinging indictment of the examining corps.”

    That they don’t want to go through the appeals process?

    Yeah, some indictment.

  24. 36

    “Except you royal prcck, the applicant has something that he wants to protect and the examiner is merely a bureaucrat getting “points”.”

    Lulz, and that has to do with what in the comment you responded to precisely?

    “the pathetic examiners have mucked up that route as well.”

    I’m not so sure why one amendment should not be enough considering you can get mini actions by doing an interview.

    Bottom line is, if you’re not getting cases allowed and are all the time appealing, you probably suc k just as much as your examiner does, though it would pain you deeply to admit it and you probably don’t even know it.

  25. 35

    R. Katznelson Protege,

    being completely right ~14% may seem low, but if you actually look at the rejection-by-rejection breakdown, the Examiner is affirmed well over 50% of the time per rejection. I’m not saying that’s great, but it’s not devastatingly horrible (I think Fed Appeals courts affirm a little over 50% of the time on de novo issues – does anyone have that number handy to verify?).

    Also, the cases being decided right now include a lot of rejections actually made in 2007-2008, which, by my experience, was the most “creative” time for Examiners seeking any means to avoid allowance. So a lower affirmance rate for now is reasonable. Let’s wait a few years and see what happens before freaking out.

  26. 33

    Let’s take a look at this picture in a slightly different manner.

    Let’s ask the question, “How often is an examiner held to be completely right in what he does?”

    The full pool of data includes not just the “end data” in Dennis’ study of final Appeals decisions, but the 75% that is kicked back after the Appeals process is started (hence the term used here of “cherry-picking” is indeed apt). That 75% completely works against the examiner.

    The remaining 25% must actually be broken into the 8% and 92% groups that Dennis notes above (the 8%, even being “other” means that those decisions do not fully support the examiner).

    Further, the 92% of the 25% is then broken down into the three groups that Dennis notes. As the “split group” of 14% is unknown, we will generously split this in half and give the examiner 7% of the 92% of the 25% (even thought the examiner is not actually completely right in these cases). The actual 52%, while on its face seems almost an even coin flip, is actually only 52% of the 92% of the 25%.

    Fully multiplying all percentages out, the examiner is only (and genreously) completely right in 13.6% of the total cases that begin down the Appeals path (and only 12% of the total cases if we do not split the 14%).

    That, ladies and gentlemen, is clearly an indictment.

  27. 32

    There is nothing more funny than a poster trying to be funny and making a joke out of something someone posts in error – two days after that poster has noted his error.

    Nice job Feigin – way to advertise that people should go elsewhere for services that include the power of observation.

  28. 30

    “Thanks for the laugh. Not sure where you’re getting your 75% number…”

    You can take a look at the numbers that Ron Katznelson put together. Or you can go on being one ingnunt examiner. My guess is you’ll choose the latter.

  29. 29

    Thanks Dennis.

    Would also be interesting to know those companies / entities with the largest raw numbers of filed appeals; as well as how said numbers correlate with the number of apps / issued patents for each company / entity during this same time period.

    Just who are the most prolific appealers?

  30. 28

    AP,

    You hit reply to, and cut and paste a quote from, “cherry picker”, but you address your comments to me. On top of that, I take as snide your nearly verbatim “parody” because you have no right to make such a parody, as the rest of my comments at Jun 17 10:25 PM show: you cannot simply switch sides because the dynamics simply are different between the sniveling bureaucrat and the erstwhile creater of value.

    Rather than playing cute, why don’t you actually address the points I make. And while you are at it, address the points of others who have also posted. Rather than merely say my logic is flawed, prove it.

  31. 27

    “This is one dumm [sic] examiner.” Thanks for the laugh. Not sure where you’re getting your 75% number, but they don’t correspond to this “dumm” examiner’s statistics. The only re-opening in my cases that has happened after a pre-appeal or appeal has been filed has been when the Applicant has filed an RCE after the preappeal response or examiner’s answer.

    @anon – there were no snide comments in my nearly verbatim parody of your logically flawed rant. The initial set of appeals from which you say the office “cherry picks” should already be extremely skewed toward the Applicant since you’re supposedly appealing in the belief the the examiner is 100% wrong on every outstanding rejection. Since you still can’t see why your logic is flawed, there is no point continuing this conversation.

  32. 26

    “But in my view much of the latter is due to companies that do not maintain effective legal-cost-control systems and maintenance-review system to make rational decisions on whether or not by then [usually years after filing, when the subject matter may already be obsolete] the rejected patent claims are even potentially patenable AND worth the costs of persuing them further. [Including in-house patent attorneys who do not care enough about regularly wasting their company’s money to want to make such decisions. Hence, in effect, giving their IP firms carte blanch to bill as much as they want for appealing anything and everything.]”

    Dude, where you been the last 10 years? In-house attorneys who don’t care about wasting their company’s money? Get a clue.

  33. 25

    PhilS,

    I believe that Dennis ran such numbers awhile back. Not surprising, the numbers showed increasing “productivity” near the end of every quarter with a marked increase near the time raises were decided.

    Just some more legal realism.

  34. 24

    The numbers are not all that surprising to me as I believe the “broadest reasonable claim interpretation” is used by the BPAI as well.

    However, the seemingly low numbers siding with the applicant could also be due to increased production pressures to reduce the backlog of cases under appeal. I’m not aware of any criteria used to evaluate board productivity.

    Anyone have info on the latter would be appreciated.

  35. 23

    And what I mean by controlling for time is the filing date of the applications in question. The decisions themselves are from a short time period and so they are controlled for time. However, if the filing dates of the applications in question span a large range of time, then time-based effects (changes in the law & the economy) can affect the statistics.

    A filing date filter (any 3 year period that did not have a significant change in direction in the law or economy) should be an easy filter to apply, and it would help eliminate time as a factor.

  36. 22

    Is the data controlled for the time period considered? That is, for a given judge, is the distribution over time more or less constant and over the same period?

    Over time, the law swings between a pro- and anti-patent stance, and if the data is not controlled for time, then that can also muddle the statistics.

    Just a thought…

  37. 21

    Paul, I fully agree.

    This somewhat echoes my own concerns about mainly defense litigation counsel who will take advantage of weak inside counsel.

  38. 20

    Somebody commits Trademark Infringement.. Makes me go broke. then puts me in Their Box.. To bad it ain’t at a UT GAME.. That will be another setting on Principle. and while your at it.. How about the RED SOX too. Life time Boxes. After all I spent so much time in one, I am comfortable that way.

  39. 17

    The appeal backlog and the rejection/reversal rates are in part a function of BOTH unwarrented examiner rejections and money-wasting applicant appeals on clear “losers.” Some of the latter may be for a good reason – applicants wanting to keep applications pending longer to be able to amend claims later. But in my view much of the latter is due to companies that do not maintain effective legal-cost-control systems and maintenance-review system to make rational decisions on whether or not by then [usually years after filing, when the subject matter may already be obsolete] the rejected patent claims are even potentially patenable AND worth the costs of persuing them further. [Including in-house patent attorneys who do not care enough about regularly wasting their company’s money to want to make such decisions. Hence, in effect, giving their IP firms carte blanch to bill as much as they want for appealing anything and everything.] Foreign companies that fail to timely respond to their U.S. law firms request for instructions are another source of such money-wasting backlog-clogging loser-appeals. A law firm cannot just make a decision on its own to abandon an application without client approval.

  40. 16

    As I see it someone took Wimpy’s Ants and made it a HILL of Beans? Wimpy said less Ants. And those weren’t the real ants Wimpy was talking about. Now “who” did that to poor Wimpy. When he finds out,Wimpy will have his Friends to help stomp the beans.
    Refer to above

  41. 15

    So as I see it, and have stated it. We are in a position because of these terrible disasters to take it back… So what do we do? We have Commercials on Toyota’s soft side giving Cars to deserving Organizations to help their bottom line. What about our bottom line? Chop Chop get to it.

  42. 14

    “The elephant in the room is the fact that these cases should be much more skewed toward the Applicant, and the entire process affords ample opportunity for the Applicant to cherry-pick the cases to proceed with (and which ones to RCE). The only time Applicants go to appeal is when they believe that the examiner is wrong on all the rejections, which should skew the appeals much more toward reversal.”

    This is one dumm examiner. In case you hadn’t noticed, about 75% of cases in which a pre-appeal and/or appeal brief are filed never make it to the Board because the examiner re-opens prosecution. That, clueless examiner, is a stinging indictment of the examining corps.

  43. 13

    “My guess is Mssr. John Jeffery and Jay Lucas are in the 30% block.”

    Add Catherine Timm to the 30% group. You know we’re right too, DC.

  44. 12

    I am not sure that the blame should be entirely placed on attorneys/agents for the affirmation rate.

    How many clients demand that cases are appealed regardless of (or with very little) merit?

    One would think that this stance might make some sense from the large client standpoint: If there is a 7% chance of complete reversal or affirmed-in-part (likely a very conservative number) on a baseless appeal and a 99% chance of that case being subsequently allowed, perhaps the numbers in the aggregate favor this strategy.

  45. 11

    Except you royal prcck, the applicant has something that he wants to protect and the examiner is merely a bureaucrat getting “points”.

    For the applicant, his choice is to forego protection (not likely – especially if tied to funding), or put up with the shtt from such bureaucrats. Your so-called cherry-picking is totally one-sided. In case you haven’t noticed, the RCE pipelines are full too, and even though the gravy train has slowed (it has not stopped), the pathetic examiners have mucked up that route as well. So yet again, your snide comments are wrong.

    Now ask me how I really feel.

  46. 10

    Faulty logic: “these cases should be much more skewed to the examiner.”

    The elephant in the room is the fact that these cases should be much more skewed toward the Applicant, and the entire process affords ample opportunity for the Applicant to cherry-pick the cases to proceed with (and which ones to RCE). The only time Applicants go to appeal is when they believe that the examiner is wrong on all the rejections, which should skew the appeals much more toward reversal.

    Even with such cherry-picking by Applicants, the fact that only a third fully find the examiner wrong is a clear indictment of the patent bar.

    While Congress fritters, the real focus of attorney/agent quality continues to be missed.

  47. 9

    it would be great if you could also identify the examiners involved. this would allow applicants to identify the examiners being reversed.

  48. 8

    A VERY interesting comparison would be a chart that shows the distribution as a function of those judges who at least partially favor the examiner. It’s not the inverse of this distribution.

    The overall number of affirmed or affirmed in part is about 2/3.

  49. 6

    I would loooooove to see the data on the individual judges.

    My guess is Mssr. John Jeffery and Jay Lucas are in the 30% block.

    For those who are interested, Jay Lucas was once the Deputy Commissioner for Patent Policy (and acting Deputy Commission of Patent Quality) — during the Dudas reign of terror. As such he was one of the architects and chief spokesperson for the “10 representative claims,” “Examination Support Document” (aka document of death), limited continuations/RCEs, and the belief that the USPTO has substantive rule making authority among other “applicant friendly initiatives.”

  50. 5

    The elephant in the room is the fact that these cases should be much more skewed to the examiner, and the entire process (pre-appeal) affords ample opportunity for the Office to cherry-pick the cases to proceed with (and which ones are kicked back to the examiner).

    Even with such cherry-picking, the fact that at least a third fully find the examiner wrong is a clear indictment.

    While Congress fritters, the real focus of examination quality continues to be missed.

  51. 4

    The “vast majority of ex parte BPAI decisions involve questions of obviousness and the remainder most often involve anticipation”, but nobody is interested in drilling down into what “teaching away” really means.

  52. 3

    Dennis, your bar chart distribution looks surprisingly tightly grouped around 40%? Compared, for example, with the widely divergent opinions between different members of the Fed. Cir. on claim constuction patent infringement decisions?

Comments are closed.