Citing References

For patents issued thus far in 2017, the average patent lists 44 references submitted by the patent applicant for the examiner to consider. The median patent, however, only has 10 applicant cited references.  The gulf between the mean and median is the result in skew in the dataset — here caused by a fairly small percentage of patents that cite a disproportionately large number of references.  Thus, if you pick a few patents at random, about half of them will have <10 references cited by the applicant.  Every once in a while, though, you’ll come across a big papi – see, for example U.S. Patent No. 9,535,563 and its 7,800 references cited by the applicant (inventor-patent attorney Steven Hoffberg).  At the other end of the spectrum, about 12% of applicants cite no prior art at all — this is down from about 20% in 2005.

One way to think about the skew is to look patents where applicants submit cite 250+ references — that represents about 3.5% of patents issued in 2017 thus far (though only 0.5% of patents issued in 2005).  Those 3.5% of issued patents account for half of the prior art submitted to the PTO.

To be clear – I’m not demonizing patentees who submit lots of references.  What I’m working toward is whether the large number of references creates any special difficulty or benefits for the examination process and whether examiners receive any special assistance in considering the large number of references submitted.

I think the chart below is interesting. I took the 2017 patents and grouped them in quintiles according to the number of references cited.  For each quartile, I then found the average pendency (in months) for the issued patents.  The results are clear – patents with more citations correlate with longer pendency.  (Note that the auto-created axis does not reach ZERO – this makes the slope appear more dramatic than reality).  Obviously we should be a bit cautious in interpreting this chart as suggesting a causal role.

ReferencesCitedQuint

 

 

 

43 thoughts on “Citing References

  1. Great observation that the number of references cited in an IDS has a direct correlation with the term of pendency! Seems obvious in hindsight (no pun intended!), but I really never thought of that. Of course, citing more references has the dual advantage of (a) being more likely that you fulfilled your Rule 56 duty of disclosure, and (b) assuming the patent is eventually granted, increasing the art that includes the benefit of the presumption of validity. However, we all want our patents sooner rather than later.

    Yes, this is a sales pitch but, SyncIDS.com has a real software solution than can give you the best of both worlds! We have a way to cite as many references as you want (automatically), and, at the same time, give the Examiner a way to quickly focus on just the relevant references without characterizing any of the references. Please send me an email to jim@syncids.com, and I will show you our solution.

    1. Re: “..we all want our patents sooner rather than later.”
      Actually, a surprising number of applicants prefer to delay issuance so as to be able to continue to amend claims to better cover competitive products or later versions of one’s own products, or to be able to make subsequently noted prior art of record. That undoubtedly adds to whatever correlation there is to prosecution time and references.
      But the Judith comment at 9 here may hit on the main factor – litigation on that or related patents.

      1. I see very very few delays of issuance, given that basic continuation practice is a much easier path to obtain all of the items that you list as reasons for delayed issuance.

        Pharma may be the rare exception (given that Pharma patent worth is much more heavily weighted to the end of the patent life spectrum).

  2. I suspect that the excessively large number of references are from cases that either are members of a large family, or are in litigation.

    Once you have one case in litigation, you will see your citation matrix explode for that family.

    If you’re doing stats, I’d be curious to see if you can separate out cases with no siblings/parents v. ones with. I suspect that would account for a significant chunk of the disparity.

    1. You should also consider applications filed in multiple countries compared to those filed only in the US, especially those countries that use a language other than English or, more particularly, a different alphabet. Each country’s search will contribute additional entries that are, ipso facto, material to patentability in this country and unlikely to be picked up by an English-language keyword search. The second or third continuation of an application filed by a joint US-Chinese research team cross-filed in 140 different countries will most likely produce a far larger number of relevant references than a single-inventor application filed only in the US.

  3. An Examiner has stated this in a rejection with me:

    “The information disclosure statements filed from X to X have been reviewed and placed of record in the file.

    Applicant submitted, from X to X, Information Disclosure Statements containing 150+ references. Due to the voluminous set of documents submitted, applicant is requested under 37 CFR 1.1 05 to provide an indication of which references are considered to be most highly pertinent to the claimed invention. The IDS and the information referred to therein has been placed in the file but not considered.

    Significantly, an applicant’s duty of disclosure of material and information is not
    satisfied by presenting a patent examiner with “a mountain of largely irrelevant [material] from which he is presumed to have been able, with his expertise and with adequate time, to have found the critical [material]. It ignores the real world conditions under which examiners work.” Rohm & Haas Co. v. Crystal Chemical Co., 722 F.2d 1556, 1573 [ 220 USPQ 289 ] (Fed. Cir. 1983), cert. denied, 469 U.S. 851 (1984).

    If applicant instead indicates that all the submitted documents are most highly pertinent, then applicant is requested to provide an explanation of the pertinence of every single cited reference in the Information Disclosure Statement. The applicants are requested to identify the relevant references including relevant sections highlighted in each of the relevant references, since they are believed to be the most knowledgeable about the content of the information included in the DS submitted.

    This information is necessary to evaluate to the large volume of prior art submitted.”

    1. Lol – here, you do my job…

      1. This was one of a couple cases I had with this guys, from a group of 50ish related cases, and he was the only one to do this. I couldn’t help but not point out the Examiner’s 3 ref 103 rejection from different technology backgrounds as the reason he should actually do his job:

        “Additionally, Applicant respectfully requests a review of all of the previously cited art that was not in the “most highly pertinent” IDS. Specifically, Applicant believes that features or aspects of the cited art may be relevant to the current (or future) claims, and that the Patent Office may take a different reading of the art, and thus implore the Patent Office to consider the cited art. While Applicant understands the Examiner’s contentions, Applicant also understands the need for the ideas and opinions of all parties to be on the record in order to ensure Patent Quality. Specifically, Applicant would not traditionally consider applications relating to [Abstract quote of Ref 1], [Abstract quote of Ref 2], and [Abstract quote of Ref 3] to be “most highly relevant” to the current case, as required by the Examiner, and thus Applicant highly suggests the Examiner uses their own independent judgment to assess the validity of EACH piece of art previously submitted in an IDS in order to determine whether the Examiner would consider any of them relevant.”

  4. The usual reason expressed for citing the entire kitchen sink contents is inequitable conduct [IC] protection. But what Fed. Cir. decision finding IC for not citing references has occurred other than Regeneron Pharmaceuticals, Inc. v. Merus N.V. (Fed. Cir. 2017) since the Court’s en banc decision in Therasense, Inc. v. Becton, Dickinson and Co., 649 F.3d 1276 (Fed. Cir. 2011) seemed to seriously blunt the IC defense?

    1. Any such “blunting” of the defense does nothing for the separate aspect of action while still in the domain of the Office.

      Paul, this point was made abundantly clear in the discussions following the case you provided.

      We do not have to ride that merry go round again.

      1. That comment would be more practically relevant if you could cite to any recent* case in which “while still in the domain of the Office” any IC action for simply not citing a reference was ever initiated by the PTO itself? Or to even a PTO OED disciplinary action initiated by the PTO itself for not citing a reference rather than in response to an IC decision in patent litigation against a third party?

        *not to two old interference decisions

        1. No need for that – just read the words of the IC case itself, as the distinction is made there.

          The practical relevance exists – regardless of the citations that you seek.

  5. The entire premise here appears to be “clickbait.”

    As to “For each quartile, I then found the average pendency (in months) for the issued patents. The results are clear – patents with more citations correlate with longer pendency. (Note that the auto-created axis does not reach ZERO – this makes the slope appear more dramatic than reality). Obviously we should be a bit cautious in interpreting this chart as suggesting a causal role.“:

    the graph is nigh meaningless.

    Not only can the “auto-created axis” be changed – it SHOULD BE changed based on the creator of the graph’s intended message. Further, lacking any basis for picking quintiles (quartiles?), there clearly is some induced error in the portrayal of data as representative. How big are those error bands? I can easily envision that the size of appropriate error bands removes the ability to fit even a simple straight line through the data that would provide the “clear” take away for the results presented.

  6. 7800 references sounds like a lot until you look at the claims of 9,535,563.

    Wow.

    1. Re 9,535,563, did Lemelson have children?
      “CROSS REFERENCE TO RELATED APPLICATIONS
      The present application is a continuation of U.S. patent application Ser. No. 13/043,411, filed Mar. 8, 2011, now U.S. Pat. No. 8,583,263, issued Nov. 12, 2013, which is a continuation of U.S. patent application Ser. No. 11/363,411 filed Feb. 27, 2006, now U.S. Pat. No. 7,904,187, issued Mar. 8, 2011, and U.S. patent application Ser. No. 11/363,393 filed Feb. 27, 2006, now U.S. Pat. No. 7,451,005, issued Nov. 11, 2008, which are a continuations of U.S. patent application Ser. No. 10/693,759 filed Oct. 24, 2003, now U.S. Pat. No. 7,006,881, which is a continuation of U.S. patent application Ser. No. 10/162,079 filed Jun. 3, 2002, now U.S. Pat. No. 6,640,145, issued Oct. 28, 2003, which is a continuation of U.S. patent application Ser. No. 09/241,135 filed Feb. 1, 1999, now U.S. Pat. No. 6,400,996, issued Jun. 4, 2002, each of which is expressly incorporated herein by reference in its entirety.” [The latter being superfluous if these were all true continuations?]

      1. Is it superfluous?

        Does not the “each of which” – being in reference to the continuation (and not just the specification) necessarily include the different prosecution histories?

        (While this seems to be a minor difference in scope of coverage when compared to the actual (non-in-the-(invited)-weeds) “getting at” topic of the scope of the IDS material, I do not think that the thrust of your comment is correct.

        1. The “thrust of the comment” was obviously humor.
          An “incorporation by reference” clause to prior continuation applications does not provide 112 or new matter support from non-specification file histories. Nor does it preclude any [examiner missed] “double patenting” rejections.
          Whatever admissions against interest of claim term definitions occurred in parent, grandparent, great-grandparent, etc., etc. file histories will occur irrespective of any “incorporation by reference” clause unless expressly disclaimed.

          1. Please Pardon Potential (re)Post…

            It is difficult to see humor in what many would consider a “valid point made.”

            Your additional distinctions only make that point even more so.

  7. Dennis, I read what it is, within the “examination process” that you are “working toward”. But why ask us? Why not ask inside the USPTO?

    For me, the interesting question is why, when you have a list of prior art references, you declare it only to the USPTO and not to any other Patent Office in an English law jurisdiction which understands equity. Could it be that the only one that has an explicit and statutory Duty of Candor is the USA?

    Parties that have done searches have art to declare. The wider one casts the net, the longer will be the list of references. Maybe record-holder Steven Hoffberg, a member of the patent attorney inventor species, had accumulated an inordinate amount of search time, based on a particularly broad and woolly definition of his invention.

    Is patent attorney Hoffberg, then, the most candid Applicant ever? Or is he the one who is better placed than any other patent owner, to see off assertions of inequitable conduct.

    BTW, Ned, your “unenforceable” point at #3 is the way this issue used to be handled by the courts in England, under equity. Asserting rights results in discovery and then, when patent owner needs to amend the claims to save the patent, he is refused permission.

    1. Max, I got that from reading the cases on “false suggestion.” Grounds for a scire facias.

      So long as the inventor was on the up and up, the crown would at times grant a narrower patent if the one he was first granted claimed more that he had a right to claim.

  8. My view: applicants should by rule not any references. If it turns out that the claims allowed are deemed unpatentable over a reference which the applicant not only knew of but which anticipated any allowed claim or which was inconsistent with any position advanced by the applicant, then the patent should be held unenforceable.

    Ditto factual evidence filed via affidavit. This should be prohibited. If the applicant wants to introduce it, do so in a trial de novo or an opposition where the evidence might be subject to rebuttal and live cross before a judge.

    1. Anybody else have any thoughts on this? I am still sort of a n00b when it comes to practicing but this seems…um…risky, to say the least.

      1. Q, what is it about filing claims that one knows to be unpatentable over prior art that is so hard to understand. This is not allowed because one files an affidavit swearing that the applicant believes the claimed invention to be patentable over the art.

        This duty should not not relieved by submitting the anticipatory reference. But a strict reading a rule 56 seems that this disclosure excuses the original sin. It should not, particularly – particularly if the anticipatory reference is submitted in a large pile of other references.

        Furthermore, during prosecution, one has a duty not to take a position that is inconsistent with references one knows about regardless of whether one discloses those references to the PTO. The violation is taking the inconsistent position, not failing to disclose.

        The PTO is the author of its own chaos by requiring a duty of “disclosure” instead of a duty of ethical behavior.

        The rest the world seems to get along just fine without applicants flooding the patent office with mountains of references that clutter and distract and provide very little useful information. What happens the United States is not only unreasonable and illogical, it is counterproductive the good prosecution.

        1. “Each individual associated with the filing and prosecution of a patent application has a duty of candor and good faith in dealing with the Office, which includes a duty to disclose to the Office all information known to that individual to be material to patentability as defined in this section”

          I’m not sure why you think there is not already a broader duty of ethical behavior in Rule 56? “Candor and good faith” is pretty broad, and duty to disclose is just part of it. In terms of declarations, etc., I would say the caselaw has been pretty merciless towards Applicants who fudge ANYTHING so I don’t see the merit there of barring them.

          With that said, I would submit that the biggest offenders nowadays of poor filing practice are large Internet companies who seem to hire dozens of different firms to handle cases, but none of them appear to be aware of any prior art in the relevant field, nor do they ask their clients about the same. Nor do any of the internal IP depts cross cite anything. The claims are drafted in a vacuum, with no consideration of any art whatsoever. This, to me, seems like a transparent ruse to draw rejections, more Office Actions, and of course MORE FEES.

          1. First, let’s make clear that I am not defending any poor practices***, but JNG, what you speak of is a direct result of driving towards a bottom line and “getting what you pay for.”

            The client controls and when the client dictates that only “X” amount of effort is to be expended, the too-easy-to-see result of which is NOT “best practices” will occur.

            You seek to place the blame on the party that is NOT to be blamed for the client’s choice.

            ***My caveat here is that in the terms of the agreement, client choices such as these are spelled out with informed consent, that is, the client is advised that their choices may run a gamut of effects including the ones that you point out.

        2. Ned,

          A few points to your comments:

          The violation is taking the inconsistent position, not failing to disclose.

          Not a minor nit: this statement is not correct.

          Violations of both have occurred.

          But Ned, a question for you: given that BRI is known to be “too wide” (that is, wider than a court of law looking at the same terms), is the punishment you seek actually commensurate with the offense? If you are going to level that degree of punishment, should not the offense be on par with an actual (actual as in court) construction, as opposed to one that merely seeks to “more clearly define?”

          It should not, particularly

          There is zero logical basis for such an overbroad statement. As long as the reference list is provided in some logical order, there is no such thing as “burying.” Your view creates a Hobson choice of omitting references with the potential penalty of omission being just as bad as the penalty of not omitting.

        3. If a first action rejection relies only on art cited in an ids, that action is properly made final. Thats the trade off.

          1. Given the (low) quality of examination,that is one of the more unsustainable suggestions that I have heard Ed.

          2. Yup. ‘First and Final Action on the Merits’. Otherwise known as a Ex parte (Judge) Dredd Office Action. But they can undo an Ex part Dredd if they file an empty IDS with the Quantum enTanglement Information Disclosure Statement (QTIPS) pilot program, which lets applicants modify their prior filings and create an alternate timeline (MPEP 3001).

        4. “Q, what is it about filing claims that one knows to be unpatentable over prior art that is so hard to understand. This is not allowed because one files an affidavit swearing that the applicant believes the claimed invention to be patentable over the art.

          This duty should not not relieved by submitting the anticipatory reference. But a strict reading a rule 56 seems that this disclosure excuses the original sin. It should not, particularly – particularly if the anticipatory reference is submitted in a large pile of other references.”

          I agree with this w/r/t claim drafting but when you draft around that art and don’t disclose that reference, and then the reference is found during litigation as a 103 reference, the patent can be held invalid for inequitable conduct regardless of whether the court finds nonobviousness…right? Or am I missing something?

          1. regardless of whether the court finds nonobviousness

            No – that was changed with the requirement of “but for”

    2. A bit of a problem when it comes to references that would make a 103 though Ned.

      1. 6, one’s duty is to file claims one believes are patentable and to not file claims that are not. The courts and common sense can take it from there.

        But this duty cannot be excused by simply disclosing a reference where the claims filed are clearly unpatentable over the reference. The obvious hope is the the examiner misses it. I consider this sharp practice.

        1. What you consider “sharp” is certainly not controlling….

          How could it be?

  9. An interesting correlation to run would be look at whether the patents that have over the mean and/or median number of references are more likely to be PCT Applications.

    We manage a lot of worldwide prosecutions and often we cite a lot of references because they were cited in foreign applications.

  10. Greetings Dennis

    Just to clarify, are you crediting ALL the references cited on the patent as “submitted by the Applicant” ? Because that’s not usually the case – a lot of Applicants cite no art whatsoever. OTOH perhaps it doesnt matter to your analysis bc Examiners rarely cite more than a handful of references.

    Also its probably better to sequester out continuations as they tend to be like snowballs, accumulating references as the various spawn roll through the PTO.

    1. Re “What I’m working toward is whether the large number of references creates any special difficulty or benefits for the examination process.”
      I’m assuming that was droll humor, since as noted many times here, application examiners do not have time to actually study hundreds of cited patents and publications and patents, and compare them to the claims, much less 7,800 references. Examiners would not get credit for all that increased time if they tried, and it would have a negative impact on an examiner overall case disposal rates negatively affecting bonuses and grade promotions. Nor do applicants pay for it.
      A large number of applicant cited references might be indicative of a well-funded application preparation and prosecution effort? That in tern may correlate to the observed longer pendency prosecution?

      1. The last paragraph of 1.1 above is consistent with the 2d paragraph of comment 2 – PCT applications are more likely to be patent applications considered important enough to file in several countries.

        Re comment 1, USPTO examiners do not cite such large numbers of cited references.

        1. If only the 892 form had a “CPC subgroups cited” section.

      2. I was an Examiner before practicing and when I got an IDS I would just text search through it using less restrictive Booleans, similar to what I would do when text searching through large CPC of USPC classes.

        If I had a hard time finding something through a regular search, I would look more closely at the IDS. At that point though, the applicant is likely going to get an allowance fairly early on because it means they found the most relevant references and they know the difference between those references and the prior art better than I do.

        Regardless, from my time at to Office, I noticed that very rarely would any Examiner read through Any reference start to finish (IDS or not) without some cursory relevance check (like reading the abstract).

        Just a few observations… take what you want from them

        1. Read a reference from start to finish? Sounds like a good strategy for not making it to the end of the probationary period.

          1. The Patent Office does not have a policy for examiners to gain extra time to examine applications with large references. If it does, it is applied on an ad hock supervisor by supervisor basis. As a result, no examiner has the time to actually consider all of the references submitted. And as a result, there is probably somewhere in the long list of references a dead ringer which adds to bad patent quality (unless the search is good enough to uncover the reference). Very bad for examiners and the patent system.

            1. Thanks, and yes that result is possible since many prep and prosecution attorneys also do not have the time [aka cannot afford the time for what they are being paid] to thoroughly read the entire specifications of hundreds of cited patents.
              [Why order a pre-filing prior art search without setting a reasonable numerical limit on the number of references cited in the search report?]

Leave a Reply

Your email address will not be published. Required fields are marked *

 Notify me of followup comments via e-mail.

You can click here to Subscribe without commenting