References Cited

The number of references cited per patent continues to rise, albeit more slowly in recent years.  For patents issued in 2016, the average patent includes more than 50 cited references.  In my view – this is great, although it would certainly help if the examiner was given some clue as to why the reference is deemed relevant  or what portion of the reference is relevant. (If an examiner has a question, they can ask the applicant).  A not-surprising facet of the growth in references-cited is that almost all of the growth is in applicant-cited.  Compared with 10 years ago, applicants cite 26 more references per patent (on average) while examiners cite only 1.5 more per patent.  As you might also note from the difference between median and average – the citation distribution is highly skewed.  Example: If we take the top-5% of patents from 2016 (those with the most references cited) – they include 35% of all of the cited references.  My experience with this skew is that patent applicants considered more valuable by their owners are more likely to submit more prior art references.

Why is all this good? (1) Our IT systems can now easily handle searching through large numbers of references and in many cases the submitted references should be the first referenced searched as a group to determine a starting point for the patent analysis; (2) Because of the skew, the more important patents have more references and a better starting point.  If these are well used by the examiner then we also get better examination and a stronger patent system in the end.


35 thoughts on “References Cited

  1. 7

    Based on the information posted by the PTO (see link below) Examiners do not spend enough time to consider IDS and there is no incentive for them to make a rejection based on art cited in an IDS after the first office action is issued (the rejection may not be final and will thus have zero count). I think this needs to change.

    link to

    1. 7.1

      Actually if the reference is used in a rejection of an unamended claim in a 2nd action it can be made final if the reference that is causing the issue was cited in an IDS.

      See MPEP section on finality i’m too lazy to look it up for you.

      1. 7.1.2

        Anon is right. There is no incentive for an Examiner to raise a new ground of rejection in a second office action based on art timely cited (within 3 months).


          I don’t know what you two are talking about. As far as I can see you have to pay a fee under 1.17 (p) for any IDS filed in the time periods under 1.97 (c-d). And you have to satisfy either b c or d under 1.97. The b section under 1.97 pretty much only applies to things before first action (or equiv stuff for international apps). Thus generally apps end up paying the fee under 1.17(p) and whatever refs in that IDS will be subject to being used in a final action.

  2. 6

    Seriously, PTO, do you really want all these references?

    I think not.

    Instead of acting to reduce the number submitted, the Office refreshes Rule 56. What they should do is forbid the citation of references entirely, only allowing a discussion of the prior art in the specification.

    The Office only needs to be informed of related cases and of litigation. The Office should cooperate with other examiners on searches and should be aware of the results of the searches of other examiners. The Office can monitor litigation.

    But what if someone gives the applicant a piece of relevant prior art? We have had this discussion before. The applicant should have no duty to turn this over to the PTO. The third party should be forced to file it publicly in order to have the Office consider it. So long as the applicant has a duty to disclose this art, the ability of a third party to harass an applicant is unbounded.

    1. 6.1

      I could live with that. But, are you overlooking prior art that the Office probably wont find on its own, such as public disclosures by the applicant? For example, the inventor may have forgotten about or may have never been aware of, some disclosure made at a trade show. If the inventor or the attorney later become aware of that, do you need to modify your proposal to require that to be disclosed, or is that covered in some way I am overlooking?

      1. 6.1.1

        Les, the applicant and the patent attorney have a duty not to file and prosecute claims they know are unpatentable over the art. It is the filing of such claims that is the fraud. It will be discovered in litigation, and the patent held invalid and unenforceble.

        No patent attorney in their right minds wants to place their client in this position. The failure is prosecuting invalid claims.

        The Office acts as if IT, and not the courts, were the final arbiter of validity that IT could hold patents unenforceable.

        The applicant and his attorney owe a duty of candor to the PTO, the violation of which renders the patent unenforceable in court. Not disclosing dozen or even hundreds of marginally relevent prior art may violate Rule 56, depending on how it is written, but such does not violated any duty of candor which is directed at the communications of the applicant and his attorney about the patentability of claims that they know to be not patentable over the art or which are inconsistent with the state of the art of which they are aware.

    2. 6.2

      only allowing a discussion of the prior art in the specification.

      A horrible suggestion that ignores what the courts have done with patent profanity.

      Ned should know better for at least the point that he has had this point explicitly provided to him (by yours truly) in past discussions.

  3. 5

    IDS for block-buster drugs often cite 100s of WO and foreign patent pubs and multiple 100s NPLs. The PTO stores these documents as image files. The IT infrastructure at the PTO does not OCR these documents in mass. If an examiner wants to text search a document they must initiate an OCR of that particular document from their desktop application viewer (DAV). Of course, there is no practical, reliable way to identify which select few of the many 100s of cited documents one should be OCRing in the less than 3 days one has to review a case and write a first office action. As it currently stands, the only way to identify documents of relevance is to (i) select, document by document, all WO and foreign patent pubs (each of said docs being around 100-1000 pages) and all NPLs (each of said docs being around 10-100 pages) from what is essentially a PTO internal PAIR and download (ii) OCR these pdfs overnight hoping that adobe does not pause to give a warning, or crash in the process and/or the PTO computer does not crash (iii) save and index the now text searchable documents. Now you can begin a proper search of the IDS cited documents. Completing (i) and (ii) is incredibly time consuming at best, often failing due to a warning that stops the OCR in the middle of the night, or adobe crashes or the OS does. So these JUMBO IDS citations offer almost no value (unless the NPL title, if applicant has followed the rules and recited it as they are required to do so, includes the keywords of interest; note that WO, For patent docs are not cited with a title).

    1. 5.1

      Yup, it’s god awful.

      I’m sure they can do something with DOIs as well to speed up the process (as pretty much every journal-NPL has a DOI, and I’m pretty sure you can retrieve the html fulltext from a DOI, except for older articles)

  4. 4

    I can see advantages for Applicant, in submitting a list of 150 references rather than 5 or 15. Is there anything to deter an Applicant from presenting a list of 150 (or more) references of marginal relevance, interesting only as technological background?

    It seems so strange, for me, based in Europe, to present a list of 150 references. In Europe, the closest prior art is usually beyond serious dispute. Does Applicant in the USA have any incentive to i) identify it or, to the contrary, ii) bury it in a grotesquely long list of prior publications?

    1. 4.1

      Deterrence? Have you ever been the recipient of a death glare from an assistant asked to prepare such a list? Have you ever prepared one yourself? Have you ever mailed copies of 150 documents to a patent Office? Have you ever scanned that many documents to submit them electronically?

      As awful as that is, it is nothing compared to actually reading them and making a judgement as to how an Examiner might interpret or worse misinterpret them and include them in a obviousness type rejection.

      Awful as that is, it is nothing compared to the feeling one can anticipate in causing ones client to spend $1,000,000 extra in litigation fees arguing about whether a document you decided was not one of the most relevant 15 was in fact actually number 12 and whether the failure to submit was inequitable conduct.

      1. 4.1.1

        Les, I see your “extra one million dollars” point but am not sure I agree.

        There always will be some doc or another, not on your List, that will get an IC attack on its feet. Given a deep enough pocket and a desire to bankrupt you, there will always be IC attacks that have to be defended against, very expensively.

        Your List is either a competent extraction of the closest art or it isn’t. If it is, you escape IC and malpractice. Regardless how long your list is, if it lacks at least one hot doc you knew about, you must worry not only about client’s million but also your own practice. Regardless how short your List is, if it contains all the hottest docs you should be OK, and it’s out of your control, but within the control of the court, to stop the other side crushing you with million dollar costs, defending against vexatious IC attacks.


          The key phrase there MaxDrei: “that you knew about.”

          That’s why a “dump them all” mentality exists.


            Tell me more about the “know about” concept. I mean, suppose you have an invention and do a quick patentability search. You identify 10 patents as of relevance. But each issued specification of each of the ten begins with a List of 1,000 References (of which a mere five have an asterisk).

            Do you “dump” all 10,000 references on the USPTO. After all, you do “know” about them, don’t you?

  5. 3

    Dennis –

    Are you able to evaluate the percentage of applicant cited documents that are actually applied by an Examiner in a rejection?

    My estimate would be less than one in 100.

  6. 2

    Dennis –
    Is there still an increase if you subtract out documents cited by applicants to the U.S. Examiner that were first cited in a corresponding foreign application?

    Is it good that Applicants have to spend time and money cross citing documents between patent offices?

    1. 2.1

      You don’t have to cross cite anything. It’s “just an option”, as the resident patent maximalist b0ttom feeder here would tell you ten thousand times a day if he wasn’t a total hypocrite.

      1. 2.1.1

        Yeah, IDSing to cross cite is an option in the same way handing over your wallet to a mugger with a gun is an option.


          Not unlike functional claiming at the point of novelty without any structure in your specification. “anon” and his s00per awesome patent attorney friends will gladly take your money if you want to exercise that option!

          And you might even get a patent. And you can take that patent to East Texas and threaten a hundred different businesses with it (yay! jobs!).

          A patent obtained through inequitable conduct is still a patent. It’s presumed valid. What’s the worse that can happen from enforcing it?

          Maybe Trump can dig up Ray Niro and make his corpse the PTO Director. That might be the perfect solution.


            You seemed to have strayed off topic.

            In software, function implies structure much the way in the mechanical arena, connect A to B with a screw implies structure even though a screw size is not specified, wood or machine is not specified, Philips or regular, counter sunk or not, thread size, screw length are not specified.

            Claim language is broad, no matter the field. There is very little point to a narrow patent.


              Except we can all identify the structure of a Phillips screwdriver but there is no objective structure whatsoever to a “DVD preference ranking module.”

              Thanks for playing, though! You’re a very serious person. You want to recite your script about the Wright Brothers now? Or are you going to pretend to be born yesterday?


                First of all, the example I gave had nothing to do with a Philips screwdriver. It had to do with the broad nature of recitation of a SCREW that did not specify any additional information that might be required. That’s because, one of ordinary skill in the art would understand the range of structure encompassed by the word SCREW.

                Similarly, ones of ordinary skill in the art would understand the range of structure encompassed by the phrase “DVD preference ranking module.” You can pretend to be baffled if you like…. and as I thought I clearly illustrated, one can pretend to be baffled by words and phrases used in mechanical claims as well.

                But,we know you know, you are a super serious person after all.

                1. the example I gave had nothing to do with a Philips screwdriver.

                  Says the guy who brought up the Philips screwdriver. Super serious stuff!

                  one of ordinary skill in the art would understand the range of structure encompassed by the word SCREW.

                  And — once again — there is no objective whatsoever whatsoever for “DVD preference ranking module.” But somehow all this manages to fly right over the deep serious head of Joseph Herndon and the other lost sheep over at Patent Cr0cks.

                2. MM:”And — once again — there is no objective whatsoever whatsoever for “DVD preference ranking module.” ”

                  Yes there is. It is a module that ranks DVD’s by preference. That is just as clear and precises as “screw.” There are different ways one can rank DVDs by preference. Just as there are different kinds of screws. Ones of ordinary skill in the respective art can select one without undue experimentation.


              In software, function is structure because the CAFC said so

              Fixed for accuracy. Don’t forget your history lesson.

      2. 2.1.2

        Maybe you should be more specific with your slander, given as you are to place “everyone else” in the same bucket and wrongly misapply your ad hominem willy nilly.

  7. 1

    Citing ten or ten thousand references doesn’t make any difference if the PTO continues to pretend that it was born yesterday when it examines logic patents.

    1. 1.1

      What exactly is a “logic patent”…?

      Keeping in mind your own (lack of) clarification between logic and applied logic….

Comments are closed.