Computing stuff tied to the physical world

TK – Resolution vs accuracy

In Hardware on Apr 19, 2012 at 00:01

Welcome to the Thursday Toolkit series, about tools for building Physical Computing projects.

This episode is not about an instrument you will normally need, but about using a high-end unit.

Once you get into measuring instruments, there’s a trap – the kick of going after models which have more and more resolution and accuracy. First, let me explain the difference – i.e. roughly speaking:

  • resolution is the number of digits you can measure
  • accuracy is how close that value is to the real value

So you could have a 3-digit multimeter which is spot-on, and in most scenarios it’d probably be much more useful than a 5-digit multimeter which delivers meaningless results because it’s not properly calibrated.

The trouble with this search for perfection is that it can be addictive – see the time-nuts site for one example of keeping track of the EXACT time. Over the top for most mortals, but hey, I can relate to this sort of craziness :)

And recently I fell into the same trap. I’ve got quite a few hand-held multimeters, but when someone pointed out some eBay offers of a 6.5 digit HP/Agilent bench-top multimeter, I simply couldn’t resist and bought one:

DSC 3050

An amazing instrument – above it’s measuring between 1.8 and 2.0 µV with the probes shorted out. It’s a second-hand unit, probably from the 90’s, so it’ll be out of calibration by now. I could send it to a calibration lab, where they tweak the thing until it’s back to its sub-ppm accuracy, but that might well cost as much as what I paid for it. So for now I’ll just assume its accuracy is decent, perhaps in the 5-digit range. More than good enough for the experiments at JeeLabs anyway. This is all for fun, after all.

One of the interesting specs of this multimeter is a selectable input resistance of over 10,000 MΩ on DC ranges up to 10V. This extremely high value is great for measuring the leakage of a capacitor. Let’s try it:

  • first, a 47 µF 25V cap is charged to slightly over 5V for a few minutes
  • then, the power supply is disconnected and it starts discharging
  • finally, we measure the time it takes to discharge from 5V to 3.16V
  • this was determined to be well over six hours (I stopped waiting!)

I picked this voltage range because 3.16V is 63.2% of 5V, so the measured time corresponds to the time constant of the T = R x C formula for capacitor discharge. In other words:

  • 20000 s = R x 47 µF
  • therefore, the internal leakage resistance R = 20000 / 47 ≈ 425 MΩ
  • this translates to an internal leakage current of under 5 V / 425 MΩ ≈ 12 nA

So without even having an instrument which can measure such extremely low currents, we can arrive at an estimate of the leakage of this particular 47 µF 25V electrolytic capacitor, and under 12 nA is not bad!

Update – see the comments below, the leakage is even lower because the discharge should be measured to 1.84V iso 3.16V – so it’s well under 10 nA for this capacitor, in fact!

  1. That’s a nice meter, hope you enjoy it! By the way, you can buy a calibrated 10V reference based on AD587LN with a short-term accuracy of +/- 5ppm, for a reasonable price. See:

    • Thanks – yes, I found out about this reference the other day and already ordered one. To be featured in a future weblog post…

  2. There’s a small flaw in the calculation of the RC time constant. If you’re measuring the time constant for a charging capacitor, it’s the time taken to go from 0% to 63.2%. The RC time constant for a discharging capacitor is the time to go from 100% to 36.8% (or to lose 63.2% of its voltage).

    • Ooh, small flaw, but pretty substantial difference. Thanks for correcting this. Since 0.368 ≈ 0.632 x 0.632, that means the discharge will in fact take over twice as long to reach 1.84V (everything is exponential) – so the leakage resistance is more like 850 MΩ, and the leakage current 6 nA @ 5V!

  3. Congrats with your new meter JC

    I got mine last saturday.

    Maybe i could borrow your 10-v ref sometime :-)

    Btw: My friend says they have quite a few of those meters on his work , and in all the years they have used them. The Calibration lab only have adjusted one of the meters (AC Voltage i think).

    All the rest is still within specs. , and passes verification on the Cal-Lab.

    So they tend to hold their calibration very well.


  4. I have one of the Geller 10V references, but I can only verify that my trusty old Fluke 77 indeed says 10.00 Volts. Now, if I had two of those references, then I could compare delta-V… and did you know there is also a “volt-nuts” mailing list? :-)

  5. Don’t JC

    John knows very well …

    If a man has one “voltage calibration source” , he knows how to adjust his meter. If he has two , he begins to doubt.


    Who’s is on his 3’rd Thunderbolt-GPSDO , 5’th Rubidum , 2’nd timing antenna system …..

    • LOL! Wise words Mr Bingo!

      Best to just have one and live in ignorance!

Comments are closed.