Cosine Error in Indicator Readings

Here we examine cosine error with an dial indicator. An indicator is commonly used by machine builders, operators, maintenance, tooling, and inspection personal to measure in 0.0001" and 0.001mm or smaller resolutions. Whenever we use any precision measuring instrument we expect that the reading we get from the instrument is the actual dimension. This will not be the case if we do not account for cosine error as demonstrated below.

This example uses a finger dial indicator to show that a correction factor is needed to account for cosine error. The indicator used in this example reads the correct dimension when the finger is parallel to a flat workpiece or tangent to a curved surface. Cosine error occurs when the finger is not parallel to the workpiece or not tangent to a curved surface.Indicator

  Correction Factor Table  
Angle DegreesMultiply by
100.98
200.94
300.86
400.76
500.64
600.50

Suppose angle a as shown in the picture to the right is 60 degrees and the indicator reads .004". According to the Correction Factor Table, for a 60 degree angle one must multiply the reading by 0.50 to account for cosine error. .004 x 0.50 = .002". The true dimension is .002" not .004" as the indicator read.

Keep in mind though, this is not true of all indicators. Some manufacturers build their indicators to read correctly with the finger at a angle to the workpiece. They do that because one cannot slide a flat workpiece under the indicator without hitting the body of the indicator with the finger parallel to the workpiece. Regardless, cosine error must be accounted for with these indicators too. The point here is one must know the indicator being used to achieve accurate results. Read the indicator's documentation or consult the manufacturer for this information.

There is an exception to this. The demonstated example has been for the most common indicator which have a ball tip. There are also fingers that have a flame shaped tipTip. Cosine error is automatically accounted for with these within a designated number of degrees as long as the finger remains within those degrees of parallel to the workpiece. This is because as the angle of the finger increases, the contact point gets farther away from the pivot point therefore lengthening the distance between the contact point and the pivot point of the finger.

Cosine error also applies to button type indicators in a different way. The difference is the button shaft must be perpendicular to the workpiece or there will be cosine error.