ACCURACY STATEMENTS USED IN PHOTOGRAMMETRY

• Document Type: CREST
• Collection: NGA Records (Formerly NIMA)
• Document Number (FOIA) /ESDN (CREST): CIA-RDP03T02228R000200450001-5
• Release Decision: RIPPUB
• Original Classification: T
• Document Page Count: 9
• Document Creation Date: December 28, 2016
• Sequence Number: 1
• Content Type: REPORT

Approved For Release 2006/05/29: CIA-RDP03TO2228ROOQ 50001-5
17 -P --) .6
TECHNICAL
PUBLICATION
NATIONAL PHOTOGRAPHIC
INTERPRETATION CENTER
ACCURACY STATEMENTS
USED IN PHOTOGRAMMETRY
fication Review by A
MAY 1974
COPY NO. 1?1
6 PAGES
TP-04/74
 Approved For Release 2006/05/29: CIA-RDP03TO2228R000200450001-5
Approved For Release 2006/05/29: CIA-RDP03TO2228R000200450001-5
 Appre-ved
For Release 2006/05/29: CIA-R 8R000200450Off
TOP SECRETI
ACCURACY STATEMENTS USED IN PHOTOGRAMMETRY
Mensuration from satellite photography plays an important role in the intelligence
community. Analysts and imagery interpreters need mensuration in such varied tasks as
calculating missile silo hardness, assessing aircraft capabilities,
I and computing the velocities of ships. In order to use mensuration correctly,
however, it is essential to understand the accuracy statements that accompany mensuration
products. The purpose of this report is to explain why accuracy statements are needed, what
they mean, how they are derived, and how they can be used.
WHY ACCURACY STATEMENTS ARE NEEDED
"A reported value whose accuracy is entirely unknown is worthless."
--Churchill Eisenhart'
Why do we need accuracy statements? In short, because mensuration results are not
perfect. A photogrammetrist does not obtain the true size of an object by measuring it with
a steel tape. Rather, he obtains an estimate of the true size, using the art and science of
photogrammetry. An estimate, by definition, can differ from the true value. This difference,
or error, arises from many sources.
The most important source of error in most measurements under 50 feet is the quality
of the imagery. The image on the film may be degraded and distorted by spread functions,
adjacency effects, halation, and other factors. These effects are particularly noticeable under
the high magnification (60X to 200X) used for mensuration. Each photogrammetrist interprets
and measures these blurred edges differently. Apart from interpretation of the edge, some
error is inherent in each photogrammetrist's ability to position the measuring reticle accurately.
This error source is usually negligible for monoscopic measurements but can be significant
for stereoscopic mensuration. Other sources of error occur in the film processing, the measuring
devices, the parameters which are input to the photogrammetric equations (math model) and,
to a much lesser degree, the math model itself. All these factors can cause the
photogrammetrist's estimate to be in error -- hence the importance of accuracy statements
to indicate the extent of the possible error.
'Former Chief of the Statistical Engineering Laboratory at the National Bureau of Standards, Washington,
TOP SECRET
F
25X1
25X1
25X1
25X1
 For Release 2006/05/29: CIA- 228800020045 -
TOP SECRETI
I 5X1
We also need accuracy statements to avoid "implied" accuracy statements. It is common
practice in science and engineering to report values to the least significant digit.
The most generally accepted type of statement of uncertainty or possible error is the
confidence interval. Since 1968 NPIC has been using the confidence interval for the accuracy
statements that accompany mensuration. An example of an accuracy statement follows:
Dimensions are accurate to withini at a 95%
confidence level. (The phrases "at a 95% confidence level" and "within a 95% confidence
interval" are used interchangeably.)
A confidence interval is composed of two parts: (1) an interval and (2) a percentage
which indicates a degree or level of confidence. The interval itself often consists of two parts
as in the example above: a foot portion, referred to as pointing error, and a percentage portion,
referred to as system error, which is dependent upon the size of the dimension. The two
parts of the interval must be added together to obtain the total interval. For example, a
25X1
25X1
25X1
25X1
25X1
 For Release 2006/05/29: CIA- 2288000200450001-5
TO ~R
25X1
The degree or level of confidence quoted in the above exam le is 95 percent. This means
that there is a 95 percent chance that the interval includes the true diameter
of the missile. This does not mean that each value within the interval can be considered
equal' like' . The most likely dimension is the one value reported by the photogrammetrist,
i.e. IThe next most likely. values are ones very close to the reported dimension.
The va ues less likely to be correct are at the extremes of the interval. The 95 percent
confidence level also means that 5 percent of the time, or one time in 20, the interval will
fail to include the true object size.
If the interval seems large, it is because there is a high level of probability (95 percent)
that it will include the true size. If the level of confidence were statistically reduced to 67
percent, the interval would be reduced to half its size, I nfidence
interval would be However,
throughout the years scientists and analysts have chosen 95 percent' as a reasonable degree
of confidence to be required of an estimate.
DERIVATION OF STANDARD ACCURACY STATEMENTS
The Photogrammetry Division, NPIC, derives "standard" accuracy statements for each
camera system, for each photographing mode (monoscopic and stereoscopic), and for each
type of measurement. These statements are called "standard" because they quantify the
measurement error for one coverage of a target measured by any photogrammetrist on any
instrument. They serve as the basis for most of the accuracy statements reported by NPIC.
Two techniques are used in deriving standard accuracy statements: empirical and theoretical.
Fundamental to the empirical approach is a good source of urn?nrl truth Thprp
. 'One exception occurs in cartography, where the national "map accuracy standard," which is the accuracy
statement for geodetic positions on a map, is given at a 90 percent confidence level.
TOP SECRET
I I
25X1
25X1
 Approve For Release 2006/05/29: CIA 222880002004
TOP SECRET
When satellite coverage of a ground truth area is obtained, photogrammetrists measure
selected dimensions on that coverage under typical working conditions. The differences between
the ground truth values and the values measured on the photography are tabulated.
After sufficient data is accumulated it is statistically reduced to give a standard deviation
which is then used in the accuracy statement.
These statistical analyses have shown that mensuration error is approximately normally
distributed about a mean error ofI u In addition, a system error not exceeding 1 percent 25X1
of a measured dimension may occur. 25X1
The theoretical approach to deriving standard accuracy statements is the use of error
propagation, which is a complex mathematical and statistical process. It requires knowledge
of the magnitude, distribution, and interrelationships of the various errors that affect
mensuration. Each of these individual errors is propagated through a math model to obtain
a final accuracy statement.
The Photogrammetry Division relies primarily on the empirical approach for establishing
accuracy statements. The theoretical approach is undertaken by the Applied Photo-Science
Division, NPIC, to predict the accuracy that can be expected from a new or modified camera
system. It also serves to indicate how data should be acquired in the empirical investigation.
The accuracy statements generated by the two techniques have been in close agreement.
A new accuracy statement investigation is conducted whenever there is an a priori
indication that the previously derived accuracy statements should be changed, for example
when improvements are made to the camera, film, or mensuration procedure. There is also
routine mensuration of ground truth on each new mission to insure that the mission meets
current accuracy standards.
An assumption must be made when applying the results of these error investigations
to measurements on foreign targets. It must be assumed that the errors observed
tatistically from the same population as the errors which occur on foreign
coverages. This assumption is valid to the extent that the variety of ground truth objects
is representative of actual foreign targets so far as contrast, size, complexity, etc., are concerned.
Also, the quality of the photography I land foreign targets, with regard to scale,
graininess, resolution, etc., should be the same. The photogrammetrist will subjectively modify
the foot portion of the standard accuracy statement when these conditions are not met.
TOP SECRET J
25X1
25X1
25X1
25X1
25X1
 Approved
For Release 2006/05/2S E C R E Tc 28R000200450~
SPECIALIZED ACCURACY STATEMENTS
The previous discussions of standard accuracy statements referred to accuracy statements
which are used for most measurements made at NPIC. These statements represent a
randomization over different instruments, photogrammetrists, types of objects, and situations.
However, such statements can be too general for certain critical measurements.
Specialized accuracy statements are often produced when multiple coverages are used
or several photogrammetrists measure the same object. These accuracy statements are based
on the fact that an average of several independent measurements is more precise than a single
measurement.
A new technique called "max-min" has been developed to provide a special, hand-tailored
accuracy statement. An experienced photogrammetrist can make a subjective measurement
of the possible error in a given image. In effect he actually measures the bounds of the
confidence interval. This technique has proven effective in assigning a suitable accuracy
statement to mensuration from imagery which is either considerably better or worse than
the imagery from which accuracy statements are normally derived.
The current effort in specialized accuracy statements involves locating and quantifying
the individual sources of error. It is known that there are different variances among instruments,
photogrammetrists, target types, image sizes, and contrast. By quantifying these separate
sources of error it will be possible to obtain not only better accuracy statements but also
more accurate measurements.
HOW TO USE ACCURACY STATEMENTS
An accuracy statement dictates how a measured value should and should not be used.
For example if the geodetic coordinates of an airfield are reported as accurate to within
?10,000 feet at a 90% confidence level, the coordinates may be used to locate the airfield
on a map but should probably not be used for If a missile length
TOP SECRET
ii
25X1
25X1
25X1
25X1
 TOP SECRET
25X1
The accuracy statement can be used to compare and contrast items. If two tanks have
been measured and their lengths are different but the confidence intervals overlap, a statistical
test can be performed by the Photogrammetry Division to ascertain, at a given level of
confidence, whether the two might actually be the same size.
Many times dimensional information is used in conjunction with information from other
sources to perform analysis. Most of these other inputs have accuracy statements associated
with them. A standard statistical procedure referred to as error propagation can be performed
by the Photogrammetry Division to produce an accuracy statement for the final result.
The accuracy statement is a statement of chance or probability. It tells how confident
one is that the true value is within a certain interval. There is not an equal likelihood that
any value in the interval is the correct dimension. The best estimate of the true value is
the one value given by the photogrammetrist.
Since accuracy statements for linear dimensions are always given at a 95 percent
confidence level, the size of the interval serves as a measure of the reliability of the
measurement. The size of the interval also suggests what decisions should be made based
on the mensuration.
Accuracy statements can also be used in a more rigorous statistical way to compare
or contrast items or to calculate an overall error through error propagation.
The methods of producing accuracy statements are in a continuing state of improvement.
NPIC is striving to improve not only the accuracy and precision of its mensuration but the
reliability and usefulness of its accuracy statements as well.
Project 750000
Comments and queries regarding this report are welcome. They may be directed t 25x1
Photogrammetry Division, Technical Services Group, NPIC, ~K}
TOP SECRET
25X1
25X1
25X1
 Approved For Release 2006/05/29: CIA-RDP03TO2228R000200450001-5
Approved For Release 2006/05/29: CIA-RDP03T02228R000200450001-5