Overview of Least Squares Cave Survey Issues

[This page is an updated web version of an article that originally appeared in "Compass and Tape"]

There is, unfortunately, a lot of confusion in the cave survey community about analysis and adjustment of survey data. The problem is bad enough that some cave survey programs have massive misunderstandings built into them.

The reasons for this are many. Most cavers usually have neither the time nor the motivation or sometimes even the chance to take a detailed theory of surveying course. Such a course may not even be taught in their area. So they hunt around in the literature for the pieces of information they can understand. What they find understandable are those things that have been simplified so as to be easy to understand. What they don't get from such articles are which simplifications were made, and how to tell when those simplifications' assumptions were being violated.

The offshoot of this is that cave survey software is often just flat out wrong. It is not uncommon for two programs that claim to do a "least squares" solution of the survey problem to produce dramaticly different answers given the same data.

Maybe "wrong" is a bit strong as a term... they correctly solve some simplified problem, but they don't state the simplifications, and they don't test for the data violating the assumptions that underly those simplifications.


Why don't more people notice problems?

A least squares solution to a problem produces a solution that is a "linear combination" of the various loops values. What this means in non-mathematical terms is that if several traverses put a point in a different location, the final point will be somewhere between these locations.

Exactly where in that area the points go is determined by the weights. Large changes in the weights don't normally produce results that look very different to the naked eye. Since even bizarre weights don't cause passages to move to strange locations there is a feeling that weights are less important than they are.

*IF* all you are doing is drawing a map, then almost ANY location for a final point (between the various places the data claims) works. Only in the case of major blunders will you see a difference... and in those cases the initial adjustment is wrong anyway [without post processing] (see below).

Why care? Correct processing gives you a LOT of useful statistical information about the survey (although many cave survey programs ignore it). This can help you watch for systematic problems. From a practical standpoint, correct processing keeps blunders from screwing up every loop to which they are attached.

If all you care about is a pretty map, almost anything will work and arguments between various methods are little more than aesthetic and religious issues.


Common Issues

Adjusting a cave survey is what mathematicians call a "non-linear" problem. (Some people say that "non-linear" is a slang term for "really ugly"). In order to deal with the data at all, the problem is generally "linearized". This is done by reducing the Distance, Azimuth, and Inclination values to a vector which is the change in X, Y, Z for the shot. This changes the problem into what is called a linear approximation to the original problem. The equations that linearize the problem also force a weighting on the problem. Because this is done, the problem being solved is NOT the original problem, but only one (hopefully) near the real problem.

A linear approximation to the original problem works perfectly well *IF* the solution lies close to the original data. For professional surveys this is almost always the case (partly becuase they resurvey blunders). For surveys with hand-held instruments, this is usually the case. For surveys with major blunders, the linearized solution is probably junk.

Least squares analysis will give statistical figures (if one checks) on how well the data matches the model. Many cave survey programs don't bother to check, and don't generally do much statistical analysis at all. Most cave survey programs don't know what to do when the data doesn't fit the model anyway.

Issue 1: Programs that don't know when the linearized model is being violated

Since a professional grade survey is almost never going to violate the assumptions (they go resurvey when there are blunders), even professional survey programs sometimes don't have checks for whether the linearized solution is good enough. This can be handled correctly, but it involves making new approximations after the first adjustment, and rerunning the adjustment [This is called "relinearization"]. This is not hard to do, but cave survey programs never do. In caving, the fit between the model and the data is often very bad because blunders are left in. For example, there is no way a 1000' long eight inch high passage with four inches of 33 degree running water is going to be resurveyed. Obviously, there is a high probability that the freezing cavers made mistakes.

When the linearized model is being violated, one has to come up with a model closer to the problem. Fortunately, the results of an adjustment of the original linearlized problem give information to do this. Unfortunately, it is not commonly done. Professional survey programs don't have to bother in most cases because they resurvey blunders...

Issue 2: Ad hoc and bad weights

Converting from Distance, Azimuth, and Inclination to X, Y, Z imposes certain *structured and computable* relationships between the uncertainties of the X, Y, and Z. Using anything else can make the weights wrong by an order of magnitude. The weights are determined by the uncertainties of the original DAI information, and the transformations used to get X, Y, Z. They are not just pulled out of a hat.

A large number of cave surveyors use the standard transformations to convert the problem to X, Y, and Z, and then don't want to accept the statistical weights this imposes on their answers. Often they don't even realize that doing the transformations has any bearing on the covariances, and therefore the weights, at all.

Throwing away the covariance information (that information that relates the uncertainties of the variables to each other) and continuing to process violates the underlying assumptions of the least squares fit. Making up weights from other assumptions (chain rule, length fit, etc.) may have intuitive appeal, but violate the underlying assumptions that made a least squares fit a good idea to begin with. In some cases ad-hoc weights can mean you are running statistics on correlated variables claiming them to be uncorrelated.

Some people even solve the unweighted problem, with any number of arguments as to why the weights don't matter.

Issue 3: "Least squares is least squares"

The Internet abounds with least square programs that take a matrix of the right form, and give you the solution to the least squares problem that that matrix describes. The underlying assumption that went into deriving them is that you are solving a LINEAR least squares problem. They have no idea you are handing them a LINEARized problem. Therefore they don't report when the linearization assumptions are being violated (such as when there are the right kind of blunders).

The cave survey program authors are prone to almost religiously defend this usage of their canned routines since they were "being used to solve least squares problems for many years by many high-powered organizations like NASA". The fact that the programs work correctly on the problems for which they were designed, means nothing if you give them a linearized problem they weren't designed for, and don't take correct care around it.

Note, however, that if the correct care is taken, any program that solves linear least squares problems at all can be used to solve the full linearized weighted least squares problem.

Issue 4: Simplifications

Many cave survey programs make "obvious simplifying" assumptions. However the authors often don't know what the true effects of of the simplifications are, since they didn't go back to the general principles to measure the effects. A good example of this is adjusting X, Y, and Z data separately. This can cause your weights to be off by an order of magnitude. [I need a good example here-jh] It may make the program simpler, but it also makes it wrong.

Issue 5: Schmidt and Snelling said... [This is a reference to a quick introduction to least squares published in the NSS bulletin.]

If a quick introduction to a subject simplifies, and doesn't cover all the details, then an implementation based on that introduction is also not going to correctly cover the details.

Good introductions are just that, Introductions. Anybody planning to implement something should get a more complete introduction.

Issue 6: "The program is god"

For some reason people tend to take an output of a network adjustment as if it were magicly "right". Some program authors encourage this. Just because a program applied good mathematics to bad data doesn't magicly make the data clean and right. If there were errors in a loop there still are, they've just been spread around in a fashion that has good statistical properties.

Issue 7: "Fancy mathematical XXX for simultaneous Least Squaeres loop closure"

There are sequential least squares techniques. There are simultaneous least squares techniques. They each produce (up to round off errors) the exact same answers. Since the sequential least squares methods produce the same answers, the argument that "simultaneous" loop closure is better than "sequential" loop closure is clearly bogus. (But I continue to see them in print.) The 'real' issue is more often "I don't like the manner in which you do sequential closures" with an appeal to a (possibly misimplemented) least squares program's output as being the magicly correct one.

Note that this doesn't mean sequential least squares can find the adjustment of one loop magicly without the ones to follow. It just means that as a new loop is added the appropriate adjustments can be made to what we have so that the adjustment is at all times correct for all the the data it has processed.

Issue 8: "This program tested with 1000 different cave surveys"

The adjustment of a cave survey has a lot of answers that can "look good". The weights could be dramaticly wrong and still produce a result that "looked good". Results that "look good" can only eliminate very gross missimplementations, not prove "correctness".

Was the program tested with data where the mathematically correct answers were known? Some never have been.

Were the parts of it checked with data sets that tested it's ability to handle problems nearly unstable? Did it notice?

Does it even notice if it is handed bad data that produces a singular matrix that gets "solved"?

What sorts of libraries does it use and how were they tested? If the inverse of the inverse of the problem matrix (as computed by the underlying library) is not close to the problem matrix, it surely doesn't matter how well the least squares setup is done.

Issue 9: "Nobody cares about the statistical part anyway"

This is my favorite issue. If the statistics are bad, why believe the adjustment is good?

EVERYBODY should care if the survey contains blunders and mistakes. A network adjustment that doesn't downweight blunders is just flat out wrong. A correct least squares adjustment does this automaticly. But you see people showing off adjustments that give blunders the same weight as good shots... and they often try to claim that is is "THE RIGHT WAY" because it came out of a least squares program.

A least squares adjustment *IS* a statistical technique. If you violate the assumptions of a statistical technique, then the result has no statistical significance. If one doesn't care about the statistics, then why use a statistical technique to begin with?

Issue 10 Terminology

As long as cavers read cave survey literature, instead of reading survey literature, cavers are going to end up with terminology that is different than real surveyors.

An immediate side effect of this is that when they search for literature they are only going to turn up cave survey papers, because they are the only papers that use those terms in that way.

This also makes real papers hard to read for them, since the terminology used in real survey papers is totally new to them.

Cave survey documentation that doesn't also give the real survey terms continues and worsens the problem.

Remember: Surveyors have been at this game a lot longer than cave surveyors. They've already dealt with many of the problems we are just now dealing with.

Issue 11: The one [simplified] paper I read is the entire world.

There are lots of presentations of any mathematical technique. Some are good. Some are bad. Some emphasize one aspect, some emphasize another. Some simplify in some areas, some in another.

Least squares is no different in this respect. If you are planning to write code for it, I highly recommend reading several different presentations of it first.

Trying to defend specific details of someone's program's approach, if you only understand one way is (at best) difficult.

Issue 12: Cave Survey is the world.

Just because something agrees with cave survey papers, doesn't mean it agrees with real survey papers.

Just because something hasn't been tried by cave surveyors, doesn't mean it hasn't been tried. It may have hundreds of years of literature about it back in the real world.

Just because the only caver implementation of something is bad doesn't mean that the idea is bad, or that there are not good implementations of it out in the real world.

Just because it is not found in cave survey papers, doesn't mean it doesn't exist and isn't in regular survey papers.

Just because cave survey papers have some opinion on a topic doesn't mean that real surveyors have that opinion.

Just because a paper has been published in a cave survey forum doesn't mean it is right.

Issue 13: We use the programs availiable to us and they all have the problems.

This is unbeliveable. Many regular survey books come with software for the basic functions. (See my bibliography for example.) Anybody that claims that there aren't implementations readily availiable hasn't read the regular survey literature.

Notes

It would not be nice of me to complain people aren't reading real books on the topics without my pointing people into some place reasonable. So, I prepared a small but relevant bibliography


Go to ...


This page is http://www.cc.utah.edu/~nahaj/cave/survey/overview.html
© Copyright 1999 by John Halleck, All rights reserved.
This page was last modified on August 8th, 2000