Briefing:

The following is a re-post of a comment I posted a few days ago which I feel deserves the visibility of being a new thread. I have also appended a comment about the earlier post and my reply.

This article presents a new way (to UFOlogy at least) to estimate the probability that a UFO really did crash at Roswell in 1947, using techniques of "decision theory" that are proven and trusted in many other fields, including communications and radar detection, among others. In the communications field, intelligent use of decision theory is what made possible the progress from 300 baud modems of a decade or so ago to the 28,800 baud modems of today. My contention is that by carefully applying Bayesian decision theory to UFO cases, we will make a similar advance in the reliability of our decisions in that arena.

If you're not familiar with Bayes's Rule or decision theory, you can find explanations of it in most elementary books on probability theory. What it does, in simple terms, is let us turn around the question of "how likely is it that a UFO crashed" and start by looking separately at the two cases (or more than two if we wish): OK, so lets assume a UFO *REALLY DID* crash. For each of the things we *KNOW* happened, or we *KNOW* people said, or we *KNOW* the Government did, how likely is it that that would have happened ... again, if a UFO *REALLY DID* crash. Then in the alternative case, let's assume that it is *NOT TRUE* that a UFO did crash. How likely would each of these *KNOWN* events be if *THAT* were the case?

In my opinion, reasonable people should be able to agree on at least rough estimates of such probability estimates for these two cases. Then by applying the formula described below called Bayes's Rule, we can turn the equation around as it were and get an estimate of the probability that a UFO crash is what happened, or is NOT what happened. And even if we can't fully agree on what are the right assumptions to make, we CAN see how sensitive the conclusion is to the assumed probability values.

Here, then, is the re-post. I hope readers sincerely interested in arriving at a better understanding of what happened at Roswell will join in the exercise of trying to agree on probability estimates so that we can see what Bayes's Rule tells us in return for our efforts.

Read on....

Former "Subject:" field: Re: Roswell is sneaky even to experienced debunkers!

Message-ID: <DqrGsp.2Hy@world.std.com>

Date: Thu, 2 May 1996 04:36:24 GMT

...>snip<...

Let U denote the UFO hypothesis, i.e. the hypothesis that a UFO crashed at Roswell in July, 1947 and alien cadavers (plus maybe a live one or two) were recovered. Let Pr{U} denote the A Priori probability that this event occurred.

Assignment #1 for all who wish to participate in this exercise: Make your own estimate of this probability, giving it any value that makes sense to you, greater than zero but less than one. In next week's class, we'll all get together and try to come to a consensus on these values, but meanwhile you can do the calculation with your own values to see how it comes out.

Let N denote the alternative hypothesis, i.e. that a UFO did Not crash at Roswell, but rather, something else happened that caused all the commotion. Let Pr{N} denote the probability of this hypothesis. Note that Pr{N}=1-Pr(U).

Let F1, F2, ... , Fk, ... , Fn (think of the numbers as subscripts) denote n independently observed Facts about the case - each of Brazell's major actions, each of the major items of testimony by alleged eyewitnesses, etc., etc. For each fact Fk let Pr{Fk|N} denote the *CONDITIONAL* probability that this event would have happened conditioned on hypothesis N, i.e. That No UFO crash occurred but some other event caused all the commotion.

Conversely let Pr{Fk|U} denote the conditional probability that event Fk would or might have happened given that the alleged UFO crash *DID* occur.

Assignment #2 for all participants is to make your own personal estimates of the conditional probabilities Pr{Fk|N} and Pr{Fk|U}, for each of the facts Fk of which you are aware. In next week's class, we will make a master list of all the events and facts that we can identify and try to come to a consensus on the conditional probabilities. Meanwhile, you can make the calculation for yourself, using your own idea of what constitutes a good conditional probability.

For test purposes, you can work with crude probabilities like Pr{U}=10^-6, Pr(N}=.999999, and Pr{Fk|whatever} = 10^-6, .1, .5, .9, or .999999 depending on which of these rough values makes sense to you.

Anybody who isn't with me to this point, feel free to ask questions, which I or someone from among the esteemed scientists from sci.skeptic will try to answer.

Now let's review Bayes's Rule. The starting point is the definition of conditional probability. Using the above notation, one defines

Pr{Fk|N} = Pr{Fk & N} / Pr{N} and similarly Pr{Fk|U} = Pr{Fk & U} / Pr{U}.

Bayes's insight was that we can use this relationship to work backward from the conditional probabilities that we can more readily estimate, i.e. The ones defined above, to calculate the conditional probabilities of the ones that are more mystifying, in this case Pr{U|F} and Pr{N|F}, where I have added some new notation by using just F to denote the entire list of individual events Fk. In other words, Bayes's Rule gives formulae for what we all want to know... what is the CONDITIONAL probability of each hypothesis (U or N) CONDITIONED on *ALL THE FACTS* F1, F2, F3, ... etc., that we have at our disposal.

Note that in this context, a "fact" means anything that happened, e.g. some witness alleged they saw an alien body, or whatever. We're no longer concerned with the A Priori probability of that event, but just the probability that the particular witness would say so *GIVEN THAT* we have already assumed a UFO crashed, or did not crash, as the case may be.

Using the above formulae for Pr{Fk|N} etc. we can state Bayes's Rule for calculating Pr{N|F} and Pr{U|F} by simple algebra as

Pr{N|F} = Pr{N & F} / Pr{F} = Pr{F|N} Pr{N} / Pr{F},

and Pr{U|F} = Pr{U & F} / Pr{F} = Pr{F|U} Pr{U} / Pr{F}.

Note that Pr{F} as used here means the A Priori probability that all the events Fk would occur, independently of any hypotheses about the events N or U. Note that we can calculate this as Pr{F} = Pr{F|N}Pr{N} + Pr{F|U}Pr{U}. We can make an "independence assumption" and calculate Pr{F|U} and Pr{F|N} as the product of all the conditional probabilities we estimated, Pr{Fk|N} and Pr{Fk|U}. Or, if you think you know how to quantify the lack of independence, be my guest.

I couldn't resist... I just did a little exercise using MathCAD(tm) to see what I would come up with.

Let's use Pr{U} = 10^-6, i.e. one in a million that it could have been a UFO.

Assuming it WAS NOT a UFO, assume that each of these events such as a Roswell undertaker SAYING he was called with a request for a hermetically sealed casket has a conditional probability of .001. (Later you can make your own probability asignments and play around with the results.)

Assuming it WAS a UFO, assume that each of these events has a conditional probability of 0.5, fifty percent.

These assumptions are just for openers - we can negotiate any of them to a consensus.

Plugging in all of the above.... I get a value of Pr{N}, the probability it was NOT a UFO... of 3.2 x 10^-8.

For Pr(U}, the probability that it *WAS* a UFO, I get... ta-da... essentially unity, 1, measure-theoretic certainty. Specifically MathCAD(tm) gives the value 0.999999968.

If I reduce Pr{U}, the A Prior probability that a UFO crash occurred, to .000000000001, or 10^-12, I still get Pr{N}=.03 and Pr{U}=0.97.

Note that Pr{F} doesn't affect the RATIO of the probabilities because it appears in both formulae.

I rest my case. (Actually, I don't, I *INVITE* discussion of the above model, and I hope it will lead to a few eyes being opened and a few others being blackened, figuratively speaking. Or, rather, maybe we'll settle for a little egg on the face of the insinsere skeptics. Maybe even some on mine, if I've made a mathematical slip-up in the above. But I don't think I have, except that the conditional probabilities are negotiable.)

So, let's get on with the discussion! (The Awareness Program *LIVES* !!!)

-John Sangster

Wellesley Hills, MA

Article 143174 of alt.alien.visitors:

From: Brian Zeiler <bdzeiler@students.wisc.edu>

Newsgroups: sci.skeptic,alt.alien.visitors,alt.alien.research,alt.paranet.ufo

Subject: Re: Roswell is sneaky even to experienced debunkers!

Date: Thu, 02 May 1996 00:45:55 -0700

Organization: University of Wisconsin

SPHINX Technologies wrote:

> But I don't think

> I have, except that the conditional probabilities are negotiable.)

Very interesting post. The most valid skeptic criticism is that the observations, F, are not all independent. For instance, they could argue that one lying idiot jumping on the witness bandwagon might further stimulate other crackpot kooks to come out of the woodwork with wild yarns to tell. In this case, the observations are no longer independent, but are conditional themselves on prior observations to the extent that one observation, like a witness's testimony, directly leads to another witness coming forward.

HOWEVER... this criticism doesn't go too far, since many of the observations are indeed objectively independent, such as the Schulgen memo, the Twining memo, the Brazel detention, the military's behavior in other bizarre aspects, and the independent, cross-corroborative testimony of the original set of witnesses.

So, what would definitely be useful is to refine the set of observations to a set that consists of inarguably independent observations like the ones I mentioned above. However, I anticipate that the results will come out nearly the same as the ones you presented. The useful aspect of Bayesian inference is that it doesn't care about whether a hypothesis is "extraordinary" or whether we need "physical proof". All it cares about are the probabilities of the independent observations conditional upon a given hypothesis.

Nice work.

--

Brian Zeiler

Article 143375 of alt.alien.visitors:

From: sphinx@world.std.com (SPHINX Technologies)

Subject: Re: Roswell is sneaky even to experienced debunkers!

Keywords: Awareness Program, UFOs, Roswell, Bayes's Rule, Decision theory

Bcc: bdzeiler@students.wisc.edu

Organization: MJ-12/AP

Date: Fri, 3 May 1996 04:52:57 GMT

In article <31886833.391E@students.wisc.edu>,

Brian Zeiler <bdzeiler@students.wisc.edu> wrote:

>Very interesting post. The most valid skeptic criticism is that the

>observations, F, are not all independent. For instance, they could argue

>that one lying idiot jumping on the witness bandwagon might further

>stimulate other crackpot kooks to come out of the woodwork with wild

>yarns to tell. In this case, the observations are no longer independent,

>but are conditional themselves on prior observations to the extent that

>one observation, like a witness's testimony, directly leads to another

>witness coming forward.

>

>HOWEVER... this criticism doesn't go too far, since many of the

>observations are indeed objectively independent, such as the Schulgen

>memo, the Twining memo, the Brazel detention, the military's behavior in

>other bizarre aspects, and the independent, cross-corroborative testimony

>of the original set of witnesses.

>

>So, what would definitely be useful is to refine the set of observations

>to a set that consists of inarguably independent observations like the

>ones I mentioned above. However, I anticipate that the results will come

>out nearly the same as the ones you presented. The useful aspect of

>Bayesian inference is that it doesn't care about whether a hypothesis is

>"extraordinary" or whether we need "physical proof". All it cares about

>are the probabilities of the independent observations conditional upon a

>given hypothesis.

>

>Nice work.

Thanks. That's exactly right, formulating the problem this way lets us make a maximum-likelihood estimate of which hypothesis is correct, based on any set of conditional probabilities we find reasonable. The a priori probabilities of the alternative hypotheses don't come into play in throwing out each "improbable" observation individually as they keep doing in the typical ad hoc skeptibunker "analysis".

I like to point out that it was this same kind of systematic analytical use of statistics to do objective maximum-likelihood decisions which gave us the dramatic increase in data rate that modems can handle, just in the last few years. In a sense, a modem cranking along at 28.8 kilobaud faces a similar problem to that of a UFOlogist, namely "digging a signal out of noise". Until the modem design community started really using all the data systematically -- doing true maximum-likelihood decisions on the transmitted sequence over a substantial constraint length -- modems were quite limited in performance compared to the true channel capacity. Similarly, the skeptibunker who tries to make the whole decision individually on each incident or report is like the old strategy of symbol-by-symbol decisions which limited data communications to 300 or 1200 baud in the old days.

This is not just a metaphor - the math of it is pretty similar!

What I hope to stimulate by that lengthy posting is to get the a.a.v and sci.skeptic community to come up with conditional probability estimates of some sort that we can agree on, and plug them into the formula and crank out numbers which illustrate how different assumptions affect the result.

We need to come up with working "a priori" probabilities for the two basic hypotheses: U=it was a UFO and N=~U, the complement, it was not a UFO crash. (Or we can use more than two hypotheses, no problem with that.) As I noted, I started out with Pr{U}=10^-6, one in a million. I think that in the absence of real information that's a reasonable starting point, but anybody who wants to can use an even smaller probability.

Of course then Pr{N}=1-Pr{U}.

We also need to come up with a list of "events" or "facts" in the sense noted in the posting. These don't have to be proven facts like UFO landing gear, just the fact that somebody reported something is fine. As you noted, all that matters for the objective Bayesian evaluation is to come up with reasonable *CONDITIONAL* probabilities of each such event for the two cases under consideration.

I think the best way to handle possible dependencies is to lump them together. For example, if Mac Brazell told his son and a few other people about what he found, and somebody wants to lump those other peoples' testimonies together with Brazell's, OK, go for it. Come up with two conditional probability estimates, for how likely all that would have been to happen (a) if what really happened was a UFO crash, and (b) if it was

something else.

I think the next order of business for this thread is for people to propose their lists of significant events Fk that we should include in our list. If we can come up with a good list, the more the merrier, that we can all agree are suitable (except Dean, of course, who is probably having apoplexy along about now since the game is obviously up), then the next thing to do is to ttake a flying guess at assigning conditional probabillities to each of

the selected events.

Since these estimates will of course be subjective, anybody is free to think up their own estimates and run out the numbers for themselves. Presumably we can come up with several sets of assumptions that we all feel are reasonable guesses, aimed at satisfying different constituencies in the group, and we'll all learn how sensitive the answer is to these

different assumptions.

To start the ball rolling, here are some of my favorite "events" to

include.

1. Brazell's initial report.

2. Brazell's incarceration and intimidation by the Military Intelligence

folks.

3. Walter Haut's release of an official Army Air Force press release

saying that a flying disk had crashed and been recovered.

4. The mortician's testimony about the caskets.

5. General Ramey's retraction of the Haut press release. Here we get into

a situation where that statistical independence assumption has to be

questioned. We may even have to split our two basic hypotheses.

But tentatively, I would say that, in the case of Hypothesis U,

this event is not wildly improbable if one makes the additional

assumption that the Government for some reason didn't want tthe truth

to come out. Thus we can throw in the additional hypothesis that

Hypothesis C (Coverup hypothesis): "If a UFO were to crash, the govt.

would want to cover up this fact" and the complementary hypothesis ~C

(read "not C"), or "It is not true that if a UFO were to crash the

govt. would want to cover it up." - That's what I meant above by

hypothesis splitting - we now have four cases, U&C, N&C, U&!C, N&!C,

which complicates the evaluation a bit but can be done if we choose to.

6. The FBI "memo to Mr. Ladd" I think it was, or in any case the one that

says a flying disk crashed but the AAF says it was only a weather

balloon, but on the other hand the FBI agent was dubious because of

other facts he was aware of.

Etc. etc.

Does somebody want to volunteer to compile a master list?

Then we need to start building a table of conditional probabilities:

Event or "fact" Fk Pr{Fk|U} Pr{Fk|N}

(optionally)---> &C &!C &C &!C

F1 - Brazell's testimony .95 .001

F2 - Brazell's incarceration .99 .5 .01

F3 - Haut's press release .5 .9 .000001 .001

F4 - Mortician's testimony .75 .0001

F5 - Gen. Ramey's retraction .95 .1 .95 .95

F6 - FBI Memo .5 .0001

The above are my own first stab at conditional probability estimates. I think you'll find it easiest to go down each column independently, as it is easier to keep in mind the scenario you are evaluating if you don't keep changing it. Also, I think for a first pass we should just *ASSUME* that the coverup policy would be in place. (Mostly because my existing MathCAD model doesn't include the C / ~C variable. But anybody who wants to include that in their own calculation can of course do so. In this case Bayes's Rule can tell you if there's a coverup, too!)

I hope this gets the ball rolling, but it's time to sign off for now. I look forward to seeing what kind of discussion ensues!

-John Sangster

Wellesley Hills, MA

More coming. Stay tuned.

Likings

• The previous information is from:

  Decision-Theoretic Analysis of
  Roswell Incident: UFO or No UFO?
  by John Sangster Wellesley Hills,

  MA 4 May 1996.

And was obteined from

alt.alien.visitors, sci.skeptic , alt.paranet.ufo

Surf the net...
...And watch our goverment