This has turned into a really interesting discussion.
A few points...
 "Exhaustive" Induction is, in a way, a deductive argument. Suppose there are only 4 instances of consideration in the domain of proposition, P: a, b, c, and d. Suppose further that P(a), P(b), P(c), and P(d) are true. Then P is true on it's domain.
Here we are reasoning from specifics to the general case. However, the specifics form independent premises which fully justify the conclusion (which is, in effect, a disjunction of all the premises).
 Statistical "Sampling," seems to also be a way to reason from specifics to generalities. We get a sufficient sample, and calculate the mean of a particular observable on that sample, and use that mean as an estimate for the mean of the observable for the whole population.
But, again, we see that the power of this method actually comes from deductionthe deduction that starts from the axioms of probability theory, and a further assumption about the population distribution. Now, along with the estimate, when "done right" with deduction, we get confidence intervals for various confidence levels based on our sample size.
Similar things are true of further tools of statistics...regression, factor analysis, cluster analysis, etc. The power of the seemingly "inductive" method is actually derived from deductionthe deduction starting from the axioms of probability and the assumptions of the probability models involved.
 Although the process of deduction that leads to many things that we widely accept to be true (the laws of physics, the laws of probability, the conclusions of geometry) is painstaking and an even ingenious at points, I am more impressed with the (often errorprone) process of abductive reasoning that produced the very axioms we use.
How did Newton come to posit the basic premises of Classical Mechanics?
How did Euclid and others create the postulates of Euclidean Geometry?
Many of of us live in structures that rely on the amazing accuracy of these incredibly well chosen premises of deductive reasoning.
Certainly, they were based on on experiments and observations. But I very much doubt that these experiments and observations were as slavishly scrutinized and analyzed, before their proposition of premises, as many of the observations and experiments leading to (far less accurate) theories of modern life and social sciences. They were incredibly scrutinized after they were proposed, and withstood those tests to amazing effect.
Like, reason, I challenge the value of inductive reasoning. It somehow seems to lack the intellectual courage that abductive reasoning takes.
As a further example, consider Quantum MechanicsAmong the most accurate theories we have. The abductive leaps taken (from "photons" to "matter waves") often seemed downright audacious.
I posit that what requires intellectual rigour and care is NOT what leads to the generalities we posit, but the intellectual rigor and care that follows. If the premises we create are malformed, then the wake of deductive reasoning and empirical testing should destroy the weaker theories.
User Tag List
Thread: Hume's problems with induction

09292009, 07:23 PM #11
Accept the past. Live for the present. Look forward to the future.
Robot Fusion
"As our island of knowledge grows, so does the shore of our ignorance." John Wheeler
"[A] scientist looking at nonscientific problems is just as dumb as the next guy." Richard Feynman
"[P]etabytes of [] data is not the same thing as understanding emergent mechanisms and structures." Jim Crutchfield

09292009, 07:37 PM #12
The power of statistics isn't just about sampling size and extending it to the general population. that sampling size jump to the general population is necessary for practicality, you simply can't sample the global population for every hypothesis and test you want to conduct. practically, it is necessary to take a sample, because most people don't have a hundred years of testing time available to conduct one experiment by testing the entire population. The power of statistics for hypothesis testing lies in being able to eliminate randomness and establish a high probability of "connection", which will never be certain 100%. you would need inductive reasoning in most of the extremely valuable areas of statistics, in terms of hypothesis generation, hypothesis testing, explanatory power, autocorrelation elimination, etc... what does seem more deductive, is filling in the lego pieces between the architecture of the statistical model. but without inductive reasoning, you are left with 10^100000 possibilities to aimlessly apply deductive logic in general.

09292009, 07:50 PM #13
I don't dispute that the statistical methods save a lot of time. What I do dispute is that that the statistical methods are truly "induction."
What I am claiming here is that the seeming "induction" is actually abduction in disguise. The abductive step is in the choice of statistical methods to use (which carries with it the assumptions of the probability models involved).
Statistical reasoning is more accurate than going by "feel" based on evidence, because it is based off of the laws of probability.
The models that lead to hypothesis generation, hypothesis testing, autocorrelation elimination, etc., are all deductive processes (that's why you can program them to run on computers).
Accept the past. Live for the present. Look forward to the future.
Robot Fusion
"As our island of knowledge grows, so does the shore of our ignorance." John Wheeler
"[A] scientist looking at nonscientific problems is just as dumb as the next guy." Richard Feynman
"[P]etabytes of [] data is not the same thing as understanding emergent mechanisms and structures." Jim Crutchfield

09292009, 07:59 PM #14
Lets say you run a simple two sample t test to see if Americans are heavier in weight than Finns.
Lets say you find with 95% confidence that this seems to be so.
Trying to determine whether this information is useful or not would require more testing. Trying to eliminate possibilities like weight per height inch, % of women vs. men, relative age, etc... and finding the right conclusion would require a lot of inductive reasoning. Is it really just that Americans are fatter? You can't just conclude that with that simple t test. If so, why are they fatter? Questions that you simply cannot solve with a high degree of confidence with deductive reasoning alone.
Trying to determine whether your sampling population is accurate and relevant would require a lot of inductive reasoning. Testing explanatory variables, and finding those explanatory variables is very inductive. How would you know which autocorrelation variable to eliminate? What falls more in line with your hypothesis and won't corrupt the results? Does that autocorrelation specific to your sample or is it not?
I think in general you can classify deductive as what the field researchers will do, and inductive as what the conclusions the professor can derive or what the professor will do in "consulting" with the field researchers if they want to have their results up to a global standard. Once you can conclude that you've reached the end of knowledge in the entire world, you can say you don't need inductive reasoning anymore. well, in that case, you wouldn't need deductive reasoning either.

09292009, 08:10 PM #15
I claim what you are doing is abductive reasoning.
Note you used the word "eliminate" not "conclude," which implies deduction IS happening. You are testing out possible explanations for the disparate things you see...generating those possible explanations is abduction, testing them afterwords is deduction.
True induction would look at the evidence, then conclude something, not generate a possible explanation. That is abduction.
Accept the past. Live for the present. Look forward to the future.
Robot Fusion
"As our island of knowledge grows, so does the shore of our ignorance." John Wheeler
"[A] scientist looking at nonscientific problems is just as dumb as the next guy." Richard Feynman
"[P]etabytes of [] data is not the same thing as understanding emergent mechanisms and structures." Jim Crutchfield

09292009, 08:44 PM #16
well, i think its both induction and abduction. because eliminating auto correlation, and looking at possible sample corruptions can be derived from previous similiar experiences, which would be inductive on wider scale.
but to say again, you can't conclude anything with 100% certainty from statistics. if you derive a 100% confidence interval, your intervals will be too wide to produce a useful result.

09292009, 09:10 PM #17
Well, I suppose I have a very rigid view of what induction is. So we'll likely have to agree to disagree here. I believe abduction works in all cases that induction does, it is simply done in a more intuitive and heuristic basiswhere as induction requires the citing of "reasons" and coming to a "best" guess.
I suppose you could use induction, with all the careful sighting of reasons, and coming to conclusions at various phases, but this seems like a more inefficient process actually. Because it seems like you generally would want to go through a deductive reaoning process afterwards anyway.
I still believe that you can forgo most induction for abduction followed by deduction without losing much. The extra time taken for deduction (which with the aid of computational devices is becoming smaller and smaller) is saved by the speed, flexibility, and possible recontextualization of abduction.
Accept the past. Live for the present. Look forward to the future.
Robot Fusion
"As our island of knowledge grows, so does the shore of our ignorance." John Wheeler
"[A] scientist looking at nonscientific problems is just as dumb as the next guy." Richard Feynman
"[P]etabytes of [] data is not the same thing as understanding emergent mechanisms and structures." Jim Crutchfield

09292009, 11:37 PM #18
well actually before this conversation i grouped everything as either inductive or deductive. but after this convo, I guess I am more prone to think that inductive is like eating a big carl, and thinking that it is similiar to a big mac and possibly copied from it, while abductive would be like eating a big carl, and seeing fat Mr. Carl Jr., and wondering if its some cruel joke by the marketing chief.
but i do believe that induction does incorporate pattern recognition more, in either logic, math, causality, etc... and i believe pattern recognition is very important in interdisciplinary research. its where a lot of University of California style research of bringing in different experts and having them collaborate really shines. i.e. anthropologist applies verbal history insights to linguists research in linguistic language derivation and makes breakthroughs... something like that.

09302009, 07:02 PM #19
To give you an idea of how rigidly I think of induction, I would say that almost all of this is abduction also. I believe the only induction is thinking that the Big Carl is similar to the Big Mac...the "and possibly copied from it" would be abduction.
Pattern recognition can come from any of the three processes.
Also, in this case, I claim most of their research programs are more of a mix of abduction followed by deduction.
I would say that if the pattern recognition was inductively done, that is, after looking at a lot of data a a formally "justified" pattern was recognized, that this alone is not worth much for the following three reasons.
 I believe that a pattern clear and rigorous enough to be recognized by strict induction, it could have been either intuited through abduction, far earlier, or recognized by automated statistical techniques (i.e. through deduction).
 The pattern will generally need to be tested through deduction afterwords anyways. The predictive power of the pattern must be tested by deducing outcomes and confidence regions around those outcomes, and checking that the outcomes do actually match.
 Without abduction giving an explanation, the purely inductively derived pattern cannot be recontextualized or generalized to other situations. You are trapped, essentially, to the phenomenon that generated the pattern.
I'll put it in other terms... getting a pattern inductively is the worst of both worlds. It combines the clumsiness and uncertainty of a subjective process (a fault that it shares with abduction), with the rigidity and lifelessness of a formal process (a fault it shares with deduction).
Maybe I am biased, because I rarely think..."example, example, example...general case." I do however often think "...phenomenon, phenomenon, phenomenon...common explanation, ok, let's check how well theory fits."
I suppose arguing that a thinking process has little value for all people is a fool's errand. Some people may use it to good effect, I just don't see much value in it, myself.
Accept the past. Live for the present. Look forward to the future.
Robot Fusion
"As our island of knowledge grows, so does the shore of our ignorance." John Wheeler
"[A] scientist looking at nonscientific problems is just as dumb as the next guy." Richard Feynman
"[P]etabytes of [] data is not the same thing as understanding emergent mechanisms and structures." Jim Crutchfield
Similar Threads

[NT] Problems with IQ, tests ect..
By God Of Emptiness in forum The NT Rationale (ENTP, INTP, ENTJ, INTJ)Replies: 22Last Post: 02052008, 12:03 PM 
Does anyone have a problem with Hyperfocusing?
By alicia91 in forum General PsychologyReplies: 8Last Post: 01052008, 06:43 PM 
A Note on the Problem of Induction
By reason in forum Philosophy and SpiritualityReplies: 3Last Post: 09192007, 08:47 AM