• You are currently viewing our forum as a guest, which gives you limited access to view most discussions and access our other features. By joining our free community, you will have access to additional post topics, communicate privately with other members (PM), view blogs, respond to polls, upload content, and access many other special features. Registration is fast, simple and absolutely free, so please join our community today! Just click here to register. You should turn your Ad Blocker off for this site or certain features may not work properly. If you have any problems with the registration process or your account login, please contact us by clicking here.

"Is there something wrong with the scientific method?"

Randomnity

insert random title here
Joined
May 8, 2007
Messages
9,485
MBTI Type
ISTP
Enneagram
6w5
Instinctual Variant
sp/sx
Yes, exactly what the "rebuttal" (more of an explanation really) said. It can be summarized by the last paragraph:
Although Lehrer makes some good points, where he stumbles, from my perspective, is when he appears to conflate "truth" with science or, more properly, accept the idea that there are scientific "truths," even going so far as to use the word in the title of his article. That is a profound misrepresentation of the nature of science, in which all "truths" are provisional and all "truths" are subject to revision based on evidence and experimentation. The decline effect--or, as Lehrer describes it the title of his article, the "truth wearing off"--is nothing more than science doing what science does so well: Correcting itself in its usual messy and glorious way.

The original article seems to be saying "oh noez, many science truths (i.e. theories) have been disproven" when this isn't exactly news, and the point of science isn't to start out 100% right and then never need correcting. In most cases even theories that end up being wrong can improve the knowledge base, leading to concrete applications as well as leading the way for the next "more correct" theory. It's not like we revert back to stone-age knowledge levels every time a theory is proven wrong.

And it bugged me when the original article was taking about "proven" theories being "disproven". If the theories are being "proven" it's not by the scientific method.
 

Octarine

The Eighth Colour
Joined
Oct 14, 2007
Messages
1,351
MBTI Type
Aeon
Enneagram
10w
Instinctual Variant
so
Not through observation nor induction, but through deduction.

What do you think motivates and guides us through such processes? (or more specifically, why would we pick such axioms in the first place?)
 

erm

Permabanned
Joined
Jun 19, 2007
Messages
1,652
MBTI Type
INFP
Enneagram
5
A computer, with no observational capabilities, using deductions to discover new truths. The old classics, like when the number zero was first thought up, negative numbers, or someone had to point out that A = A, and then used such concepts to build larger, more complex truths.

Empiricism does not enter the picture any more than food does. Sure our deductions are both motivated and sustained by food, just as they are by empiricism and a variety of other things. Formal logic motivates and guides empiricism and our behaviour around food as well. Trying to prove the primacy of one over the other won't work.

No truths deduced through formal logic need to be proven through empiricism, and formal logic does not need to be "trimmed" (made more efficient) by empiricism in order to discover truth. No matter how varied or inefficient the system is. Empirical testing is just one method ("meta"-deduction, trial and error and such also work) of trimming them down, which is useful for saving time and energy, but not necessary.

The counterbalance to all this (and the inherent greater reliability of deduced truths), is that for a lot, if not most truths, formal logic alone is not enough to discover them. Some truths seem only able to be discovered through empiricism (how would we have learned of the stars if we weren't able to see them?). This is where science steps in, a method essentially there to reduce some of the inherent unreliabilities found in empiricism and induction. A good one, but far from perfect.
 

Octarine

The Eighth Colour
Joined
Oct 14, 2007
Messages
1,351
MBTI Type
Aeon
Enneagram
10w
Instinctual Variant
so
I think you are underestimating the importance of recursion in reasoning. I'm not trying to prove primacy of empiricism, simply that empiricism is inherently a part of the system.

The computer experiment is an excellent example. A computer with "no observational capabilities" is just an unprogrammed bunch of transistors. Clearly no truths are going to be derived at all.
For any interesting output to be determined, the computer must either have an observational capacity to learn, or alternatively be programmed by a human. Programming a computer places the computer into such an recursive loop. The rules by which the computer is programmed are determined through experience of the programmer. Secondly, the programmer may (will) modify the output until the desired result is obtained.

Formal truths are impotent, because they are reliant on the truths of the underlying axioms. For either the axioms or the derived truths to be considered universal requires outward examination.

By the way, have you examined any alternative formal systems (fuzzy logic, Bayesian logic or more recent ideas)?
 

erm

Permabanned
Joined
Jun 19, 2007
Messages
1,652
MBTI Type
INFP
Enneagram
5
The computer experiment is an excellent example. A computer with "no observational capabilities" is just an unprogrammed bunch of transistors. Clearly no truths are going to be derived at all.
For any interesting output to be determined, the computer must either have an observational capacity to learn, or alternatively be programmed by a human. Programming a computer places the computer into such an recursive loop. The rules by which the computer is programmed are determined through experience of the programmer. Secondly, the programmer may (will) modify the output until the desired result is obtained.

Empiricism is inherently part of what system? Of course it is inherent to the whole system, but again, taking the first Euler problem:-

Find the sum of all the multiples of 3 or 5 below 1000. Using a computer and a human together, when will empiricism be used to find the truth? When have they observed the result? When I solved it, I don't recall observing data of any kind. I sat in a room, some stuff went on in my mind, typed a few lines of code, compiled and ran, and a number came out.

Do you agree that that is the relevant entirety of the process? If yes, what part is empirical?

Formal truths are impotent, because they are reliant on the truths of the underlying axioms. For either the axioms or the derived truths to be considered universal requires outward examination.

Why must axioms be proved empirically? Some premises must be, but axioms are tautological. It can even be said that they need not be proven nor disproven at all.

By the way, have you examined any alternative formal systems (fuzzy logic, Bayesian logic or more recent ideas)?

Yes. Fuzzy logic, and any system that denies the Law of Excluded Middle I would dismiss, but the details of such don't seem relevant tot he topic.
 

Octarine

The Eighth Colour
Joined
Oct 14, 2007
Messages
1,351
MBTI Type
Aeon
Enneagram
10w
Instinctual Variant
so
Of course it is inherent to the whole system

So you do agree then. A formal system may prove formal truths, but the rules that go into inventing/choosing such a formal system in the first place are derived empirically - through experience.

An infinite number of formal systems can be constructed and therefore an infinite number of truths can be proven with such systems. Why did you pick the example that you did? Given the many possible example, the reality is you were only able to suggest a problem that you were familiar with.

I would suggest that mathematics cannot exist without a mathematician.
 

erm

Permabanned
Joined
Jun 19, 2007
Messages
1,652
MBTI Type
INFP
Enneagram
5
So you do agree then. A formal system may prove formal truths, but the rules that go into inventing/choosing such a formal system in the first place are derived empirically - through experience.

The details of the system are chosen through practice/empiricism and deduction. That does not effect the truths derived from such a system, nor how they are derived. It's merely a matter of efficiency.

An infinite number of formal systems can be constructed and therefore an infinite number of truths can be proven with such systems. Why did you pick the example that you did? Given the many possible example, the reality is you were only able to suggest a problem that you were familiar with.

How can an infinite number of truths be derived from infinite systems? Infinite systems for arriving at the same truths as each other. Just as the different empirical methods discover the same truths, the different deductive methods discover the same truths. Truth does not change according to your method of discovering it. It's the falsities that change according to the method.

I would suggest that mathematics cannot exist without a mathematician.

Of course. All the methods we've discussed cannot exist without the practitioners of them.
 

Octarine

The Eighth Colour
Joined
Oct 14, 2007
Messages
1,351
MBTI Type
Aeon
Enneagram
10w
Instinctual Variant
so
Sorry, I didn't clarify that point as I thought it would be obvious what the implications were. The infinite number of systems includes infinite combinations of axioms. (it is also merely a technicality that not all truths will be able to be proven at the same time with a given system and different systems may produce contradictory truths).

This discussion isn't really going anywhere, so I'd like to get back to the original topic. Is there something wrong with how science is practised today?
 

phoenix13

New member
Joined
Mar 31, 2008
Messages
1,293
MBTI Type
ENFP
Enneagram
7w8
Interesting article here:

http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer?currentPage=all

"The decline effect is troubling because it reminds us how difficult it is to prove anything. We like to pretend that our experiments define the truth for us. But that’s often not the case. Just because an idea is true doesn’t mean it can be proved. And just because an idea can be proved doesn’t mean it’s true. When the experiments are done, we still have to choose what to believe."

This article was mind-blowing for me. Having been a lab rat for 4 years, I knew research was imperfect, but I believed the p-value was a legitimate indicator of a real effect. Now, my faith in p<0.05 is utterly shattered. If randomized clinical trials with statistically significant p-values are so frequently inaccurate, what the hell can we be sure of?!

I aggree that null results need more exposure (maybe on an online database). I've seen null results go unreported many times, and no one even thinks of turning it into a paper, so it isn't completely a bias on the part of the journal. Nevertheless, this imbalance is something we need to correct, considering pooled data from clinical studies are the basis for clinical guidelines, and ultimately, the health of the nations (oooo, dramatic).

I thought of the decline effect article as I was reading this: http://www.nytimes.com/2011/01/06/science/06esp.html?_r=1&ref=science
 

Randomnity

insert random title here
Joined
May 8, 2007
Messages
9,485
MBTI Type
ISTP
Enneagram
6w5
Instinctual Variant
sp/sx
I think stats courses should be mandatory when you start research. The p-value is completely arbitrary, and by definition it means the results seen will be due to random variation 5% of the time (at p=0.05), which can add up when thousands of studies are published yearly. It's only "magic" because the top journals decided to set that as a standard. They could easily have chosen 10% (p=0.1), or 1% (p=0.001) instead.

It isn't a flaw of science that the p-value isn't absolute, though. The popularity of "the cult of the p-value" signals a flaw with the teaching system and perhaps even our general understanding. Knowing that "statistical significance" isn't everything and being aware of bias just means you actually have to use your brain to interpret results, be objective and look at more than one study. Shouldn't be that hard for a scientist :cheese:
 

erm

Permabanned
Joined
Jun 19, 2007
Messages
1,652
MBTI Type
INFP
Enneagram
5
Hmm....this is why I think stats courses should be mandatory when you start research. The p-value is completely arbitrary, and by definition it means the results seen will be due to random variation 5% of the time (at p=0.05), which can add up when thousands of studies are published yearly. It's only "magic" because the top journals decided to set that as a standard. They could easily have chosen 10% (p=0.1), or 1% (p=0.001) instead.

It isn't a flaw of science that the p-value isn't absolute, though. The popularity of "the cult of the p-value" signals a flaw with the teaching system and perhaps even our general understanding. Knowing that "statistical significance" isn't everything and being aware of bias just means you actually have to use your brain to interpret results, be objective and look at more than one study. Shouldn't be that hard for a scientist :cheese:

How is that easy for a scientist, or anyone? Sounds like you are asking for perfection.
 

BlueScreen

Fail 2.0
Joined
Nov 8, 2008
Messages
2,668
MBTI Type
YMCA
I couldn't be bothered reading the whole article. If they stated their contention clearly and got to the point, I might have.

From what I did read: Cocaine+mouse test is ridiculous, aren't you meant to at least try to limit possible variables? Even if they get a statistical distribution, the test is stupid. It's like saying, A=BCDEFGHIJKL, lets leave B,C,D,E,F,G,H,I,J and K to do what they want, and see what distribution we get when changing L. Again in the other cases, they look for tooth fairies or make up catch phrases rather than trying to see what they are leaving out of the system. Maybe no one has the time or resources to be thorough, I'm not sure. Still their method seems so obviously flawed that I wouldn't use it as evidence of anything.

Apart from that, the way it was written reminded me more of a new age business book than a scientific analysis. Or maybe one of those websites where the writer explains how they've disproved relativity. :sleeping:
 

phoenix13

New member
Joined
Mar 31, 2008
Messages
1,293
MBTI Type
ENFP
Enneagram
7w8
Hmm....this is why I think stats courses should be mandatory when you start research. The p-value is completely arbitrary, and by definition it means the results seen will be due to random variation 5% of the time (at p=0.05), which can add up when thousands of studies are published yearly. It's only "magic" because the top journals decided to set that as a standard. They could easily have chosen 10% (p=0.1), or 1% (p=0.001) instead.

It isn't a flaw of science that the p-value isn't absolute, though. The popularity of "the cult of the p-value" signals a flaw with the teaching system and perhaps even our general understanding. Knowing that "statistical significance" isn't everything and being aware of bias just means you actually have to use your brain to interpret results, be objective and look at more than one study. Shouldn't be that hard for a scientist :cheese:

Thank you for assuming I graduated without knowing what a p-value is. I wasn't talking about accepting the results of a shitty study because p<0.05. I was talking about accepting the results of a randomized controlled trial with p<0.05 (or p<0.01). If the p-value is used to support contradictory or highly variable results 41% of the time in a research design that minimizes sources of bias/error (the "gold standard"), that blows me away. RCTs are not infallible, but the percentage is much higher than I expected. Of course, that number could diminish over time... :)
 

Randomnity

insert random title here
Joined
May 8, 2007
Messages
9,485
MBTI Type
ISTP
Enneagram
6w5
Instinctual Variant
sp/sx
Sorry, I wasn't talking about you specifically, just the hordes of grad students/undergrads/etc who don't understand what a p-value is (and stats in general). I see that it came off as directed to you and that wasn't my intention, just using you as a spring board for ranting. :)

Is the 41% from the article? That's another good point to show that p-value isn't the important thing at all, you can get a nice p-value from contradictory or highly variable results as you say. RCT or not. This underlines the importance of repeating studies and comparing your data to all the data in the field (i.e. from independent labs).

edit: I've edited my earlier post to remove the reference to you, to better reflect my intentions.
 

phoenix13

New member
Joined
Mar 31, 2008
Messages
1,293
MBTI Type
ENFP
Enneagram
7w8
Sorry, I wasn't talking about you specifically, just the hordes of grad students/undergrads/etc who don't understand what a p-value is (and stats in general). I see that it came off as directed to you and that wasn't my intention, just using you as a spring board for ranting. :)

Is the 41% from the article? That's another good point to show that p-value isn't the important thing at all, you can get a nice p-value from contradictory or highly variable results as you say. RCT or not. This underlines the importance of repeating studies and comparing your data to all the data in the field (i.e. from independent labs).

edit: I've edited my earlier post to remove the reference to you, to better reflect my intentions.

Ah, I see, and I agree with your point. The 41% study was mentioned in the article, but I don't think he gave the source. I plan on tracking it down when I have time.
 

Octarine

The Eighth Colour
Joined
Oct 14, 2007
Messages
1,351
MBTI Type
Aeon
Enneagram
10w
Instinctual Variant
so
What are your thoughts on Science Based Medicine, as contrasted with Evidence Based Medicine. The charge is that evidence based medicine unnecessarily trusts the results of Randomized controlled trials, without requiring a plausible pathology. A consequence for example, is the placebo controlled homeopathy trials which found significant (p<0.05) effects are deemed to be evidence of efficacy. Science Based Medicine proponents suggest this is insufficient.

The downside is that it means that more controversial (but still plausible scientifically) hypothesis (eg vaccine injury risk) might not be researched due to ideological or political reasons.

There is discussion about the differences at the following blog. http://www.sciencebasedmedicine.org/
I would post the specific links, but that site seems to be down right now. (To be updated)
 

Randomnity

insert random title here
Joined
May 8, 2007
Messages
9,485
MBTI Type
ISTP
Enneagram
6w5
Instinctual Variant
sp/sx
I haven't read the blog much, but read a few articles there in the past and found them very interesting. Mostly those were on how homeopathic "practictioners" prey on desperate people like cancer patients and encourage them to avoid medical treatment. Haven't read any of their comments on EBM, it sounds interesting though, I'll have to remember to look it up when it's up again.

I think that in theory, both science and evidence are pretty essential, although some areas can be "flexed" with more than others (sore muscle remedies vs. cancer treatments, and so on). I can't really say more than that without knowing a more precise description of the two approaches and seeing them in action, I think.
 

phoenix13

New member
Joined
Mar 31, 2008
Messages
1,293
MBTI Type
ENFP
Enneagram
7w8
What are your thoughts on Science Based Medicine, as contrasted with Evidence Based Medicine. The charge is that evidence based medicine unnecessarily trusts the results of Randomized controlled trials, without requiring a plausible pathology. A consequence for example, is the placebo controlled homeopathy trials which found significant (p<0.05) effects are deemed to be evidence of efficacy. Science Based Medicine proponents suggest this is insufficient.

The downside is that it means that more controversial (but still plausible scientifically) hypothesis (eg vaccine injury risk) might not be researched due to ideological or political reasons.

There is discussion about the differences at the following blog. http://www.sciencebasedmedicine.org/
I would post the specific links, but that site seems to be down right now. (To be updated)

The site must still be down because it isn't working for me. I read something from the Sceptic's Dictionary site (http://www.skepdic.com/sciencebasedmedicine.html), and I'm not sure I understand the process. Specifically, I don't know how someone goes about calculating "prior probability". It sounds too subjective... the prevailing opinions of the day should not determine the likelihood of something being true or false .
I'm reminded of how the first body of evidence of the nature of light "proved" that light acted as a wave via Young's Double Slit experiment. If my understanding of prior probability is correct, the prior probablility of light being a particle would be low, rendering those results less powerful/significant than those saying light is a wave when in reality, light acts as both. In this example, the concept of prior probability would not lead us to more accurate results, but would instead skew the results towards what we think they should be.

Now, regarding clinical RCTs: overall, we haven't mastered the translation of the interactions between medical interventions and the body at the molecular level to the resulting clinical effect. Until we can do that and thus determine and control significant confounding variables, RCTs will never be truely unequivocal, but in the meanwhile, we should press on with the best tools we have available while acknowledging its weaknesses.

EDIT: I'm so wordy! Let me know if you need clarification on these long sentences.
 
Last edited:

BlueScreen

Fail 2.0
Joined
Nov 8, 2008
Messages
2,668
MBTI Type
YMCA
Well, why don't you explain how some simple formal logic, like 2 + 2 = 4, is based on empiricism?

I know it isn't quite relevant to the argument, but 2+2=4 is one of the 45 equations you have memorised to allow you to perform addition in base 10 of the Arabic number system. Its logic is a fundamental of natural numbers (and reality) though.
 
Top