User Tag List

123 Last

Results 1 to 10 of 22

  1. #1
    Symbolic Herald Vasilisa's Avatar
    Join Date
    Feb 2010
    Posts
    4,127

    Post Over half of psychology studies fail reproducibility test

    Over half of psychology studies fail reproducibility test
    Largest replication study to date casts doubt on many published positive results.
    Monya Baker
    27 August 2015
    Nature

    Excerpt:
    Don’t trust everything you read in the psychology literature. In fact, two thirds of it should probably be distrusted.

    In the biggest project of its kind, Brian Nosek, a social psychologist and head of the Center for Open Science in Charlottesville, Virginia, and 269 co-authors repeated work reported in 98 original papers from three psychology journals, to see if they independently came up with the same results.

    The studies they took on ranged from whether expressing insecurities perpetuates them to differences in how children and adults respond to fear stimuli, to effective ways to teach arithmetic.

    According to the replicators' qualitative assessments, as previously reported by Nature, only 39 of the 100 replication attempts were successful. (There were 100 completed replication attempts on the 98 papers, as in two cases replication efforts were duplicated by separate teams.) But whether a replication attempt is considered successful is not straightforward. Today in Science, the team report the multiple different measures they used to answer this question1.

    The 39% figure derives from the team's subjective assessments of success or failure (see graphic, 'Reliability test'). Another method assessed whether a statistically significant effect could be found, and produced an even bleaker result. Whereas 97% of the original studies found a significant effect, only 36% of replication studies found significant results. The team also found that the average size of the effects found in the replicated studies was only half that reported in the original studies.

    There is no way of knowing whether any individual paper is true or false from this work, says Nosek. Either the original or the replication work could be flawed, or crucial differences between the two might be unappreciated. Overall, however, the project points to widespread publication of work that does not stand up to scrutiny.

    Although Nosek is quick to say that most resources should be funnelled towards new research, he suggests that a mere 3% of scientific funding devoted to replication could make a big difference. The current amount, he says, is near-zero.

    Replication failure
    The work is part of the Reproducibility Project, launched in 2011 amid high-profile reports of fraud and faulty statistical analysis that led to an identity crisis in psychology.

    John Ioannidis, an epidemiologist at Stanford University in California, says that the true replication-failure rate could exceed 80%, even higher than Nosek's study suggests. This is because the Reproducibility Project targeted work in highly respected journals, the original scientists worked closely with the replicators, and replicating teams generally opted for papers employing relatively easy methods — all things that should have made replication easier.

    But, he adds, “We can really use it to improve the situation rather than just lament the situation. The mere fact that that collaboration happened at such a large scale suggests that scientists are willing to move in the direction of improving.”

    The work published in Science is different from previous papers on replication because the team actually replicated such a large swathe of experiments, says Andrew Gelman, a statistician at Columbia University in New York. In the past, some researchers dismissed indications of widespread problems because they involved small replication efforts or were based on statistical simulations.

    But they will have a harder time shrugging off the latest study, says Gelman. “This is empirical evidence, not a theoretical argument. The value of this project is that hopefully people will be less confident about their claims.”

    Publication bias
    The point, says Nosek, is not to critique individual papers but to gauge just how much bias drives publication in psychology.

    < Full Story >
    the formless thing which gives things form!
    Found Forum Haiku Project


    Positive Spin | your feedback welcomed | Darker Criticism
    Likes Cellmold, Qlip liked this post

  2. #2
    Senior Jr. SD45T-2's Avatar
    Join Date
    Feb 2012
    MBTI
    ESTJ
    Enneagram
    1w2 so/sp
    Posts
    3,387

    Default

    Yet another reason why we are all doomed.
    1w2-6w5-3w2 so/sp

    "I took one those personality tests. It came back negative." - Dan Mintz

  3. #3
    Member
    Join Date
    May 2015
    MBTI
    InTJ
    Posts
    68

    Default

    Still better than religion.
    Likes Poki, Riva liked this post

  4. #4
    Senior Member
    Join Date
    Dec 2008
    MBTI
    STP
    Posts
    10,583

    Default

    That's because alot try to follow black and white because of statistics. Averages, means, medians all belong in math. Psychology is the understanding of one of the most complicated subjects on earth as we are driven by what's around us as well as what's inside of us and they both have a cause and effect on each other which further compounds things. So while there may be a obsessive compulsive disorder it may have a different reason, desire, drive based on each person. To assume there is only a single path in and out is closed minded. Things can even compound on each other and look like something else. It is a very complicated subject that has to be taken on a situation by situation basis. Group concepts and theories are jumping points to dig deeper, not the answer or end all be all. Everything should be a "this is what's witnessed and observed" psychology shouldn't have conclusions, just shared knowledge of experience. It's to complicated to be simplified into a step a...b...c...d. it's more like a complicated Web with loops and back tracks and dead end paths.
    Im out, its been fun
    Likes Lark liked this post

  5. #5
    Senior Member
    Join Date
    Jun 2009
    Posts
    27,534

    Default

    Quote Originally Posted by Stek View Post
    Still better than religion.
    How so?

  6. #6
    Senior Member
    Join Date
    Dec 2008
    MBTI
    STP
    Posts
    10,583

    Default

    Quote Originally Posted by Lark View Post
    How so?
    Because religion just says "gay is bad" and uses a bible that has a conclusion that "gay is bad" as the reason. You can't even prove that the analysis is correct as there is no analysis, just conclusion. It is propogation of conclusions with no supporting evidence other then restatement of conclusion from another source. Atleast science, science of psychology included, has supporting evidence that denotes an attempt to reason and understand.
    Im out, its been fun

  7. #7
    Senior Member
    Join Date
    Jun 2009
    Posts
    27,534

    Default

    Quote Originally Posted by Poki View Post
    Because religion just says "gay is bad" and uses a bible that has a conclusion that "gay is bad" as the reason. You can't even prove that the analysis is correct as there is no analysis, just conclusion. It is propogation of conclusions with no supporting evidence other then restatement of conclusion from another source. Atleast science, science of psychology included, has supporting evidence that denotes an attempt to reason and understand.
    That sounds unlike any religion I know.

    It sounds a lot like what atheists say about religion though.

    Should I accept the as an authority on something the people who declare a hatred for it? I'm not sure that is wise, I would not expect it to be balanced and unbiased.

  8. #8
    Senior Member
    Join Date
    Dec 2008
    MBTI
    STP
    Posts
    10,583

    Default

    Quote Originally Posted by Lark View Post
    That sounds unlike any religion I know.

    It sounds a lot like what atheists say about religion though.

    Should I accept the as an authority on something the people who declare a hatred for it? I'm not sure that is wise, I would not expect it to be balanced and unbiased.
    Lol...you realize aethiest is based off of questioning due to lack of validity, proof, and reasoning. Compare this to all the blind "amen" followers that associate something good with God unquestionably because its what they want to believe, not because there is analysis, data, and proof to back it up. God is great, he allowed my car to drive with no engine....halleluja...amen...god is awesome. This man was stoned because he questioned god....halleluja...amen...he will smite those who go up against him. I am biased, biased against stupidity associated with blind faith. Which is exactly why this study was done against psychology. Can you imagine this being done against religion. It would not be embraced, all hell would break lose.

    Accept what you want...this is nothing more then my reasonings based on my observations of so called "religious" people.
    Im out, its been fun
    Likes Hard liked this post

  9. #9
    Senior Member
    Join Date
    Jun 2009
    Posts
    27,534

    Default

    Quote Originally Posted by Poki View Post
    Lol...you realize aethiest is based off of questioning due to lack of validity, proof, and reasoning. Compare this to all the blind "amen" followers that associate something good with God unquestionably because its what they want to believe, not because there is analysis, data, and proof to back it up. God is great, he allowed my car to drive with no engine....halleluja...amen...god is awesome. This man was stoned because he questioned god....halleluja...amen...he will smite those who go up against him. I am biased, biased against stupidity associated with blind faith. Which is exactly why this study was done against psychology. Can you imagine this being done against religion. It would not be embraced, all hell would break lose.

    Accept what you want...this is nothing more then my reasonings based on my observations of so called "religious" people.
    You think that is a good basis from which to generalise?

    I think you are talking about religion at its worst, are you prepared to consider religion at its best?

  10. #10
    Tempbanned
    Join Date
    Jan 2014
    MBTI
    ENFJ
    Enneagram
    1w2 sp/so
    Socionics
    EIE Fe
    Posts
    8,589

    Default

    Quote Originally Posted by Lark View Post
    You think that is a good basis from which to generalise?

    I think you are talking about religion at its worst, are you prepared to consider religion at its best?
    The point made is very simple: Would you rather the study of the human mind be based off religon, or science?

    The correct answer is science, even if there are flaws.
    Likes Julius_Van_Der_Beak liked this post

Similar Threads

  1. The value of random on-line personality tests
    By Ilah in forum General Psychology
    Replies: 6
    Last Post: 09-26-2008, 08:33 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
Single Sign On provided by vBSSO