I think there is a conflict here only if you go into it with the expectation that we know all there is to know, and the only question is whose version is right. What if you assume, instead, that all our knowledge is imperfect? Then your critical thinking skills are directly relevant, because you are always evaluating which version of current scientific knowledge is the least imperfect.
For me, I start with triage: how much do I care/how much does the answer matter to me/would the difference between the options be meaningful? The level of effort I put into figuring out the best approach is directly proportional to the import of getting it "right." So when I was wondering about what a good stroller/carseat was, I asked my SIL (who had done all the research); if she hadn't been available, I'd have pulled up Consumer Reports or somesuch and called it good. OTOH, when I was struggling to meet my daughter's needs, I spent literally years reading all sorts of parenting books and researching the state of the science until I figured it out.
Assuming it's something I care about and there are conflicting versions of "right" out there, I look for problems that might undercut the findings on each side. What is the sum total of the evidence/research/sample size we have (more is better)? Is it clinical or epidemiological (epidemiology is fine, but if we don't have some clinical explanation of why/how the purported cause leads to the noted effect, that leaves open the possibility that it's correlation, not causation)? How have they address possible confounding factors, and have they left anything out? Is there bias built into the findings (e.g., old studies that relied on white men being extrapolated to other races and genders)? Does the study come from someone with an ax to grind (which increases the likelihood of a finger on the scale)? Have the study results been peer-reviewed (although even this is caveated, as it's very easy to get into Groupthink). Is there some other way the same data could be construed to draw some other conclusion (the Freakanomics kind of approach -- e.g., does the marshmallow study mean poor kids have no self-control, or does it mean they are acting rationally given their life experience?).
Actually, if it's a newspaper headline, the first question is "does the study really say that?" Because more often than not, the study doesn't say what is reported, or all of the caveats and qualifications are completely ignored.
After I do all that -- again, assuming I care enough to do so -- I am usually left with an imperfect answer on one side and a slightly-less-imperfect answer on the other. And usually by that point, I go with what seems to make the most sense to me, based on my own understanding of the issues involved and how well the author has tied in his/her points to solid scientific principles and data. Which is not always majority rule; for ex., I was a huge fan of behavioral economics back when the guys who initiated that branch of the science were still by and large treated as pariahs, because what they were saying just made sense (I'd never bought the whole "rational man" precept that traditional economics is based on; I just didn't have a framework for any other alternative).
