Cosmological bias

[unixronin]

There's a very interesting article in the current New Scientist about the large-scale structure of the Universe.  It's interesting, because it exposes some of the ... shall we say, self-deception involved in current dark-matter cosmological theory.

To briefly summarize, current mainstream cosmological theory posits that the distribution of matter in the Universe is homogeneous on a large scale.  Recent observations, though, tend not to support this.  The debate is somewhat divided into two camps; one, exemplified by David Hogg of New York University, says that the overall structure of the Universe is homogeneous when you take cold dark matter into account.  We'll call this school the DMC, for Dark Matter Cosmologists.  The other camp, exemplified by Luciano Pietronero at the University of Rome and the Institute of Complex Systems, says that the overall structure of the Universe appears to be fractal.  We'll call them the Fractal Structure Cosmologists, or FSC.

Now, the currently-mainstream DMC cosmological theories are founded on the assumption that the distribution of matter through the universe is more or less uniform at the very largest scales.  If this assumption turns out to be wrong, then the models break and the theories fail.  Unfortunately for this viewpoint, observations seem to contradict the assumption.  Stars clump together in arms within galaxies, which clump together to form galaxy clusters, which clump into superclusters, which clump into filaments and sheets, forming structures hundreds of millions of light-years across.

The DMC camp says that if you continue to zoom out, the distribution of matter becomes homogeneous.  But the results are now in from the largest-scale astronomical survey of the universe conducted so far, the Sloan Digital Sky Survey.  One of its notable findings is what is now known as the Sloan Great Wall, a sheet-like megastructure more than a billion light-years across.  Given what we know about the age of the universe, and what we can infer from that about its size, there's not a whole lot of room left to zoom out further.

The DMC camp says that the Sloan Digital Sky Survey results prove that the universe is, in fact, homogeneous.  The FSC camp disagrees, pointing out that the statistical methods used by the DMC camp to extract "proof" of homogeneity are flawed, because they start out with the initial assumption that the Universe is homogeneous.  The FSC, indeed, points to the Sloan results as evidence that even on such vast scales, the Universe has visible fractal structure.  The DMC camp says that the distribution smooths out at scales just over 200 million light years; the FSC camp says the apearance of smoothness is a statistical artifact caused by the limited range of the Sloan survey — structures that vast are likely to be far enough apart that the Sloan survey simply did not look deeply enough into the sky to see very many of them.

Why this is a problem is because the existence of large-scale structures the size of the Sloan Great Wall calls into question the accepted models of how those structures formed.  The DMC theorists say that all the observed structures in the Universe formed due to gravitational effects.  The FSC theorists point out that 14 billion years isn't enough time for structures the size of the Sloan Great Wall to form by gravitational effects alone.  This throws serious doubt upon the entire cold dark matter model.

The cold dark matter model says that visible matter makes up only 15% of the matter in the universe, and the remaining 85% is unobservable cold dark matter.  The DMC camp says that the new observations do not undermine the standard model, and that the cold dark matter model accurately explains the Sloan data.

As long as, that is, you apply a "bias factor".

Now let's stop and talk about this a moment.  This bias factor represents the difference between the distribution of matter in computer simulations of the cold dark matter model, and the actual observed distribution of luminous matter.  So by saying that the theory "accurately explains" the Sloan observations as long as you apply a bias factor, the DMC camp is saying "The theory matches the data perfectly, as long as you fudge the data."

So how big is this fudge factor?  Are we talking 1%?  2%?  5%?  10%?

It turns out the "bias factor" is 2.  Which is to say, the visible galaxies are clumped twice as tightly as the predicted distribution.  In other words, the error in the theoretical predictions that needs to be fudged isn't a 5% error, or 10%, or even 20%, it's 100%. The error in the theoretical prediction is equal to the actual observational data.

When the theoretical predictions of your model are off by 100%, that should be a clue that there's something wrong in the model.

But wait!  It gets better!

"Mainstream cosmologists [that's the DMC camp] ... feel that the bias is justified, assuming that galazies cluster in regions of space that are replete with excess dark matter."  The DMC theorists say that dark matter is everywhere, including throughout the voids, but galaxies only shine in the rare regions where it is densest.  In other words, although the visible matter is clumped into strands, sheets and filaments, the distribution of the underlying dark matter is much more homogeneous.

"If the cold dark matter model is correct, then there should be dark matter in the voids," Hogg says.

Uh-oh.  It turns out the Cosmic Evolution Survey Project has just completed its own sky survey.  They combined observations from the Hubble Space Telescope, the Subaru Telescope in Hawaii, and the Very Large Telescope in Chile, using large-scale gravitational lensing to infer the presence and distribution of cold dark matter.  In very bad news for the DMC theorists, those voids are ... well, VOIDS.  They're empty.  According to the Cosmic Evolution Survey, the distribution of dark matter is almost identical to that of luminous matter.

"The first thing that strikes me is the voids," [Richard] Massey [of the California Institute of Technology in Pasadena] says.  "Vast expanses of space are completely empty.  The dark matter makes up a criss-crossing network of strings and sheets around these voids.  And all the luminous matter lies within the densest regions of dark matter."

Although this seems to favor the FSC idea of an inhomogeneous fractal structure to the Universe, the DMC theorists aren't convinced.  The measurements of dark matter, they say, are much less precise han the data on galaxy distributions.  But they're willing to base their own theories on the very inobservability of that dark matter, on the gravitational-lensing observations being wrong, and their initial assumptions — which are not only unsupported by any observational or observable data, but are contradicted by recent observations — being correct despite the observational data.

In short, the DMC is saying, "Cold dark matter is everywhere throughout the Universe, and you can't disprove it because you can't see it."  Now I don't know about you, but this seems to fly in the face of the scientific method that says you don't have a real theory unless it's falsifiable.  (In fact, this is almost approaching the creation-science camp and their assertions that the lack of falsifiability of creation science proves it's correct.)

(The concept of falsifiability, in the scientific method, does not mean that you have to show that a theory is false before it's a valid theory.  It means that in order to accept it as a valid theory, you have to be able to define some objective, unambiguous criterion which, if met, would prove it false.  In other words, you don't have to prove a hypothesis false for it to be a theory; you have to show that a way exists to prove it false.)

Here's where the DMC camp really starts getting defensive.

"My view is that there's no reason to even contemplate a fractal structure for the universe until there is a physical fractal model," says Hogg.  "Until there's an inhomogeneous fractal model to test, it's like tilting at windmills."

Or, to look at this viewpoint a bit more honestly, there's no point considering the possibility until there's a fractal model, and as long as you don't consider the possibility, there'll never be a fractal model, so cold dark matter will remain as the only model.  The way I learned the scientific method, scientists should welcome competing theories, because it provides new ways to test and verify their own.  But it seems the DMC camp — or Hogg, at least — simply doesn't want anyone to even think about any competing theory, because as long as there's no competing theory, then any challenges to homogeneous cold dark matter can just be swept aside and dismissed.

Just one problem:  There is a fractal universe model, the theory of scale relativity proposed by Laurent Nottale at the Meudon Observatory in Paris.  Nottale suggests that the structure of the Universe is fractal because space itself is fractal.  It's unproven as yet, but new data expected to be released by the Sloan survey in 2008, extending the range of observation out to 650 million light-years, may provide more evidence one way or the other.

So how is this debate shaping up?  So far, overall, it looks something like this:

• DMC:  The universe is homogeneous and there's dark matter everywhere.
• FSC:  What if it's fractal?
• DMC:  It isn't.
• FSC:  The Sloan Digital Sky Survey shows structures too big for your theory to explain.
• DMC:  Nonsense.  The Sloan survey proves it's homogeneous, as long as you fudge the numbers a bit.  ... Well, OK, a lot.  You just can't see the cold dark matter everywhere, because it's unobservable.
• FSC:  But the Cosmic Evolution Survey says cold dark matter isn't everywhere, it follows the distribution of visible luminous matter.
• DMC:  The observations are inaccurate!  You can't observe cold dark matter, so you can't prove it isn't there.
• FSC:  But doesn't that mean you can't prove it is there, either?
• DMC:  Well, anyway, it's pointless to talk about it or even think about it unless there's a fractal universe model to consider.
• FSC:  But there is a fractal universe model.
• DMC:  LA LA LA LA LA LA LA LA, I CAN'T HEAR YOU!!!

---

Addendum:

The problems of homogeniety at large scale, large-scale structure, and dark-matter voids discussed above are not the only problems with the homogeneous-cold-dark-matter cosmological theory.  Another problem is that dark matter theorists assert that anomalies in the observed rotational speed of galaxies relative to the predictions of Newtonian gravity are explained by every galaxy being enveloped in a globular cold-dark-matter "halo" more massive than the galaxy itself, with which it interacts only gravitationally.  Dark matter theory, however, fails (to my knowledge) to explain why this dark-matter halo has not coalesced into a disc under the influence of gravity the same way that the visible luminous matter has.  This problem becomes more pronounced when one considers the results of the Cosmic Evolution Survey showing that dark matter (if that's what is in fact responsible for the increased gravititational lensing) clumps together just like normal luminous matter does.  In fact, since dark matter does not interact with light except gravitationally, one would expect that if anything, cold dark matter associated with a galaxy would be clumped more tightly than the visible galaxy, since there is no outward force on it due to radiation pressure.

(I say "if dark matter is reponsible", because there is a competing theory, TeVeS, or Tensor-Vector-Scalar gravity, which posits that instead of having to presume dark matter and the even more vaguely defined dark energy, gravity is a more complex force than we have historically assumed it to be, and is slightly stronger at very large distances and very low accelerations than Newtonian theory would state.  TeVeS, like dark matter, accurately accounts for the observations upon which the existence of cold dark matter is based.)

This brings us to the Bullet Nebula, claimed by both TeVeS and dark matter theory as evidence of the correctness of their opposing theories.  The Bullet Nebula, dark matter theorists will tell you, matches perfectly theoretical predictions based on dark matter and dark energy.

Except that it's not that simple.  Like the "bias factor" mentioned above, there is "small print".  The "small print" in the case of the Bullet Nebula is that in order to make dark matter theory fit the observations, they found it necessary not only to invoke both dark matter and dark energy as factors, but to invent a new and mysterious fifth basic force in the Universe, one which acts only upon dark matter.  It's nature is unspecified, its mechanism is unspecified, its place in the Standard Model is unspecified; it's just a magic mystery force invented to make the theoretical predictions match the data.  "These would match perfectly, if there was a hitherto-unknown fifth basic force in the universe that acted in such-and-such a way, but only affected dark matter."  In short, it's yet another "fudge factor", making the theory seem to work by cooking the books.

Your mileage may vary, but if you ask my opinion, the homogeneous cold dark matter theory is starting to look shakier and shakier all the time.

Comments

[7leaguebootdisk.livejournal.com]

Frankly, I always thought that astronomers were chaining too many asumptions and measurments. You can triangulate distances to nearby stars, everything else is gueswork stacked on gueswork.

[unixronin.livejournal.com]

Actually, there's a lot of it that's much better than guesswork. A classic example of that is the Type 1a supernova — it turns out that once you understand the mechanism of the type 1a, it's possible to know the intrinsic brightness of any type 1a with quite reasonable precision, and it has a particular optical signature which makes it possible to unambiguously identify type 1a supernovae even in distant galaxies. Those two factors together enable it to be used as a standard candle, and knowing the intrinsic brightness and the and the apparent magnitude of one, you can calculate its distance quite closely. (This incidentally serves as a nice cross-check against the Hubble red-shift, just like bristlecone pine tree-ring dating has been used to cross-check and calibrate carbon-dating.)
So then if you know how far away another galaxy is by using type 1a supernovae within it as standard reference candles, you can calculate the intrinsic brightness of other events within it, and everything cascades into a big series of "If we know A and B, we know C, and from that we know D ..."

[smandal.livejournal.com]

Some thoughts and questions:

* Our cosmological evolution has never been structure dominated. We had hot radiation, then hot matter, and just as structure was forming we appear to have become dark energy dominated. I.e., you need dark matter to get the little structure we do see given such low matter density and rapid expansion. Can TeVeS explain the structure formation in this setting?

* The consensus is that the "Sloan Great Wall" is an artifact of analysis, but I'd like to know if this is false. Can you post some references arguing against this consensus?

* I read the Bullet Cluster paper, and don't recall anything hinky. Nevertheless, MOG and some varieties of TeVeS are not excluded. The lesson is that we have direct observation of DMC phenomenology. Could you explain more how they might have fudged the validity of DMC?

Particle cosmology people like me don't really care either way, but need some reliable data on consequences in order to constrain our high energy models. If the conventional picture of structure formation is wrong, that could allow interesting possibilities for particle theories.

[unixronin.livejournal.com]

1. I don't know whether TeVeS explains structure formation or not. As far as I know, it doesn't directly attempt to. It certainly appears that if the fractal-universe theory and scale relativity are correct, then dark matter isn't required to explain any of these macrostructures; the fractal large-scale structures we see are there because space itself is fractal, and the dark matter distribution mirrors the luminous matter simply because the fractal nature of space forces all matter, dark and luminous alike, into those structures. Whichever is correct, I don't believe it makes sense to just assume that the mainstream theory is correct and dismiss all others as unworthy of consideration at all, which is Hogg's stated position.

2. I don't have detailed references yet on the Sloan wall beyond the New Scientist article. But you could try looking to see what Pietronero has published; his camp maintains in turn that what the DCM camp claims is large-scale homogeneity in the Sloan results is an analytical artifact, and that it'll vanish when longer-range data at larger scales is available. The additional data due to be published in 2008, hopefully, will shed more light on this question.

3. Again, I can't point to specific references, but the most recent articles I've seen have said that both DCM and TeVeS fail to fully account for the Bullet Cluster data without making additional assumptions. So it could be that neither theory is wholly correct, or it could be that there's more going on in the Bullet Cluster than we realize. I don't recall what respect TeVeS fell short in, and I haven't seen a detailed explanation of what exactly the Mystery Fifth Basic Force posited by the DCM explanation of the Bullet Cluster is supposed to actually be or do; I just recall reading that invoking such a Mystery Force, acting only on dark matter, was the only way they could make the predictions of DCM theory fit the observations. (I'm curious to know whether DCM theorists have re-evaluated the rest of their predictions taking this mysterious fifth basic force into account, and if so, whether they still fit the observations.)

MOG is not an acronym I'm familiar with, and I'm unable to find a reference to it that appears to be in the correct context.

I don't know enough about the interaction between large-scale cosmic macrostructure and structure formation, and particle physics, to know how one affects the other. Whichever way it shakes out, I'm pretty sure the results will turn out to be interesting.

I'm just becoming increasingly dubious about homogeneous-DCM theory as they add more and more layers of matter and forces and fudge factors trying to make their model fit the observations, and I can't help but think of the geocentric astronomers and their addition of epicycles within epicycles within epicycles within epicycles in efforts to prop up a theory of the universe that we now know to be completely and utterly wrong.

At the current count, homogeneous DCM now posits unobservable cold dark matter, detectable only by its gravitational influence; mysterious dark energy whose nature and operation are unexplained, but which is forcing space to expand; a mysterious fifth basic force that acts only on dark matter, with no proposed mechanism; and roughly homogeneous distribution of dark matter at macroscale, even throughout the great voids. So when the CES project maps dark matter distribution by its gravitational influence, which the DCM theorists say is the only way to observe it, and comes up with a result saying that the dark matter that DCM theory says must be present in the voids isn't there, Hogg's group don't try to come up with an explanation within their theory as to why this could be so; they just dismiss the conflicting data as inaccurate.

There's an interesting doublethink there: DCM theorists say that they can map dark matter distribution with sufficient accuracy to support dark matter theory, but that other researchers can't map it with sufficient accuracy to throw doubt on the theory. To me, it looks awfully like they're cherry-picking their data and discarding the data that doesn't fit their theoretical model. And right or wrong, that's just plain bad science.

[smandal.livejournal.com]

It comes down to predictive power: in the set of maps between measured inputs and observed outputs, how much "data compression" does your theory do on that set? This is greatly complicated in areas like cosmology w/o laboratory experiments to control and maximize your measured inputs.

So, the Lambda-CDM (dark energy + dark matter) model does well I think, esp. since we probably understand the young universe better than our present one through collider experiments. Certainly, all the scientists I interact with have an open mind for alternate cosmologies, and some even for modified gravity (though the recent "precision" gravity tests all align with minimal Einsteinian gravity). Dark matter is also attractive because we think that stable, gravitational-only particles are quite plausible at around 1 TeV. Finally, WMAP strongly confirms the canonical picture of rapid, dilute, homogeneous expansion at the time of the radiation decoupling (though perhaps not necessarily the present).

Of course, the apparent necessity of dark energy confuses everyone. Really, the only theoretical motivation for it is that Einstein's equation allows it :)

[unixronin.livejournal.com]

Oh, sure. And as long as open minds prevail, I don't have a real problem with it. Theories grow, change, evolve, and sometimes die to be replaced by better ones. When I have a problem is when scientists dismiss competing theories as not even worth thinking about, while simply ignoring and discarding observational data that contradicts crucial points of their own.

[smandal.livejournal.com]

Yes, it's most commonly referred to as "having an ax to grind." It manifests most amusingly as yelling matches at conferences :)

Kind of sad really -- it means that you don't own the subject, nor your own desires. OTOH, I imagine that researchers at schools/institutes/labs outside of the Top "N" feel like they have to fight to hang on in the community.

[ilcylic.livejournal.com]

Hunh. These dark matter cosmology guys sound like the anthropogenic global warming folks. Except, of course, you didn't mention the DMC folks calling for tribunals against the FSC adherents. Or them referring to the FSC people as "dark matter deniers". So they haven't quite reached that level of insane ideological adherence yet.

[unixronin.livejournal.com]

Personally, I think the people who say "It's all anthropogenic" and the folks who say "None of it can possibly be anthropogenic" both have their heads up their asses.

Like it or not, we are at a solar maximum, and are receiving more energy from the Sun. However, according to IPCC data, that's an increase of 0.12 W/m², while they say anthropogenic causes are contributing 1.6W/m². And on a fiddler-on-the-roof third hand, we do know there are long-term fluctuations in the global climate whose causes we don't yet wholly understand.
The correct answer is surely that it's a mixture of all of these causes. But it's not the causes that are going to hurt us; it's the effects, and if we decide that it can't possibly be our fault and use that as an excuse to not do anything, we're going to be hurting. The rate of loss of ice sheets is doubling every six years, and the entire Siberian permafrost is thawing, with a possibly huge resultant methane release.

.As far as I can tell, the principal argument of the people who say global warming can't possibly be anthropogenic is that, gee, the planet is so huge we mere humans can't possibly be affecting it. But you know, we used to think that the sea was effectively infinite relative to anything we could ever do to it, so we could just keep on fishing to capacity and dumping waste into it forever with no ill effects.

But now, here we are, with something like 70% of the world's fisheries crashed or crashing from overfishing, the entire Mediterranean is polluted, and the Irish Sea is radioactive. Things don't look so infinite any more.

[ithildae.livejournal.com]

I don't have a horse in this race. Dark matter makes sense to me in many ways, how it is distributed is not affected by my bias.

The model question is crucial. We use models to predict findings. (Seems like the DMC folks have forgotten that, but so have the FSC group.) Models also answer certain questions, often mathematically, that used to stump us. Without going back and reading, I don't remember the list of stuff that the DMC folks were able to put to rest with their model, but I remember it being some fundamental observations. That caused many people to buy into that particular model. I don't necessarily condemn them for not throwing away a working model without even seeing the mathematical implications of something new. It will take time to chew over the newer stuff.

As an example, chemists still use the Bohr model of the atom. It only works for Hydrogen, but it is such a useful model to teach and understand that it would be foolish to abandon it, even though it is wrong.

I remember hearing many years ago that science changes one funeral at a time. That is more true than anyone would ever care to admit. The only way to slay the old dragon is with a bigger and better dragon. It does get ugly at the edge of knowledge.

[unixronin.livejournal.com]

Sure, I'm not questioning any of those points.

I don't know if TeVeS is correct. I don't know if FSC is correct. I see increasing evidence that DMC, at best, needs more work (especially when one of its leading supporters says that "if the cold dark matter model is correct, then there should be cold dark matter in the voids", but the best observational data available at this time says there's no cold dark matter in the voids).

I don't basically have a problem with this. Theories have flaws. They evolve, they grow, they mature, they get fine-tuned. Sometimes they get discarded and replaced with better theories. Sometimes a competing theory arises, and the debate between the two results in the original theory being amended and emerging stronger, with a better model that makes closer and more accurate predictions.

What I have a problem with is when supposedly objective scientists fudge the numbers in their theoretical predictions to make the predictions fit observational data by applying arbitrary correction factors as large as the actual observational data, and simply dismiss out-of-hand observational data that they cannot make fit into their theory on the basis that if it doesn't fit the theory, it obviously must be wrong. When you come across observational data that doesn't fit your theory, the scientific method says you re-examine your theory in light of the new data — not sweep the conflicting data under the rug and pretend it doesn't exist.

There's also the Occam's Razor factor. As we all know, Occam's Razor says that if you have two possible explanations for a phenomenon or event, then the one which requires the least new assumptions is the more likely to be true. At this point, the cold dark matter theory requires a new type of matter that we don't know anything about and lack any kind of model for, except that the theory requires it interacts with the rest of the known universe only gravitationally; a new kind of energy that we don't know anything about and lack any kind of model for, except that the theory requires a repulsive force pushing the universe apart to make the predictions match the observations; and a fifth basic force acting only on the mysterious dark matter, that — again — we don't know anything about and have no model for, but without which the theory cannot be made to explain the data. In short, the entire theory is based upon the existence and nature of three almost total unknowns, only one of which — the cold dark matter — we have any actual observational data for, and that observational data is entirely inferential: it basically only supports the existence of cold dark matter if you assume the existence of cold dark matter that is only detectable by inferring its presence from that observational data.

By my count, cold dark matter theory is making no less than three new and otherwise completely unsupported assumptions, all three of which are basically simply defined as having the properties required to account for the observational data, with no experimental confirmation and no theoretical explanation of why they should be that way except that that's what the math requires (and even then, they still have to fudge the numbers), with no evidence for their existence except inference from a class of observational data which currently appears to show that at least one of the required conditions of the theory is, in fact, not true. That seems to me like a pretty shaky foundation for a theory.

I'm not saying that the competing theories are, at this point, on any more solid ground (though in some cases, TeVeS for one, they do seem to require less assumptions). I'm just saying that it's too early in the game to be dismissing the other theories out of hand and declaring cold dark matter to be the One Truth, especially when observational data is throwing doubt upon several of its core requirements.

[ithildae.livejournal.com]

I am not arguing any of your points. What I wanted to say is, give it some more time. These conversations on changing basic theories take place over years, not months, and certainly not from a single source of conflicting data.

Physicists are the most pliable about accepting new theories because they get bombarded with new data so often. It is hard to maintain a swelled ego in the face of many sources of new data that conflict with your baby, single sources of data are easy to ignore.

Be grateful that we are talking about physics here, there are some branches of "science" where the prevailing wisdom is so rigidly enforced that any departure results in immediate loss of position/funding/status.