The arguments can get complex but return repeatedly to rest on a couple of basic issues. Here is a question to start with: What, if anything, are we allowed to take for granted when we describe the beginning of the universe? The obvious rejoinder to any proposal that "X caused (or is a reason for) the universe, so the existence of the universe is explained by X" is to say, "Alright, but where did X come from?" (Or, if X is a law or principle, why does X obtain? What makes it true?) This rejoinder is extremely effective when X is, say, God: as Richard Dawkins, among many others, has pointed out, the religious "explanation" of the universe – God made it! – is entirely unsatisfying unless one can also explain who made God. One can hold that God does not need an explanation, of course; but then, why not just say that about the universe itself? As one would expect, there have been attempts to show that God, by his nature, is special and does not need any such explanation; the attempts offered thus far, though, are hopelessly unsatisfying for any questioner not already committed to a religious framework for thought.
But the "So where did X come from?" rejoinder is not only a problem for religious accounts; it also raises issues for any scientifically oriented explanation that aspires to completeness. 'The Higgs boson did it" is as unsatisfying as "God did it" unless we can explain why the Higgs boson and other elementary particles exist at all. (Actually it's not clear that the Higgs boson, if and when its existence is conclusively confirmed, will shed any light on questions of the universe's origins, despite its somehow managing to get itself nicknamed "the God particle.") More seriously, saying that the universe began with the Big Bang doesn't really explain much of anything unless we can explain why the Big Bang happened; if it's just something that came out of nowhere, for no reason, then it can't be counted as a complete explanation. Similarly, a number of physicists have argued that quantum mechanics allows a universe to pop into existence out of something extremely minimal – an energy-packed void, a quantum fluctuation, or something of that ilk. But these accounts still start with something, and so we can still ask where the energy that packed the void came from, or why the laws of quantum mechanics that permit such fluctuations should be as they are. A complete explanation of the universe's existence, as many people understand it, would have to explain those, too.