Physicists are more accustomed in thinking of the behavior of the universe in terms of 'laws'. These are usually formulated into mathematical equations, and from there the thought is that the universe always behaved accordingly, even an infinite time ago.
But, philosophers are sometimes more concerned about the status of a law rather than the equation itself. That is, does the law only describe 'regular' movements of matter (e.g., statistical ensembles of material stuff), or do the laws exist, and therefore material stuff behaves accordingly. It is quite clear that many physicists, including Einstein, preferred the prior perspective of physical laws.
However, the prior perspective is inconsistent with materialism. Laws can only be describing the regularities of the universe, and not somehow forcing material stuff (i.e., some fundamental stuff that all things are made of) to behave 'mathematically' or 'logically', etc. In other words, logico-mathematical behavior is entirely an emergent phenomena in a materialist universe. First the 'stuff' does something, and then the ensemble of stuff takes on certain 'lawful' standards of behavior which only appears logical, or appears mathematical. The reason it behaves such is because it happened to do so, there's no logic or mathematics that force it to behave so.
Well, this puts a random spin on things when talking about 'laws of physics'. The 'stuff' that regulate themselves without restraint (except to their own axiomatic behavior) could presumably had different set of axiomatic behavior, and therefore regulate themselves entirely differently had it just happened to be the case. Or, we only see a 'slice' of their ensemble behavior at a certain 'time', and for all we know, the tomorrow or the day after the consistent laws that we all know and trust will sporadically change, perhaps even with our knowing since we'll be gone like a twister hitting an outhouse in Kansas during tornado season.
What this says to me is that our universe and the timeline of sequential events are ordered by pure happenstance. There's a few cases to explore, but each have the same answer:
1) Infinite universe is eternal: The concept of time is a pure illusion. Our past, present, and future of the universe just 'exists', and this whole (our timeline) is just one of many infinite collections that are 'out there'. Sort of a hodgepodge world of stuff, made up of random assortments of timelines where anything happens in and out of 'sequence'. If this is the case, then it is odd that we happen to live in a universe that appears sequential (from the earliest time at the big bang to the trip that I just came back from). Why not even a deviation? Why no unexplained odd events in our history where the 'future' (or a future-like event) co-mixed with our past. In such a hodgepodge scenario there's nothing that forbids this kind of regularity, and there's no reason that we should be prevented from having such a odd mix of history. The fact that we do not have such experiences, tells me that we can safely rule it out.
2) Infinite universe is eternal, but growing: In this case, there are infinite timelines that are 'finished', but such a universe is getting new arrivals of timelines (complete with past,present,future). This situation is problematical. That is, an infinite collection that is 'growing' is already infinite so how can it increase its size? This would only makes sense if it is increasing its cardinality, but how can that be the case? For example, if you have an infinite collection of integers, it can grow larger if you start adding reals to that group one at a time, but the moment you add a member to the list of infinite integers you are viewing the infinite set of integers within the set (i.e., you are viewing each member of the set), and as anyone who is familiar with infinite sets, if you start counting the infinite set as individual members, it is impossible to count to infinity. The infinite set only is plausible if you never view the set within the context of each member, but only in context of the sets' properties (e.g., infinite number of all integers). If you start adding real numbers within the set of all integers, the presumably you could also count the set of all integers, which you cannot do. Hence, it seems something unphysical about increasing the cardinality of an infinite set. It either possesses a certain level of cardinality or it doesn't.
Well, that's debatable I suppose, but even if you say 'yes' to this idea, you still cannot explain why our universe has an ordered set of events (i.e., in temporal sequence). Even if we just 'popped' into existence (e.g., a real number is just added to an infinite collection of integers), we still have no reason to expect for us to be in such a special position as to being an ordered set. After all, unordered sets have a higher cardinality over ordered sets, so there are many more possible unordered sets than ordered sets. If you acknowledge the 'Copernican principle' (a principle that we are not in a special location of worlds), then you would have to discredit this notion.
3) Infinite universe is locked in the present: In this case, the reason that we are in an ordered universe, versus an expectant non-ordered universe is because the nature of the infinite universe is that there is no eternal past or eternal future that exists. Rather, there is only the present, and the present evolves for an infinite collection of universes. All of these infinite collection of universes are infinitely old, and will exist in the nfinite future as well.
The problem here is not a violation of the Copernican principle, rather it is treating an infinite past as something that actually 'happened'. That is, since there is no first moment, if you travel back to the past, you can never reach the past with a finite speed of time travel, hence there is no infinite past to travel to. Hence, you have to consider the infinite past as an infinite set of events that occurred, and the present and future events are constantly being added to the infinite set of events. The problem, though, is that the infinite set of events do not exist. That is, if all there is is the present, then the past technically doesn't exist, so there is no members in this infinite set of past moments (it is a null set). Okay, one might argue that an infinite set of past moments refers to what once 'did' exist, but if this is so, then it is not a physical set. It points to things that specifically do not exist. Even if you say they are moments that once did exist, so therefore they were once physical, this is not a good argument. An infinite time ago is not physical unless there is something in the infinite set that still exists (e.g., an eternal universe that is 'out there'). In this case, how can you say that an infinite moment ago is a physical thing when, in fact, you are saying the universe is only in the present and you can never reverse time to arrive at that infinite moment ago? What about it is real? It is only a concept in principle, nothing about it is more than a concept. This possibility contradicts the basis of materialism which is that material stuff 'exists' or had once 'existed' which led up to the present ensemble. In this case, there is nothing material about a moment that existed an infinite moment ago.
Well, I'm sure there are other configurations that can be suggested, but I believe the above covers the major arguments against an infinite universe (i.e., as a means of a materialist to explain why it is that the universe exists without a beginning).
My view is that materialism is just not well thought out. It relies on the confusion created by not defining the nature of physical law, infinity, and the nature of time. When you consider all three in context, it seems to me that materialism falls apart.