Let's focus on the issue of randomness. I think I found a good way to explain where I'm coming from. I'll give you a simple example, but you should be able to understand how the basic principle applies to problems of any size and complexity (if you can't see how then I'll give up)
I have this little 'system' which has one input and one output. What goes in or comes out is irrelevant to the argument so I'll just use numbers to represent them. My job is to figure out if the system works in a logical way or if it's random. I'll start by assuming the system is logical and search for a logical way to describe its behavior; if I fail to do that I'll conclude the system is random. I will propose a function V = f(a) which describes the output of the system (V) for every input 'a'. Here's what my first measurements tell me:
Measurement #1: a = 1; f(a) = 0
Measurement #2: a = 1; f(a) = 1
That would indicate the system is random, but as a scientist I'm moved by the conviction that the universe is ruled by a logic which comes from some bigger picture, so I'll try harder. I'll assume there's another variable, 'x', whose value is unknown to me but can be discovered by inference. So I'll rewrite the system's function as f(a, x):
V = a - x
Using my new function I can infer that x = 1 in meas. #1 and x = 0 in meas. #2. Not only have I "proved" that the system's behavior is logical, I discovered a new entity 'x' whose value can be indirectly known and subject to rational study.
A few points are important here:
i) No matter how complex your problem is, no matter how many real variables are involved, you can always propose the existence of imaginary entities whose values can only be known indirectly
ii) Failure to do (i) is what characterizes a system as random
iii) The power of our imagination is the only thing which stands in our way of doing (i) for any system whatsoever
iv) It is a serious mistake to think the imaginary entities we propose to make sense of random data are anything other than products of our imagination
v) It's extremely easy to lose sight of (iv)
vi) Scientists often resort to that strategy; no wonder they can explain everything!
The most important point concerning our debate is, as long as you allow the inclusion of imaginary entities in your explanation you will observe as much order and logic as your imagination allows you; if you don't then you'll see very little order in the world, if at all.
Don't rush to reply. If you think this is trivial or easy you are missing the point.