Well, if the dimensions aren't at least close, it doesn't matter which end you do first, you are going to end up doing major rework. I think that most of us would prefer to redo the holes at the primary end of the scope than cut another huge hole in the top of the scope and 4 more for the secondary holder.
I do agree that you really do have to know approximately where all of the parts need to go before you start cutting.