You are right. I used the wrong notion of completeness when I talked about Goedel's Theorem. The correct notion is, as you say, the idea of the provability of all theorems, or true statements, within the system of axioms.
But this notion is not the opposite of the notion I used. It's not "like saying that 5/4 can be found in the set of integers, but cannot be derived in that set." 5/4 cannot be found in the set of integers, and it cannot be defined in a consistent way in the set of integers. Derivation does not apply since you can't define it.
The notion of completeness I had in my mind when I came up with that example is what might be called "Algebraic Completeness". (I'm not sure about that because my recollection is very dim at this point.) Algebraic Completeness means that all algebraic expressions, i.e. combinations of sums, differences, products, and quotients, excluding divisions by zero, can be found in the set of numbers.
And in this sense you are right that no system of numbers is complete except for the Complex Numbers which is Algebraically Complete. In fact, it is this type of incompleteness that drives the successive definitions of the integers, the rationals, the reals, and the complex.
So let me back up and change my answer to your question
"Is the finite system of arithematic I am using therefore complete?"
In the Algebraic sense, it is incomplete. For example, the infinite set of integers is incomplete because it does not contain the answers to all division questions. For example, 5 divided by 4 does not appear in the infinite set of integers. And including infinity doesn't make it Algebraically complete.
In the Foundational sense, it is incomplete because it is finite. For example, if the biggest number in your system was 100, then you couldn't prove the theorem 50 * 4 = 5 * 40 because neither of those products can be consistently defined in your system. In this case, if we include infinity so that all products are defined, then Goedel's theorem kicks in and hangs the jury. The answer to your question becomes, "We just don't know if your system is complete. But we can be sure that if your system is consistent, then it must be incomplete. However, we can't know if it is consistent."
Sorry for the confusion -- both the previous confusion and any new confusion I have caused.