Write $ x $ in the form $ 10a + b $, where $ b $ is the units digit of $ x $ and $ a $ is the "remaining" part of $ x $. (For example, if $ x = 5718 $, then $ a = 571 $ and $ b = 8 $.) Then \[x^2 = 100a^2 + 20ab + b^2.\]Since the older brother was the last brother to receive a whole payment of $ \$10$ , the tens digit of $x^2$ must be odd. The tens digit of $100a^2$ is 0, and the tens digit of $20ab$ is even, so for the tens digit of \[x^2 = 100a^2 + 20ab + b^2\]to be odd, the tens digit of $b^2$ must be odd. We know that $b$ is a digit from 0 to 9. Checking these digits, we find that the tens digit of $b^2$ is odd only for $b = 4$ and $b = 6$ . We also see that the units digit of $x^2$ is the same as the units digit of $b^2$ . For both $b = 4$ and $b = 6$ , the units digit of $b^2$ is 6. Therefore, the last payment to the younger brother was $\boxed{6}$ dollars.