Actually it’s a subtraction relation between the units, rather than a division relation.

Charlie has 5 times as many stamps as Ryan. I presume when you mentioned 5 times more, you are referring to the statement I just wrote previously. “5 times more” can get misleading, as I interpret “5 times as many times” as “4 times more”, or “400% more”, since the excess is 4 units more.

So, we are looking at 4 units at our final expression.

Charlie has 1072 stamps more than Ryan.