Whenever you have a high denom glass, the payout is listed in dollar amounts. The glass says $10 for 3 blanks, which would be 2 credits on a $5.00 machine. It shouldn't pay $10.00 in nickels. Keep playing and record the pays for some other wins. So far, the machine is paying correctly for a $5.00 denom machine.
If you get 2 credits for $10 win, 5 credits for $25 win, 20 credits for a $100 win, then all is good. You will either need to find the glass (that states wins in credits, not $$ amounts), or just go coinless and stick with the $5.00 denom, and only load credits with bills.