MY PONY IS LOSING WEIGHT!! HELP!!?

My pony is losing weight. He gets feed one a day 2 dippers of mixed chaff, one dipper of pony mix and a bail of hay every 3 days. He only gets ridden once a week so I don't think this is the reason for it.He is also in a paddock with a shetland. Should I be feeding him more? Any suggestions would be appreciated.