One idea (not so thought out):

1. Vesting rewards are made somehow modestly superlinear (as curation rewards are now)
2. If an account upvotes, it forfeits its share of vesting rewards (what you call inflation) for some period of time. This becomes a market (i.e. you don't need a fixed limit) because the rate of return would increase the less stake "opts out" in this manner and larger accounts are more incentivized to not upvote and instead take the vesting rewards since these rewards are superlinear. Small accounts would not care and still upvote because their vesting rewards would be tiny or zero (currently curation rewards are literally zero for the smallest accounts because of rounding).
3. Downvoting does not affect the vesting rewards, and might or might not have its own downvoting incentive  (designing a workable one seems hard)