When I started my headphone journey, I couldn’t tell why everybody insists on using amplifiers as my ears couldn’t pick up the difference. After years of listening, I am now convinced that all amplifiers, good or bad, alter the signal in some ways (read: it can improve, or degrade the sound).
The new rule of thumb on amplifiers is that if you are using a big circumaural headphone (the one that covers your ears), regardless of the impedance or sensitivity rating, an amplifier in most cases would help. This is extremely true if you are talking about driving the headphones from a portable player, a mobile phone, or a laptop. Even if you can get the volume to go loud enough, a separate amplifier will do a better job in delivering current which will improve bass punch, among other things.