An SSD will definitely improve your overall experience.
I have a late 2012 Mac Mini with an internal HDD.
Even running a separate install from an external SSD (via Thunderbolt) is a lot faster.
That said, the RAM being limited to 4GB is something you will have to keep in mind.
(But should be...