While I like the conveniencde it’s a tad annoying that Flatpak never re-uses dependencies but I found how to clean them up.
Just run flatpak uninstall --unused after using flatpak remove NAME.
Turns out that ‘flatpak uninstall NAME` also gets rid of dependencies.