That's fine
Normal USB wall warts / sockets are not actually "chargers" anyway, they are just 5V power supply units.
Anything connected will only draw the current it needs.
The one i use on my desk is 6A rated.
The "Charger" name is a misnomer, a leftover from earlier devices, where the battery regulation and control electronics were in the external unit.
Any device that charges from a USB socket has the actual charge control electronics in the device itself.
USB-C power units can have the facility to select different output voltages via handshaking, but the charge regulator is still in the device being charged.