When you test the solar panel and measure 7V, what is the load on the solar panel?
I'm guessing that the load is a high-impedance voltmeter, which says nothing about what the solar panel will deliver into a LOAD. To be useful as a solar charger, the panel would have to deliver ~ 50mA at ~6V, which by Ohms Law is R = E/I = 6/0.05 = 120 Ohms. Connect a 120 Ohm resistor across the panel, and go measure the output voltage in sunlight. I'm guessing you will see less than 2V, which means the panel is not suitable for this task. Rule of thumb: the open circuit voltage of the panel must be about 50% higher than the voltage you want it to deliver under load!