How do you use a Gameboy Advance charger to charge the Gameboy Advance?

I have both of these, but it says tells me that I should only use an approved adaptor, that I should only use with rechargeable batteries, and that I should not dismantle. Apparently you cannot just plug it into an outlet and charge it...