I don't think glue makes paddle faster. I would put just enough glue so that it firmly sticks, but not extra glue. Most people I see put too much glue for my liking, and it starts to impact the feeling of the rubber.
So, I don't know specifically what you're doing wrong. But why don't you try...