Would you mind my talking with you if i speak English badly

From strictly a translation standpoint, it would be better if you changed your sentence to --> Would you mind chatting with me even though my English is poor?
Your version seems a bit awkward and wordy. On the other hand, if you say this sentence or any variation, it means that you are not confident in your language ability. Even if your English is extremely poor, you should still try hard to communicate with others to the best of your ability.If you are approaching a foreigner, you should just say, "Hi, do you have a minute? I would like to discuss (such and such.)."