Is he the bandit chief in a fort somewhere near orphan rock/helgen? I had the same problem..he was using a battle axe of damage stamina and I couldn't run away from him or use any power attacks. He killed Lydia with one hit.
I eventually got him by reloading a previous save file, sneaking up behind up naked (armor make noise) and got a sneak bonus critical hit with my elven greatsword.
I think I know which part you are talking about. There was a bandit marauder which just destroyed me if I try to to head to head with him.
Sneaking + archery is the way to go. I think the game is generally a lot easier if you go sneaking plus archery route. You are usually able to pop in 4-5 shots with sneak attack bonuses before he gets close. That's how I killed the guy.