Couldn't an entire group of people like yourself be more like a loosely knit honor bound group that keeps to themselves and only comes to another's aid when it matters? An Alliance of sorts, a Pact, an Agreement?
I'd offer my loyalty and my skills for protection and eventual attacks. People naturally tend to come to me with their problems and make me the center of their group, but this can be a disadvantage if I am seen as a threat by the existing leader too soon. Sometimes it's instant mutual rivarly rising out of seemingly nowhere between me and other men.