It's his natural protective instinct coming to the fore.
You and your family are part of his pack/his family, he thinks he needs to protect you from these strangers and hence attacks them when you are around.
You will have to instruct him to not do so by being firm(not aggressive/screaming )etc say no to him when you can see he will attack. Dogs always give signs before they are about to attack, their hackles rise, show their teeth in a sneer, growl,tail go's up etc
It's important for you to tell him that all is well and this person means no harm
best of luck