Teach him "leave it" or "drop" command. Follow the yips & he will learn. U must remember that he is just a small pup & pups are by nature playfull & full of curiosity...thats why they want to examine every new things they see...& of course teething adds to his nature....so dont punish ur pup or this behaviour, rather teach him some basic commands & give him plenty of toys to play with....
To teach "drop" command u need to proceed gradually.
1st when he tries to pick up something, say a loud "Nooooooooooooooooooooo" & quickly grab that object before he can reach it.....if he still tries to get it again say a "Nooooooooo" & keep that object out of his reach...give him a lovely treat when he settles down & no more tries to get it.
2ndly if he does manage to grab something then show him a treat (do not give it at this point) & say "drop it" repeatedly...point is to get his attention & make the treat more attractive than the object he is holding....give him the treat ONLY AFTER he has dropped the object.
3rdly if he still doesnt drop, hold the treat in closed palm & place right in front of his nose so that he can smell the treat but cant take it......again keep on saying "drop it"....let him have the treat if he let the object go....
U will have to be persistant...gradually he will learn the "drop it" command....then u can command him to drop if he grabs something without giving him a treat....but first teach him the command.
Best of luck.