My general rule is that if I needed to take a tool from my house to the boat for mechanical or electrical maintenance, it stays there, or a similar and suitable replacement is obtained. Carpentry tools, no.
That's pretty much how my boat tool box got populated as well.
Another thing - While those tool sets in the blow molded plastic boxes look pretty and it's easy to tell if something is missing, they take up a lot of room and there will always be something you need to add (like a universal joint or an extension) and there won't be a place for it.
I just keep similar tools (like sockets or wrenches) in zip lock bags in my tool box. If I need a wrench I just open the appropriate bag and empty it on a flat surface. The zip lock bags also allow me to spray corrosion preventative into them to keep the tools rust free.