Is it possible to buy a yacht and go to other countries with it?

Yes you can. I've done it with my boats. Been to Mexico and Canada. You'll need all of your boats paperwork, insurance forms and doentation. Everybody aboard will need their pport and a second form of ID. There may be additional requirements for other countries so be sure to check into that Port of Call.