You can also easily get to Cave from Galliot Cut either from the ocean or bank, which is how I go. I find Emerald Bay to always have surge in...
Cave Cay has a well protected, nice marina, but nothing else there. Island is nice. Emerald Bay sucks due to the constant surge.