You can kayak in quite a few places in the area, such as to the sea caves off of Meyers Beach, or near the shoreline on Madeline Island, or off Bayfield.
Take a guided kayaking trip from Living Adventure (close to the cabin) or Trek and Trail near the ferry in Bayfield.