There is a pool complex that is part of the SV Wacker sports club that is open to the public for a daily fee. You do not have to be a sports club member to swim there. Purchasing 10 visits at once (Zehnkarten) or season passes will save money if you intend to go more often. The indoor complex has an Olympic-size pool and a separate diving pool; there are always lanes roped off for lap swimming. During summer months, the indoor pool is closed and the outdoor pool located right behind it is open. It has a separate play pool with a huge water slide. The grounds are grassy with trees and it is a nice spot on a warm summer day. But beware of crowds! There are also very similar complexes in Burgkirchen and Altötting.
The lake located just below the castle, the Wöhrsee, is open for swimming during summer months. You can buy a season family pass for a very reasonable fee or pay a daily entrance fee. The facilities have a playground, diving boards, lots of grass for picnics, and a nice Biergarten as well. Paddle boats are available for rent. During the winter you can ice skate on the lake if it gets cold enough to freeze sufficiently!