There's a Pro Among Us
First off, it's super common for former pro athletes to pivot into finance, so it's not as strange (or unique) as you may think. I'm assuming your boss knows this already (as it was likely on your resume?) so the question is really about how to handle this with co-workers and clients? The answer is simple - leverage the fuck out of it. It's not bragging - it's part of who you are, part of your history, part of your style and personality. Fucking OWN it dude! It's cool, so don't make it weird.
For clients, as you're getting to know them in the small talk phase, it's super easy to add this in as part of your historical run down that we all share with each other when we're first getting to know each other. And for existing clients, when you find yourself in one of those tangential conversations about personal lives, throw it out there. I know it sounds a little strange, but this past experience (and you sharing it) will make you more trustworthy, which is critical in your industry.
For your co-workers, if you haven't told them before now, I'd wait until you're at the next happy hour, and then let it drop. Don't make it a thing that you haven't said anything before now. And if someone calls you out on your lack of disclosure, simply say you didn't want to brag. This little deet makes you interesting, gives you plenty of material to have more engaging conversations with co-workers and clients, and should help you perform better at work. So use it.
One note of caution - there's this sort of judge-ey "It's so cool that he was a pro, too bad he totally sucked" double edged sword thing that can come with your situation (people WILL google your stats). So when in doubt, downplay your athletic performance. You being graciously modest about your pro career will make it harder for the haters (who are just jealous) to hate.