Then tell him and get it over with. I know that's easier said than done, but if you wanna clear your conscience then you might as well do it now.
Or you could wait a whole bunch o' years down the line and just drop it in the middle of a discussion...just say no biggie.