go by the board
To go away or disappear forever, be forgotten or not used.
Tom had several chances to go to college, but he let them go by the board.
Grandfather said he was too old to go to the beach. "Those days have passed by the board," he said.
Compare: DOWN THE DRAIN.