Millions of tech and finance workers around the world have enjoyed a year-long break from the office because of pandemic lockdowns but one key bank CEO say that needs to end before it becomes the "new normal."

Goldman Sachs CEO David Solomon said working-from-home, or anywhere but the office, needs to stop at his company, especially for new employees who would not become immersed in company culture if joining remotely.

"I do think for a business like ours, which is an innovative, collaborative apprenticeship culture, this is not ideal for us. And it's not a new normal. It's an aberration that we're going to correct as soon as possible," he said at a business conference Wednesday.

On the other hand, big tech firms like Twitter, Facebook and Microsoft have made plans for some staff to work from home permanently if they want, but maybe at lower pay than they'd get in the expensive Silicon Valley hub.

Goldman Sachs has seen "less than 10% of our people" in the office in 2020, and Solomon said new employees who traditionally join in the summer after graduating from university, about 3,000 new recruits, need "direct mentorship."

"I am very focused on the fact that I don't want another class of young people arriving at Goldman Sachs in the summer remotely," he said.

For banks like Goldman Sachs, advances in remote working are welcome to a point, but "as we get out of the pandemic the overall operating mode of the way a business like ours operates will be vastly different," he said.

Fellow banking chief executive Jamie Dimon of JP Morgan Chase has echoed the sentiment that remote working hurts productivity.

But at least two other banking giants, Lloyds and HSBC, plan to cut rented office space to save money as more and more of employees work from home.