Prince Harry and Meghan Markle have decided to quit royal life to focus more on their family and charities. But, just like most decisions, theirs also come with a price.

Although Queen Elizabeth showed support to Prince Harry and Meghan Markle’s decision, she still has the last say regarding the things that the couple wanted before they go. Her Majesty allowed the couple to step back, but she gave them a couple of deals and the Duke of Sussex appeared to be really upset with one of them.

After announcing their decision, Prince Harry and Meghan Markle’s website stated that they would “carve out a progressive new role” and “adjust their working model.” But after their discussion with Queen Elizabeth, Prince Charles, and Prince William, it was decided that the Duke and Duchess of Sussex will completely step back from royal duties as of spring of 2020.

And even though the couple will still be known as the Duke and Duchess of Sussex, they can “no longer formally represent” Queen Elizabeth in their new lives. Also, Prince Harry and Meghan Markle will not only lose their HRH titles, but Prince William’s brother will lose his military titles as well, which deeply upset him.

“I think the loss of his military links, especially that of Captain-General of the Royal Marines, will have deeply upset Harry, who continued the royal family’s tradition of military service by his two tours of duty in Afghanistan,” royal commentator Richard Fitzwilliams said. “Since he can no longer wear uniform, this is something he will feel deeply.”

He then pointed out that Prince Harry showed sadness during his Sentebale speech. He may have accepted Queen Elizabeth’s ultimatum, but it was a hard pill for him to swallow.

“Harry’s speech, in which he made clear his unhappiness at the Queen’s decision about their future was another sign of them doing things their way,” he said. “Although the agreement will be reviewed in a year’s time the Sussexes have, in reality, been exiled.”

The royal commentator added that although it appears that Prince Harry is ready to move on from his senior royal obligations to being a non-working royal, he may miss his old life in the future. The Duke of Sussex is royal by birth, and being a prince had always been a part of him.

“Independence has its price! They may have a meteoric rise with top Hollywood celebrities to back them, but they will always be followed by the press,” he said. “Meghan may be in her element, Harry may prove totally adaptable but he also could miss what he has given up.”