When you empower people, you should (occasionally) be able to sit back and enjoy the peace of your workspace, and the contents of your cookie jar. Other people are doing the work. Not you. That’s great. But try and keep it that way.
When an empowered team walks into your office and asks you to decide on an issue, find a way to have them solve the problem themselves. I once heard of a manager who tossed a coin for every decision his team asked him to make. This quickly motivated the team to make their own decisions, as they balked at being ruled by a penny. I sometimes use a mirror as a metaphor. As a manager you can act as a mirror to the team. You can help them with their own thinking processes. If they look at you for guidance, you hold up the mirror and help them to find guidance in themselves.
When a team member walks into your office and asks you to do something for which you had delegated responsibility to someone else, make it clear to her that this is now the job of that other person. Tell her that trust is meant to be a transitive relation. If employee A trusts manager M to make a decision, and manager M trusts empowered employee B to make such decisions, then by agreement employee A should also trust employee B. Never betray your trust in employee B by making decisions for him, and certainly not behind his back!
And finally, when nobody walks into your office, then don’t criticize them for not consulting you about their decisions, even if they turn out to be terrible. If you want to be consulted in advance you must clearly communicate those expectations. Of course, if you have communicated such a requirement, and the team hasn’t lived up to it, then they have broken the trust, and will need to repair it. A contribution to the cookie jar will do nicely, I think.