The Drunken Lizard Pub

I'm in trouble! I have a problem in the Goblin Citadel. Im at first level with only three keys. I went through all the corridors, but could not get into the main hall - it was closed.
Then I realized that I need another key, I found it in the Vela and back again, but now still I can not get over the doors to the main hall of 1st level. I have recovered all four keys, but the journal note about keys does not disappear. What do I need to do?

Thank you! Strange that I did not see this switch when I come into this room the first time.