I don't find fault with the mechanism so much as the implementation. The reason your advanced options only appear once is you remove the options_frame from the root window. It doesn't matter if you add another label to options_frame because the options_frame is no longer visible! This should have been really obvious and the only reason I can think of for you not seeing this is because you are trying to design using a keyboard. Bad! Bad!
Step away from the keyboard and think about what you want to do. Not what you think Python or Tk is going to allow you to do. Write what you want down on a sheet of paper so you can refer back to it. Think of it as a requirement for your application.
Example
I want controls for setting advanced plotting options. The options are.... I want the user to be able to have access to the options, but I don't want them to take up valuable space in the plotting window.
There are multiple ways I could implement this requirement. I could have a section of the panel that changes to display or hide the advanced options. Maybe with a little button that looks something like this:
v Advanced Options ------------------------------------
The user could click on the arrow (v) button to display or hide the advanced options. To do this I could place the option controls in a frame that I add/remove from the layout, OR I could collapse/expand the size of the frame holding the controls. I don't know a lot about Tk, but in Qt, GTK and wxPython this is a really easy thing to do. All you have to do is set the visibility of the frame holding the advanced options and the window layout manager shifts everything to make it all fit.
Another way to solve the problem is do something like the ribbon on Microsoft Word where you have several different views that you can tab between.
Another way to solve the problem is place the advanced graphing options in a menu or in a subpanel that is drawn when you make a menu selection or type a keyboard shortcut.
I'm sure you can think of other ways your advanced plotting options idea could be implemented. Write them all down and try the ideas out using a storyboard. Which way works best?
Now you have Requirements and Design. You are ready to code. If you don't have at least rudimentary requirements and a rough design you usually end up with a poor solution. It may be that once you start coding you will come up with ideas how your design can be improved, or you may learn there were some problems with your initial requirements. It is ok to revisit both requirements and design and make changes. You may also find it is really hard to implement some designs, but if you have to weigh ease of implementation against good design, try to always pick the good design.
Coding is pretty easy. At first it will be horrible looking code and 10x to long, but it will get better with experience. The hardest part of writing software is gathering requirements and designing the solution.