I do not speak as a Joomla guru only a learner, but it seems that the nature of Joomla makes us all learners because it is in a constant state of flux. My opinion about your question is that if you have a joomla site up and running to your satisfaction you should be most concerned about security releases, because php sites are notoriously easy to hack. This problem is ameliorated by installing good backup systems.It is good to keep up to date on the forums of your most important extensions such as CB VM and other big extensions as they can have their own holes.
Personally I am very leery about updating or adding anything to a site that is working properly. Joomla is fascinating. I feel like a caveman trying to build a watch.