The standard system notifications about available extension upgrades are only displayed for extensions that are available to install via the UI. However there are many extensions at civicrm.org/extensions that you can install that are not available to install via the UI, and you likely want to know when upgrades are available for those too.