How the Web Filter Handles Unknown URLs
All Rocket appliances ship with a base categorization database of the most commonly used sites. When a site is unknown by a customer’s on-premise Rocket, a request is sent to our Dynamic Database (DDB) servers for categorization. This process works very similar to a DNS lookup. Our master databases will respond to the on-premise Rocket with the proper categorization for the site and a Time to Live (TTL). (This is designed for maximum efficiency and speed.)
The TTL tells the on-premise Rocket how long to store this looked up entry in their local database. The TTL varies by category but most are set for two weeks. This allows your Rocket to maintain only the categorizations in your database that your end users are visiting. If a site were to change categories before the TTL expires, our emergency update process will alert the Rocket appliance to recheck for categorization prior to the TTL expiration. This is useful if a website were to be compromised by malware. When this is discovered, we can signal all the Rocket appliances to update their categorization within minutes to ensure that customers are protected.
Since the internet is very dynamic and sites are added every day, there are cases where a customer Rocket will request a site that is not in our master database. If this is the case, it will be returned as unknown with a short TTL. Our DDB sends this information to our categorization engine with top priority. Most sites are fully categorized within a few hours. Then the site gets categorized and when the TTL expires, the new categorization will be pushed to your on-premise Rocket.