Content Database

The Content Database is the core of the Web Filter module. The Web Filter module uses a database consisting of millions of domains, IPs, and URLs combined, sorted into categories appropriate for education.

About our Database

Our database is education specific. This means that in almost all cases any website that is necessary for the classroom is already categorized properly for education so the need to allow unknown or unverified websites for day-to-day classwork is not necessary.

And our database is dynamic. We use a variety of methods for categorization including:

  • Internet crawlers that are constantly scanning new and existing websites for malware and inappropriate content
  • Human review team that not only verifies change requests from users but also validates the crawlers
  • And our education community worldwide. Feedback from classrooms all over the world is sent back to our database team and processed. These changes are shared with all customers, so we are working together to create the most complete and accurate database for schools

With our dynamic database, we can update all of the Rocket Web Filters worldwide within minutes if necessary. This takes the burden of real-time scanning off of your individual Rocket and moves it out to the cloud where our resources and shared customer base can all help each other.

Categorized Sites

Use the Categorized Sites page to search the Content Database for a URL. The search results will show how the URL is currently categorized.

From the search results, you can add a domain to your Local-Allow and Local-Block lists, create redirection rules for built-in or local categories, or add a domain to one of the default categories.

To update the category for a domain:

  • 1. Search for the domain, URL, or IP address on the Categorized Sites page.

searchdatabase

  • 2. In the search results, click to edit the domain you want to recategorize.

recategorized

  • 3. Click to open the Category dropdown list, and select a local category or an existing default category, then click Save.

Note:

These exceptions are stored locally on your Rocket, and are not sent to Lightspeed Systems. To request a review of a site by Lightspeed staff, please submit the request via the Database form.

Tip:

Open the Rule Sets page and click Default to see the complete list of content categories and the default Allow and Block actions.

How Web Filter Handles Unknown URLs

All Rocket appliances ship with a base categorization database of the most commonly used sites. When a site is unknown by a customer’s on-premise Rocket, a request is sent to our Dynamic Database (DDB) servers for categorization. This process works very similar to a DNS lookup. Our master databases will respond to the on-premise Rocket with the proper categorization for the site and a Time To Live (TTL). (This is designed for maximum efficiency and speed.)

The TTL tells the on-premise Rocket how long to store this looked up entry in their local database. The TTL varies by category but most are set for two weeks. This allows your Rocket to only maintain the categorizations in your database that your end users are visiting. If a site were to change categories before the TTL expires, our emergency update process will alert the Rocket appliance to recheck for categorization prior to the TTL expiration. This is useful if a website were to be compromised by malware. When this is discovered, we can signal all the Rocket appliances to update their categorization within minutes to ensure that customers are protected.

Since the internet is very dynamic and sites are added every day, there are cases where a customer Rocket will request a site that is not in our master database. If this is the case it will be returned as unknown with a short TTL. Our DDB sends this information to our categorization engine with top priority. Most sites are fully categorized within a few hours. Then the site gets categorized and when the TTL expires, the new categorization will be pushed to your on-premise Rocket.

Categories

See details on our categories here.

Many customers can use Lightspeed Systems default blocked and allowed categories without modification. However, you can override the default categories by adding Domains, IPs, or URLs to the Local-allow and Local-block categories. You can also create your own local categories containing Domains, IPS, or URLs that should be handled differently.

Local Categories

The simplest way to manage local categories is to use the Reviews report to categorize the sites your users are attempting to access. You can also edit the local categories by clicking the category name, and then adding sites manually.

  • Local-allow – The local-allow category allows you to unblock sites that would ordinarily be blocked by the Lightspeed Systems database
  • Local-block – The local-block allows you to block sites that are normally allowed by the Lightspeed Systems database

Add Local Category

Click the Add Local Category button to add a new Local Category.

  • Name – Enter a meaningful name for your local category in this field
  • Description – Briefly describe the category here, for example “search engines”
  • Redirect URL – When a user requests a site within a blocked category, the URL you enter here will be displayed instead of the requested page. Enter the destination URL for sites in this category. For example, you can redirect requests for Google.com, Yahoo.com, and other search engines to a child-safe site such as RefSeek. Leave this field blank if you do not want Lightspeed Systems Web Filter to redirect URLs in this category
  • Set default behavior to block – Select (check) this check box to block all sites in this list. Instead of opening the requested page, the Web Filter will display the Access Page

Categories with Local Changes

You have the option to use one of the standard Lightspeed categories instead of adding a domain to a local category. There are a couple of reasons to use this option:

  • Reports — recategorized domains will be associated with meaningful category names, without requiring you to duplicate the list of default filter categories.
  • Block Page — users will see a meaningful category name when accessing a blocked page, again without requiring you to duplicate the list of default filter categories.

Domains are added to local and standard categories from the Categorized Sites page.

Redirected Categories

You can redirect any of the predefined blocked content categories to a URL of your choice. For example, you can redirect all sites in the category kids_and_teens.chat to a single chat site such as Kidzworld. Note that local categories with redirect URLs will also appear in this list.

Add Redirected Category

Click the Add Redirected Category button to add a redirected category.

add-redirected-category

  • Category – Select a content category from the dropdown list.
  • Redirect URL – Enter the destination URL for sites in this category.

Locked Categories

Locked categories are content categories that should always be blocked, even to users with rule sets or advanced rule sets that would otherwise allow access.

Add Locked Category

Click the Add Locked Category button to add a new Locked Category.

add-locked-category

  • Category – Select a content category from the dropdown list.

Categorized sites

The Categorized Sites page is where you search the Lightspeed Systems Content Database for web sites, IP addresses, and URLs, to determine their category and blocking status.
categorized-sites

Note:

Use the wildcard character ( * ) to find subdomains. For example, to find subdomains of “cnn.com,” search for “*.cnn.com.”

Adding a site to a local category

    • 1. Click the site name in the list to add the site to a local  category.

edit-categorized-site

  • 2. Enter the following information:
    • Category – From the dropdown list select the local  category.
    • Apply to selected URLs – Check (select) subdomains you want to add to the local category.
    • Comment – Optional. Enter a comment for this local category.
  • 3. Click Save to save your changes.

Tip:

Search for URLs at http://archive.lightspeedsystems.com to see detailed information about a web site’s classification.