Report: Clairview Internet Sheriff
An Independent Review

Last Updated: 25 June 1999

Introduction

In March 1999, the Australian Government announced intent to enact Internet censorship legislation which would require Internet Service Providers (ISPs) to block adults' access to particular content on the Internet. Since that time, and the tabling of the Bill in Parliament, much of the public discussion about the merits, or otherwise, of the Bill has concerned the availability, if any, of blocking technologies which could be used by ISPs to comply with the legislation.

Several Senators have commented that they believe blocking technologies have improved considerably since the preparation of the CSIRO Report in 1998. The Internet Sheriff Technology by Clairview Internet Pty Ltd has been referred to in this context on a number of occasions. However, little information about the product is available in the public arena.

In early May 1999, Internet access accounts were purchased with Clairview's Brisbane ISP, CVue Internet with a view to evaluating the technology in terms of its stated objectives.

Independent tests, by a team of three people, have revealed that while Internet Sheriff Technology does block pornographic material (although not all), it will also block a wide range of non-pornographic material, for example:

Entire major sites, including home pages, eg:

Ozemail: www.ozemail.com.au
Powerup: www.powerup.com.au
Iinet: www.iinet.net.au
Geocities: www.geocities.com Geosites shutdown, read more
AOL members' site: members.aol.com
and many others.

Individual examples of sites found to be blocked include:

National Party of Australia - Federal Party Home Page
Access Economics - Consultants
Mick's Whips: Australian handcrafted kangaroo and crocodile leather goods
Agfest - Tasmania's Rural Trade Fair
Christian Bookselling Association Australia Inc
St Luke's Lutheran Church Nambour, Queensland
Optometrists Association Australia.
Marina Mirage Port Douglas Qld
Glen Dhu Country Retreat, Tasmania
Crossroads Australia & AD2000 Disability Network
Wangaratta Primary School
Orchid Society of NSW
Coalition for Positive Sexuality - Safe Sex
Womens Net - Andrea Dworkin
and many others.

Contents

Background

In March 1999, the Australian Government announced intent to enact Internet censorship legislation requiring Internet Service Providers (ISPs) to block adults' access to content on the Internet that is either classified X (non violent erotica) or refused classification (banned). Since that time, and the tabling of the Bill in Parliament in April 1999, considerable community and industry attention has been directed towards the availability and accuracy of blocking technologies which could be used to that effect by Internet Service Providers.

Senator Alston, Minister for Communications, Information Technology and The Arts, has indicated on several occasions that he believes that such technology is now available, or will be in the near future. For example, on 24 March 1999 on Radio National, Senator Alston said:

"...I've had a couple of new technology companies comes to see me in recent days and it really opens your eyes. One of them are using what is called eye filter. It's based on an American technology where they have physically tracked eight and a half million web sites and are able to then check them and determine whether they would offend against a classification regime. And they can then guarantee you clean sites.

Another doesn't even need to inspect the sites in the first instance. It can monitor the data stream by what's called a guessing engine and that will then match up with the known characteristics of pornographic sites; give you say a 90% probability that that material is emanating from one of those sites. You can then physically check it and put it on your black list. And, if it's coming from an international source, then there are backbone service providers with international gateways who could be required to use their routers to check the IP addresses."
[Source: "Australia talks back with Sandy McCutcheon, Radio National, 24 March 1999" available on the Department of Communications, Information Technologies and The Arts' web site]

The first mentioned product, referred to as "eye filter" in the transcript, is "iFilter" and is marketed by Infopro in Brisbane. iFilter is also known as BESS, a product of the USA company, N2H2. This product has been available for several years and aims to provide access suitable for primary school children. It was recently announced that iFilter would be available to Victorian schools through the Schoolsnet service. As such, it seems unlikely that Senator Alston is suggesting that this product is suitable for mandatory use by Australian adults. Information about the types of material BESS blocks, and/or has been known to block, is available on Peacefire's site.

The second product mentioned by Senator Alston is believed to be Clairview's Internet Sheriff Technology. This product was discussed during the Senate Select Committee on Information Technologies' public hearings in late April/early May, by Mr Gareth Grainger, Deputy Chairman of the The Australian Broadcasting Authority (ABA) as well as representatives of Clairview Internet Pty Ltd. During one of the public hearings, Clairview representatives demonstrated the product by dialling into their "Seaview" ISP in Brisbane. It is assumed that "Seaview" is an incorrect spelling of CVue in the Senate Hansard transcript.

About Internet Sheriff - from Clairview

The Aim of the Product

Marketing information, on the CVue Internet start-up CD ROM pack, states:

"A clean, fresh Vue of the Net. Stop pornography poisoning your home or business!
...
CVue is a product that is committed to providing an Internet experience that can be enjoyed by everyone, at any age. Unlike other unsatisfactory screening options which have only "ON/OFF" porngraphy functions. CVue's aim is to prevent anything that can't be aired through an Australian public medium from being viewed on the Internet*.
...
CVue's special Internet Sheriff Technology is a software that searches out and profiles possible pornographic sites on the Internet for classification. After inspection, if we feel that the site regularly contains material, pictures etc that would not be viewed in an Australian public place, then that site is blocked on all Clairview's Servers and Networks."
...
*IMPORTANT DISCLAIMER - whilst CVue is committed to providing a pornography free internet service, CVue and Clairview Internet Pty Ltd cannot and do not warrant that the internet service will be free of images otherwise not available (either freely or by sale) in any Australian public place at all times. Please refer to our terms and conditions of use before accessing the service."

How the Product Works

According to Mr Allan Jones (Chief Executive Officer, Clairview), in testimony to the Senate Select Committee on Information Technology (Canberra 3 May 1999), the technology scans data in real time and uses various automated tests to guess whether or not a site contains pornographic material. If the software decides it does, the site is automatically added to a blacklist and access blocked.

Mr Jones stated:

"...it is like a list based service which can provide a black list but which can also dynamically add to that black list depending on the probability of it actually finding a match..."

"...if you look at the overall traffic consumption of data from a particular site, such as the text to picture ratio-how much text and how much picture is in this document-or whether certain words appear in association with the site or whether it has a rating service like PICS, all those things together form a model of what a site should look like. One could probably argue that a medical site would have words which are anatomical, but it would not have words which might be found on a pornographic site. Conversely, words by themselves may not be enough to tip a site over in terms of the model..."

"...A very good descriptor is: does this site have links to related sites? The way these porn sites work is that they have chains of sites that are related. If I have a site and I have, say, 20 links to sites of a similar category, then there is a high probability that I am actually of that category as well-if I am not a search engine. There are what are called 'link sites', which are typically like that. Another thing is words. Some of the words that appear in these sites are pretty much not found anywhere else on the Internet. Also, there is the picture-text ratio: how many pictures do they download in real time to the ratio of text? These sites are consumed in a certain fashion. They are consumed differently. If, say, I am reading a Dickens novel, do I read it differently from the way I read a Playboy? Behaviourally, I read it differently. We can take that into account in our modelling. We can put all these factors into a normal linear model and draw up a probability out of that model.We do it in real time..."

An Independent Review of the Product

How the Tests were Conducted

In early May 1999, Internet access accounts were purchased with Clairview's Brisbane ISP, CVue Internet. CVue start-up packages, including 20 hours access, are available from resellers, albeit apparently very few.

The package includes a pre-established user-id and password to access the system, i.e. customers using these accounts are anonymous to Clairview, and the reseller, unless the package is purchased with other than cash.

These customers then used the CVue service, as any cautious and reasonably Internet familiar parent would do, to check the reliability and suitability of the Internet Sheriff blocking technology for the needs of themselves and/or their children. Approximately 20 hours testing was undertaken by three people over a period of one week.

It is unknown how many other persons were using the CVue Internet service at the time.

Findings of the Investigation

Blocking of Pornographic Material

Initially, investigation focussed on the reliability of blocking of access to pornographic material. A narrowly-tailored search was conducted on a well-known search engine to obtain a list of material likely to be at the higher end of the "pornography scale". Access to many of the pages listed was blocked.

However, other pages were accessible. On returning to accessible pages during a period ranging from approximately 5 minutes to 2 days, researchers found that some had been subsequently blocked, and others had not. It appeared that pages which escaped automatic blocking were those which have an insufficient combination of factors, such as links and/or specific words and/or phrases, etc, to be automatically profiled as pornographic material. The contents of images on some not-blocked pages are similar, and the dimensions of images are sometimes considerably larger, than those on blocked pages.

These findings are consistent with advice provided by Clairview to the Senate Select Committee. For example, Mr Jones advised:

...When a site appears for the very first time on the Internet, will a person see pornography using this service? The answer is yes, definitely-because by seeing it, it exists. The site does not exist before it is first seen; then we can model the site, detect it and block it automatically in a high-speed router.
...
Some sites would take up to about three or four minutes of surfing before we could detect them. But if we have not actually worked out what the site is within about half an hour of viewing, we give up; it is not going to happen.

It was also observed that full frontal images of nude men and women were readily accessible, and remained accessible, on the sites of some museums, art galleries and professional photographers, etc. However other sites of that type, containing similar images, were blocked. This may indicate, amongst other things, that the automatic scanning technology is incapable of identifying the difference between a photograph that would be classified Unrestricted, or Category 1, or Category 2 or Refused Classification under Australian law.

The blocking mechanism was able to be by-passed using free anonymiser-type services available on the World Wide Web, that is, blocked sites could be accessed while using the CVue service.

Collateral Damage - Blocking of Non-pornographic Material

Investigation subsequently focussed on the blocking of access to innocuous material and to potentially controversial material, rather than effectiveness in blocking pornography. The objective was to ascertain whether, if this automatic scanning system could block access to all, or even most, pornography, would it also block access to vast numbers of non-pornographic pages.

The site of the USA National Organisation of Women was not blocked on first accessing it. It is well known that this site is, or has at one time or another, been blocked (inadvertently) by other products. The homepage links to the page Lesbian Rights which was also not blocked. An attempt was next made to access the page "Supreme Court Victory Protects Lesbian / Gay Rights; Activists Fight New Civil Rights Threats (October 1996 National NOW Times)". This page was blocked. Within minutes, each of the preceding pages were blocked, as well as the entire site.

A similar example was observed with the site of the USA law firm Schnader Harrison Segal & Lewis. Content on this site includes the Brief of Amici Curiae American Association of University Professors, et al. in support of the Appellees in ACLU v. Reno. This case is commonly referred to as the CDA (Communications Decency Act) case. The Brief contains a list of exhibits submitted to the Court in support of the view that the CDA was overbroad in banning the speech of content providers and would criminalise publication of information such as that contained in the exhibits, i.e.

  • Medical Information on the Internet
  • Artwork on the Internet
  • Literature on the Internet
  • Social and Political Discourse on the Internet
  • Journalism on the Internet
  • Education on the Internet

The list of exhibits links to pages, on the law firm's web site, which contain copies of pages (exhibits) from other sites on the Internet. Initially it appeared that all of the pages were accessible. However, within minutes, all pages on Schnader Harrison Segal & Lewis's site had been blocked, including their home page.

It seemed clear at this point that Internet Sheriff technology applies an exceptionally broad brush in deciding, not only what it deems pornographic, but also in blocking entire sites (domains).

This was verified following further tests, when it was found that the Internet Sheriff technology had blocked the entire sites, including home pages, of:

Ozemail: www.ozemail.com.au
Powerup: www.powerup.com.au
Iinet: www.iinet.net.au
Geocities: www.geocities.com
AOL members' site: members.aol.com

On first access the above sites were accessible. The sites became blocked at varying times between the evenings of Friday 7 May and Sunday 9 May. They remained blocked at 2.00 pm on the afternoon of Monday 10 May, 1999. By approximately 10.00 pm that evening, the sites had been unblocked.

As would be expected, obviously Clairview's staff can manually unblock sites once the problem comes to their attention. However, that entire sites were blocked by Internet Sheriff, was verified by three individuals using the system.

By approximately 10.30 pm on the evening of Monday 10 May, one of those sites was again blocked. This site remained blocked at 6.30 pm the following evening. It seems a trivial matter to trigger the technology to block sites once a basic understanding of the automated blocking process is gained simply by use of the system.

It is noted that Clairview's Chief Executive Officer, Mr Allan Jones, advised the Senate Committee that:

"...You can make a mistake by letting a site through that should not be let through. You can make a mistake by blocking a site that should not be blocked. The question is: how aggressive do you want the model to be? Can we make mistakes blocking things artificially? You bet we can. We can dial it right up and block most of the Internet out..."

It is not known whether the system was dialled right up at the time tests were carried out. However, findings indicate this to have been likely.

It seems probable that if such technology were to be mandated by any government for use by adults, large portions of the Internet would be frequently blocked.

Questions Regarding the Capabilities of the Technology

In considering whether technology is sufficiently advanced to reliably replace human classification of content, a number of questions arise which cannot be answered by the type of tests reported on herein. In regard to the Internet Sheriff Technology these include, but are not limited to, the following:

  • Can the product be modified to ensure entire sites are not indiscriminately blocked? If so, would the product then be likely to allow access to a larger number of pages containing pornography material?

  • If the product could be adjusted so as not to block entire sites, and instead blocked specific pages, would the resultant size of the blacklist detrimentally affect system performance, eg. result in longer delays in accessing content on the Internet, etc?

  • How regularly is the blacklist database checked by a human and how quickly are blocked sites viewed by a human and manually unblocked if wrongly added?

  • If the product were to be mandated for use by adults, would the number of adults attempting to access pornographic material cause constant blocking of sites containing primarily non-pornographic material?

  • Given that some sites containing pornographic material are not automatically identified and blocked, if the product was mandated for use by Australian adults, would web site designers (either in Australia or overseas) quickly become adept at designing pages and sites in a manner that the automatic scanning technology does not currently detect as pornographic, eg. more similar to non-pornographic sites?

  • If the product is able to be constantly updated to take account of the changing profile of pornographic sites seeking to avoid blocking, would the product be more likely to block non-pornographic sites?

  • How could access to "R-rated" material by adults using a password or other adult verification mechanism be enabled? At the present time, it appears that once an adult using a password had accessed a page deemed pornographic by the technology, the automatic scanning process would block it, resulting in that page, and potentially entire site, being inaccessible to adults using passwords to access R-rated material.

Appendix 1

Screen Image: Internet Sheriff page denying access to The National Party of Australia - Federal Party Home Page

[Screen shot of Internet Sheriff denying access to the The National Party of Australia - Federal Party Home Page]

Appendix 2: Examples of Material Found Blocked/Not Blocked

(Added 25 June 1999)

Continued >>

[Cyberspace is Burning - What are you going to do about it?]
Go to: EFA Stop Censorship Campaign page

Report issued: 11 May 1999
Index added: 25 June 1999
Appendix 2 added: 25 June 1999

You are here: EFA Home » Issues » Censorship » Internet Censorship in Australia » Review of Internet Sheriff