Creating an HTTP Testing Tool

Written by: Christoph Wille
Translated by: Bernhard Spuida
First published: 5/8/2000

Testing a Web site with various Browsers is part of a Web developer's daily work - at least when not working on an Intranet project where the Browser is company policy.

Browser tests allow for the recognition and removal of layout problems and of client-side scripting problems. This however is no guarantee that you can manipulate all details of the connection between Web server and client, which can be important for testing security and stability.

What details am I aiming at? For example the Browser automatically sends its ID, which e.g. for Internet Explorer under Windows 2000 would be:

Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)

Netscape Navigator sends a similar ID, however without the string compatible. When not using a component for the detection of Browser capabilities, you will parse the string yourself - and occasionally make mistakes, such as not taking subversions into account or ignoring later versions:

If Mid(strBrowser,31,1) = 5 Then

This code would only work for Internet Explorer Version 5.x, in case of an upgrade to version 6 or later, the Web pages would have to be re-programmed. And it is rumored to have happened that such mistakes have been overlooked for long periods of time...

Further points of interest for testing outside of Browsers: simpler testing of GET and POST, the possibility to directly investigate headers (for cache-control, Cookies, etc), manipulating Content Types, etc.

In the further course of this article I will demonstrate how an interactive testing application can be created. This app can be adapted to specific needs or even be turned into an automated testing tool for Web sites..

Programming

For programming I used two tools: Visual Basic 6.0 (ASP programmers have the advantage of not having to learn a lot of new stuff), and the AspTear 1.5 component by AlphaSierraPapa. With VB I program the front end and AspTear is used for conversing with the Web server - i.e. requesting the pages with the details in question:

• URL
• Port
• User Agent
• Protocol
• Data (GET or POST)
• Content Type

The server returns headers and content (the HTML document) - as well as eventual error messages.

The Program W3Test was programmed as a standard EXE (source code and executable are contained in the download). In the Visual Basic development environment the finished User Interface looks as follows:

To insert the code for the AspTear component, we only have to reference the type library. This is done with the References command in the Project menu. Using the Browse command you then search for asptear.tlb (1.2.5; 1.5 ships with the type library inside asptear.dll) and bind it into the project:

Now we can write the code for fetching a Web page from the server:

Private Sub cmdGo_Click()
On Error GoTo Err_CmdGo
    Dim strURL As String, nPort As Integer, strRetVal As String
    Dim nAction As Integer
    
    ' I simply get the text here, you can add whatever check you want
    strURL = Me.txtURL.Text
   
    ' get the port - simple check if it is numeric
    nPort = 80
    If IsNumeric(Me.txtPort) Then
        nPort = Me.txtPort
    Else
        MsgBox "The port number you entered is non-numeric."
        Exit Sub
    End If

    Dim tearObj As New ASPtear.ASPtear
    tearObj.ContentType = Me.txtContentType
    tearObj.FollowRedirects = Me.chkFollowRedirects.Value
    tearObj.HttpVersion = Me.cbProtocol.Text
    tearObj.Port = nPort
    tearObj.UserAgent = Me.txtUserAgent

    If Me.radioGET.Value = True Then 
        nAction = ACTION_GET
    Else
        nAction = ACTION_POST
    End If
    strRetVal = tearObj.Retrieve(strURL, nAction, Me.txtData, "", "")

    Me.txtResponse = Replace(strRetVal, vbLf, vbCrLf)
    Me.txtHeader = tearObj.Headers
    MsgBox "Retrieval finished!"
   
    Exit Sub
Err_CmdGo:
    MsgBox Err.Description & vbCrLf & "Error number: " & Err.Number
    Exit Sub
End Sub

I exercised a lot of restraint in checking user input to keep the source code short - and also to allow for seemingly meaningless requests like e.g. an empty User Agent string (this is the downfall of many a Browser-checking code...).

Usage

Using the new tool is straightforward, the author inquiry page of AspHeute.com will serve as an example:

http://www.aspheute.com/Service/contactauthor.asp?
        author=Christoph+Wille&
        AuthorEmail=christophw@alphasierrapapa.com&
        artikel=20000502

Entering this address into the Browser's URL field will take you to the contact page for the German article "Monitoring ASP".

To call this URL from the W3Test application, the following settings should be entered (the result also is shown in the screenshot):

Now you can go and use POST instead of GET, try out User Agents, change some parameters of the Query String to check the code of the page for weaknesses.

I want to emphasize that the format for passing parameters is the same for GET as for POST, thus the Query String also is identical. Binary POSTing (File Upload) is not possible.

The W3Test program is capable of following Response.Redirect instructions when the checkbox Follow Redirects has been checked; otherwise an error message results, indicating that the server has requested a redirect.

Conclusion

The W3Test program presented here does not exhaust the full range of of functionality of the AspTear component. It also provides user validation, SSL client certificates and faking referers (the page the visitor has originally come from).

Another possible use of the AspTear component would be a script which visits one's own Web site daily and compares the main page to a template - checking for possible defacement by hackers. Of course, the component can be used on the server itself, e.g. for verifyinging credit card numbers.

Downloading the Code

Click here, to start the download.