Creating an HTTP Testing Tool
Written by: Christoph Wille
Testing a Web site with various Browsers is part of a Web developer's daily work - at least when not working on an Intranet project where the Browser is company policy.
Browser tests allow for the recognition and removal of layout problems and of client-side scripting problems. This however is no guarantee that you can manipulate all details of the connection between Web server and client, which can be important for testing security and stability.
What details am I aiming at? For example the Browser automatically sends its ID, which e.g. for Internet Explorer under Windows 2000 would be:
Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)
Netscape Navigator sends a similar ID, however without the string compatible. When not using a component for the detection of Browser capabilities, you will parse the string yourself - and occasionally make mistakes, such as not taking subversions into account or ignoring later versions:
If Mid(strBrowser,31,1) = 5 Then
This code would only work for Internet Explorer Version 5.x, in case of an upgrade to version 6 or later, the Web pages would have to be re-programmed. And it is rumored to have happened that such mistakes have been overlooked for long periods of time...
Further points of interest for testing outside of Browsers: simpler testing of GET and POST, the possibility to directly investigate headers (for cache-control, Cookies, etc), manipulating Content Types, etc.
In the further course of this article I will demonstrate how an interactive testing application can be created. This app can be adapted to specific needs or even be turned into an automated testing tool for Web sites..
For programming I used two tools: Visual Basic 6.0 (ASP programmers have the advantage of not having to learn a lot of new stuff), and the AspTear 1.5 component by AlphaSierraPapa. With VB I program the front end and AspTear is used for conversing with the Web server - i.e. requesting the pages with the details in question:
The server returns headers and content (the HTML document) - as well as eventual error messages.
The Program W3Test was programmed as a standard EXE (source code and executable are contained in the download). In the Visual Basic development environment the finished User Interface looks as follows:
To insert the code for the AspTear component, we only have to reference the type library. This is done with the References command in the Project menu. Using the Browse command you then search for asptear.tlb (1.2.5; 1.5 ships with the type library inside asptear.dll) and bind it into the project:
Now we can write the code for fetching a Web page from the server:
Private Sub cmdGo_Click() On Error GoTo Err_CmdGo Dim strURL As String, nPort As Integer, strRetVal As String Dim nAction As Integer ' I simply get the text here, you can add whatever check you want strURL = Me.txtURL.Text ' get the port - simple check if it is numeric nPort = 80 If IsNumeric(Me.txtPort) Then nPort = Me.txtPort Else MsgBox "The port number you entered is non-numeric." Exit Sub End If Dim tearObj As New ASPtear.ASPtear tearObj.ContentType = Me.txtContentType tearObj.FollowRedirects = Me.chkFollowRedirects.Value tearObj.HttpVersion = Me.cbProtocol.Text tearObj.Port = nPort tearObj.UserAgent = Me.txtUserAgent If Me.radioGET.Value = True Then nAction = ACTION_GET Else nAction = ACTION_POST End If strRetVal = tearObj.Retrieve(strURL, nAction, Me.txtData, "", "") Me.txtResponse = Replace(strRetVal, vbLf, vbCrLf) Me.txtHeader = tearObj.Headers MsgBox "Retrieval finished!" Exit Sub Err_CmdGo: MsgBox Err.Description & vbCrLf & "Error number: " & Err.Number Exit Sub End Sub
I exercised a lot of restraint in checking user input to keep the source code short - and also to allow for seemingly meaningless requests like e.g. an empty User Agent string (this is the downfall of many a Browser-checking code...).
Using the new tool is straightforward, the author inquiry page of AspHeute.com will serve as an example:
http://www.aspheute.com/Service/contactauthor.asp? author=Christoph+Wille& AuthorEmailfirstname.lastname@example.org& artikel=20000502
Entering this address into the Browser's URL field will take you to the contact page for the German article "Monitoring ASP".
To call this URL from the W3Test application, the following settings should be entered (the result also is shown in the screenshot):
Now you can go and use POST instead of GET, try out User Agents, change some parameters of the Query String to check the code of the page for weaknesses.
I want to emphasize that the format for passing parameters is the same for GET as for POST, thus the Query String also is identical. Binary POSTing (File Upload) is not possible.
The W3Test program is capable of following Response.Redirect instructions when the checkbox Follow Redirects has been checked; otherwise an error message results, indicating that the server has requested a redirect.
The W3Test program presented here does not exhaust the full range of of functionality of the AspTear component. It also provides user validation, SSL client certificates and faking referers (the page the visitor has originally come from).
Another possible use of the AspTear component would be a script which visits one's own Web site daily and compares the main page to a template - checking for possible defacement by hackers. Of course, the component can be used on the server itself, e.g. for verifyinging credit card numbers.
Downloading the Code
Click here, to start the download.