Beyond Hard-to-Reach Pages: Interactive, Parametric Web Macros

The changes in the WWW including complex data forms, personalization, persistent state and sessions have made user interaction with it more complex. Research is needed into resuing and sharing user interaction with the WWW. We describe Web Macros, a system for interactively recording and playing scripts that produce pages that cannot be directly bookmarked. Our system supports parameterized macros, interactive playback, verification of downloaded pages based on structure similarity, and the concept of public and private persistent state. We describe the advantages and challenges of the pure proxy architecture.

Keywords:

1. Introduction

From its first days, the World-Wide Web was more than a just vast collection of static hypertext, and now its character as a Web of interactive applications is essentially defined. A significant number, if not a majority, of what we think of as "Web pages", are not stored statically, but rather generated on request from databases and presentation templates. Web servers check user identity, customize delivered HTML based on it, automatically track user sessions with cookies and session ids, etc.

Interaction with the Web has become more complex for the user, too. One has to remember and fill out user names and passwords at sites requiring identification. Requesting information such as car reservations, flight pricing, insurance quotes, and library book availability requires filling out forms, sometimes complex and spanning several pages. If a user needs to repeat a request with the same or somewhat different data, it must be entered again. We claim that there are tasks involving Web interactions that are repetitive and tedious, and that users will benefit from reusing these interactions and sharing them. Interactions can be reused by capturing (recording), and reproducing them. We illustrate the repetitive aspects of interacting with the Web and potential for reusing and sharing tasks in the examples below.

A typical scenario of interaction with a Web information provider may involve navigating to the site's starting page, logging in, providing parameters for the current request (which can be done on one but sometimes several pages), and retrieval of results, such as the list of available flights or matching citations. Figure 1 shows a typical session with the University of Minnesota Housing Services database. The login page is bookmarked directly; no steps to navigate to it are shown.

Figure 1 - A Session with the U of M Housing System Search

Since the housing database may change daily, the user performs the housing search multiple times, perhaps on a regular basis. The interaction can be repeated exactly or with variations. For example, user may specify a different rent range or availability date. These can be thought of as parameters of the interaction. Flight reservations, real estate, rentals, or car classifieds change often, so the same task can be performed daily with the same parameters. These are all examples of hard-to-reach pages [FAK+00] that are not identified by any bookmarkable URL, but must be retrieved by a combination of navigation and form filling. There is a considerable amount or repetitive interaction that potentially can be reused by automating it - recording this interaction so that it can be "replayed" by the user in the future.

Reuse of Web interaction is not limited to a single user. Consider a scenario in which a college lecturer determines a set of textbooks for a class she is teaching. She recommends that her students purchase these at Amazon.com. To make it easier for students to find the books and complete the purchase, she would like to be able demonstrate the interaction with the bookstore using her browser: navigate to Amazon.com home page, perform searches for desired books, and place them in a shopping cart. If these steps can be saved and replayed by students, they will instantly get a shopping cart filled with textbooks for the class.

In this work, we concentrate on reuse and sharing of Web interaction. We describe Web Macros, a system that automates the Web for repetitive tasks by recording and playing parameterized, interactive macros. The paper is organized is follows. First, we discuss prior and related work on end-user Web automation. Second, we describe goals we set for Web Macros that distinguish it from other systems, and our approaches to addressing them. Then, system architecture and implementation are explained. Finally, we discuss system limitations, lessons learned, and directions for future work.

2. Prior and Related Work

Histories and bookmarks, or favorites, were the original tools to simplify interaction with the Web. They have been extensively studied [ABC98], and, along with their usefulness and wide acceptance, their fundamental limitations have been identified. Both rely on URLs as identifiers for Web resources, an approach applicable only for static pages and for dynamic ones where request parameters are encoded in the URL. Histories and bookmarks are therefore useless for pages that are retrieved by sending an HTTP POST request, as many form pages are. It is not possible to associate specified form values with a bookmarked page, so that a user would only have to click the form submission button. Histories and bookmarks do not have the notion of state encoded in session tokens or cookies. These tools do not show if, or how, the retrieved page has changed since they were bookmarked or visited (Netscape has the "Update bookmarks" menu command that checks all bookmarks for dead links and modified links; the latter use the HTML Last-Changed metatag). Although a user can organize bookmarks and favorites in a concept tree (or a graph, if "shortcuts" are used), each bookmark or history item is associated with a single Web page.

Three types of approaches were developed to address different shortcomings of traditional tools: Web trails and tours; web clipping; and web automation. Our work falls in the third category, but we discuss representative work on Web trails and clipping also.

Application of research in hypertext trails and paths [MI89] to the Web produced systems allowing users to create Web tours by typing URLs that comprise the tour, or by demonstrating them through navigation ([FSM+97], [JJG98]). The tours can be stored on either the server or the user's computer. Typically, interfaces for viewing tours were provided by wrapping tour pages in frames, and displaying tour navigation controls in a separate frame. These systems still supported only static URLs and pages. Recently, SpotOn has offered a product and a subscription service that allows users to demonstrate WebTours. WebTour pages are cached on the SpotOn server for archiving and fast retrieval, are automatically updated on the server if source pages are modified, and can be annotated with HTML and voice for presentations.

The Turquoise [MB97] project was one of the first "web clipping" systems, allowing users to compose pages by navigating, specifying regions of interest, and inserting or manipulating them in the target page. It used a customized AOLpress browser. Turquoise attempted to make its scripts generic with respect to changes in source pages by generalizing user actions. Identified HTML patterns, described in a simple, human readable pattern description language, were matched against a pattern template database. The interesting contributions of Turquoise were its inference engine and the database of HTML pattern templates. Turquoise supports navigation between pages used for clipping, but only for pages with static URLs.

Internet Scrapbook [SK98] is a personal "web clipping" system similar to Turquoise. Internet Scrapbook users create personal pages by graphically specifying regions of interest on the page. Using the text in the region's heading and the position of selected data in the page, the system extracts matching content from changing Web pages, such as electronic newspapers. The reported user study shows that the Internet Scrapbook is robust to common changes in the markup structure, such as added list items.

AgentSoft's LiveAgent [Kru97] was an early proprietary system for automating repetitive browsing and data entry tasks on the Web. LiveAgent scripts that automate tasks are authored by demonstration, with the user explicitly initiating the recording session. Novel features of LiveAgent include HPDL, an HTML Position Definition Language that describes the link to be followed in terms of its absolute URL, or its number on page, text label, and a regular expression on the URL. The user is prompted with a graphical dialog to specify HPDL for each link. The LiveAgent system records user actions by attaching JavaScript onClick handlers to links, buttons and input fields. LiveAgent provides a MasterAgent interface for authoring multiple "user surrogate" agents that can be run in parallel.

The original implementation of the Web Macros system [SKC99] provided means for users to demonstrate interaction with the Web through a browser. Web Macros saved the verbatim interactions as WebL scripts that could be replayed later, via a browser or using a command-line interpreter.

WebVCR [FAK+00] is a applet-based interactive tool for authoring and replaying user interaction with the Web. It supports changing content and markup, as well as URLs with embedded session ids by using a set of heuristics for matching the recorded link or form action with the actual Web page retrieved during replay. [FAK+00] discusses alternative architectures for Web automation systems and merits of client-based and server-based implementations for recording and replay. The paper includes an analysis of Web automation tasks that sets the standards for future Web automation systems.

We build on our earlier work, as well notions explored in other Web automation systems: LiveAgent and WebVCR. Our approach to making recorded macros robust with respect to changing pages is a subset of the one used in WebVCR. Web Macros extends the concepts of recording and replaying Web interaction with parameters, interactive replay, result verification, and manipulation of persistent state encoded in cookies.

3. Goals for the Web Macros System

Our earlier work on a Web automation system prototype [SKC99] produced a proxy-based system that allowed users to demonstrate Web navigation and form filling that were recorded and replayed. Continuing this work, we have identified four additional goals that we wanted to address in the next version of the Web Macros system. The first two are support for parameters and interactive playback. Verification of result pages alerts the user if an "unexpected" result page was produced during playback, while allowing structurally similar pages with different content. Finally, support for remote use and sharing permits a user to record and replay interactions on different computers, and to control how personal information is communicated to other users sharing these tasks.

3.1. Support for Parameters

Hard-to-reach pages are often dynamically generated based on information provided by the user. This information includes flight destination and dates for a ticket reservation system, author names in a bibliography search, or available rental units. In the housing search examples above, a user may repeatedly search for rentals matching different criteria. For users to automate reuse of similar interaction scenarios with different data, a Web automation system should support parameterization.

Most elements of user's interaction with the Web can be considered parameters. Examples are values that the user types into a form, and links page clicked on a page. The questions are, what parameters are most likely to change between user interactions, and how these parameters can be determined from the interaction.

In the current version of Web Macros, macro parameters correspond to certain form fields specified during macro recording. We support two types of parameters: variable, and private. Variable parameters get their default values from the recorded ones, but can be overridden by the user. Private parameters are not stored with a recorded macro; they must be specified by the user at playback.

3.2. Interactive Playback

A Web automation system should be able to do more than execute the recorded steps in the background and present the user with the resulting page. In some cases, results produced by the intermediate interaction steps can be of interest to the user. A primary example is a tutorial recorded by an expert user and played by casual users. A Web automation system must support recording and replaying tutorial-type interactions, by providing step-by-step playback and controls to navigate between steps.

3.3. Verification of Results

During the replay of recorded interaction steps, Web automation systems have no control over what the servers actually send back as results. A page representing results of playing a macro step can be different from the one produced during recording for several reasons. The recorded URL may become obsolete, or the requested page may be unavailable due to the server downtime. The recorded URL might contain session information that the automation system does not correctly factor out. The retrieved page may have a different HTML markup because it (or the template from which it was generated) was updated on the server. Finally, the page may be generated from the same HTML template but with different data, as can be the case with flight listings or available rentals.

Ideally, a Web automation system should be able to distinguish between cases when a desired page is retrieved during playback, and when the macro result is not what the user intended. While to completely solve this problem, a Web automation system must have near-human intelligence, some useful solutions can be approximated. In particular, it is desirable to recognize that a retrieved page has HTML structure similar to the recorded one, but perhaps a different content. Such structurally similar pages are likely when a server generates pages from a template and a database. For example, for a housing search where search location and rent range are parameters, the content of the search results page will be different for different parameter values.

3.4. Support for Remote Playback and Sharing Among Users

If a Web automation system captures complex, repetitive interactions, it is likely that users will reuse captured interactions from multiple computers or locations. For example, a researcher may run a bibliography search from her office computer, home computer, and her laptop. Moreover, a user may record an interaction to be reused by other individuals, as in the example of a lecturer populating an Amazon shopping cart with textbooks for her class. If any persistent state is maintained on one's computer that determines what pages are retrieved as the result of the interaction, a Web automation system must provide this state for the macros to run correctly, and must let users control what personal information is shared.

HTTP was designed as a stateless protocol; however, as browser developers realized it was often needed to maintain user identity and other state in HTTP, they introduced HTTP cookies, which later became part of the standard. Servers send cookies to browsers, which save them on a local computer. When subsequent pages are requested from a site, a browser uses a set of rules to determine which previously received cookies from this site must be sent back with the HTTP requests. Cookies are typically used to track user identities, user preferences, and site visitation patterns. Though cookies can be turned off or restricted in most popular browsers, many sites require them for normal functioning. Cookies are perhaps one of the aspects of Web browsing least understood by casual users.

Cookies encoding persistent state needed to correctly play the macro may only exist on the computer where the macro was recorded, and they may have expired there. Web Macros system should support including cookies, both existing ones and those received during the recording, with a macro.

When a macro creator and macro users are different individuals, macro creator may want to hide all persistent browsing state when recording a macro, since it can contain private information. In the scenario where the instructor populates a shopping cart with textbooks, a macro should be recorded in a "sandbox" environment: recording is started with no pre-existent state, and no side effects of the recording are preserved.

4. Implementation

4.1. A Proxy-based Architecture

We have selected a proxy-based architecture for the Web Macros system. The Web Macros system is written in Java and based on the Muffin HTTP filtering proxy. The proxy, running on either a local or a remote computer, performs both recording and replay. The proxy uses HTML rewriting to annotate pages for recording and playback. We describe the details of annotating pages below.

Figure 2 - Web Macros Recording Controls

User interacts with the system through special URLs that are interpreted by the proxy, and HTML pages generated by it on the fly. For example, in response to the URL http://webmacros/controls/ (which can be bookmarked or placed in a browser shortcut bar), the Web Macros proxy returns a simple HTML page shown in Figure 2. When user clicks on the "Start Recording" button, the proxy receives the http://webmacros/do?action=Start+Recording URL and initiates a recording session. The "Get Me Here" button is part of an experimental interface to infer state dependencies and generate a Web macro that would produce the last retrieved page, without explicitly demonstrating the complete interaction.

4.2. Recording of Web Macros

Unless the user has initiated a recording of her interaction, the proxy tunnels the HTTP stream without any changes and without a significant performance hit. Once the user has started the recording of a macro by clicking on the "Start Recording" button, the JavaScript code associated with the button opens a new empty browser window, and the controls page is regenerated (via an HTTP redirect) to display the "Stop Recording" button.

The Web Macros proxy now starts to track all user navigation and interaction done in the browser. The proxy has no access to the browser (as they can be running on different computers), so tracking is done by rewriting URLs in all links and form actions on each retrieved page. As the proxy receives a page during the recording, it replaces URLs of all links and forms actions with the special URL that it will intercept when a link is clicked or a form is submitted. The original link or form URL becomes a CGI-style parameter of the rewritten one, along with the Document Object Model (DOM) information for that URL. For example, the following link (from the bibliography search example):

In this example, the CGI arguments of the rewritten URL include action (record) instructing the Web Macros system to append this step to the current macro, the original URL, its type (link), and the index of the form in the DOM array document.forms. No rewriting can be done on URLs specified by the user directly, for example, by typing them into the browser address bar. Such URLs are recorded without any DOM information.

The user must currently demonstrate all macro steps in a single browser window, the one opened by Web Macros when user initiated the recording. The reason is that Web Macros only rewrites links and forms in pages displayed in that window, so the system cannot distinguish links followed in other browser windows from URLs specified directly, for instance, by typing or pasting into the browser address bar.

When the user clicks on a link or submits a form, the Web Macros proxy receives the HTTP request with the rewritten URL, parses and records it. The proxy then strips away the augmenting information from the URL, returning it to the original format, and sends the HTTP request to the Web.

When the user has demonstrated all the steps that are to be automated by the current macro, she fills in the macro name and description fields and clicks on the "Stop Recording" button. Web Macros generates an internal representation of the macro (Section 4.4), and redirects is control window to display the "Start Recording" button again.

4.3. Support For Macro Parameters

In the current version of the Web Macros system, certain form elements can be specified as parameters during recording. Based on the bibliography search and similar scenarios, we selected TEXT, TEXTAREA, and PASSWORD inputs, and SELECT elements. A user recording a macro may specify the type of each supported form parameter by selecting one of three radiobuttons, as shown in Figure 3.

Figure 3 - Form Fields Augmented For Parameter Type Selection

During recording, as the proxy retrieves requested pages, it adds radiobuttons to the above form elements by rewriting the HTML of the page. Names of form elements are used to generate names for radiobuttons. For example, for the text input Username in the login form on Figure 3, the following HTML is inserted into the form for the third radiobutton:

When the user submits the form during recording, the inserted annotations are parsed by the proxy and removed before the request is sent to its destination.

Our experience shows that these annotations are not usually visually disruptive. However, because of a lack of layout control in HTML, the three radiobuttons sometimes are not rendered on the same line. An alternate solution (which is also easily supported by HTML rewriting) it to augment the page with JavaScript code creating popup menus for specifying parameter type when the user right-clicks on a form element.

The Web Macros system currently uses simple rules for inferring parameter types. By default, all TEXT, TEXTAREA and SELECT elements are constant, so if the user wants to be able to run the macro with different values for them, she selects the "variable?" radiobutton. PASSWORD fields are private by default. A private field means that a user will be prompted to enter its value each time during execution. If there is a TEXT input immediately preceding a PASSWORD input in the same form, as in the example, the field is considered to be a username and also made private. Private parameter type is appropriate for secret data such as passwords, as well as parameters that do not have a reasonable default value.

4.4. Internal Macro Representation

In the initial implementation of the Web Macros system [SKC99], macros were stored as WebL [KM98] scripts. The script-based representation worked for macros that did not support parameters, check for page identity at playback, and manipulate state in cookies. However, when we added macro parameters, and started storing fingerprints of the recorded page, we needed to parse WebL code to extract this information. This became onerous as new features were added.

DbId	URL	Type	DOM index	Method
101	http://rentals.tc.umn.edu/	Ext	NULL	GET
102	http://rentals.tc.umn.edu/renter/	Link	1	GET
104	http://rentals.tc.umn.edu/searchrentals.asp	Form	0	POST

Table 1 - Relational Representation of Macro Steps

Currently, all macros data is stored in a relational database, with separate tables for steps, form elements, and form element data. Three rows of a relation storing information about recorded steps are shown in Table 1. DbId, the database-generated unique id (leftmost column) is the primary key for this table and a foreign key in other tables. DbId links a macro step with the tables storing pages produced at recording and form elements. The Type column determines whether the URL originated on the "previous" page (Link and Form), or was specified externally (Ext). DOM index is only defined for on-page URLs (4th column). Some of the columns, including the recorded User Agent, and a fingerprint representation of the recorded page, are not shown.

Table 2 shows the rows from the form field table describing parameters for the form in Figure 1c (macro step with id 104 in Table 1). DbId and field name together form the primary key. Field name and field type come from the HTML source of the page. Parameter type is defined for TEXT and SELECT fields, and is NULL for the SUBMIT button. Two of the fields - unittype and maxrent - are variable and can be changed by the user when the macro is run.

DbId	Field Name	Field Type	Parameter Type
104	campus	select	const
104	unittype	select	variable
104	bedrooms	select	const
104	maxrent	text	variable
104	next	submit	NULL

Table 2 - Relational Representation of Form Fields

Recorded values of the form fields are stored in a separate table, since a SELECT field may have multiple values. Table 3 shows values of form fields unittype and bedrooms from Table 2.

DbId	Field Name	ValueNum	Value
104	unittype	0	apartment
104	bedrooms	0	1
104	bedrooms	1	2

Table 3 - Relational Representation of Form Fields Values

The database identifiers of steps comprising a macro are currently stored in an HTML file, which also contains human-readable macro description entered by the user when the macro is created.

4.5. Interactive Playback of Macros

When a macro is played, the system retrieves relational data about each macro step and constructs an internal representation in the form of Java objects. WebL code for each step is generated from Java objects on the fly. The generated code is passed for execution to the WebL interpreter embedded in Web Macros.

Web macros can be played in either batch or interactive modes. In batch mode, the system assumes that the user is only interested in a page produced by the last step of the macro. Intermediate pages are presented to the user only if they contain private parameters. Interactive playback mode is appropriate when the user intends to change parameters of the macro by providing different values in some of the forms, or when she is interested in viewing results of intermediate macro steps.

Figure 4 - Pre-filled Form Fields and Playback Controls

In the interactive mode, Web Macros fills recorded values of variable parameters and other form fields into appropriate forms. Username is a variable parameter - its value is pre-filled in the form, but it can be overridden. The Password field in Figure 4 was marked private when the macro was recorded, so it is not pre-filled and must be specified when the macro is played. The proxy pre-fills form fields by rewriting the page before it is presented to the user; matching between recorded and actual form fields is done based on the forms' DOM indices and field names. Updating constant form fields is disabled by adding JavaScript handlers and currently, due to incomplete JavaScript support in Netscape Navigator, works only under Internet Explorer 5 or newer.

Figure 5 - HTML for the Next button

For continuing the playback of the recorded macro in the interactive mode, the proxy adds the Next button (Figure 4) at the bottom of each retrieved page. User modifies variable and private form parameters after the Next button is appended by the proxy. Therefore, it cannot encode form parameters in the Next button since they are not known yet. Instead, the Next button has an onClick JavaScript that submits the appropriate form on the page.

The page in Figure 1a contains two forms, one for creating a new user, and the second one to log in as an existing user - the latter was submitted when the macro was recorded. Both have the same action URL. The proxy determines that the second form corresponds to the recorded one, by using its DOM index and comparing the names of the form fields between two forms. The onClick handler generated by the proxy for the Next button contains therefore a reference to document.forms[1], the second form in the page.

4.7. Verification of Web Macro Results

Each step of a macro can produce a page that is different from the one retrieved when this step was demonstrated during macro recording. Web Macros attempts to distinguish between pages with updated content and all other changes - the latter require some recovery or at least user notification of failure.

Web Macros detects server errors during playback checking for HTTP return codes, searching the page content for human-readable error message, or a combination of both.

The other potential causes for page changes (see Section 3.3) are more difficult to address. It is clear that verbatim comparisons between recorded and retrieved content are overly restrictive. Indeed, a page with a list of apartments matching some parameters will fail this test if different parameters were specified, or the rentals database was updated. Comparing HTML parse trees directly will also fail if the numbers of items (found apartments, in this case) are different.

We have studied measuring page structure similarity, and implemented a prototype verification mechanism for Web macro results based on it. When a page is retrieved and recorded during macro creation, its HTML parse tree is constructed (it is a side effect of retrieving a page in WebL), and path expressions [ABS00] of all text elements on the page are hashed into 64-bit fingerprints ([Rab81], [Bro93]). HTML path expressions capture the structure of a page, and are independent of page content. Fingerprints provide a compact representation that can be efficiently compared.

The set of fingerprints for each retrieved page is saved in the database with the macro step that produced it. During playback, Web Macros computes HTML path expressions and their fingerprints for each retrieved page. The two sets - the recorded and the actual one - are then compared using a Jaccard similarity measure (relative set overlap): the cardinality of set intersection divided by the cardinality of the set union. The formula is given in Figure 6.

Figure 6 - Jaccard Similarity Formula

Sufficient similarity between the recorded and retrieved pages means that page parse trees contain paths that are the same, while the number of such paths is ignored. Pages produced by the same search engine on different queries will be highly similar according to this measure, even if the number of search results is different.

If the similarity between the recorded and retrieved page is above a threshold, Web Macros considers the retrieved page to be the desired one, and macro playback continues. Otherwise, the user is presented with a dialog prompting to continue or cancel macro playback. The value of threshold to be used for verification is an open issue, and is in general site- and page-dependent. We have used the threshold values between 0.4 and 0.5 with some success. These similarity threshold values usually are sufficiently high to identify user login failures and significant page layout changes. We are conducting more experiments with groups of template-generated pages from popular sites.

4.8. Sharing and Manipulation of State in Cookies

We formulated the following issues in manipulating cookie-contained state in Web macros. First, should recording-time cookies be used during recording and included with a macro? The answer is yes if the macro is played by the same person who recorded it, but on a different computer. Second, when a user plays a macro, should her cookies be used, and whether they take priority over cookies included with the macro? Finally, does recording and playing a macro have side effects, that is, are new or updated cookies propagated to the browser cookie jar? Our implementation does not handle expired cookies; we plan to detect what HTTP requests refresh cookies, and include them in macros as hidden steps.

By default, when user initiates the recording, the Web Macros proxy retrieves her existing browser cookies and initializes the macro "cookie jar" with them. As a macro is being recorded, new cookies can be received from the servers, or values of existing ones updated. We argue that not all pre-existing cookies should be included with a macro when it is recorded, for privacy reasons and to save space. It should be sufficient to include only those of the existing cookies that were sent when the macro was being recorded, and new cookies set during recording. Web Macros keeps track of cookie additions and updates to included a minimal set of cookies with a macro.

If at the start of recording the user checks the "Safe Mode" checkbox on the Web Macros controls page (Figure 2), use of existing cookies in a macro is disabled. In this case, a macro being recorded starts with an empty "cookie jar" that may get filled with cookies if they are set explicitly during recording. The "Safe Mode" option, if checked, also disallows propagation of cookies to browser storage after the recording is complete. The safe mode of recording provides a "sandbox" model, appropriate when a macro will be recorded and used by different people.

Figure 7 - Interface to Control Cookie Use at Playback

For a playback of a macro not recorded in a safe mode, the user has the option of enabling macro cookies, her personal browser cookies, or using both. If both macro and personal cookie are used when playing a macro, personal cookies take priority by default. The interface to alter the default settings is shown in Figure 7. Checkboxes and radiobuttons in the HTML-based interface are enhanced with JavaScript to enable and disable the radiobuttons as the checkboxes are selected and deselected.

After the user stops macro recording, or when playback completes, the Web Macros proxy updates the local browser cookie storage by directly modifying Netscape or IE cookie files if the proxy is running on the local machine. We currently support both Netscape and Internet Explorer cookie formats, though we do not query the registry information under NT to determine the current browser and a non-default location of browser cookies. It is not possible to update cookie files from a remotely running proxy using the HTTP "Set-Cookie" header because of the security restrictions.

The implementation of the cookie management in Web Macros is based on associating a "cookie jar" with a macro being recorded or played. It is initialized based on the requested recording or playback mode, and serves as surrogate for a browser cookie database. This cookie jar is later saved with a macro, propagated back to browser cookie files, or discarded. To determine what cookies must be sent with each request (emulating browser functionality), Web Macros uses the WebL implementation of the HTTP standard for matching URLs against the cookie jar.

5. Evaluation and Future work

The Web Macros system demonstrates that a pure proxy approach works for authoring and playing Web interaction scenarios. HTML rewriting is a fundamental model used by Web Macros during both demonstration and playback. During macro demonstration, the Web Macros proxy annotates links and forms with the information that allows it to intercept and record user-initiated page requests. During interactive macro playback, pages are augmented with the correct values of form parameters, and with the Next playback control button.

All the interfaces to control the Web Macros proxy are in HTML, and there is no need to code the GUI in the system itself. Unlike Java applets, HTML-based pages are static, so some pages generated by the Web Macros include JavaScript code for increased interactivity, such as form validation.

The advantages of a proxy over an applet include the proxy's independence of an operating system, and of an idiosyncratic Java Virtual Machine in a browser. The proxy approach does not require the user to confirm security privileges for an applet. Ideally, a proxy is independent of a specific browser brand or version. Our system, however, is browser-specific in two areas. First, Web Macros uses HTML- and JavaScript-based interfaces for controlling recording and playback, and so relies on the browser to correctly render HTML and interpret JavaScript code. Second, cookie saving and loading code in Web Macros has been specifically written to work with Netscape Navigator on Unix and Microsoft Windows, and with Internet Explorer on Microsoft Windows, since formats for cookie storage are not standardized.

A proxy architecture has some issues that a applet-based approach [FAK+00]avoids. Because the proxy intercepts an HTTP stream and not browser events directly, it must distinguish user-initiated requests for pages from requests for inline objects, such as images or frames. The Web Macros system handles this by building a hash table of frame and image URLs for each pages that is being requested during recording. Subsequent HTTP requests are checked against the table.

A limitation of a pure proxy architecture is that it has no way of detecting client-side script events. However, the proxy can attach its own JavaScript handlers that would communicate back to it through LiveConnect. Interaction between author-provided and proxy-provided client-side scripts is an open issue.

Web Macros allows macro creators to select which form fields can be overridden at replay. Such macro parameters make macros more generic. Private parameters, which must be specified by the user during playback, ensure that macros can be shared without disclosing secret information. We are working on automatically extracting parameter fields and their text labels and then presenting them to the user in the macro launch interface.

Web macros can be played back either in the batch mode, or interactively. In the interactive mode, the Web Macros proxy annotates pages presented for each macro step with the Next button. The structure of each retrieved page is compared with the structure of the recorded one, and if the similarity is below a threshold, Web Macros alerts the user that an unexpected page has been retrieved.

We would like Web Macros to detect iterative interaction, when users repeat the same task multiple times with different parameters, and later merge results into a single page. For example, a user may search multiple bibliography databases (INSPEC, Compendex, ACM Digital Library), or query flights into different airports in the same area (San Francisco, San Jose and Oakland airports). Eventually, Web Macros will support iterations over multiple links, not just form field values. Web macros will also merge results of iteration automatically. Initially, we aim to support detection of iteration and merging for multiple queries of the same service. Here we may build on some approaches developed in web clipping systems ([MB97],[SK98]).

6. Bibliography

[ABC98] David Abrams, Ron Baecker, Mark Chignell, and M. Information archiving with bookmarks: personal Web space construction and organization. Proceedings of CHI '98, ACM Conference on Human factors in computing systems, 1998.

[ABS00] Serge Abiteboul, Peter Buneman, Dan Suciu. Data on the Web. From Relations to Semistructured Data and XML. Morgan Kaufmann Publishers, 2000.

[Bro93] Andrei Broder. Some applications of Rabin's fingerprinting method. In R. Capocelli, A. De Santis, and U. Vaccaro, editors, Sequences II: Methods in Communications, Security, and Computer Science, pages 143-152. Springer-Verlag, 1993.

[FAK+00] Juliana Freire, Vinod Anupam, Bharat Kumar, Daniel Lieuwen. Automating Web Navigation with the WebVCR. In Proceedings of the 9th International World Wide Web Conference, Amsterdam, Netherlands, May 2000.

[MI89] C. C. Marshall and P. M. Irish. Guided tours and on-line presentations: how authors make existing hypertext intelligible for readers. In Proceedings of ACM Hypertext '89, 1989, pp. 15-27.

[Rab81] Michael O. Rabin. Fingerprints by Random Polynomials. Harvard University Technical Report TR 15-81.

[SK98] Sugiura, A. and Koseki, Y. Internet Scrapbook: Automating Web Browsing Tasks by Demonstration. Proceedings of ACM Symposium on User Interface Software and Technology, 1998.

(1a)	(1b)
(1c)	(1d)