Fred Brack   Fred Brack  
Raleigh, NC
Picture of the Cape Hatteras lighthouse

Using AJAX, the XMLHttpRequest Object API,
to Dynamically Populate Web Page Data

by Fred Brack
Last Updated

Introduction

I maintain a website whose Home Page is updated once per week; but other pages on the site are updated several times a week.  I wanted to extract a number from one of those pages dynamically to display as part of the Home Page, and that's how I got into learning about AJAX.  This page shares the result of my learning how to achieve my objective, and hopefully it will help you also.  It took awhile, as I am a JavaScript novice; but even it you are a novice, hopefully this will get you most of the way there.

Subsequent to writing this standalone article,
I wrote a series of articles on JavaScript.
In those articles, I discuss another use for AJAX,
imbedding HTML from one web page in another
web page by using Document Object
Model (DOM) methods.  To read that article, see
Using AJAX With JavaScript to Access Another Web Page.

What Does the AJAX XMLHttpRequest Object Do?

Per TutorialsPoint

XMLHttpRequest (XHR) is an API that can be used by JavaScript, JScript, VBScript, and other web browser scripting languages to transfer and manipulate XML data to and from a webserver using HTTP, establishing an independent connection channel between a webpage's Client-Side and Server-Side. ... Besides XML, XMLHttpRequest can be used to fetch data in other formats, e.g. JSON or even plain text.

The highlighting of "or even plain text" is mine, because that's all I'm going to talk about on this page.  XHR (the abbreviation for XMLHttpRequest) was designed primarily to give you access to a "database" of information coded in XML (Extensible Markup Language) format, and that's the default assumption.  But since the API (Application Program Interface) must first read the file of XML, it can also just pass you the raw file data for you to process.  And that's what we want:  the data from a file of our choosing, be it HTML, or TXT, or something else.

Limitation

A fundamental limitation of XHR is that it will only access a file from the host's server on the same domain.  You cannot use it to read just any old file.  This is standard web protocol known as Same-Origin Policy (SOP).

Methodology

Here then is the approach we will use to access a "piece of data" contained in a file on our web server.

  1. Create or identify a file (of any filetype) on our server that contains the data we wish to access.  For example:  mydata.txt.
  2. In the HTML file where we wish to display (for our example) or otherwise use (you are on your own!) this data, we will:
  3. Separately code that JavaScript routine (myscript.js) which does the following:

That's it!  Two statements in your main HTML; two functions in JavaScript; plus your data file.  Easy when you know how ...

The Data File and How It Is Accessed

Let's spend a moment talking about the data file.  In my case, I wanted to extract a single number from somewhere in one HTML page and display it on another page which was updated less frequently -- thus a dynamic presentation of the number.  To understand the approach I took, there is one thing you should understand about how XHR operates:  XHR reads an ENTIRE file and returns ALL the data in one object!  In my case, it would have returned nearly a million bytes of data just to extract 4 of them.  I felt that was too inefficient, so I chose a different approach.  The number I wanted was also contained in a file of a little over 1,000 bytes.  It turns out that the HTML file where that number would have been accessed is programmatically created, so I simply added one line to the program to copy the data file to the server at the same time as the HTML.  And my JavaScript specifies the URL of the data file.  For your purposes, you may simply wish to access an existing HTML file, and that's fine.  [With the speed of modern computers, my reservation about reading in a million bytes was probably unwarranted.  I subsequently wrote another routine which reads that same file every time it is invoked to search for something, and it is fast!]

How XHR Returns Data To You

As mentioned above, XHR returns all the data in the entire file as one long stream in a single object.  While you can simply search the stream for what you want, I chose to break the stream down into the "lines" that originally made up the data.  To do this, I search the stream for the end-of-line character ("\n" in JavaScript), first counting all of them to determine how many lines there are, then to look for the "next" one to be able to pick off only the current line.  It is that line, the current line, that I examine for the specific data I am seeking.

In the data file (called ADPvariables.ini in my example below), the line I am looking for looks like this:

@mastertitles = 4321 /* date */

So I will be searching first for each new line, then examining each line for "@mastertitles" so I can pick off the number associated with it (4321).  I don't care about the date field which follows.  In JavaScript notation, numbering starts with 0, not 1; so when I break the line into an array (simply one way of approaching the problem), the 4321 is field 2, not field 3.

IMPORTANT!  XHR defaults to returning data in XML format.  You MUST override the expected data type to force "plain text," as seen below, if you are not processing the file as XML.

How You Know When the Data Is Ready

After you "send" your request to XHR, you need to wait until XHR is done successfully transferring your data.  Along the way, it changes its "state" several times, returned in the 'readyState' property as five values ranging from uninitialized (0) to sent (2) to completed (4).  You only care about completion.  At that point, you care about the 'status' property, and there you only care if it is OK (code 200).  As a programmer, you might care about code 404 ('Not Found'), but you are unlikely to send that message to your user's screen, so we don't check it here.  Thus we only check for success with the following line, where the term "this" is JavaScript for the object that received the event (XHR in our case):

if (this.readyState == 4 && this.status == 200)

For reference, here is the chart of status codes taken from w3schools.com:

Property Description
onreadystatechange Defines a function to be called when the readyState property changes
readyState Holds the status of the XMLHttpRequest.
0: request not initialized
1: server connection established
2: request received
3: processing request
4: request finished and response is ready
status 200: "OK"
403: "Forbidden"
404: "Page not found"
For a complete list go to the Http Messages Reference
statusText Returns the status-text (e.g. "OK" or "Not Found")

Note from experience:  There is another status code of interest:  0.  It means an error occurred, which could be lots of things including some limitation of your server in the amount of data allowed to be returned.

The JavaScript For XHR

Here then is the JavaScript file I am using to accomplish the objective described above:

// GetADPCounts.js: Read the ADPvariables.ini file to acquire current AD title counts
const url = "ADPvariables.ini"; // 'url' is set to the desired URL
// The getData function issues the XHR requests against the URL
function getData(url) {
  let request = new XMLHttpRequest();
  request.overrideMimeType("text/plain"); // must override default of 'text/xml' to avoid err msg
  request.onreadystatechange = function() {
    if (this.readyState == 4 && this.status == 200) {
    processData(this); // Call processData with all the data from the file at URL
    }
  };
  request.open('GET',url,true);
  request.send();
}

// processData does whatever I want when the URL data is successfully read in
function processData(xml) {
  var data = xml.responseText; // ALL of the input file's data
  var lines = (data.match(/\n/g)||[]).length + 1; // determine number of lines by counting line breaks
  // Break up the lines by searching for the line break and dividing into current line and remainder
  for (i=1; i<lines; i++) {
    var cr = data.indexOf("\n");    // find position of first CR in input
    var line = data.substr(0,cr-1); // current line
    var data = data.substr(cr+1);   // remainder of file
    var line1 = line.substr(0,1);   // first char on line
    if (line1 == "@") {
      var x = line.indexOf("@mastertitles")
      if (x>-1) {
        var array = line.split(" "); // returns @variable = value /* date */
        var count = array[2];        //         0         1 2     3  4    5
        var text = count; // just the # of Master AD titles
        document.getElementById("masternum").innerHTML = text; // replace the contents of the specified area
        break; // terminate the function, as we found our data
      };
    };
  };
};

// Execute the above functions to modify the web page
getData(url);
// END

And In Conclusion ...

  1. If you wish to see my code in action, visit this page:  https://acb.org/adp/index.html.
  2. If you wish to see an extension of this technique reading in an entire file and searching for a hit on a specified title, visit:  https://acb.org/adp/findavideo.html.  Note that this particular page may be invoked from any other page on the website using the same input method (a FORM), as all the processing is done from this page by examining the URL to see if query was passed first.
  3. This is not meant to be a complete tutorial on AJAX XHR!  Please do a Google search for more information:  https://www.google.com/search?q=ajax+xhr.
  4. There are obviously multiple ways to find my data in the file and place it on a page of HTML, and my JavaScript could no doubt be improved.  This is just one simple example to get you started.
  5. If you wish to offer corrections or make suggestions for improvements to this page, please contact me via the email address below. 
  6. I hope this is helpful to ... someone!

Fred