Upgraded jQuery Spy

September 30th, 2006

Following many requests, I have upgraded the jQuery spy code to support multiple items returned from the AJAX response and custom timestamp functions - so that requests can be completely tailored.

Download the latest jQuery spy

Examples

Import the plug-in, and call the code on the spy container div (assuming you've imported jQuery already).

<script type="text/javascript" src="spy.js"></script>
<script type="text/javascript">
  $(function() {
    /* returns a selection of HTML DIVs to be inserted in to 
        the #spyContainer in a spy-style */
    $('#spyContainer').spy({'ajax': '/ajax/news.php'}); 
  });
</script>
<div id="spyContainer"></div>

Working Examples

Example spy with dummy data
The AJAX URL being used to return the dummy data (view the source of the page)
Source code to the HTML example
Same example as above, except using a JSON driven spy
Source code to the JSON example
Real life example on Arsenal web site: Arsenal news spy

Support

Multiple items from AJAX request and shows one at a time, reducing the load on your servers
Custom timestamp function available upon AJAX request to control the content appearing in the spy
Custom is duplicate function to allow better handling on JSON objects from the server

Prerequisites

You must be using the latest stable version of jQuery.

If you return JSON content, then you must provide the following:

Method: 'json'
Custom push function
Custom isDupe function (if you don't want duplicates sitting next to each other)

These are explained below.

Usage

Simple Parameters

ajax - required - the URL to server side script that will return the content
limit - default 10 - the total number of rows to show
fadeLast - default 5 - the number of rows to fade out, i.e. the last 5 rows
fadeInSpeed - default 'slow' (600) - the speed, in milliseconds to use for the first row to fade in with
timeout - default 3000 (3 seconds) - the time in milliseconds between showing a new row
method - default to HTML - should be the AJAX response type, valid values are: HTML or JSON

For example:

$(function() {
  $('#spyContainer').spy({
      'ajax': '/jquery_spy/out.php',  // return type is HTML
      'fadeInSpeed': '1400', // crawl
      'timeout': 2000 // new item every 2 seconds
  });
});

Custom Functionality

push - custom push on function, this must be customised if using JSON
timestamp - custom timestamp function, defaults to return seconds from 1978.
isDupe - custom duplicate check function. This will work if not specified for HTML, but not JSON.
pushTimeout - control the time between individual items being pushed on to the spy stack.

Custom Push

If you don't want to return HTML to the spy, you can write your own push method.

Your custom method will have access to 'this' which is the DOM element (i.e. the contain div) and 'response' which is the response in plain text.

Here is an example using JSON:

$(function() { 
  $('#spyContainer').spy({
    'ajax': '/jquery_spy/out_json.php', 
    'push': custom_push });
});

function custom_push(response) {
  eval("var json = " + response); // convert to JSON

  // I'm being lazy here, but you get the idea:
  // Build the HTML
  var html = '<div style="display:none">'
  html += '<span class="ctr">' + json.num;
  html += '</span><span class="content">';
  html += json.string + '</span></div>';
  // Prepend the HTML inside the container
  $('#' + this.id).prepend(html);
}

See the an example JSON push in action.

Custom timestamp

Most developers have their own system for tracking the latest items, so I had this in to it's own pass in function. The example should explain the usage. The return value from the 'timestamp' value will be what is passed in to the AJAX request via the 'timestamp' posted value.

$(function() { 
  $('#spyContainer').spy({
    'limit': 25,
    'fadeLast': 3,
    'ajax': '/jquery_spy/out.php', 
    'fadeInSpeed': 'slow',
    'timeout': 3000,
    'timestamp' : myTimestamp }; 
});

function myTimestamp() {
  var d = new Date();
  // formated as yyyy-mm-dd HH:MM:ss
  return d.getFullYear() + '-' +
    pad(d.getMonth()+1) + '-' + // because month starts at zero
    pad(d.getDate()) + ' ' +
    pad(d.getHours()) + ':' +
    pad(d.getMinutes()) + ':' +
    pad(d.getSeconds());
}

function pad(n) {
  n = n.toString();
  return n.length == 1 ? '0' + n : n;
}

Custom isDupe

This function allows you compare the latest item against the last item to flag for a duplicate. This customisation really comes in to it's own if you're using JSON. If give the JSON object an ID, then you can compare the ID between the latest and the last items and chose not to show it.

Continuing in the Digg spy fashion, this spy does allow duplicates to appear in the list, but not next to each other.

If the return value is true the latest item is not shown. If the return value is false, the item is shown.

The function takes two parameters: latest item and last item.

Here's an example using JSON (assuming our JSON output object has an ID attribute):

$(function() { 
  $('#spyContainer').spy({
    'limit': 25,
    'fadeLast': 3,
    'ajax': '/jquery_spy/out_json.php', 
    'fadeInSpeed': 'slow',
    'timeout': 3000,
    'method': 'json',   // JSON output
    'isDupe': myIsDupe };
});

function myIsDupe(latest, last) {
  return !!(latest.id == last.id); // return boolean
}

Custom pushTimeout

The custom pustTimeout is particularly useful if you want to show all the items returned from the Ajax call to be pushed on to the stack straight away (rather than evenly one at a time).

The best way to do this is:

$(function () {
  $('spyContainer').spy({
    ... normal settings ...
    'pushTimeout': 1 // gives the appearance of all coming in at once
  });
});

Pause and Play Buttons

Included in the plug-in are pause: pauseSpy() and play: playSpy() functions.

These will, as the digg spy does, pause the flow of new content and start it up again. You can see these in use in the simple example

Usability

I touched on this before, but I would strongly recommend that you think about those users that will have JavaScript disabled for whatever reason. Your spy page should still work.

Here's what I would recommend:

Pre-populate the page. If you are going to limit to 10 items in the spy, start off with 10 items on the page.
Use a meta refresh in a <noscript> tag to automatically have the page reload.

Here's an example:

<script type="text/javascript">
  // the index starts at zero - so fade down all 
  // the divs after the 5th one
  $(function() { 
  $('#spyContainer > div:gt(4)').fadeEachDown(); // initial fade
  $('#spyContainer').spy({
    'limit': 8,
    'fadeLast': 3,
    'ajax': '/jquery_spy/out.php', 
    'fadeInSpeed': 1400,
    'timeout': 3500 };
  });
</script>

<noscript>
  This browser does not support JavaScript - 
  which is used to automatically update the content.
  Therefore this page will automatically refresh
  every 1 minute.
  <meta http-equiv="refresh" content="60" />
</noscript>

Posted September 30th, 2006 | Tags: digg, javascript, jquery, plugin, spy

Comments

Remy said on September 30, 2006:

Check out a working version using the new spy on Arsenal Mania

The main difference is the load their servers are taking – rather than 1 AJAX request per item, it shows 10 items, then does another AJAX request.

free_IPTV said on September 30, 2006:

Nice work, digg +1

Alejandro Sena said on September 30, 2006:

Thank you very much for this upgrade! I’ll test it soon =)

Alejandro Sena said on September 30, 2006:

How would the out.php look like? I’m using the same that the last version and it only shows the last row.

Marco said on October 7, 2006:

Hi, great handy tool explanation!

I do have one (style) question: when looking at the example at “http://leftlogic.com/jquery_spy/spy.html” in firefox (1.5.0.7), I notice that after scrolling down, all rows after the first one scroll a little bit up!
This does not happen in IE, so I guess it is a css thing.
Anybody fix that one already?

Also, I found that the IE (6) javascript engine doesn’t like it when you start with an empty div id=”holder” element. You’ll get an “object required” javascript error. I just put in some dummy text for now.

Thanks,
Marco

joel said on October 14, 2006:

If you don’t have a timestamp to check for newness you can use an ID. If you have a field with a unique auto increment ID then implement your timestamp function like so:
var id;
function mystamp() {
newid = $(‘div.id’).id();
if (newid == ‘’) {
return id;
}

id = newid;
return id;
}

and be sure to have an empty div for each row like so:
<div class=”id” id=”<?php echo $row[‘id’]; ?>”></div>

The idea is to get the latest auto increment id from the content and pass that to the source. The source would then add ”.. where ID>$timestamp.. ” to get the next items.

Anon said on October 18, 2006:

Hmmm I still seem to be having troubles.

I am using the following code to avoid any old rows from coming up:

function myIsDupe(latest, last) {
if (latest.id > last.id) {
return false;
} else {
return true;
}
}

and of course adding

‘IsDupe’: myIsDupe to the query options part, but it doesn’t seem to be working out.

I tried what joel said but it just straight out didn’t work.

Anyone?

Remy said on October 18, 2006:

@Anon – I would recommend adding some debug to the ‘myIsDupe’ function.

If you’re using Firefox with Firebug or Safari, change the function to read:

function myIsDupe(latest, last) {
consoloe.log(“dupe check”, (latest.id > last.id), latest.id, last.id);
if (latest.id > last.id) {
return false;
} else {
return true;
}
}

zsolti said on October 19, 2006:

Hi,
Is anybody know, why it doesn’t work, if there are other onLoad functions on the same page?

You can see here:
http://www.blogsearch.hu/index_alpha.html
For example at the left side, in the “alpha, be patient” box the spam-prevented e-mail dosn’t turn into the right format. However, here, where I don’t use the spy.js, and the jquery.js, it works fine:
http://www.blogsearch.hu/belepes

Thanks, and sry for the long post…

khelllo said on November 7, 2006:

i have a problem when i use this script with others who use prototype library…

do u suggest any solution for this?

khelllo said on November 8, 2006:

thanks,
really it worked well with me….

but now am facing and i hope it’s the last problem with Rico library!

there r another conflicts with Rico library and spy plugin…..

so do u suggest anything???

Micky Bou said on December 15, 2006:

Well, you can see your excellent Spy application working on our main page. We added you to the site credits. Keep on the good work!

Remy said on December 15, 2006:

@Micky – that’s great to hear my Spy is getting out in to the wild!

Nilesh S. Tailor said on January 17, 2007:

Any suggestions on how to do this on ASP.net

Logan Koester said on January 21, 2007:

Has anyone noticed any resource consumption problems leaving this running for an extended period of time?

If I leave my page open in Firefox and come back a few hours later I find it is using 99% CPU and 200MB of RAM. I don’t know if it is the Spy or some other js I’m using, however.

Remy Sharp said on January 22, 2007:

@Logan – I have noticed something similar, not particularly with the spy, but with repetitive AJAX requests – the if I am using Firebug in Firefox, and have the XMLHTTPRequests turned on, the CPU goes pretty wild after a few hours.

I would run the same check in IE, or check the flag (if you’re using Firebug). Let me know how you get on.

Nilesh S. Tailor said on January 24, 2007:

Great job, thanks.

dragon said on February 26, 2007:

any coder out there wanna do a walk through on how to get this to work with e107?

cmelb said on March 23, 2007:

Do you have any recommendations for timeout frequency. Anthing under a 1000ms seems to send the scroll a tad doolally in firefox (IE sems to handle it better).

Remy Sharp said on March 30, 2007:

@cmelb – ideally you should be pulling back more than one row of content on each request.

Though the spy appears to be real time to the user, behind the scenes it should actually be showing slightly “stale” data – otherwise you have to poll at a frequency that will give your user’s browser problems.

I would recommend every 1 second, but make sure I return 10 or so rows of content, as each of these are added to the spy on each second, rather than polling each second, it ends up polling every 10 seconds (i.e. 10 rows x 1 second).

Hope that helps.

entropie said on April 7, 2007:

Thanks for sharing.

I edited the plugin to fade the first elements out and pushing to the end of the list. (chatlike).

if anyone is intrested mail mit to mictro / this fancy ‘a’/ gmail.com

acidofil said on May 11, 2007:

Hi there, there’s small bug in myTimestamp example function as d.getMonth() is counting months from zero.. so myTimestamp returns date one month ago ..

akomabantot said on June 18, 2007:

how can i parse the content in a div?

IgoS said on June 21, 2007:

I want to create something like http://www.ubuntustats.com/. How to create 2 or more spy instances?

Bernie said on June 22, 2007:

First I'd like to say this is extremely cool, and very easy to implement, even for a dunce like myself.

Now, predictably, I have a question!

I would like to not 'loop' the results returned, but rather keep the list 'static' (or rather, just stack new results on top of the old results).

What I mean by this, is if I query the database now, and it returns three additional entries to add to the output, it will loop those three entries, until another query is made.

I would like it to pull those entries and display them, then stop, and wait for more results, not loop through the results until the next query is run.

What I want to do is have a "live" list of forum posts on my website, and it's sort-of working, but it looks odd looping through three results (for example) when I have a list of seven. I'd like it to just add those three to the list (bumping off the oldest three), then stop and wait.

Could someone advise on how to achieve this?

Remy Sharp said on June 26, 2007:

@Bernie - I've added a new option to the spy settings to control the speed the new items are pushed on to the stack.

This was the easiest way to achieve what you're after, otherwise it would require a significant change to some of the internal spy code.

If you download the latest spy and follow the instructions for the pushTimeout it should do the trick.

Let me know how you get on.

Bernie said on June 26, 2007:

Thanks, is pushTimeout in seconds?

Remy Sharp said on June 26, 2007:

@Bernie - milliseconds. However, if you want to make the items appear almost all at once, set it to '1' - this is lowest timeout you can give it (the spy won't register zero).

Emin said on June 27, 2007:

When i got data i want to start javascript function for edit new data.
How can i start to work javascript function automatically when data got

Ryan Lynch said on June 30, 2007:

I have been working with this for a few hours, and it's really impressive. Say, would there be an easy way to do alternating row colors?
Best,
Ryan

Luke Byrne said on July 12, 2007:

Remy,

Fantasic piece of work! I have been playing around with it and is very easy to implement.

I have a requirement whereby I want to pass some paramaters from a form that is being monitored to the ajax page via the specified URL, much the same as the actual Digg Spy.

I am under the impression that AJAX URL is loaded once and from then on it is set? Would that be correct?

If so, can you envisage a way whereby I am able to pass form parameters to the server side page?

I look forward to your response.

Kind regards,

Luke Byrne

Linus said on July 14, 2007:

how can i use it with jsp instead of php?
how should the source code be?

thanks

murki said on July 14, 2007:

Is there any way to use the "ajax" parameter with an external (different server) URL in firefox? the firebug gives a "uncaught exception: Permission denied to call method XMLHttpRequest.open" error.

Remy Sharp said on July 14, 2007:

@Ryan - yes. It's not ideal, but something like this will do it (note that I've not tested this - but it's along these lines):

Use the custom push method and colour the new DIV's background depending on whether it's an odd or even DIV:

function custom_push(html) {
  // Prepend the HTML inside the container
  $('#' + this.id).prepend(html);
  var l = $('#' + this.id + ' > div').length;
  if (l % 2 != 0) { // then it's odd
    $('#' + this.id + ' > div:last').css('backgroundColor', '#c00');
  }
}

@Luke - It should be fine - but you'll need to mod the spy.js file, and the code I've suggested is a bit of hack, bit it should work.

You need to add a new method inside the spy (within the return this.each(function () { block) to change the Ajax URL (I assume you have some kind of click handlers when the user changes the form settings).

var e = this;
e.setAjaxURL = function (url) { o.ajax = url; };
... // continue as normal

Then to change the URL:

$('#spyContainer')[0].setAjaxURL(url);

@Linus - sorry, no idea. Have you tried Google?

@Murki - that's a browser feature. Your browser won't let you access content, through scripting, from different URLs (i.e. the protocol and hostname must match the source request). What you're trying to do is called Cross Site Scripting

murki said on July 16, 2007:

So, in order to overcome that problem, I should implement some kind of "proxy script" to retrieve the feed, right?
Something like a php file (in my server) outputting the json feed with

print file_get_contents($json_feed_url)

...

murki said on July 16, 2007:

I have another question... It says that the code automatically handles "multiple items from AJAX request and shows one at a time, reducing the load on your servers", so how do we take advantage from this using JSON replies? I mean, you have to override the 'push' function, but how do we define the rows in a JSON request so they can be handled automatically? Am I missing something here?

murki said on July 17, 2007:

Seems http://leftlogic.com/jquery/spy_json.html doesn't update unless you have firebug enabled

Remy Sharp said on July 17, 2007:

@Murki - you should find the JSON spy working now. You're right, I had a specific console.log which was breaking in other browsers.

In reply to how you could get around the cross site scripting - yes, you could create a proxy-type script - that would overcome the problem.

murki said on July 20, 2007:

Thanks a bunch Remy!

Is there any chance you could help with my previous question? About setting/displaying multiple items with JSON?

murki said on July 24, 2007:

What is the right way to "return a selection of HTML DIVs to be inserted in to the 'spyContainer' in a spy-style" for JSON response types? If the JSON feed returns a lot of items, should I loop through them and build the HTML rows in the 'push' function? (Because it's not working for me that way).

cactus said on July 27, 2007:

IgoS said on June 21, 2007:

I want to create something like http://www.ubuntustats.com/. How to create 2 or more spy instances?

To create more than two instance of spy, from what i have tried you will have to make a copy of the current spy.js, rename every instance of SPY to something else as it is a variable eg SPY2,

so in the new file you will have
var spy2Running = 1; instead of
var spyRunning = 1;

same goes for other variables

spy2.last, spy2.epoch, spy2.json etc

make another copy of out.php to out2.php for example. change the content of the divs so into something different.

also create a new copy of test.php to test2.php or any name. In the new file test2.php, call the jquery and instead of calling spy.js now call spy2.js

In the function section where you have

   $('#spyContainer').spy({'ajax': 'out.php' });

change the variable names and id to match what u previously named your variables in spy2.js and also make a new id for the div tag e.g.

   $('#spyContainer2').spy2({'ajax': 'out2.php' });

finally create a new php demo.php file to call both test.php and test2.php

to make it look like ubuntulivestats, instead of placing in a table you will want to place it in draggable divs.

miniajax.com. ajaxian.com u can find tutorials on draggable divs

good luck

Stelios said on August 29, 2007:

any live examples? the one on arsenal mania i dont think so is working!

mike said on October 30, 2007:

I've made a diff against spy 1.4 to make sure it will not allow multiple spy instances on the page with the same object ID. If .spy() is called again, the old timer will be cleared and the new one (with new settings) will take over. It should still allow multiple different spies on the same page, just not two of the same thing (I was having an issue where it would keep reloading the same ID over and over because of a .click() event changing the configuration settings) - this allows real-time changing of the settings (say, the AJAX URL) without spawning additional timers.

I also added in a Math.random() parameter to force reloads every call and changed $.post to $.get - those can easily be removed if desired. :) Essentially I just create an array that has a list of all the spy IDs that are called, if a dupe is detected, the old timer is disabled and the new one starts like normal. Oh, and I changed the epoch behavior, for some reason on my browser it wasn't reporting the right time. I don't see why it needed the spy.epoch calculation at all.

Here's my blog post about it with a link to the patch file:
http://michaelshadle.com/2007/10/29/jquery-spy-improvements/

Kenyansoul said on November 17, 2007:

How can you implement pause on mouse over?

Sven said on December 10, 2007:

Thank you, really great Script ...
I've got one problem, i tried to use a ajax-loader gif, cause my SPY starts with an empty div.

I tried to show thy gif with document.getElementById("xyz").innerHTML but it didnt work ... Can anyone help me ?

Sorry for my English ;-)

Jason Salas said on January 22, 2008:

Hi Remy,

Great work. Above, you mentioned that to give the script more of a realtime effect, you'd need to do polling to bring in fresh data according to some interval. The problem, might appear to be t he CPU cycles and RAM allocated for long-running scripts with repeated AJAX remote calls.

Nonetheless, could you add-in a property that would allow for programmatic polling? Maybe something like:

var o = {
        limit: (settings.limit || 10),
        fadeLast: (settings.fadeLast || 5),
        ajax: settings.ajax,
        timeout: (settings.timeout || 3000),
                poll: (settings.poll || 120000)   // refresh the data every 2 minutes
               // ....more code here

Thanks!
jason

Jason Salas said on January 22, 2008:

A-ha...I just noticed that AJAX calls through Spy are made automatically after the dataset iterates through all the members of the collection. This is a nice touch, but I still think a polling option would be helpful, too. :-)

Ali A. Akbar said on January 30, 2008:

Has anyone got this working with RSS yet?

Can't we just combine it with LastRSS and let it feed from there?

Cesar Prieto said on January 31, 2008:

Hi, from Spain, your project looks fantastic, thanks for sharing, my question:
I only work with ASP not .NET, how to adapt the code for work with my project. The project is a website for online auction where all the user can see on real-time all the bids from other users, normally auctions work only for one hour each.
Thanks, and great job.

David Jacques-Louis said on February 9, 2008:

Great. Need more demos

Remy Sharp said on February 26, 2008:

@Ali A. Akbar - I'm pretty sure you should be able to combine it with Google's feed API - then you could have quite a nice effect.

Luiz said on March 11, 2008:

Remy,

What your suggestion about SQL statements with timestamp ?

Luiz said on March 11, 2008:

Remy,

The fadeIn/last using json dosen't work. Because?

joe2 said on March 17, 2008:

Has anyone got this working with RSS yet?

joe2 said on March 17, 2008:

How can you implement pause on mouse over?

Remy Sharp said on March 17, 2008:

@joe2 - for RSS feeds, I can suggest pushing your feed through http://pipes.yahoo.com to have it converted to JSON - then add your own parser in the spy.

On your second question - try adding something like this:

$('#spyContainer').mouseover(pauseSpy).mouseout(playSpy);

Owen said on March 20, 2008:

If anyone is having a problem with JSON not working in IE, make sure your JSON code it perfect or it will not work. I got stuck at that for a few days.

ryancoughlin said on April 3, 2008:

Hey, I have been reading these comments and can't seem to figure out why my code isnt working. Remy do you hae any ideas?

http://rksdesignstudios.com/magnet/spyTest.php

That is my test page that I am trying to get it working on, it is setting the display to none if you view the generated source code.

I am not sure why, I have read through your doc's. Anything I can do to it?

Thanks a ton! Great work as well :)

Ryan Coughlin

ryancoughlin said on April 3, 2008:

If you need any of my PHP to look at if thats the case, let me know!

Thanks again,

Ryan

Owen McAteer said on April 7, 2008:

In response to Luiz question about fadeIn not working for JSON, you need to give your custom div the style property of "display:none"

Axel said on May 21, 2008:

Hello Remy!
I have the same 'problem' like Bernie.
I don't want to loop thoose database entries untill another querry is made. Stop and wait.

What shall I do ?
Anyway, great script!

jj said on May 29, 2008:

is there any way to stop looping?

Francesco said on June 13, 2008:

My little example:
http://yagg.org/spy.php

Kaiser said on June 26, 2008:

Ey, GREAT work! This script was exactly what I was looking for ;)

I´ve done some modifications to the original code to avoid repeated items to appear simultaneously, not only consecutively. It´s not a very "portable" patch, because it requires to define new elements in a javascript array to control if the new "appearing" item is already printed on screen.

But anyway, if you are interested, just take a look:
http://www.xonsuns.com

Morad said on June 27, 2008:

Remy,
In my "outjson.php" file, I load my source xml file with simplexmlload_file.. Then I pull random array from the xml file..and pass it as json.title.. also I pass json.num which holds unique id

that works fine and everything, but then my spy doesn't loop after it displays all xml arrays.

Any clue?

People Search Dude said on July 24, 2008:

This is so off the hook...however, I have the same question as JJ about the looping.

Good work however...

Left Logic - Brighton web development, design and consultancy

Style

Navigation