How to create a Digg-like spy with no work at all
August 14th, 2006The jQuery spy has now been superseded by the upgraded jQuery spy. Please see the upgraded jQuery spy article for details.
Yes, that's right kids, you know you love that Digg spy effect. The real time news, as it rolls in, and how cool does it look? It looks pretty cool, that's how.
Now there is a plug-in for jQuery that means you can have your very own spy, for anything, with only a few lines of JavaScript†.
† Obviously you'll need something on the server side to respond to the AJAX request, but really it's still kinda only a few lines...
Download the jQuery spy plug-in
Prerequisites
Okay campers, here's what you'll need to make this work:
- A container div to hold your 'spy' - this can be pre-filled or empty
- A server side script that returns the HTML of your spied content (article or what have you) - note that this can only be one row at a time (something I may work to upgrade)
- A single div must wrap the content returned from the AJAX (or AJAH) - and it should contain a style attribute of 'display: none;' - this is explained in more detail later
The Examples
Import the plug-in, and call the code on the spy contain div (I'm assuming you've imported jQuery already).
Simple eh?
<script type="text/javascript" src="spy.js"></script>
<script type="text/javascript">
$(document).ready(function() {
$('#spyContainer').spy({'ajax': '/ajax/news.php'})
});
</script>
<div id="spyContainer"></div>Here are a couple of examples of the spy in action.
- Example spy with dummy data
- The AJAX URL being used to return the dummy data (view the source of the page)
- Real life example on Arsenal-Now.com
Usage
Now you've seen the examples, I can hear some of you thinking: "that's all well and good, but what if I don't want to return the HTML, what if I want to return JSON?". No problem, use a custom push method.
Simple Parameters
All the parameters (the few that there are) are passed in via object notation (i.e. with curly brackets).
- limit - default 10 - the total number of rows to show
- fadeLast - default 5 - the number of rows to fade out, i.e. the last 5 rows
- ajax - required - the URL to server side script that will return the content
- fadeInSpeed - default 'slow' - the speed, in jQuery notation to use for the first row to fade in with (currently: crawl, xslow, slow, medium, fast, xfast, normal - though note that the pre-version 1 jQuery only currently supports slow, normal and fast)
- timeout - default 3000 - the time in milliseconds between requests for a new row
For example:
$(document).ready(function() {
$('#spyContainer').spy(
{ 'limit': 25,
'fadeLast': 3,
'ajax': '/jquery_spy/out_json.php',
'fadeInSpeed': 'slow',
'timeout': 3000 }) // 3 seconds
});Custom Push
If you don't want to return HTML to the spy, you can write your own push method.
Your custom method will have access to 'this' which is the DOM element (i.e. the contain div) and 'r' which is the response in plain text.
Here is an example using JSON:
$(document).ready(function() {
$('#spyContainer').spy(
{ 'ajax': '/jquery_spy/out_json.php',
'push': custom_push });
});
function custom_push(r) {
eval("var json = " + r); // convert to JSON
// I'm being lazy here, but you get the idea:
// Build the HTML
var html = '<div style="display:none">'
html += '<span class="ctr">' + json.num;
html += '</span><span class="content">';
html += json.string + '</span></div>';
// Prepend the HTML inside the container
$('#' + this.id).prepend(html);
}See the custom JSON push in action.
Pause & Play Buttons
Included in the plug-in are pause: pauseSpy() and play: playSpy() functions.
These will, as the digg spy does, pause the flow of new content and start it up again. You can see these in use in the simple example
Spy Detail
This section is a bunch of quick bullet points to help explain a few key points.
- Each new row should be wrapped in a single DIV with the style coded inline to 'display:none;'
- The plug-in is specifically searching for the child div elements to the container to correctly limit and fade the rows away
- The 'display: none' is required to fade the new row in properly, otherwise you might see some strange flashing effects
- The spy only supports one row at a time. If I get requests to upgrade this, I'll take a look
- The AJAX request is made using a POST
- The AJAX request is posted one variable: timestamp - note that this is not the current time, and in fact the time of the last request - so that you may grab the latest row in a database after the timestamp
- If the same content appears more than once in sequence, it is ignored, i.e. don't show the same row twice
- I have purposely tried to keep this plug-in simple and easy to use, as this is the fun nature behind jQuery (that I interpreted)
For accessibility, I would recommend loading the page you plan to use the spy on, with the limit number of rows (e.g. 10 news stories), and detect whether JavaScript is enabled. If it is not, using a noscript tag, insert a 'meta refresh' tag and display a message explaining the page will auto-refresh to offer the latest content.
If you plan to pre-load the page with the rows, you can apply the fade down effect, as shown in the code example below.
<script type="text/javascript">
// the index starts at zero - so fade down all
// the divs after the 5th one
$('#spyContainer > div:gt(4)').fadeEachDown();
</script>
<noscript>
This browser does not support JavaScript -
which is used to automatically update the content.
Therefore this page will automatically refresh every 1 minute.
<meta http-equiv="refresh" content="60" />
</noscript>Feel free to add comments, suggestions, ask questions, point out any errors - and I will try my best to get back to you.
Dude I was looking at your Jquery thing - is this the file that gets your content?
/jquery_spy/out.php
The thing I don't get about AJAX is how it queries the databse on the fly.
Chris - The 'AJAX' response doesn't have to be intimidating, think of the page (in my example out.php) as a very mini HTML page.
The requested script my be something:
[psudocode]
$row = query("select * from articles where timestamp > $POST['timestamp'] order by timestamp desc limit1");
printhtml($row);
[/psduocode]
The AJAX part is done in the browser, just calling your script and processing the output - and in the simplest case, adding the HTML to the page.
This look great ! i wonder how i could apply this for my Joomla! website..is it possible to pull out data from an rss/xml feed ?
Looks awesome!
Awesome script! Very cool effect. Though it is worth mentioning (and I'm sure the author knows this but left it out for the sake of simplicity) that pinging the server every single time a new item scans down isn't a good idea. Every iteration of the animation a POST is done to the server to get the new item. Digg's spy uses a slightly more complicated batch retrieval of items, so there isn't a 1:1 ratio of postbacks to items. I also don't think digg actually displays every single item, it shows enough items to give the perception of a real-time display. If the "digg throughput" exceeds the animation speed, there's no way for it show them all :)
Ben - yes, absolutely, you can write a short push function and parse the RSS to grab the latest news.
Zac - yep - I certainly don't expect Digg to switch to this any time quickly ;-) but it would be interesting to look in to throttling the processing.
Could you perhaps show us the actual PHP code for this example/how to write a push function to parse the RSS?
Here's the PHP used for the example:
http://leftlogic.com/jquery_spy/out.php.txt
As for a push function for the RSS, I'll post a follow up soon (I'm on hols soon - so next week).
i was also wondering how to get this to display a rss feed, if you could help me out i would be for ever greatfull. thanks
Javascript!? Heard it doesn't add search engine value to the site..
Is this available in PHP or so?
Would be nice if it it was possible to show what page people are currently viewing instead of the whole content /rss. I mean showing an RSS feed is kind of pointless because the RSS feed, for the most part, is stable. It generally only gets updated hourly (if not longer) - so it would show like 8 items and then fade away or loop again showing them, just use a marquee with some javascript fading to show the rss feed - very simple to create.
What I'd like to do with this, is show what page of a blog people are viewing - so as ppl click you can view what pages are being viewed. Problem is that my blog is flatfile with server side.. so i don't use a database which makes it more difficult.
I'm trying to get my head around how one would pull rows from a database based on a timestamp. I know the implications of such a method, but being able to display the latest updated rows would be very neat indeed.
Any ideas?
Great tutorial regardless! Thanks a lot.
@Ben Borges
Wait for a new joomla module for this! Perhaps next week. Check out www.webpr.gr/joomla for updates.
@Remy Sharp
Excellent work. When our Joomla implementation is finished, I will send you a link, if it's OK with you.
Looks great, tnx u so much. I m going to try it for my blog (:
Great piece of tutorial, hope I will be able to put to use here.
Speaking to John Resig of jQuery, you can use 'fadeInSpeed: 1400' to make the fade in effect uber smooth and slow.
@Sam
That's what I was curious about too, but after examining the source code, I realized how the concept works. What happens is the JavaScript code sends a timestamp among other things to a script you've created in the language of your choice (I used Perl) as often as you specify (in this case abut a second or two) and for the example I created, the Perl code selects the row from a field in a table that has a timestamp later than the one sent by the JavaScript and sends that back to the spy page. Hope this helps.
@Henry
Thanks for the reply Henry, I figured it out and basically did the same as you but in PHP.
I decided to try out your script -- great work! I implemented it using my forum's database as recent data as to provide a dynamic feed of recent posts on the main page! Take a look:
http://programming-designs.com
Great tutorial. I have tried the code and it worked on my PC. But it doesn't work on the Linux server. Help!
Can you show the actual code for
/jqueryspy/outjson.php? I wonder what is the difference between your out.php and out_json.php,
It really is a drag that the output file needs to... output one item at a time. As you said the ajax pings the output file in fixed intervals, requesting for new items.
So, this makes it somewhat difficult to apply this script in a real situation, in a CMS. It is practical only for RSS feeds or sites with huge input, like Digg. You cannot use it to display in cyclic order the 10 most latest items in your website!
What is nice about the script -and I guess everyone agrees on that- is the effect produced. So why not modify it to "read" several items at a time and display them in some order (by date, alphabetical etc.)...
So at the moment I can't find a practical implementation of the script -although I'd love to!!- inside Joomla, as I previously rushed in to do.
Don't get me wrong, it's very good what you did, but unless it has practical applications you won't be seeing it implemented in many websites.
Even the "demo" site you provide (Arsenal-Now) seems to fail to deliver (e.g. it displays double items)...
Hi, I wanted to preload the latest 30 news entries then it to keep updating as new ones where made and fade out the old ones as new ones are made. So it keeps only 30 on screen.
How is this possible with the script?
Thanks, Jack.
I would certainly alos like to see the php push for rss...
Remy, what do you recommend for the issue of not wanting to query the database constantly? I'd LOVE to get something like this working on our forums, but with over 2.5 million posts, have EACH reader of the page that this jSpy is on, querying the database ever 2-3 seconds, we are going to kill the system. Even with just me doing that on a sample testing page, I drove down system performance pretty badly.
Two ideas I had were to query the database once every 5-10 minutes with a PHP script to get the last 150 posts (5 minutes at 2 seconds each) and put that into either a flat file or populate a small database. Then each user would either access the flat file via the "out.php" file or query this much smaller db in the "out.php" file. The only downside is that the only way to "cycle" through the results would be to keep a cookie on the users system to tell the out.php file what row of the db or file it last saw, so the out.php can grab the next one.
Do you see any other options for that?
Okay, after looking at this page some more, i see that you have something you say: if you want to preload the page with rows, use this code...
Well, that sounds like what I want to do, but I can't seem to get it to work. Where should the script code go? In the "out.php" file or in the "spy.html" file? And what is "#spyContainer"? Does that need to be renamed to something? And do the div's need to be named something with an index? (it has "div:gt(4)" there).
thanks for any help!
Hi,
Though I've replied to most of these comments directly via email - I thought I might summarise my suggestions/feedback:
@Simon - here's the source to the json example:
http://leftlogic.com/jqueryspy/outjson.php.txt
@Jack + @Ryan - you should pre-populate the page as if you didn't have the spy functionality. So long as each item is in a container div (the spy ignores the class or ids) and the collection of items is in a single container div that has an ID that you pass to the spy function it will work correctly (see the 'spy detail' code example).
You don't need any naming on the item divs - the 'div:gt(4)' means all the divs after the 5th one.
@Fotis - you've got my long reply! But for those who had similar questions - I will be looking at a version of the spy that reads several items in at once in the near future.
For those of you who want to use the Spy to track user activity on a website, I've set this up:
Arsenal Now site activity
Hi Jonathan,
Could you please share how did you achieve this.... I am having a real hard time querying the DB. the bugger just dosen't work. Displays the first listing and then nothing after that.
If you can help out, that would be great!
my email is:
Matt55_22@Yahoo.com
thanks.
Hi
I did a simple example of out_json.php only in perl/cgi. It had three different lines of text which it randomly selected and print. Now to my problem, it printed the same line of text two times in a sequence.
That shouldn't be possible should it?
I'm pretty new to javascript but I can't seem to see the point in this:
var epoch = new Date(1970, 1, 1);
and then minus'ing that from the current date/time. It just seems to make the timestamp a month late...
Maybe someone can help me out with this. In a php file, how would I define the spycontainer around a variable? It will then be parsed out in a template.
Hey, great tool!. I've just finished this site (http://www.hayequipo.com/) for football (soccer) discussion. The "spy" shows users comments and blogs trackbacks, a lot of things can be done with this.
Only one problem, the spy must support more than one row, that's the only thing left.
I've just find something strange: the $_POST timestamp is wrong.
I was trying to grab more than one row at a time and find out that the $_POST timestamp give me this value 1155370514 (08/12/06)and the timestamp php function 1158059700 (09/12/06) in the same moment.
Is this a problem in my server configuration? Really don't know, but I can't grab more than one row because the timestamp is one month behing the real date.
I don't understand much about JS, that's why I can't modify the code, but the timestamp() in PHP is different than the $_POST["timestamp"] value.
Hope you can help me in this one, greetings.
Alejandro Sena:
I found the same thing it's the minusing of the 'epoch' which causes the timestamp to be a month late... Very confusing...
time() - $_POST['timestamp'] is the amount of seconds missing.
$_POST['timestamp'] + that number of seconds will give you the actual timestamp, just substract the refresh time and you have the correct time.
Something like this (in a 5 seconds refresh period):
$correct = $POST['timestamp'] + ((time() - $POST['timestamp']) - 5);
$correct will be the actual time minus your refresh time. You can put just $correct = time() - 5 and the result will be the same.
It's not the right way to do it, but well, just have to wait for the creator answer. At least now I can load any number of rows at a time =)
I've just written my own version of this, it supports multiple rows and doesn't require jQuery so i'm happy.
Sooo.... what are you waiting to share it?
This will probably solve the timestamp problem correctly:
var timestamp = Date.parse(now) / 1000;
That timestamp is the same that the time() function of PHP.
The problem is that the specified month for epoch is 1, which is February (Jan is 0, Feb is 1, Mar is 2 etc). All you have to do is to replace
var epoch = new Date(1970, 1, 1);
with
var epoch = new Date(1970, 0, 1);
It works on me! Thanks for the great plugin :)
Hi all,
Thanks for pointing the bug in the timestamp. I'll update the source code on this site very shortly.
I've also been working on a new version of the spy that support multiple rows from the AJAX request and also supports custom timestamp functions.
It also supports collecting multiple items from the AJAX, and showing them at the specified timeout interval - giving the appearance of 'real-time'.
I'm just ironing out the last of the bugs in the timeout "thread" and then I'll release it.
It does use the version 1+ of jQuery - you will need to upgrade.
I'll be in touch soon!
If the script can collect multiple items the only thing left is to do it from an XML file. That way you release a lot of server resources.
I'm working on that but in other script, if everything goes right soon I'll post it here.
Greetings.
Softius, the difference between the time() in PHP and the timestamp of the script (with the fixed epoch) is ~10800 (abour 3 hours).
There must be something else. If you use a one-row system it's impossible for you to notice, 'cause the script don't repeat the last row, but in a multiple system (like the one that I'm trying to do) gives 3hs ago results with the epoch fix.
Crazy, uh?
I found that even a slightly incorrect time on a users machine can mean that they miss rows because their timestamp is different to the one on the server. My solution is to send out a request to compare the users time to that of the server and then return the difference which is used to adjust the timestamp sent by further request.
Darkip, the difference with the users time changes all the time. Now is 2 seconds and in a few seconds is 3, 4 or even 10.
The solution is to store the last timestamp sended by spy.php (the database timestamp) and in the next query send the newest compared to it. Never use the client time, always the database one, just store the last time sended to the client.
Hi, I’m using this script and it works great, however is there anyway to keep the news from starting over? I want it to wait until there is a new news entry to add another ”row”.
Help would reeeally be appreciated. Thanks in advance!
The latest version of the jquery spy supports multiple rows – and should do the job.
Its cool.. but, I had a same problem as described by ryan:
“Too many requests to the db.. solution – Two ideas I had were to query the database once every 5-10 minutes with a PHP script to get the last 150 posts (5 minutes at 2 seconds each) and put that into either a flat file or populate a small database. Then each user would either access the flat file via the “out.php” file or query this much smaller db in the “out.php” file”
@Varun – if you use the latest jQuery spy you see that you can return multiple items from the AJAX request reducing the load on the database.
Is it possible to add sound alert with every new entry ?
Hi there, I wonder if there is a way to run the script twice in a page (no iframes) ?
I have madethis in ASP.But i get problem while update database..Or any new data entry ocurre.I use prototype.js …Any one have suggestion..
I want add RSS! but i can’t :(
please help me!
Hi Remy, this script looks great. Although I haven’t tried to implement it quite yet (I’ll wait till I’m on the clock =) ), I was wondering if you had an example similar to that of Arsenal-Now.com’s site where they show the spy in action, with reference to particular URLS that people are visiting – I would like to know how to use this same functionality so that I can get a live view of where users are at, at any given time on our site (aka live webstats). Thanks!
– Collin
ps – email me the response if easier for you seeing as its probably really indepth.
Como hago funcionar el script de php out.php? disculpen sino escribo en ingles
love the script. how can i make it pull from an xml file?
@priscilla – you can, but you’ll need the latest jQuery spy and you’ll have to patch the ‘spy.parseItem’ method to support XML.
Let me know if you get stuck, I might be able to help.
Man, great Script !
There's just something i dont understand, my Script is always in a loop. What i mean is it takes if there are 5 entrys in the DB it shows these 5 and startswith 1 again, how can i stop this, so that evere entry isshown once and the sriptonly updates, when there is something new?
Sven, did you ever figure this out? I'd like to do the same.
How can we show text in input ?
for example i tried something like below but it dind't work;
$(document).ready(function() {
$('#text1').spy({'ajax': '/get.php'})
});
hi, can provide a tutorial for xml data? will be useful for rss
@Sven + Linus - if you check out the latest jQuery spy you might be able to achieve what you're after.
would anyone be interested in working on a memcached version of this. If 300 people are viewing the spy at the same time i think that would be pretty DB intensive.