To get this project rolling, we are going to start by building a really basic queue system structure. It will start of simple and not very cool, and over the next few blogs we will evolve it into awesomesauce.

What is a Queue?

A queue is just a list of things that need to get done, just like when you stand in line at the cafe and the counter girl can only take one order at a time. Imagine the line of people as an Array, and the counter girl some processor function, and the entire store itself as a giant loop that always exists. The only difference is instead of selling us coffee the counter girl is going to look up website IP addresses for us.

Here is what the initial queue system is going to look like before we evolve it in later posts.

 


To prevent the loop from using all our CPU, that sleep function will cause it to pause for 10 seconds. Later on we will replace that sleep with something much cooler, but for now it will do the job.

The initial loop

In the most basic form stripped down to basic, here is what the queue system looks like in code.

PHP Code:
<?php

//. this function is our counter girl.
function process() {
    return;
}

//. this queue is our line up.
$queue = array();

//. and this is the store itself which is open 24/7.
while(1) {
    
process();    
    
sleep(10);
}

?>

We have an empty processor function, an empty queue array, and an infinite loop to process items. The next step is to modify the loop so that it actually checks if the array has jobs. If it does have a job, it will process it and will need remove it from the queue.

Handling the queue

To do this we will take advantage of PHP’s ability to iterate over arrays. If you are unfamiliar, basically PHP arrays have little pointers inside that point at the first element in the array. You can then ask for what is being pointed at, and you can slide the pointer up and down the array. In this case we have an array with the pointer at the beginning, and we are just going to slide down the array until we hit the end.

PHP Code:
<?php

while(1) {

    
// fetch the job the array pointer is on. if we are
    // at the end of the array we will get nothing back
    
$job current($queue);
    
$jobkey key($queue);
    
    if(
$job) {
        echo 
'processing job '$jobPHP_EOL;
        
        
process($job);
        
        
// push the array pointer to the next job
        
next($queue);

        
// release the completed job to free memory
        
unset($job,$queue[$jobkey]);
    } else {
        echo 
'no jobs to do - waiting...'PHP_EOL;
        
sleep(10);    
    }

}

?>

Now the loop will ask the array for the current job to process via the current() function. If there is no job we will get nothing back so we will sleep for 10 seconds.

If we do have a job, we will process it and then delete it from the queue array to free up memory. Where current() fetches the current element of the array, key() fetches the array key of the current element. With the key we can delete the specific element we want Рin this case is the one we just processed. Then we push the array pointer to the next job with next() and the loop begins again to process the next job.

All together so far, this is what the full program looks like.

PHP Code:
<?php

function process($job) {
    return;
}

$queue = array();

while(
1) {

    
$job current($queue);
    
$jobkey key($queue);
    if(
$job) {
        echo 
'processing job '$jobPHP_EOL;
        
        
process($job);
        
        
next($queue);
        unset(
$job,$queue[$jobkey]);        
    } else {
        echo 
'no jobs to do - waiting...'PHP_EOL;
        
sleep(10);    
    }

}

?>

Running this script will result in every 10 seconds seeing the output “no jobs to do – waiting…” as we have done nothing to allow our queue server to accept input. In the next post we will deal with fetching input so that we can actually add things to the queue. There are many ways to fetch input (files, databases, etc) and the method we are going to use is called a Named Pipe. Named Pipes are special files that operate on the FIFO principle, First In, First Out. This fits perfect since our queue system should process the first thing added to it first.