posts with the tag: queue

Posted by: bob on Thursday July 28th 2011 1 comment
Tags: fork, queue

The next logical step in the development of our queue service is to allow the application to push itself into the background of a system and run detached from a terminal.

What is forking?

Forking is when you take one process, and clone it to create another with the same state as the original. When our application executes we will clone it, let the new copy take ownership of itself, and terminate the original. This leaves a background process or daemon running in the background still ready to process our queue.

PHP by default does not have the functions we need enabled. In order to enable them PHP will need to be compiled with the option –enable-pcntl. This will make the pcntl_* function set available to us.

Wanna fork?

Forking is a fairly simple process. The best time to fork is early on in the program, because when we do it literally everything about the application state gets copied over. This means file handles, database pointers, everything… and this usually causes problems. Fork before you open your files and databases, and you will find your life running much smoother.

Continue reading about forking our process

Posted by: bob on Wednesday July 27th 2011 2 comments
Tags: named pipe, queue

In this part of the series we will add the ability to accept input from named pipes to the queue server, an important feature so that the queue can actually be given things to do. Again, I would like to point out that it is assumed we are running on a Unixesque operating system such as Linux, BSD, or Solaris.

What are Named Pipes?

Named Pipes are special files that the kernel can create allowing for buffered byte streams. They operate on the principle of FIFO, where the first thing put into it is the first thing it spits out. This is perfect for our queue system as we want to process the first thing put into it first. To store data into a named pipe, it is as easy as `echo lol > /path/to/pipe` and reading the data back out is as easy as `cat /path/to/pipe`. An interesting note about named pipes is that by default they block until there is both a reader and a writer. This means the echo command will hang until another terminal is opened and the cat command is executed.

Here is a quick video demonstrating how the queue system will work by the end of this post. We will be able to launch the server, and send input to it from another terminal via the named pipe.

Continue reading about using Named Pipes

Posted by: bob on Tuesday July 26th 2011 no comments
Tags: array, iteration, queue

To get this project rolling, we are going to start by building a really basic queue system structure. It will start of simple and not very cool, and over the next few blogs we will evolve it into awesomesauce.

What is a Queue?

A queue is just a list of things that need to get done, just like when you stand in line at the cafe and the counter girl can only take one order at a time. Imagine the line of people as an Array, and the counter girl some processor function, and the entire store itself as a giant loop that always exists. The only difference is instead of selling us coffee the counter girl is going to look up website IP addresses for us.

Here is what the initial queue system is going to look like before we evolve it in later posts.

 

Continue reading about building the base queue loop…

Posted by: bob on Tuesday July 26th 2011 no comments
Tags: daemon, fork, named pipe, queue

So to be hip, I am going to write one of these multipart blog things that describe building a simple project from start to finish. Are you ready? O.K. here we go.

What is a Queue Server?

A queue server is a server that processes a queue, which is why it is so aptly named. You send it a list of things to do and it does them over time based on your parameters, be it “as fast as possible” or “when system resources allow” or “once an hour” it is your call.

Over the course of the next week or so I am going to demonstrate building a simple queue server that will be able to fork itself off as a background daemon, accept input through a named pipe, and process things as fast as it can until it runs out of things to do – and when that happens it will sit there waiting with the upmost desire for you to provide it with more jobs.

Continue reading about this project…