Overview of the Application

Let me summarise the functionality of the application, incase you just want to try it out. The app allows you to enter PHP code and then execute it without having to save it to file. You see the results instantly in the bottom half of the window.

There is also a built-in, simple (single-threaded) web server. This means you can view the results of execution in your browser—again, without having to save the file.

I’m going to explain some of the technical aspects of putting the app together. If you’re just interested in using the app, scroll down to the bottom of the post for the download link.

Working with the Command Line from Java

PHP can be run from the command line. Details about this are on the PHP website, but in short, if you run php with no arguments, then it will take in source code from the standard input, execute it, and print out the result.

This can be achieve programatically from within the Java environment:

// Get the runtime object
Runtime rt = Runtime.getRuntime();
 
// Run the command
Process process = rt.exec(PHP_COMMAND);
 
// Setup buffered input and output streams
BufferedReader input = new BufferedReader(new InputStreamReader(process.getInputStream()));
BufferedWriter output = new BufferedWriter(new OutputStreamWriter(process.getOutputStream()));
 
// Send the source code and close the stream (to make EOF)
output.write(code);
output.close();
 
// Read in the result, line by line and print it to the console
String line;
while((line=input.readLine()) != null) {
    System.out.println(line);
}
 
// Wait for the process to terminate (not really necessary, because the process will terminate once it has closed the input stream anyway)
int result = process.waitFor();

Note that rt.exec(...) and process.waitFor() can both potentially throw exceptions—IOException and InterruptedException respectively—which must be dealt with.

We could get the standard error stream aswell—using the same approach as with the output stream—but PHP doesn’t send anything to the error stream, so we don’t need to worry about it. If we wanted to do this, we’d need to read both streams in separate threads.

Building the Swing User Interface

Let’s look at how to setup the user interface. I’ll talk through a simplified version of the final application. We’ll look at a few of the Swing components individually.

JFrame

We’re going to make use of the JFrame class as our base. A common practice is to extend this class with your own, however, generally a more practical approach is to encapsulate the JFrame object by keeping track of the JFrame as a property within your class, and exposing methods as necessary.

We create a JFrame, set a layout (refer to the next section), a preferred size, the behaviour to associate with the window’s ‘close’ button, pack it (to make sure the components get arranged properly) and then finally make the frame visible to the user:

frame = new JFrame(FRAME_TITLE);
 
frame.setLayout(new BorderLayout());
frame.setPreferredSize(new Dimension(500, 500));
 
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
 
// Add components to the frame here
 
frame.pack();
frame.setVisible(true);

BorderLayout

Once the JFrame has been setup, we’ll use a BorderLayout to arrange our components on the window. Border layouts are a fairly common concept in user interface design. One component can be attached to each of five regions. Each region defines the position that the component appears at, and how its size changes as the parent component is resized. Border layouts can be nested within each other. The regions are:

• PAGE_START – generally at the top of the page, resizes to fill the width of the parent component.
• PAGE_END – opposite side to PAGE_START.
• LINE_START – generally on the left of the page (but actually dependent on the user’s language), resizes to fill the height of the parent minus the height of any components at PAGE_START and PAGE_END.
• LINE_END – opposite side to LINE_END.
• CENTER – the remaining space in the middle of the layout, resizes to fill the available space.

See the example below, which shows the positioning of the components:

JSplitPane

For our user interface we will divide the window into two: the top half for editing the PHP code, and the bottom half for viewing the results of the execution. We will allow the user to move the separator between the two components using a JSplitPane.

Using a JSplitPane is fairly straight forward. Pass each of the two components you’re splitting into the constructor, along with a specified orientation (either VERTICAL_SPLIT or HORIZONTAL_SPLIT). The location of the divider can also be given.

JSplitPane splitPane = new JSplitPane(JSplitPane.VERTICAL_SPLIT, inputPanel, outputPanel);
splitPane.setDividerLocation(200);

JTextArea and JScrollPane

Both the editor and the area for displaying execution results will be based on JTextAreas. On their own, these components don’t support scrolling (e.g., for when there are more lines of text than will fit in the component). To support scrolling, we need to wrap them with JScrollPanes.

JScrollPane inputScrollPane = new JScrollPane(inputBox);

JButton and Actions

When the user wants to execute their code, they will click on an ‘Execute’ button (which will be a JButton). There are a few ways to handle button clicks in Swing. One such method is to define an Action that will be associated with the button. This is a nice way to separate logic from the design of the user interface.

A button can be created like this:

JButton executeButton = new JButton(new ExecuteAction(inputBox, outputBox));

The easiest way to create the action is to extend from AbstractAction:

public class ExecuteAction extends AbstractAction {
 
    private JTextArea inputBox;
    private JTextArea outputBox;
 
    public ExecuteAction(JTextArea inputBox, JTextArea outputBox) {
 
        // Save these components for later
        this.inputBox = inputBox;
        this.outputBox = outputBox;
 
        // Set the name of the button
        putValue(AbstractAction.NAME, "Execute");
    }
 
    public void actionPerformed(ActionEvent e) {
        // Event will be handled here
    }
}

Any objects that the action needs to access get passed to the constructor, and stored as properties of the action for use later. In this case, we’ll want to be able to get the source code from the input box and then put the result into the output box.

Putting Together the PhpRunner Class

We will put our earlier PHP-running code into it’s own class and package (runner.PhpRunner). The class will be constructed with two parameters:

• The PHP code to be run
• An event listener (an object implementing the OutputConsumer interface)

The execute() method can then be called. This will start the execution, and notify the event listener every time we get a new line of output back. This approach means that if a PHP script takes a long time to complete, we’ll be able to see the output as it happens, rather than having to wait for the whole script to finish executing.

The code is similar to the code at the top of the post, so I won’t paste it again. I have added calls to the event listener though. Refer to the source code if you’re interested.

Threads and Thread Safety

Introducing threads into a Swing application can be dangerous, since Swing’s components are not thread safe. Only one thread should be allowed to access the Swing components at any one time—this is known as the event-dispatching thread (EDT).

However, we don’t want the user interface to freeze whilst the PHP code is being executed, so it’s important that we don’t wait for output from within the EDT. The swing library provides a SwingWorker class, which makes working with threads a bit easier and safer. Fortunately, our PhpRunner fits nicely into this pattern. A cut-down version of the code looks like this:

new SwingWorker<String, String>() {
 
    @Override
    protected String doInBackground() throws Exception {
        // Setup the PHP runner, and listen for any lines of output
        PhpRunner runner = new PhpRunner(code, new OutputConsumer() {
            @Override
            public void output(String line) {
                // Let the SwingWorker know there are more lines
                publish(line);
            }
        });
        // Execute, and return the result
        return runner.execute();
    }
 
    @Override
    protected void process(List<String> chunks) {
        // Add any output to the output box
        for (String chunk : chunks) {
            outputBox.append(chunk + "\n");
        }
    }
 
    @Override
    protected void done() {
        // The operation has completed
    }
}.execute();

Adding a Built-in Web Server

Sometimes it’s useful to be able to test the PHP output in a web browser. So we will add a very basic web server to the application. The user will be able to request execution of the code by loading a URL in their web browser.

First of all, we need to add a button to the user interface to allow the user to manually start and stop the web server. When the associated action is performed, we will either create a new HTTP server object, or stop an existing one.

The HTTP server doesn’t need to be very complicated at all. We don’t have to handle POST requests (just GET and HEAD requests) or even URLs since all requests to the server will execute the same script. We also only need to handle one request at a time—so the server can happily run on a single thread.

I don’t want to go into details of the server implementation here, but it’s worth nothing that this use of a web server does bring up the slightly unconventional approach of trying to get the contents of a Swing component from outside the EDT:

final String[] value = {""};
try {
    SwingUtilities.invokeAndWait(new Runnable() {
        public void run() {
            value[0] = inputBox.getText();
        }
    });
} catch (Exception e) {
    // Do nothing
}
return value[0];

The value variable must be declared final, since it’s accessed from a different thread. However, we want to be able to update the value, so the use of a single-element array is a bit of a hack to bypass the compiler’s warnings. A more elegant solution to this might involve wrapping the method in it’s own class, and using a class property instead of a final variable.

Saving User Settings with the Preferences API

Java provides an API that makes it very straight-forward to store and retrieve user preferences. These are used in our application to save a number of different parameters:

• The position and location of the window on the screen.
• The position of the resizable split pane.
• The last-used PHP code.
• The specified font size.
• The directory of the last file that was loaded.

We can wrap the Preferences class in our own Settings class, which provides methods to get the required properties. Attaching a WindowListener to our JFrame means that we can attempt to save the settings when the application is closed.

Conclusions

I ended up making a few more changes to the application to those mentioned above. The buttons that are referred to above have been move to the menu bar (which was easy to do, since I was using Actions), and have associated keyboard shortcuts which make the application much easier to use. It’s now also possible to load files from disk, change the font size of the input and output boxes. I’ve also added a status bar which shows status messages for a period of time, and a ‘preferences’ dialog which allows you to change the PHP command path and the web server port.

If the implementation of any of these features interests you, I suggest you download the source code to have a look.

To get a grasp of the structure of the project, the layout below may help:

I’m very pleased with the final product. The application’s concept is simple, but I think it adheres well to the “do one thing and do it well” philosophy.

I’ve also made a few minor tweaks to the application that will affect Mac users: moving the menu bar to the top of the screen, and setting the application’s title.

Download

If you’d like to download the application to give it a try yourself, or if you’d like to download the source code in its entirety, use the links below:

• Download application (packaged as a .jar file)
• Download source code (.zip archive)

If you do try out the application, please let me know what you think.

Requirements

The application requires that you have both Java (version 1.6—although 1.5 might work) and PHP installed. The PHP executable must be accessible on your ‘path’. I have tested this on OS X and on a vanilla install of Vista (after installing Java and PHP).

The thing I like about using a framework is being told how to do things, but for many the big attraction to CodeIgniter seems to be that it’s lightweight and you can do things how you like. I think this is the reason for the inconsistencies and low quality of example code. So I thought that I’d put together an article and example project showing a few of the tricks I’ve learnt, and my preferred CodeIgniter style.

I’m going to put together a simple application which shows off the search capabilities of MySQL, whilst exploring some of CodeIgniter’s features.

Getting CodeIgniter Ready

This is pretty basic stuff. Download, unzip, and dump the files somewhere into your web server’s directory. Setup a MySQL database and associated user and fill in the connection details in application/config/database.php.

Open up application/config/config.php and edit the base_url setting. This needs to be the web-visible URL to the directory that you put the CodeIgniter files into. If you did put them into your local web server’s root directory, this may be as simple as ‘http://localhost/’.

Setting Up the Database

You should already have a database setup, but we need a table to store the data. I’ve created a simple ‘pages’ table. To make use of MySQL’s FULLTEXT search capabilities, you must setup an index on any fields that you want to be searchable. See the the SQL below:

CREATE TABLE pages (
  id int(10) UNSIGNED NOT NULL AUTO_INCREMENT,
  url text NOT NULL,
  title text NOT NULL,
  content text NOT NULL,
  updated datetime NOT NULL,
  PRIMARY KEY  (id),
  FULLTEXT KEY content (content)
) ENGINE=MyISAM

I’ve setup an index on the ‘content’ field so that it will be searchable. Note also that FULLTEXT search is only available when using the MyISAM engine.

Some Test Data

In order to perform a decent evaluation of the searching mechanism, I realised that I would need some substantial data. I decided Wikipedia would make a good source of this data. Rather than export all three million articles, I limited myself to about two thousand of the featured articles. I setup a script to scrape the list of featured articles and then used the Wikipedia export feature to automatically copy across all the data into my database.

After over-coming an issue with the character encoding it soon because obvious that the Wikipedia markup wasn’t quite the format that I wanted. Hacking together a few regular expressions meant I could strip most of this out, leaving plain and readable content.

Searching the Data

At this point, the magic SQL to use takes the form: MATCH (columns) AGAINST ('terms'). Note that when MATCH ... AGAINST is used in the WHERE clause, the results are automatically sorted, so there’s no need to add an ORDER BY clause. An example:

SELECT *
FROM pages
WHERE MATCH (content) AGAINST ('test') > 0

There are a few FULLTEXT gotchas you should be aware of:

• Something that I was stung by: if you have fewer than three rows of data, you won’t get any search results back.
• Search strings have a minimum length. This is generally three or four characters, but depends on configuration. See below.
• There is a collection of ‘stopwords‘ that MySQL will ignore (words like ‘the’, ‘however’, ‘hello’). If you try a search for any of these, you won’t get any results back.

You can find out some of the configuration details by executing SHOW VARIABLES LIKE 'ft%'. You should get something like this:

A few different ‘modes’ are available for searching. By default, a natural language search is performed. A boolean search is also possible, and allows a number of powerful operators to be used – refer to the documentation for details.

It is also possible to use query expansion. This roughly means that the search may return items where the search terms aren’t actually in the content, but instead there are words that are deemed similar. This is achieved by performing two searches. Where the second search uses content from high-ranking rows in the first search to perform the second search. This only tends to work well for short queries, otherwise less-relevant results will be returned.

More details on the full-text functionality can be found in the MySQL documentation.

CodeIgniter and the MVC Pattern

Let us return our attention to CodeIgniter…

The framework takes advantage of the ever-popular model-view-controller (MVC) pattern. Everyone knows you’re supposed to separate your domain-logic from presentation code, but the nice thing about the MVC pattern is that it’s constantly there to remind you.

I stick to a few basic rules:

• you can only touch the database from within the model
• nothing should be echo-ed from the controller (or the model)
• only one statement can appear inside each set of <?php-tags and it should generally be either en echo or a control structure (use the alternative syntax)

If you find yourself breaking any of these rules, it’s probably a sign that it’s time to step back and re-think what you’re trying to achieve.

Let’s start off with the model. I only have one method here, which performs the searching. You may note that I’m using raw SQL here rather than using CodeIgniter’s Active Record implementation. This is just a personal preference – I think the CodeIgniter implementation is a bit messy compared to something more RoR-like.

The following code needs to go into application/models/page_model.php:

class Page_model extends Model
{
    function Page_model()
    {
        parent::Model();
 
        // Make the database available to all the methods
        $this->load->database();
    }
 
    function search($terms)
    {
        // Execute our SQL statement and return the result
        $sql = "SELECT url, title
                    FROM pages
                    WHERE MATCH (content) AGAINST (?) > 0";
        $query = $this->db->query($sql, array($terms, $terms));
        return $query->result();
    }
}

Next up is the view. Note that I’m religiously using the form helpers. This needs to go into application/views/search_results.php:

<?php $this->load->helper('form'); ?>
 
<?php echo form_open($this->uri->uri_string); ?>
<?php echo form_label('Search:', 'search-box'); ?>
<?php echo form_input(array('name' => 'q', 'id' => 'search-box', 'value' => $search_terms)); ?>
<?php echo form_submit('search', 'Search'); ?>
<?php echo form_close(); ?>
 
<?php if ( ! is_null($results)): ?>
    <?php if (count($results)): ?>
        <ul>
        <?php foreach ($results as $result): ?>
            <li><a href="<?php echo $result->url; ?>"><?php echo $result->title; ?></a></li>
        <?php endforeach ?>
        </ul>
    <?php else: ?>
        <p><em>There are no results for your query.</em></p>
    <?php endif ?>
<?php endif ?>

Finally, pulling both the model and view together with the controller. The following code should go into application/controllers/pages.php:

class Pages extends Controller {
 
    function search($search_terms = '')
    {
        // If the form has been submitted, rewrite the URL so that the search
        // terms can be passed as a parameter to the action. Note that there
        // are some issues with certain characters here.
        if ($this->input->post('q'))
        {
            redirect('/pages/search/' . $this->input->post('q'));
        }
 
        if ($search_terms)
        {
            // Load the model and perform the search
            $this->load->model('page_model');
            $results = $this->page_model->search($search_terms);
        }
 
        // Render the view, passing it the necessary data
        $this->load->view('search_results', array(
            'search_terms' => $search_terms,
            'results' => @$results
        ));
    }
}

At the beginning of the method, there is a reference to a redirect function. This is a helper function, and it’s part of the ‘url’ group of helper functions. To use it, we could load the helper manually using: $this->load->helper('url');. However, because this group of helper functions is quite commonly used, we can load them automatically. To do this, we need to edit the application/config/autoload.php file:

$autoload['helper'] = array('url');

We should now have a working prototype. Make sure you have some data in your database (remember, at least three rows). Then steer your browser to the relevant URL, which should be something like: ‘http://localhost/index.php/pages/search’, enter some text to search (something that you know is in the database) and hit the ‘Search’ button.

You should get something like this:

Adding More Features

To show off a bit more of CodeIgniter, I’ll add a few more features, including pagination, custom configuration files and custom helpers.

The CodeIgniter user guide boasts that the pagination class is “100% customizable”. I wouldn’t go that far, but you can certainly change a few things.

First of all, we need to modify our model so that it only returns a portion of the results, and also so that we can find out how many results in total our query would return.

The first problem is easily solved by adding an argument to the method, and a LIMIT to our SQL query. For the second problem, there are a few solutions, none of which seem very elegant:

• We can add a separate method to the model, which will return the total number of results. This means that generally we will always have to call both routines. Certainly not a disaster, but it just doesn’t seem very refined.
• Alternatively, we can calculate this count from within our ’search’ method, and return both a collection of results and the total count together in some form of array structure. This is a bit abusive of the only-return-one thing methodology.
• Or, we can pass a reference parameter to the method, set the value inside the method, and then access this variable from the controller. I think this works quite nicely because it’s discrete, but possibly a bit too subtle to be obvious to other developers.

To keep things simple, I’m going to go with the first option. Let’s refactor the model:

class Page_model extends Model {
 
    // Constructor as before
 
    function search($terms, $start = 0, $results_per_page = 0)
    {
        // Determine whether we need to limit the results
        if ($results_per_page > 0)
        {
            $limit = "LIMIT $start, $results_per_page";
        }
        else
        {
            $limit = '';
        }
 
        // Execute our SQL statement and return the result
        $sql = "SELECT url, title, content
                    FROM pages
                    WHERE MATCH (content) AGAINST (?) > 0
                    $limit";
        $query = $this->db->query($sql, array($terms, $terms));
        return $query->result();
    }
 
    function count_search_results($terms)
    {
        // Run SQL to count the total number of search results
        $sql = "SELECT COUNT(*) AS count
                    FROM pages
                    WHERE MATCH (content) AGAINST (?)";
        $query = $this->db->query($sql, array($terms));
        return $query->row()->count;
    }
}

Note that I’m also returning the ‘content’ field in the SQL – this will be useful later. The next thing to do is to modify our controller. We need to load and initialise the pagination library. Here’s the modified ’search’ method:

class Pages extends Controller {
 
    function search($search_terms = '', $start = 0)
    {
        // If the form has been submitted, rewrite the URL so that the search
        // terms can be passed as a parameter to the action. Note that there
        // are some issues with certain characters here.
        if ($this->input->post('q'))
        {
            redirect('/pages/search/' . $this->input->post('q'));
        }
 
        if ($search_terms)
        {
            // Determine the number of results to display per page
            $results_per_page = $this->config->item('results_per_page');
 
            // Load the model, perform the search and establish the total
            // number of results
            $this->load->model('page_model');
            $results = $this->page_model->search($search_terms, $start, $results_per_page);
            $total_results = $this->page_model->count_search_results($search_terms);
 
            // Call a method to setup pagination
            $this->_setup_pagination('/pages/search/' . $search_terms . '/', $total_results, $results_per_page);
 
            // Work out which results are being displayed
            $first_result = $start + 1;
            $last_result = min($start + $results_per_page, $total_results);
        }
 
        // Render the view, passing it the necessary data
        $this->load->view('search_results', array(
            'search_terms' => $search_terms,
            'first_result' => @$first_result,
            'last_result' => @$last_result,
            'total_results' => @$total_results,
            'results' => @$results
        ));
    }
 
    function _setup_pagination($url, $total_results, $results_per_page)
    {
        // Ensure the pagination library is loaded
        $this->load->library('pagination');
 
        // This is messy. I'm not sure why the pagination class can't work
        // this out itself...
        $uri_segment = count(explode('/', $url));
 
        // Initialise the pagination class, passing in some minimum parameters
        $this->pagination->initialize(array(
            'base_url' => site_url($url),
            'uri_segment' => $uri_segment,
            'total_rows' => $total_results,
            'per_page' => $results_per_page
        ));
    }
}

Let’s look at the changes we’ve made…

We have added an extra argument to the method. This means the URL will have another (optional) segment to it. It will be a number representing the record that we are starting on, and will get set automatically by the pagination code.

In order to determine the number of results to show on a page at a time, I have used the config library. In order to setup custom configuration for your application, it’s best to add a new file to the application/config directory, then set it to be autoloaded. Edit the application/config/autoload.php file again, and add ‘application’ to the $autoload['config'] array. Then create the application/config/application.php file, and put the following line in it:

$config['results_per_page'] = 10;

The next change we have made is to setup the pagination. I have put this code into a separate method. Note that the method name is prefixed with an underscore. This prevents the private method being called as an action on the controller. The code here is fairly self-explanatory.

We also pass some extra information over to our view. Which leads us to our next task of updating the view. But first, we will write our own custom helper functions for use within the view. These go into application/helpers/search_helper.php. The first function – search_highlight($text, $search_terms) – provides a way to highlight search terms in a string (there is a similar CodeIgniter helper, but it doesn’t quite do what we want). The second function – search_extract($content, $search_terms, $number_of_snippets = 3, $snippet_length = 60) – is a flaky attempt to generate an ‘excerpt’ to accompany each result. An excerpt is made up of a number of ’snippets’ which should each contain at least one of the search terms. I won’t paste the code here since it’s far from elegant, but it’s available in the download.

Finally, the updated view. Note that we have to manually load the helper before using the helper functions. The pagination links are created using the create_links method.

<?php $this->load->helper(array('form', 'search')); ?>
 
<?php echo form_open($this->uri->uri_string); ?>
<?php echo form_label('Search:', 'search-box'); ?>
<?php echo form_input(array('name' => 'q', 'id' => 'search-box', 'value' => $search_terms)); ?>
<?php echo form_submit('search', 'Search'); ?>
<?php echo form_close(); ?>
 
<?php if ( ! is_null($results)): ?>
    <?php if (count($results)): ?>
 
        <p>Showing search results for '<?php echo $search_terms; ?>' (<?php echo $first_result; ?>&ndash;<?php echo $last_result; ?> of <?php echo $total_results; ?>):</p>
 
        <ul>
        <?php foreach ($results as $result): ?>
            <li><a href="<?php echo $result->url; ?>"><?php echo search_highlight($result->title, $search_terms); ?></a><br /><?php echo search_extract($result->content, $search_terms); ?></li>
        <?php endforeach ?>
        </ul>
 
        <?php echo $this->pagination->create_links(); ?>
 
    <?php else: ?>
        <p><em>There are no results for your query.</em></p>
    <?php endif ?>
<?php endif ?>

Here’s the result of trying it out in the browser again:

Evaluation

CodeIgniter comes with benchmarking and profiling features. We can mark points in the execution, enable profiling, and have them have them output to the browser. For example, I might add marks around the calls to the model:

// Mark the start of search
$this->benchmark->mark('search_start');
 
// Load the model, perform the search and establish the total
// number of results
$this->load->model('page_model');
$results = $this->page_model->search($search_terms, $start, $results_per_page);
$total_results = $this->page_model->count_search_results($search_terms);
 
// Mark the end of search
$this->benchmark->mark('search_end');

The output at the bottom of the page would look something like this:

We can see that the search on 1800 rows took 0.0138 seconds. This is 49% of the total execution time.

It is also possible to measure the time it takes to render the view by putting marks in the controller on either side of the $this->load->view(...) call. The view takes about 0.007 seconds to load. Without the generation of a page extract, it takes significantly less time: 0.002 seconds.

Download

I have packaged up the code for this example. Feel free to download it.

Other Search Solutions

There are, of course, other search solutions available. Two such popular projects are Sphinx and Lucene. However, getting them compiled, installed and running doesn’t appear to be any simple task.

I’ve been working on a project recently that has a rather messy deployment strategy. This resulted in a minor glitch occurring during a recent deployment to a fairly high-profile site when a database table wasn’t updated.

Clearly the best solution to this problem would be to come up with a better deployment strategy, which is something we have been looking into. However, the problem has inspired me to write a small tool which collects information on the schema of a database so that it can be compared with other databases.

A slight twist to the problem comes from the requirement that the databases are more than likely to be hosted on different servers. Since the database servers are setup to only allow connections locally, we aren’t able to access both databases at the same time. Additionally, some of these servers are running PHP 4, so are lacking some newer PHP features.

Fetching Schema Information from MySQL

The MySQL queries to get information about tables are fairly straight-forward.

To get a list of the tables in a database, we can do:

SHOW TABLES

To get information about the fields in a specific table, we can do:

SHOW COLUMNS FROM table_name

This returns a collection of information on each field:

• the names of fields,
• the data type,
• whether null values can be stored,
• whether the column is indexed,
• the default value of the field, and
• whether the field uses auto_increment

Full details can be found on the manual page.

The PHP Script

The tool consists of:

• a simple class for extracting the schema and performing the comparison between extracted schemas
• a user interface to the class

Since we need to be able to access databases on different hosts, the solution I am using is to serialise the schema information and give it back to the user. The user is then required to copy and paste this data from separate instances back into a form so that the comparison can be performed.

The user interface provides the user with a list of pre-configured databases to choose from, and also allows the user to enter connection details in the form. Arguably, the first option is more convenient and more secure.

The actual comparison of schemas works on three levels:

1. We check both databases have the same tables
2. For each table, we check that the same fields are present
3. For each field, we check that the field’s parameters are the same

Any discrepancies are then displayed to the user on a per-table basis.

Download this Tool

If you have a use for this tool, or are just interested to see how it works, feel free to download the code.

Ext JS is yet another JavaScript library. However the emphasis is on UI widgets that allow you to build user interfaces in the browser. The concept is far from new, but the way Ext JS stands out from the contenders must be embarrassing. The library’s widgets are visually stunning, feature-rich, and yet still maintains high performance.

I will admit that I hadn’t heard of Ext JS before working on a project last year that used it. I’ve been doing freelance web development for about five years and I tend to keep quite up-to-date with web technologies, so I’m surprised that it had slipped under my radar.

Fairly recently, the team announced the latest release candidate of Ext 3.0. I wanted to experiment with some of the lesser-hyped new features, and so I’ve been working on a small experiment to display stock indexes using the new charts feature.

Hello, world!

Let’s start by walking through a really simple Ext example before we look at the charts.

Download the latest Ext release. Setup a simple HTML page and add a few includes:

1
2
3

<link rel="stylesheet" type="text/css" href="ext-3.0-rc3/resources/css/ext-all.css" />
<script type="text/javascript" src="ext-3.0-rc3/adapter/ext/ext-base.js"></script>
<script type="text/javascript" src="ext-3.0-rc3/ext-all-debug.js"></script>

At this point, we should be able to load the blank page in the browser without any errors.

The next step is to write some code to do something with the Ext library. So as not to upset the gods, we’re going to make a pop-up ‘hello world’ window. The following is JavaScript code, so put it in a <script> tag or an external .js file.

1
2
3
4
5
6
7

Ext.onReady(function() {
  new Ext.Window({
    title: 'Hello, world!',
    width: 280,
    height: 120
  }).show();
});

We use Ext.onReady(...) to setup a callback for when the DOM is loaded. This callback will create a new Ext.Window object, and show it. The parameter to the Ext.Window constructor is a configuration object.

The concept of configuration objects is used extensively throughout Ext, and it can be rather fiddly and difficult to get used to. You have to keep the Ext documentation open (see below) to know what parameters to use. In our simple example, all we do is specify the title and dimensions of the window.

Load the page in your browser. The result should look something like this:

Notice that you can drag the window around the screen and resize it to your hearts content.

See the demo or download the code.

For a nice collection of more advanced examples, check out the samples on the Ext site (they are also contained in the Ext download). These samples are great. My only criticism is that they can be a bit inconsistent in the way they implement features.

Another indispensable resource is the Ext documentation (this is also included in the download).

Ext JS Charts

In a way, it’s a shame that Flash has been used to implement the charts feature, since the library has achieved so much without resorting to it in the past. Nevertheless, the actual Flash applet has been well contained and even resizes smoothly with it’s container.

We can build on our simple example from above to add a chart to the window.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

Ext.onReady(function() {
 
  var store = new Ext.data.JsonStore({
    url: 'data.php',
    baseParams: {
      symbol: 'GOOG'
    },
    autoLoad: true,
    root: 'data',
    fields: ['date', 'close']
  });
 
  new Ext.Window({
    title: 'GOOG',
    width: 400,
    height: 300,
    items: [
      new Ext.chart.LineChart({
        store: store,
        xField: 'date',
        yField: 'close'
      })
    ]
  }).show();
});

We are introducing two new concepts here. First of all, the use of a ’store’. In this case, a JsonStore. To quote the documentation:

The Store class encapsulates a client side cache of Record objects which provide input data for Components [...]

So basically, the store takes care of fetching and storing our data. The JsonStore uses a JsonReader to fetch the data in JSON format, keeping the data completely separate from the UI component. The data is then represented internally as a series of Records.

Let’s look at the configuration parameters we have set. The first parameter, url, is the URL that we will be loading the data from and baseParams specifies the parameters to send to this URL. By default the store doesn’t attempt to load the data straight away, so we specify the autoLoad parameter so that we don’t have to call store.load() ourselves later on. The root parameter tells the store where to look in the returned object for the bulk of the records, and fields is a short-hand way to specify the structure of the individual records.

The second new concept we have introduced is the addition of a component to the window. We have done this by specifying an array of components in the ‘items’ property. The single item that we are adding is a LineChart component. The configuration here just links the component to the store we have defined, and specifies the fields to bind to the axes of the chart.

Once the store has been loaded, we should end up with this:

See the demo or download the code.

The current Ext documentation for styling the charts is fairly minimal. Since the underlying Flash applet for the charts is based on the one from the Yahoo UI Library, further information on styling the charts can be obtained from their documentation.

Fetching the Data

Let’s take a look now at the server-end of the process.

The data is generated by a PHP script which in turn gets the stock data from Yahoo. Yahoo kindly provide a CSV-formatted data set of the daily stock prices (including open, close, high and low values) which go back about five years. We don’t want to be pumping this much data to the browser, so we will take a few steps:

1. Fetch the data if not already cached
2. Extract the closing price for each day
3. Cache the data
4. Choose an appropriate range of dates to display
5. Sample the remaining data to leave us with a suitable number of points that can be plotted
6. Return the data encoded as JSON

We now have a simple web service that allows us to specify a number of parameters:

• symbol — The stock ticker symbol. Required. Note that because we are using the Yahoo stock data we are limited to stocks listed on the NASDAQ.
• start — The start date. Specified as a Unix timestamp. Defaults to roughly one year ago.
• span — The timespan to get data for. Also given as a Unix timestamp. Defaults to roughly one year.
• points — The number of points to sample. Defaults to 20.

An example result from the service would look a little like this:

1
2
3
4
5

{"symbol": "MSFT", "data": [
  {"date": "2008-07-24", "close": 25},
  {"date": "2008-08-12", "close": 28},
  //...
]}

I won’t paste the PHP script here. It can, however be downloaded.

Once the data has been successfully loaded, an event is raised, which the chart has been listening for. The chart can then re-draw itself using the new data.

Taking Things Further

In an attempt to mature our example a little, and to highlight a few of Ext’s other features, I’ve put together a more substantial example. See the screenshot below, and refer to the live demo.

I won’t go into much detail here, other than to list some of the features that I’ve used:

• A Viewport together with the BorderLayout to ensure that the interface takes over all the space offered by the browser window.
• A modal pop-up window which contains an auto-complete ComboBox. A list of the NASDAQ-100 are presented for the user to choose from.
• A TabPanel to organise the different stocks that have been loaded. Selecting a different tab will change the chart being displayed, and closing a tab will unload the data and chart.
• A ListView component, which is the new lightweight version of the GridPanel.
• Some more advanced features of the Chart object—including making use of the TimeAxis, linking mouse movements to the ListView component, and making some changes to the chart’s style.

I have left the JavaScript un-minified so feel free to take a look at it.