Go to the Network Merchants API PHP Class »

To start off with, the chart image must be delivered from a secure connection. Google doesn't allow this plain and simple, so we need to figure out how to host it from our own site. We accomplish this by fetching the image from google using the standard API, writing it to a file, and then calling it on our own script. We basically make a image handling proxy.

Let's take a simple google chart to experiment with.

Next we need to make a function to fetch and save the google chart locally. It will check the chart against the local copy and save it if the chart doesn't exist, or the image has changed. This way we aren't re-writing the same chart on every request, but if the chart changes, it will be updated appropriately.

Lastly we tie it all together so that it is usable in our application. Im using this within a class, but this could just be used as a function as well. Your image directory will need to be writable for this to work.

You'll need to implement your own error handling, and adjust this to meet the paths and specifics of your server, but the image can now be called from:
<img src="/images/charts/some_chart_image.png" alt="" />

If you need help creating your base chart image, this tool is a great place to start.

Unlike Internet Explorer and older Firefox versions, Firefox 3.5 gives this error if there are any missing images on a webpage. Meaning, that if an image link or css reference is to a non-existent image, Firefox warns that the page is not secure. To my knowledge, Firefox versions previous to 3.something, did not behave like this. Internet explorer, Google Chrome, Safari, and every other non-Firefox browser that I tested don't care about broken images. Not to say that Firefox is wrong, but this made the problem more difficult to diagnose because it couldn't be reproduced in another program.

The difficulty in diagnosing this, is that background images that don't load also don't show up in the page info section on Firefox. Firebug didn't provide any immediately useful information, and there was no documentation that I could find regarding this situation.

After several hours of searching, I realized that there was a small broken background image. Deleting or correcting the image path immediately corrected the certificate error.

Anyway, if anyone has been pulling out their hair trying to diagnose a mystery ssl error, and this is a possibility, definitely look into your image paths.

Having needed a way to force a SSL connection for certain pages, I modified a few scripts that I found online, and created a very simple filter to handle this. This was inspired by this script, and the unacceptably poor example in the Symfony 1.2 book.

To start off with, we need to modify our app.yml file to specify what modules and/or actions need to be secure. Leave the action completely blank if you want the entire module secure. Also change ignore_non_secure to true if you don't care if non specified pages are server over a ssl connection. Basically, from the app.yml below, setting this to false, will redirect any module/action to the non-secure version if it is not specifically defined under secure_actions. Setting it to true will allow a user to request any page over https, even if it is not listed in app.yml. Let me know if this is confusing in any way.

Next we add this filter. Save this under MyProject/apps/MyApp/lib/sfSslFilter.php

Finally, add to the MyProject/apps/MyApp/config/filters.yml file:

Clear the cache (symfony cc), and there you have it. Let me know if you have a better or different way of dealing with this on a per-module or per-action basis. Hopefully sfSslRequirementPlugin will get ported to work with Symfony 1.2, as the method above will not alter routes on your application.

Additionally, I specifically used 301 redirects to make this more search engine friendly, in case Google or another bot gets on a ssl page. This will help prevent getting duplicate pages indexed due to http and https versions of the same page.

Cheat Sheets:

1.) DZone has created professional Cheat Sheets for a number of PHP, and web development topics. These can save a ton of time from digging into a manual or doing a search online. These aren't by any means the only cheat sheets out there, but you will find PHP, MySQL, XML, ASP, Java, Design Patterns, Joomla, and about 50 more easy to read, well designed cheat sheets.

IDE:

2.) NetBeans is an IDE that stands out above all others. It's completely free. It offers a built-in FTP client (Far superior to Dreamweaver) or supports subversion publishing. It is very small, fast, and has many built-in features that help create better code, more efficiently. Netbeans is hands down the best IDE, with the fewest bugs (Not completely bug free), and virtually no learning curve (Try that Eclipse...), that I've used.

If you are holding on to Dreamweaver because of the FTP, or some other function that Eclipse PDE or Zend Studio is lacking, I urge you to take a look at Netbeans. When I'm not using SVN, I can save the file I'm working on and it is automatically uploaded to the web or testing server. Try it, you won't be disappointed.

Database Management:

3.) Navicat is by far the best database GUI, and my experience the best tool for database management on the market. Premiumsoft offers MySQL, PostgreSQL, Oracle versions, and now a single package that includes all three. Navicat provides an intelligent and efficient way to create, backup, and manage complex MySQL databases. It allows multiple methods of connecting, including a SSH tunnel, which protects your database from being accessed publicly. Apart from a good IDE, this is the #1 tool in my development toolbox. Navicat also offers a custom report building application, which can export data into Excel, Access, Text, XML, Lotus 123, or through an ODBC connection. This software is not free but they do offer a free 30 day trial. If you do a lot of database management, this software will probably pay for itself in time the first week you use it.

4.) Monyog is a fantastic real-time MySQL reporting program. It offers visual graphs of a database's performance, and stats on just about everything that can be measured in MySQL. It is an excellent way to monitor a Database and is indispensable for benchmarking and optimizing MySQL performance.

5.) MySQL Workbench is a free MySQL development program which can create visual diagrams of a database's structure, among many other features. This can be very important when you are working with large or just new databases. A picture can be worth much more than a thousand words when you are diving into a complex database.

Here's a diagram of a moderately complex database, which could take a lot of time to understand without anything but the table structure.

6.) SQL Code Formatter is a quick tool that formats SQL code into an easy to read format. This tool can be indispensable when you are debugging other developers code, or trying to break-down a massive jumbled SQL statement.

Turn junk like this:
SELECT users.id, users.name, users.address, billing.order_num, billing.subtotal, billing.shipping, billing.tax, SUM(billing.subtotal + billing.shipping + billing.tax) AS total FROM users LEFT JOIN billing ON users.id = billing.user_id WHERE billing.order_date > '2009-07-31' AND billing.complete = '1'

Into this (example query may not make any sense):

Debugging:

7.) HTML Validator is a Firefox Plugin that provides an instant status of a page's w3c validation as you browse. It also replaces Firefox's built-in source view, with an intuitive and significantly improved interface. This is one of the best tools out for quickly finding problems with a site's (x)html structure.

8.) Firebug is an outstanding debugging program that provides a ton of information about structure, scripts, and problems. Firebug is an amazing tool, but can be quite complex, which is why I also use the HTML Validator plugin.

Version Control:

9.) Git is a very fast version control system, that supports quick branching, merging, and development capabilities. There are a number of public GIT hosting sites such as github that allow developers to publish and collaborate on their projects.

Git gives each developer a local copy of the entire development history, and changes are copied from one such repository to another. These changes are imported as additional development branches, and can be merged in the same way as a locally developed branch.

10.) Subversion or SVN is another version control system, ironically despised by the creator of GIT, but is very commonly used and should be in any programmers repertoire. It is currently one of the most popular version control methods, and works with just about any platform.

Quick comparison of GIT vs. SVN.

11.) WinMerge is a fast file comparison program for Windows. It allows the user to quickly compare 2 files, and merge or edit as needed. This is by no means a version control method, but can quickly help determine differences between 2 documents.

Graphics:

12.) ColorZilla is a very fast color picker plugin for Firefox. When you need to get a color from a website, ColorZilla is probably the quickest way to do it. Not only will it show the HEX color code, but it shows the RGB values, and exactly what element is using the color. If you ever take on an existing website for further development it can be invaluable in breaking down somebody's poorly designed css stylesheets.

13.) Color Scheme Designer is a no BS way to create compatible, attractive color schemes for websites, and user interfaces. There are dozens of color scheme generators out there, but this one does what is needed efficiently, for free, and without any extra fluff. Several pre-defined color patters are a click away, and you can see quick examples of a web page using the color schema that you've generated.

Communication:

14.) X-Lite is a free VOIP Softphone application. It offers all the features that a very advanced phone would have, including video.

In a world where many people have forgotten what a phone is, this is the perfect compromise. You can get a Plantronics headset and USB adapter, a low cost VOIP service, and have unlimited calling anywhere in the US, and possibly the world, as long as you have an internet connection. Don't ignore the good old fashion telephone. Businesses still use these things...

15.) Trillian is a great way to combine all of your IM clients into a single, light-weight app. This program can free up a considerable amount of system resources compared to running multiple IM clients at the same time. It also lets you access all of your IM contacts through a single interface.

Miscellaneous:

16.) ScreenGrab is a screen capturing program that's slightly better than the "Prnt Scrn" button on your keyboard. It can either copy or save a capture to a PNG file. It allows capturing a complete page (including below the fold), only the visible portion of a screen, by selection, or the entire window. The only drawback is that it only works within Firefox. Any screen captures you see on this or my other blogs are done with ScreenGrab.

17.) Roboform is a username/password management program, and is the most important browser add-on that I use. I currently have over 2,000 website profiles stored in it (One of many pages shown in the image), every one with a very secure and unique password like "fz96Dr%PpaZjaBfk".

It supports a variety of very advanced features and configurations to help keep your login credentials secure and easy to use.

You can download a free trial here.

18.) SUN Virtualbox is the best virtual computer deployment package that I've come across. It installs on a number of host operating systems, not just Linux, and supports just about every OS on the market for Virtual Appliances. It is extremely easy to install, is very efficient, and can be customized as required. Virtual hosts can easily be backed up, or cloned and distributed.

If you've been afraid of exploring into Virtualization, I highly recommend looking at Virtualbox. There are so many benefits to this sort of technology, it's something that every developer should be familiar with.

19.) Ultramon is a multi-monitor taskbar program. unfortunately Windows XP does not include for multiple taskbars if you multiple monitors. Ultramon fixes this in a fast, predictable manner, and includes many additional features such as desktop back ground stretching. Ultramon is not free but is far superior to the free multi-taskbar products that are available.

20.) PDF Creator is a PDF printing application. It allows printing any file directly into a PDF. If you don't need the ability to edit existing PDF documents, this is an excellent alternative to purchasing any other PDF software.

Let me know if you've got your own set of essential tools that help in your development.

Unfortunately, by using these methods, PHP's restriction on private and protected properties is bypassed, making all properties public. This is completely unacceptable from my coding perspective.

This class model overrides the magic __get and __set's ability to alter and access private and protected properties. Public properties are unaffected. This script also allows the class to set and access private and protected properties.

Extended classes will not have access to __get or __set protected properties. I will alter this snippet when I find a suitable method of handling extended classes.

I'm hoping that php alters the way it handles private and protected properties through the magic methods but until then, this is a way to semi-preserve private and protected properties.

PHP Snippets:

MySQL Snippets:

This is a simple example using sleep() to demonstrate the output.

The script outputs:

Time 1: 1.001833 seconds.
Time 2: 2.001427 seconds.
Time 3: 3.001124 seconds.
Time 4: 1.001720 seconds.
Total time: 7.006104 seconds.

It's a good idea to comment each time you record a microtime so that you know which section of script took that amount of time.

This can be used to generate random passwords or keys or just about anything else that needs a random string. You can also throw this directly into a class and use it as a static method.

Examples:

This guide is by no means all inclusive, but should help to cover some of the major gaps in knowledge and commonly overlooked fundamentals that I run into on a daily basis.

The checklist is separated into 5 equal sections: Server Setup, Schema Design, Table Design, Index Optimization, Query Optimization, and a 6th Bonus Tips section.

You can also download a simplified summary on PDF form.

Section 1 - Server Setup

1. Root User
   For security reasons, the root MYSQL user must be setup with a secure password, and should only have access from localhost. It is a bad idea to allow outside access to the root account. Create additional users if you need to access the database remotely!
2. Backup and Restore
   Before allowing a database to be used in a production environment, there should be a usable backup and restore process. I use the phrase "in case the database server is completely destroyed" because the backup location and method needs to be completely independent from the database server. Note: Even a weekly database backup is better than no backup at all.
3. Benchmarking
   There's no easy way to determine bottlenecks and trouble unless a method to benchmark performance is in place. The slow query log should always be enabled, and it's a good idea to install a benchmarking program. Monyog is an external program that provides a number of real-time reports useful for monitoring and performance tuning.
4. DNS
   If you do not allow outside access, or you can access your server from known IP addresses, disabling DNS look-ups can speed up server operations. Additionally, if the MySQL server loses it's DNS look-up service, the usability of the entire Database can all but halt.
5. Privileges
   When adding users to a database, only give them the permissions that are absolutely required, and be specific in where they can access from. "GRANT ALL ON EVERYTHING TO USER@ANYWHERE" is a really bad idea. If you do need to give users full permissions for installations or another purpose, it's a good practice to change them back to the minimum once complete.

Section 2 - Schema Design

6. Naming
   A standardized naming scheme should always be used. The best practice is to use lowercase letters with an underscore connecting names, such as `my_personal_database`. Tables, and individual column names should carry the same naming convention. Use descriptive names for every column including id columns. `id` is not descriptive whereas `contact_id` is.
7. Collation
   Use the same collation for all parts of the database, and avoid using UTF-8 or multi-byte formats unless you specifically need them. Keeping the same format on all tables and columns can help prevent data corruption and conversion problems. UTF-8 requires significantly more disk space and overhead which can reduce performance. If you need UTF-8, use it , but don't make your entire database UTF-8 because you're lazy.
8. Foreign Keys
   Always use foreign keys to ensure that bad / incomplete data stays out of the database. Nothing replaces good application level programming, but foreign keys are the best way to prevent putting bad data into your database in the first place. You will need InnoDB to use foreign keys, but the benefit is worth it.
9. Logically Segmented Data in Tables
   Tables should be segmented logically by the data they contain and their association with other tables. In this manner, there may be more total tables, but will help eliminate tables with a huge number of columns which can really hurt performance. Additionally, it will make querying easier as it's unlikely that every column is needed for every query. This also allows for Single-to-many relationships such as storing multiple addresses related to a single entry. Don't be afraid of 20 tables with 20 columns each, be afraid of 1 table with 400 columns!
10. Reserved Words
    Avoid using reserved words for any name in your database schema. Words like date, time, decimal, etc. are often used, and can wreak havoc trying to get queries to work properly, and can cause even more difficulty in debugging. You can technically use these words if they are placed in back-ticks (`date`), but this is a bad practice and should always be avoided.

Section 3 - Table Design

11. Data types
    MySQL has many data types, probably more than any other database. Using the correct data type for the data being stored is one of the most important aspects in design. Failing at this step can break a database's speed and the usability of an application.

    Whole Numbers - BIGINT, INT, MEDIUMINT, SMALLINT, TINYINT
    Decimals - DECIMAL, FLOAT, DOUBLE, REAL
    Dates - TIMESTAMP*, DATETIME, DATE, TIME, YEAR
    Strings - CHAR, VARCHAR, BINARY, VARBINARY, BLOB, TEXT, ENUM, SET

    Additionally, using the correct data type allows for the use of MySQL's built-in functions which can sort, do math, comparisons, date conversions, etc. For example, I often see dates stored in VARCHAR columns, which completely prevents MySQL from sorting, or using any date related function.

12. Large Numerical Keys
    It's common for new programmers to use a BIGINT(20) when they need a key column. While admirable, this is a waste of disk space. An UNSIGNED INT(10) has over 4 billion possible numbers, which is more than most will ever use. Even so, by that time, you will want to look into partitioning, and will have a variety of other problems on your hand. Don't use BIGINT's unless you need to store very-very large numbers.
13. Smallest Length
    Using the smallest length data length is important. Every byte of savings adds up when a database's size and usage goes up. Lazy programming by using VARCHAR(255) or DECIMAL(20,2) creates unnecessary overhead and causes problems down the road. Give yourself one extra byte of space if needed, but 100 is a overkill.
14. Avoid TEXT and BLOB Columns
    TEXT and BLOB type columns can eat up a server's resources when being selected. While these are most certainly needed to store larger amounts of data, they should only be used for that purpose. VARCHAR can hold up to 255 bytes and should always be used before TEXT whenever possible.
15. Non‐Relational Storage
    A huge design mistake is storing data in a non-relational format. It's common to see data stored in a CSV format like (value1,value2,value3) in a single column. This effectively bypasses MySQL's ability to use the data. It's best to use multiple tables for single-to-many relationships, as this allows for MySQL to handle the data in an elegant manner. There are some situations where storing csv-like data would make sense, but for all intensive purposes, avoid storing data like this.

Section 4 - Index Optimization

16. Use proper indexes
    MySQL supports several types of indexes (PRIMARY, UNIQUE, NORMAL, PARTIAL, and FULLTEXT). It is important to use the correct type of index for the job. It is also important to only use indexes when needed, and not to create duplicate indexes. For example a primary key column already has an index, so adding a second UNIQUE INDEX on the primary key is a complete waste of overhead and disk space.
17. Multi-Column Indexes
    If there is a data set that is constantly queried with more than one column in the WHERE clause, it may be a good idea to create a multi-column index. If you have an index on (`user_id`,`user_category`) the index will work when both are in the where clause or the first column (`user_id`) is in the where clause. However, the index will not be used if only `user_category` is in the where clause.
18. Modifying Indexed Fields During a Query
    Unless you specify the length of an index, modifying an indexed column will prevent the index from being used.
    For example, if there is an index on `credit_card_number` and you perform a query like this:
    SELECT `user_id` FROM `my_table` WHERE LEFT(`credit_card_number`,4) = '5666';
    The index will not be used. If this was a common scenario, you could create a partial index of (`credit_card_number`,4), and the above query would use the index.
19. Indexes With a High Cardinality
    Indexes work best when there are many unique values in relation to the total number of rows. This allows the database engine to quickly reduce the number of possible rows in the result set. Indexes on columns with only a few unique values are inefficient and will end up being a waste of overhead.
20. Unique and NULL Column Indexes
    Allowing NULLS in index columns adds an additional byte of storage per row to the index. This again equates to a waste of space and overhead and will slow down MySQL's performance. It's better to use no value rather than NULL.

Section 5 - Query Optimization

21. Specific Column Names
    Always use specific column names instead of * when querying a table. SELECT * is lazy programming. While it is completely valid syntax, you won't know the columns that will be returned. If you don't know what you're going to get with a query, there's no reason to use it.. right? Write out any column names that you need data from. This way your code is intuitive, you won't have problems trying to use data from a column that doesn't exist, and the next person using your script wont hate you.
22. MySQL's Built-in Functions
    MySQL has a variety of very advanced, and very fast, built-in functions. They probably are much more efficient than php or most other application level scripts. These functions can greatly increase your application's speed, and reduce its complexity. MySQL has everything from mathematical operations, date comparisons, even spacial functions for calculation geographic equations. Learn to use them.
23. Selecting TEXT and BLOB Columns
    When a TEXT or BLOB column is select in a query, MySQL will create a temporary internal table. If large result sets are selected with TEXT or BLOB columns, this can create a major load on the database, and unnecessary overhead. This relates back to SELECT *, don't select a TEXT or BLOB type column unless you actually need to use the data.
24. Use Transactions
    Transactions are another great way of preventing incomplete or corrupted data while inserting or altering data. When using a transaction, you can insert or alter any number of rows of data. If there is an error, all of the queries in the transaction will be aborted. Think of inserting 50,000 rows into a report table, and having 10 arbitrary rows not insert correctly. That entire set of data is now corrupt, and a transaction would have prevented that.
25. SQL_NO_CACHE
    SQL_NO_CACHE is a great way to prevent MySQL from caching a query's result. This is important for results with a rapidly changing data, or very large result sets. Both of these situations can eat up server resources without any gain to the application or end-user.

Bonus

1. TIMESTAMP vs. DATETIME
   TIMESTAMP and DATETIME store dates in the exact same format (YYYY-MM-DD HH:MM:SS) but TIMESTAMP uses less space to do so. The only limitation is that TIMESTAMP cannot be used for dates older than Jan 1st, 1970.
2. SIGNED INT
   Unless you need to store negative numbers, only use UNSIGNED INT and other numerical data type fields. There's no reason to allow for negative numbers if they will never be used.
3. Collation: _ci vs. _cs
   The _ci in a collation stands for "case insensitive". If you care about case sensitivity use a collation that ends in _cs. The can be very important for searching and other operations where John ≠ john!
4. InnoDB vs. MyISAM
   If you're using MyISAM as a storage engine only because it was the default, you may be making a mistake. InnoDB is superior in several areas (Reliability, Backups, Foreign Keys, and Performance in many situations) and while maybe not always the best option (Full Text Indexing), you should know why you're using the engine you're using. You can also mix the 2, but this can make performance tuning especially difficult.
5. Consult a Professional
   When you get a project and the database design, usage or other factor is just over your head, it's a good idea to consult a professional. It may cost a fair sum, but the cost down the road could be substantial. Planning is always cheaper than reacting.