The Ender Tech

Is Your Data Lost Among the Clouds?

2010-09-14T14:52:00.000-07:00

Within the cloud, I don't know where my data is and I don't care because I'm protected. The name of the game is virtualization and just like the software apps on your computer, they don't know where in the computer's memory their data is stored and they don't care as long as it works.

I hope that my data is within secure facilities, transmitted across secure networks, and stored on secured servers. But I still don't know where or how. I haven't seen the facility. I haven't seen the servers. I have yet to see a VLAN on the private networks. I don't even know their hardware disposal policies. Do they destroy hard drives with issues or simply toss them free for anyone to pick up?

So then, how do you protect your data in the cloud? Well, as with life, your data is either moving or it's sitting still. These two require two different methods of protection.

Protecting your data in-transit (moving) requires securing the communication between the endpoints. The common method that every user knows is SSL (HTTPS). This provides a secure connection between the user's browser and the web server. Within the cloud, this same process applies to communication between servers. With servers you can use SSH. With MySQL you can use SSL between the client and server, or between servers if using something like replication.

Protecting your data at rest (sitting still) requires securing the storage of the data. The storage can be on the cloud server's hard drives, remote block device (Amazon EBS), or even on a file-based storage such as Rackspace Cloud Files or Amazon S3.

For data while sitting in roughly one place and that is constantly in use like a database, the encryption needs to happen on the fly. TrueCrypt is a common method but be aware there may be a performance penalty or integration issues with virtual servers. Be sure security is a top priority to catch these potential issues early.

Protecting data that can be wrapped into a single file, such as database backups, can make use of GnuPG. This method is important for protecting your storage on object/file-based storage services like Cloud Files or S3. You can either pipe the data into GnuPG to be encrypted before being stored to a file or feed the file to GnuPG. Then store the encrypted file on your storage service free for them to replicate and transmit through their networks safely.

You'll still want make sure no sensitive data gets written to disk unencrypted because then you have to scrub the area to make sure it's gone (if you even can). Just because you deleted the file doesn't mean the data doesn't exist. Also, be careful of remote storage services like Amazon EBS. EBS is actually data that is also in transit. The virtual server storage itself may be backed by a remote SAN using an unknown network.

Vendors don't always want to give the details so who knows, but it doesn't matter all that much if you take the necessary precautions.

Symfony 1.4 Doctrine - Admin Generator - M:M - Many to many relationships

2010-07-19T17:16:00.000-07:00

After two days of bad googling.. I think I have finally arrived at a solution.

Problem:

Creating M:M relationships in Symfony 1.4 - Doctrine. These relationships, saving methods, and admin double lists should all be populated.

Solution:

Correct schema
Correct form classes
Modified routing

Implementation:

schema.yml


Book:
  columns:
    id: { type: integer(4), primary: true, autoincrement: true }
    title: { type: string(100), notnull: true }
  relations:
    Authors:
      class: Author
      local: book_id
      foreign: author_id
      refClass: AuthorBook //link to AuthorBook class (creates list relation)

Author:
  columns:
    id: { type: integer(4), primary: true, autoincrement: true }
    title: { type: string(100), notnull: true }
  relations:
    Books:
      class: Book
      local: author_id
      foreign: book_id
      refClass: AuthorBook //link to AuthorBook class (creates list relation)

AuthorBook
  columns:
    author_id: { type: integer(4), primary: true } // foreign link to author
    book_id: { type: integer(4), primary: true } // foreign link to book
    content: { type: clob, notnull: true } // wanted to attach information onto the linking row

routing.yml - modify column field [Array]


author_book:
  class: sfDoctrineRouteCollection
  options:
    model:                AuthorBook
    module:               author_book
    prefix_path:          /author_book
    column:               author_id/:book_id // by default it gets generated as Array
    with_wildcard_routes: true

AuthorForm.class.php - using sfFormExtraPlugin


$this->widgetSchema['books_list']->setOption('renderer_class', 'sfWidgetFormSelectDoubleList');

BookForm.class.php - using sfFormExtraPlugin


$this->widgetSchema['authors_list']->setOption('renderer_class', 'sfWidgetFormSelectDoubleList');

References:

http://www.doctrine-project.org/documentation/manual/1_0/en/defining-models:relationships:join-table-associations

http://melikedev.com/2009/12/09/symfony-w-doctrine-saving-many-to-many-mm-relationships/
"It took several weeks and perusing several different google searches to piece together enough information to finally get my head around this process, hopefully this post will save some babies from being punched due to frustration." - Mike Purcell

http://forum.symfony-project.org/index.php/m/88120/
"I think admin_double_list is not available anymore in the core since Sf 1.1." - theredled

http://tohenk.wordpress.com/2009/04/22/using-multiple-primary-keys-in-admin-generator/
"I’ve been dealing with this problem for two days now. I’ve found two solutions (working with symfony 1.4)
1) create a getter for composite primary key and use this ( return $keypart1.’-’.$keypart2.’-’.$keypart3) and use this in the url…
2) today I’ve found another solution, I’ think it’s better…For you try to change :Array (in the routing settings) to the url part compound from collumns of your composite primary key …" - tomor

http://groups.google.com/group/symfony-users/browse_thread/thread/9c783fc59dc281ae/
"I don't think the admin generators support composite foreign keys." - Jonathan Wage (wrong)

Simple Database Record Locking - Propel

2010-05-29T11:55:00.000-07:00


$c = new Criteria();
$c->add(InspectionPeer::ID, $ie->getId());
$c->add(InspectionPeer::IN_PROCESS, 1);
if(InspectionPeer::doUpdate($c) === 0) {
    //field was not modified therefore it was already 1 and
    //is being handled by another process so skip
    continue;
}
$ie->setInProcess(1);

Simple record-level locking is achieved by atomically updating a lock field to grab the lock. This is done by updating the field and checking if it was actually locked by return of doUpdate which indicates the number of rows modified If no rows were modified then it returns 0. If you were able to grab the lock, then it returns 1.

This is important because it avoid the race condition between checking if the lock is free and actually grabbing the lock, which much the same as a typical get-set race condition with regular memory values.

This form of locking is known as "pessimistic offline" due to the fact that:

it grabs the lock first instead of only checking afterwards if it has changed
it actually stores the lock in the database rather than in the DBMS

Beanstalkd Poison Jobs/Messages

2010-05-14T20:21:00.000-07:00

Be careful with handling poison jobs/messages in beanstalkd:

http://github.com/kr/beanstalkd/issues#issue/11

Currently when a client disconnects while it has a reserved job, that job is instantly released onto the front of the queue, without respect of its remaining TTR.

Particularly:

A job that causes a worker to crash constantly hogs the front of the queue, blocking subsequent jobs from coming through.

Be sure to set reasonable timeouts, use separate tubes, and add job stats checking on reserves and releases metrics so the job can be properly logged and discarded.

Propel ORM Does Not Cascade Soft Deletes

2010-04-28T23:32:00.000-07:00

OK... super annoying... if you set the soft_delete flag in your schema for a Propel project, and that table is related to others via a foreign key... those related rows won't be deleted (or soft deleted) if the parent row is deleted.

Solution? Well, override the delete method and implement the cascade yourself in software... annoying...

Perhaps this is one of the reason's that Doctrine is taking over... it automatically implements this solution for you.

Here are some references:

http://eatmymonkeydust.com/2009/05/doctrine-gotchas-delete-cascade-sofdelete-and-foreigntype/

http://www.doctrine-project.org/documentation/manual/1_1/en/defining-models:transitive-persistence

dwCryptPlugin Defaults Not So Secure

2010-03-17T17:30:00.000-07:00

Thought installing dwCryptPlugin with the defaults to encrypt sensitive data was enough? Almost.

The defaults of dwCryptPlugin are 3DES and ECB. 3DES is still "strong enough" yet it is aging. The default block mode ECB is considered inappropriate for all encryption tasks.

However, dwCryptPlugin doesn't provide the ability to get or set the initialization vector. That works fine for ECB because it isn't used, but doesn't work at all for other block modes like CBC.

Solutions? Modify dwCryptPlugin, use a PEAR class like Crypt_BlowFish, or use Zend Framework's Zend_Filter_Encrypt (which defaults to CBC). While you're at it, might as well use AES-256 (Rijndael-256) too.

Here's some more info for the hungry:

http://pear.php.net/package/Crypt_Blowfish

http://framework.zend.com/manual/en/zend.filter.set.html

http://en.wikipedia.org/wiki/Block_cipher_modes_of_operation

http://en.wikipedia.org/wiki/Block_cipher_modes_of_operation#Initialization_vector_.28IV.29

http://en.wikipedia.org/wiki/Advanced_Encryption_Standard

Adobe Reader PDF Streaming Referer

2010-03-02T11:01:00.000-08:00

While maintaining a legacy application that contains Referer header checks we noticed an issue with the loading of PDF files. Saving the PDF files to worked fine but loading them within the browser via the Adobe Reader plugin would not work.

Turns out that the streaming rendering of Adobe Reader does not send Referer headers on the subsequent requests. From what I could see, it appears that:

The browser makes the request to the server
The server tells the browser that the content type is application/pdf
The browser notifies the Adobe Reader plugin that it has a file to handle
Adobe Reader takes control and makes its own request that accepts partial content responses but doesn't send the Referer header

After the initial connection from Firefox succeeded, the rest would fail. Since no Referer was sent at all and this was a simple .htaccess HTTP_REFERER check, it had to be disabled. It is important to note, however, that the Cookie header is sent which is a superior to the Referer.

Google Earth Plugin Oddities

2010-02-20T23:49:00.000-08:00

A couple issues I experienced:

Body tag or plugin div tag need a margin
IE8 (IE7?) cannot test plugin-object properties

For some strange reason, in Firefox 3.5.x (maybe 3.5.6 and later?), the Google Earth Plugin would not render if the body tag and the plugin container div (commonly "map3d") were both set to margin: 0;. You would only see the loading image and that would disappear, yet javascript-wise, everything loaded fine.

Weirder yet is that if you resized the browser then the plugin would render itself. I tested this on two different computers with no difference. If you give either tag a 1px margin, then the plugin would show up fine.

In IE8, there was a test: if(event && event.preventDefault){} as a measure of defensive programming. event being the native KMLEvent or KMLMouseEvent object mapped into javascript. However, it appears that IE8 either couldn't test, or flat out choked on testing for the preventDefault function and thus had to be changed to a simple: if(event){}. I'm assuming this is because IE's plugin/javascript is fairly weak and doesn't have the ability to grab properties on mapped objects.

Finding Competitors Backlinks

2010-02-16T11:45:00.000-08:00

For people that are new to SEO, they might have no idea how to do some competition analysis. How do you find your competitors backlinks?
The most popular command for finding backlinks is the 'link:' command, which you can use in both Google and Yahoo! Site Explorer. For better in-depth backlink analysis, more people tend to use Yahoo! Site Explorer because it lists more backlinks, whereas Google will only list a small percent.

In addition to the 'link:' command, you can use many other queries and commands to find references to your competitor links. The following commands may be helpful:
"www.competitorURL.com"
inurl:www.competitorURL.com
inanchor:www.competitorURL.com
intitle:www.competitorURL.com
allintitle:www.competitorURL.com
allintitle:www.competitorURL.com site:.org
site:.edu inanchor:www.competitorURL.com
site:.gov inanchor:www.competitorURL.com

You can also use variations of these commands. Although some of these commands may not yield backlinks per se, you will at least be able to see where these competitors are mentioned. The more you know about your competition, the better.

Investigating JavaScript "Concurrency"

2010-01-22T15:39:00.001-08:00

(This article was written with knowledge gained from tests againsts FireFox 3.5 HTML-served JavaScript. I'm interested in input from others on the correctness of this article and their findings. Thanks.)

You may notice that can have JavaScript running while seemingly also handling a UI event or processing the result of an AJAX call. Is it running concurrently? Actually, to a degree, no it isn't.

When researching this topic, I came across a couple posts that stated "js is single-threaded." Ok, but what does that mean? From my tests, essentially it means that any asynchronous events are only executed after the currently existing script has yielded.

Does yielding sound familiar? Cooperative multitasking. The application has to relinquish control for the system to continue. If you ever get stuck in a while-loop the web browser hangs because it never yielded to the UI.

Nowadays preemptive multitasking is common. The operating system forcibly removes control at any point within the application. This effectively makes all shared state subject to a race condition. Even if you're simply incrementing a variable that's three operations: get, increment, and store which can be preempted at any point between them.

JavaScript is a dialect of ECMAScript (most being ECMA-262, Edition 3 compatiable; latest being Edition 5). While ECMA-262 says nothing about threads, concurrency, or even atomic operations they can actually exist via extensions -- much like other programming languages that use a library or the operating system to properly handle concurrency.

But ECMA-262 also says nothing about yielding. So how does this supposedly cooperative multitasking implementation yield?

Well, from my tests, the global executing block has to end. As soon as that block of execution has reached its end control is given back to the browser to then execute asynchronously scheduled tasks like AJAX calls and UI interactions.

A simple example is if you schedule a 2-second-long AJAX call within your <script></script> block that takes 4-seconds to run, your AJAX event handler will be waiting 2-seconds for its turn to run. The actual AJAX shuffling will occur in its own browser thread but that's outside of JavaScript's scope.

W3C Web Workers is coming onto the scene as a form of "threading" but it's scope is private and you only communicate with it via basic messages. You still probably don't need any locking protection because it communicates just like any other async event -- waiting to be scheduled into execution when others have yielded.

Maybe one day we'll see <script type="application/javascript;preempt=true"> in the JavaScript engine's continuing quest for speed.

Admin Generated List and Sorting

2010-01-14T18:53:00.000-08:00

I noticed that trying to sort a column in an admin generated list causes a white screen of death. I traced it down to the call_user_func_array() function and did a search on google.

I found this article here http://forum.symfony-project.org/index.php?t=rview&goto=90728&th=24521 which is the exact same problem I was having.

The suggested fix is to add


class_exists($peerclass);

before


return call_user_func_array($callable, $args);

in the propel/util/BasePeer.php in the getFieldnames() function

There is another function translateFieldname that also uses the call_user_func_array but I’m not sure if that function needs call_exists($peerclass).

Multiple Inserts with Prepared statements in Symfony 1.4

2010-01-14T09:54:00.000-08:00

I have a symfony 1.4 project in the works and I needed to create a function that would insert 5,000 records into a database.

In a previous symfony 1.0 project I would typically create a custom query and manually create the SQL statement but I recently just started working with symfony 1.4 and found that they use PDO and from my understanding and experience so far, I couldn't create my custom query that would contain

What I had to do was create a prepared statement. Here was the basic set up with Propel.


$connection = Propel::getConnection();

$sql = 'INSERT INTO table_1 (name, date) VALUES (:name, :date)';

$statement = $connection->prepare($sql);

foreach ($my_array as $key => $value) {
  $statement->bindValue(':name', $value['name']');
  $statement->bindValue(':date', $value['date']');
  $result = $statement->execute();
}

The above set up was faster than using a doInsert and inserting by creating an object and using ->save().

I found that the above setup was able to insert all 5,000 records without timing out where as using doInsert() only inserted ~4,600 records until it timed out.

Creating an object and using ->save() in a loop was the slowest method. I was able to insert ~3,700 records before it timed out.

I tested this out several times and was able to replicate the results.

Summary (Speed of inserting records from fastest to slowest)
1) Prepared statements
2) doInsert()
3) $object->save()

Benefits to using a prepared statement (ref: php.net)

The query only needs to be parsed (or prepared) once, but can be executed multiple times with the same or different parameters. When the query is prepared, the database will analyze, compile and optimize it's plan for executing the query. For complex queries this process can take up enough time that it will noticeably slow down an application if there is a need to repeat the same query many times with different parameters. By using a prepared statement the application avoids repeating the analyze/compile/optimize cycle. This means that prepared statements use fewer resources and thus run faster.
The parameters to prepared statements don't need to be quoted; the driver automatically handles this. If an application exclusively uses prepared statements, the developer can be sure that no SQL injection will occur (however, if other portions of the query are being built up with unescaped input, SQL injection is still possible).

Multiple Select & Fillin in Symfony

2010-01-08T11:39:00.000-08:00

So I came across an annoying issue the other day. I have a select box with multiple select as true in a Symfony 1.0 project with fillin turned on. If I select multiple options in the selectbox and submit the form incomplete so that I have errors, the form re-displays with only the first option I choose selected.

The title of this blog links to a forum post with the exact same issue. Unfortunately there isn't much feedback on the issue.

I've tried excluding the field from the validation yml with no success but my field is setup as an array so I'm not sure if I'm excluding it properly.

I'd appreciate any info/feedback on this issue.

IE 8 and Website screen issues.

2010-01-05T11:06:00.000-08:00

Well I've come across this a couple times on two different sites with two different issues.

First issue I came across dealt with, I believe it's called, "Infinite Menus" a javascript based menu system. Well in IE8 there were major display issues that would cause the whole menu to overlap one element throwing everything off. The site I was working with was built on Ender Technology's own frame work.

Second issue I came across was in a site built off the Symfony framework. The issue was that ajax'ed items were not properly resizing it's outer container. So if you ran an ajax function and had it fill a div, the outer container would not resize causing overlapping issues.

So the fix my colleague and I found was to use the tag shown below in the sites layout.

<meta http-equiv="X-UA-Compatible" content="IE=7" />

The link I provided in this blog points to a more in-depth analysis and use of the tag but it basically sets IE8 browsers to display the site in compatibility mode for whatever version of IE you'd prefer.

If anyone has any input on the downsides of using this please feel free to contribute.

IE 6 display problems

2009-12-10T16:46:00.000-08:00

Because of display issues on a recent project I researched how many people still use IE6. I was not expecting that it still has a user base of about 13%. (Nov. 2009). Here are some interesting thoughts / reasons why some people cling to IE6. From CSS-tricks.com:

1. Because they have to (30%)

I hear fairly regularly from people using IE 6 because that’s what is on their computer at work and they are not allowed to change it. Big companies are slow to change, that has always been true. I’m no security expert, but I’m told IE 7 is a much more secure browser, I would think that alone would be intensive for big companies to get their employees on IE 7. But still, from an IT perspective, I can understand how rolling out a change like this can take tons of man hours for seemingly little benefit, especially if using the web isn’t core to the companies business.

People might also hold onto IE 6 because they use a website that either does, or claims to, only work on that browser. These websites are getting fewer and far between but they are certainly out there.

2. Because they have an old computer. (20%)

If they are using Windows 2000 IE 6 is the most current browser for that operating system as far as I know. Not to mention ME and 98.

3. Because they actively don’t care / dislike change (35%)

I say “actively” because if they just “passively” don’t care, Windows has probably automatically upgraded them without them even knowing. If they are actively not caring, they are probably clicking “No” and “Cancel” on whatever dialog boxes come up suggesting upgrades. They are probably creating more work for themselves by doing so, but they don’t know that.

If anything does change on their system, they may even take drastic steps to have it restored to it’s original version. I bet Geek Squad people could tell your stories of people demanding their web browser be returned to IE 6 after accidentally upgrading. People get used to using software in a certain way to get what they need to do done, and they are prone to getting upset if anything interrupts that.

4. Because they don’t know any better (15%)

I believe this is the smallest portion. People that could or would upgrade to a better browser, if convinced, but just don’t know the advantages yet.

Check out the latest statistics on browser usage here:
http://www.w3schools.com/browsers/browsers_stats.asp

ALOHA FRIDAY !!

2009-12-10T14:10:00.000-08:00

ALOHA FRIDAY !!

Fireworks observations

2009-12-10T11:22:00.000-08:00

Here are my findings so far and keep in mind this is with little to no Fireworks experience. These are basically notes I made for myself that are hopefully helpful to others.

From what I found out so far is that Fireworks in not the solution to everything. It's a tool that's part of a greater tool. I would personally just use Fireworks to splice a layout in its basic structure export the css and images from there and work with either Dreamweaver or some other html editing tool.

By basic I mean, background images, containers, logos, icons, and maybe even a few form items.

This of course is coming from my experiences so far so my opinion and knowledge of Fireworks will most likely change in the future.

Copy and Pasting Text
If you copy and paste text from PSD files over to a fireworks file. Be sure to name the actual element appropriately as if it were a html element otherwise it'll name the element as the text you just pasted.

Splicing
When copying text from a PSD and pasting it into a FireWorks document make sure that the text container doesn't have an additional line break in it. I had an issue where I didn't realize there was an extra line break at the end of my text when I was thinking it had to do with the Leading aka line-height.

Note: 120% Leading is normal

When you're splicing images, I found that it defaults to an 80% quality, so you may want to increase the quality level or make your own option when 80% doesn't cut it.

Layers

I find it really helpful for me to set up layers into sections of a page ie. footer, header, main content section. I found it helpful to also place any rectangle container sections within the appropriate layer.

When selecting multiple layers by holding ctrl make sure you click on the name and not to the left of the layer otherwise it won't work.

When implementing a PSD layout using Fireworks be sure to merge all layers appropriately to unsure the project is organized once it's imported into Fireworks.

For instance if you have separate layers that make up one button you'll want to merge those layers into one and maybe just leave out the text for the button as a separate layer.

Also don't merge header elements together unless they're absolutely necessary. For instance if you have a header bg image and you can't tile it make sure you don't merge
logo's onto that header bg image. Separate it into one large background image.

Right click the whole page to duplicate it... Set the top page to master by right clicking the page and selecting "Set as Master Page". From there you can modify the copied page to your liking and maintain layer structure. Simply hitting ctrl+c and ctrl+v would not work if you're trying to maintain layer structures.

Windows

When minimizing windows such as the layers window, single click it to minimize otherwise you may end up clicking other buttons, like the trash can button like I was doing. *sigh*

Save As
Make sure you save your work as a png, otherwise saving it as psd might cause some issues when reloading it in fireworks.

Fireworks Trick - Be Precise

2009-12-06T15:08:00.000-08:00

Just a quick note to those of you out there using Fireworks CS4 and trying to get a quality XHTML / CSS / Images export that is easy to read and doesn't need a ton of tweaking afterwards.

BE MATHEMATICALLY PRECISE

When you're just quickly comping up wire-frames, it is fine to just rough up proportions, margins, padding, and spacing in general... but as you approach later revisions of the wire-frames, and they start to materialize as true page templates, start paying attention to the precise relative positioning of elements.

Let your containers, backgrounds, images, headers, and other content be spaced with dimensions you would imagine wanting in a CSS file. I.E.

Keep all row elements having precisely the same Y coordinates for containers, don't let it be off by a pixel.
Same with columns, but of course, let them all have the same X coordinates.
Let all objects within containers be separated from each other by precisely proportional amounts (i.e. 10 px, 15px).
Keep duplicated object groupings using precisely the same proportions, but at their respective X & Y coordinates.

The primary benefit of being so precise before you export to CSS & Images is the readability of your final CSS and XHTML files. The numbers will make sense, especially if another developer is involved. Knowing Fireworks, probably all you'll have to do is group the elements similarly, and maybe reduce a few redundancies, but you won't have to worry about adjusting pixels.

If you have to adjust pixels, make sure you do it in the fireworks file and re-export. Let the CSS developer override the Fireworks CSS in his own file rather than edit the export file. The goal is automation.

A potential automation, if you can keep your Fireworks file this precise, is an automated refactoring of the CSS pixel based export into relative spacing and sizing by calculating the proportions in the document into percentage based spacing / sizing. You could whip up a quick PHP script to do this.

Just some ideas on a Sunday afternoon working in Fireworks CS4.

Xml and php

2009-12-03T14:17:00.000-08:00

What is Xml?

XML stands for EXtensible Markup Language
XML is a markup language much like HTML
XML was designed to carry data, not to display data
XML tags are not predefined. You must define your own tags
XML is designed to be self-descriptive
XML is a W3C Recommendation

I have Xml, but I want to read it... What do I do?

The easiest thing to do is get SimpleXML on your server. If your server support is >= php 5 then it should be able to support it.


$file = 'YourFile.xml'
$xml = simplexml_load_file($file);

This will convert the well-formed XML document in the given file to an object.

*WHERE TAG_NAME IS THE TAG (not a constant or property)*

XML will be accessible through $xml->TAG_NAME
XML arrays will be accessible through $xml->TAG_NAME[0]
XML arrays can be iterated through using simple foreach statements


foreach ($xml->TAG_NAME as $tag) {
    echo $tag;
}

Values must be cast into an appropriately to get a non SimpleXML object


(string) $xml->TAG_NAME;
(int) $xml->TAG_NAME;

Easy Cheesy, 123 done!

Typography beyond fonts

2009-11-16T14:06:00.000-08:00

I do really appreciate smashing magazine and studies they do for the realm of web design… I reviewed this great study today and decided that it is a good follow up to my blog entry about what fonts to use in web design.

The fonts I was lamenting about previously are really just a small part of an overall larger field: Typography. Typography includes the font, font size, font color, background color, character and line spacing, etc.

Good typography will allow the website’s audience to comprehend the website’s content quickly and easily. It is therefore an extremely important part of web design, regardless of the type of website (unless it’s purely graphical)… after all it doesn’t matter if you are reading a friend update on a social network, a blog entry, a product description on an e-commerce site or a news post…. you want to read / comprehend the information as quickly and effortless as possible.

So using smashing magazines study and its analysis as a base, I have come up with the following rules of thumb for good typography:

Font Type: As discussed in the previous post. Here a quick refresher: When using a sans serif font for headline or body text use Verdana if possible, use Arial as a backup. Body text should usually be a sans serif font unless explicitly stated not to be. Please use Georgia as a Serif font. The contrast in between a sans serif font and a serif font makes Georgia a good choice for headlines if your body text is Verdana.

Background Color: The general rule is that the bigger the contrast (difference) in between background and text, the better the readability. That is why all content heavy websites have one thing in common: white background with black text! Sounds boring? Maybe so… but unless you want to make a bold design statement and people visit your website for mostly aesthetic reasons, it is advisable to keep at least the background of the body content white with black or dark grey text so users can get the info they came to get as easily as possible.

Headline Font Size: A font size of 18 to 20 points is ideal, but it can go up to 26.

Body Copy Font Size: A font size of 12 to 14 points, with 13 being the most popular.

Line Height: Best readability is achieved when the line height (and therefore the space in between new lines of text is increased) is 1.5

Line Length: you may have noted how cumbersome it is to read straight across a 1000px wide page. Line length should not to exceed 600px. The ideal line length is in between 500 and 600px.

Maximum Characters: The ideal amount of characters per line is 73 to 90.

Joomla experience with breadcrumbs, menualias, and SEF

2009-11-14T00:52:00.000-08:00

So today in my Joomla experience I began to seek out a problem dealing with SEF and Routing. I had specific goals in mind and wanted to execute them... so what were they?

SEF urls (search engine friendly)
custom urls
custom breadcrumbs
modified active menu to accommodate for the custom changes

So where to start?? My first stop.. JRouter and http://docs.joomla.org/Routing

[Component Routing]
Component routing is handled in the router.php under a specific component.

components/{yourcomponent}/router.php

There are two functions that make up the router.php file. These are a build function and a parse function. What does this mean for you? In a sense, you get to customize what is already in the component url and add your own flare.

[Component Routing - What I wanted to do]
http://www.oghead.com/component/flash/flash/80.

This url is what I started with after turning on the SEF feature in the administration. Ignoring the first part "component/flash", the routing.php file accepts the parameters "flash" and "80".

The "flash" is the component view your looking at and "80" is the id of the item being displayed. I wanted to be able to have the item name displayed in the url. To accomplish there are two functions listed under http://docs.joomla.org/Routing

function [componentname]BuildRoute( &$query )
function [componentname]ParseRoute( $segments )

My new url became http://www.oghead.com/component/flash/flash/80-mygamename.
*Done*

[Custom Urls]
Custom urls are a funny thing. I never did figure out what the "proper" way of doing them is. I spent a good 8-16 hours searching forums and threads and articles for a solution, but alas no help. I even went to the lengths of creating my first community account EVER and posted a thread on forum.joomla.org! As detailed a post as it was... no response!

The idea of custom urls is whatever your imagination leads you. If you want http://www.yoursite.com/michaeljackson to go to the Articles component more freedom to you! BUT HOW!

If you want http://www.yoursite.com/jane/josh/joe/jill/amy to go to the contacts page, congratulations! BUT HOW!~

Taking a step back. Urls are driven by the JRouting class that Joomla provides. Urls get changed based on the system's global configuration. If you have SEO Settings > Search Engine Friendly URLs turned on, then Joomla displays all links and urls differently.

The SEF makes urls go from...

http://www.yoursite.com/index.php?option=com_hello&view=hello_world&id=5

To...

http://www.yoursite.com/component/hello/hello_world/5

The second url is definitely a much prettier url. Urls are also driven by menus. This makes things problematic. If you click into a menu item, your urls become dependant on the menu item that is currently active.

[Custom Urls - What I wanted to do]
Turn...

http://www.oghead.com/component/flash/flash/80-mygamename
http://www.oghead.com/action/flash/80-mygamename

Into...

http://www.oghead.com/play/flash/80-mygamename

As you can see this is problematic. You do not want multiple urls pointing to the same end. This means that to get to item 80, there can be an N amount of URLs pointing to the same place! This means google is going to index N amount of urls to that location and a N amount of instances where that itself happens.

I tried looking for a Routing solution but I came up empty handed. Therefore that left me with a .htaccess solution.

In the .htaccess file located at the root web dir I added


#Force play flash into 1 URL
RewriteEngine On

RewriteRule ^play/flash/([\d]+)-(.+) index.php?option=com_flash&view=flash&id=$1

This grabs any url that has play/flash/{a number of some size}-{any characters following said hyphen}, it translate the url given into what Joomla can read.

Now...

http://www.oghead.com/play/flash/80-mygamename

Successfully points to...

http://www.oghead.com/index.php?option=com_flash&view=flash&id=80 (where $1 is the first parathesis match)

I had to change all my links to accommodate for the new url by calling "play/flash/id" instead, but now I have one pretty url per page.
*Done*

[Custom breadcrumbs]
Breadcrumbs are created based on menu items.

[Custom breadcrumbs - What I wanted to do]

http://www.oghead.com/component/flash/flash/80-mygamename (Breadcrumb: Home)
http://www.oghead.com/action/flash/80-mygamename (Breadcrumb: Home > Action)

==After the modifications above my url is now==

http://www.oghead.com/play/flash/80-mygamename (Breadcrumb: Home)

I wanted each link to have the breadcrumb of Home > Category > Mygamename

To fix this I made some modifications to my component view file.


$pathway =& $mainframe->getPathway();
$pathway->addItem({menu name}, 'index.php?Itemid={menu id});
$pathway->addItem({mygamename});

This adds two breadcrumbs onto the existing "Home" breadcrumb. The first parameter is the name, the second is the link the breadcrumb goes to.

Now my breadcrumb for this item is Home > Action > Mygamename
*Done*

[Active Menu Issue]
I set up my menu as different category views inside my component. These categories were Home, Action, Strategy, Etc.

An active class gets put on menu items in which are active. If you are at the "Home" page, then the cooresponding menu item "Home" gets the class of "active". Now you can style your template based on the menu item that is active.

However, this poses some problems with items we have already addressed.

http://www.oghead.com (Home is active)
http://www.oghead.com/component/flash/flash/80 (Nothing is active - If no menu item is active then it means you navigated to item 80 from the Homepage the default menu item becomes component/flash)
http://www.oghead.com/action (Action is active)
http://www.oghead.com/action/flash/80 (Action is active)
http://www.oghead.com/play/flash/80 (Nothing is active)

Notice the first two links. They should both have Home active just like Action is active both times. Since we changed the url to be play/flash however, we need to accommodate for this change and make the correct menu item active.

This is done by figuring out which menu item you should be at and overwriting the active menu item.


$menu = &JSite::getMenu();
$menu->setActive($flash->item_id);

Now since play/flash/80-mygamename is an Action game. We need to get the item_id that corresponds to Action and use the code block above to set it as the active menu item!
*Done*

Testing, Testing…

2009-11-09T13:28:00.000-08:00

I recently came across some interesting studies in regards to project testing. We already have a pretty powerful method with our alpha and beta testing procedures, as well as with our online issue / bug tracking system, but what I’ve read may help us to be even more efficient during this important part of a given project.

The most informative study that I read was one done by Jacob Nielsen, the so called guru of Web Page Usability, and his company Nielsen Norman Group. Although the study is older, I still believe its core findings are true… because if anything, websites / web apps have gotten more complex since 2000 and therefore offer more room for bugs.

The study finds that rather than one specific method of testing, the trick to effective testing is the use of multiple testing individuals (besides the original engineer). The benefits of multiple user testing as revealed in the study are astonishing: Just adding one other person (that is not deeply involved in the project) to test it out, finds a third of all bugs. Adding a second person brings that number up to 50% of all bugs found. At a level of 5 individual testers, the optimum ratio of testers to bugs found is achieved (about 85%). Any larger group of testers will not contribute significant enough amounts of bugs found to justify the added cost.

Conclusion: Always have your coworkers check out your work before showing it to the client, so you can gather their (most likely very helpful) feedback. Ideally have at least two other people that where not deeply involved in the project examine it and walk through the site / app as a general user of it would do.

Sending email with attachment in php

2009-11-08T22:33:00.000-08:00

Programmers using PHP language can the mail() function to send emails.

The purpose of this blog entry is to demonstrate how to send emails with attachment using the mail() function.

mail() function definition is the following:

bool mail ( string $to , string $subject , string $message [, string $additional_headers [, string $additional_parameters ]] )

To send an email with attachment we need to use the multipart/mixed MIME type that specifies that mixed types will be included in the email. Moreover, we want to use multipart/alternative MIME type to send both plain-text and HTML version of the email. Have a look at the example:

//define the receiver of the email
$to = 'receiveraddress@example.com';
//define the subject of the email
$subject = 'Test email with attachment';
//create a boundary string. It must be unique
//so we use the MD5 algorithm to generate a random hash
$random_hash = md5(date('r', time()));
//define the headers we want passed. Note that they are separated with \r\n
$headers = "From: webmaster@example.com\r\nReply-To: webmaster@example.com\r\n";
//add boundary string and mime type specification
$headers .= "Content-Type: multipart/mixed; boundary=\"PHP-mixed-".$random_hash."\" \r\n";
//read the atachment file contents into a string,
//encode it with MIME base64,
//and split it into smaller chunks
$attachment = chunk_split(base64_encode(file_get_contents('attachment.zip')));
//define the body of the message.
ob_start(); //Turn on output buffering
?>
--PHP-mixed-
Content-Type: multipart/alternative; boundary="PHP-alt-"

--PHP-alt-
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

Hello World!!!
This is a sample text email.

--PHP-alt-
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

Hello World!

This is an HTML email.

--PHP-alt---

--PHP-mixed-
Content-Type: application/zip; name="attachment.zip"
Content-Transfer-Encoding: base64
Content-Disposition: attachment

--PHP-mixed---

//copy current buffer contents into $message variable and delete current output buffer
$message = ob_get_clean();
//send the email
$mail_sent = @mail( $to, $subject, $message, $headers );
//if the message is sent successfully print "Mail sent". Otherwise print "Mail failed"
echo $mail_sent ? "Mail sent" : "Mail failed";
?>

In the example above, to include an attachment to our message, we read the data from the specified file into a string, encode it with base64, split it in smaller chunks to make sure that it matches the MIME specifications and then include it as an attachment.

On the side note: make sure you don't have any white spaces like tabs or spaces before Content-type.

Rackspace Cloud Sites Gotchas

2009-11-03T14:33:00.001-08:00

Rackspace Cloud Sites Gotchas.

I'll share a few gotchas we came across on a recent deployment that are:

Symlinks aren't supported! Sure, your symlink may work today, but Rackspace isn't concerned or responsible if it magically breaks tomorrow.
Default session storage /var/lib/php/sessions is not cloud persistent. Cloud node goes down or too busy to handle the request? Session lost. Be sure to store sessions in web/sessions for security and persistence.
No standard error messages. Cloud down? (Yes, the cloud has gone down.) You'll get a nice "No suitable nodes are available" error page. No error branding here.
Do you like a shell, rsync, or scp? We do too. Too bad Rackspace doesn't. Use SSHFS + Rsync to at least minimize the frustration. Use PHP Shell for a shell.
Using sshfs and want to move your secure passwords to a secure publickey? Shucks. You can't.
Developed your app with the amazing Symfony framework? Good. Just you can't symlink and you can't change DocumentRoot or add an Alias. So rename that web/ to content/! I hope you used sfConfig::get('sf_web_dir') in the code ;)

A whole lot of space… white space that is…

2009-11-02T12:47:00.000-08:00

Smashing Magazine explains white space well:

Most designers know the value of white space, which is the empty space between paragraphs, pictures, buttons and other items on the page. White space de-clutters a page by giving items room to breathe. We can also group items together by decreasing the space between them and increasing the space between them and other items on the page. This is important for showing relationships between items (e.g. showing that this button applies to this set of items) and building a hierarchy of elements on the page.
White space also makes content more readable. A study (Lin, 2004) found that good use of white space between paragraphs and in the left and right margins increases comprehension by almost 20%. Readers find it easier to focus on and process generously spaced content.
In fact, according to Chaperro, Shaikh and Baker, the layout on a Web page (including white space, headers, indentation and figures) may not measurably influence performance but does influence user satisfaction and experience.

So why do you as an engineer / project manager need to consider this?

Giving your layout / design adequate white space starts with the wire frames / prototypes that we create for most of our projects. Since prototyping is essentially the first step of page design (because you determine a rough layout, elements on the page, etc), giving your items on a page adequate white space will help you and your client to determine the right amount of content and layout of a page. It will help your architectural decisions, because you can determine from your prototype how good the page flows, how well information is perceived, what elements are more prioritized than others, etc. So please allow for enough room when prototyping and make sure that clients with a lot of content to cram onto one page, understand why it is important to let the content breath.

Webserver Batch Printing to Networked Printers

2009-10-30T06:10:00.000-07:00

This will be the first of a series of posts that covers a solution that allows printing of multiple documents stored on a webserver to individual user printers.

The Need
Recently a client requested a way for their office personnel to be able to print multiple documents at once. The documents were stored on a webserver, and the way they had been doing it was to download each individual document, open it up, and then issue the print command from the application. A slow and tedious method for sure, especially when they had to print more than a few documents at a time.

The Solution
After some thinking, I determined the best way for this to happen was to have the webserver itself send the print jobs directly to the user's printers. This would remove the need for the user to download and open anything, and the server could send multiple documents to the printer at once. The only requirement was that each of the user's printers had to have a network interface. I considered first just having a printer directly connected to the webserver, but for this situation it was not practical because:
1) the webserver was not accessible to the users.
2) a single printer receiving multiple batch print jobs would get bogged down, and users would have to wait.
3) the users would be inconvenienced by having to retrieve their print outs from another location other than the printers at their desks.

Now that I had an idea of the solution, I needed the technology to make it happen. I found that CUPS (Common Unix Printing System) had the capability to turn the webserver into a network printserver, and in turn direct output to specific network printers. I installed CUPS, and then through its painless management interface (accessed through a browser on port 631), I set up each of the network printers by specifying a name and IP address for each. Now that the printers were setup, I could simply print to them by sending 'lpr -P <printer_name>' from the command line. CUPS also takes a variety of command line arguments that can help fine tune the printout, if needed (to be covered in a later post).

For the batch print interface, I added a 'Print' button to the documents list page. The page has a filter so the users can specify which document set they wish to print. When they hit the print button, any documents returned by the filter are sent via CUPS to their personal printer. The print button itself is a submit button that executes a routine on postback that takes the current filter query, selects the documents, and then loops through the result set issuing the 'lpr -P <printer_name> <file_location>' command for each.

The only trick left was how to determine which printer should receive a job based on the user. I accomplished this by adding a field to the user table that stores the CUPS printer name for the assigned printer. Since user info is stored in the session after they login, the printer name is easily accessible at any time.

Conclusion
That covers the overview of the solution. In future posts I'll go over the more technical details of the solution. I'll be covering CUPS printer setup and management, installation of printer drivers, and queue management.

Above the fold

2009-10-26T15:24:00.000-07:00

Ok, so we are all on board with the ideal page width because of my recent blog entry, right (that would be nice). So let’s get into the height of a page, positioning of main content and the users willingness to scroll…

We’ll start with the ideal page height, or where you can assume a page break will show for the majority of users. That magic number is 768 pixels. So if a client is set on having a no scroll page, no content should go below 768px from the top. A clients insistence on no scroll often times leads to cramping of the content, loss of adequate white space and therefore to poor readability of the content.

Rather than completely restricting page height, we should try to guide these types of clients towards prioritizing content. The basic rule for what should always be above the fold is this: Name / logo CI stuff; Navigation for the main sections; Value proposition of the website (i.e. what benefit users will get from using it).

These thoughts above where spurred by some articles / research I recently read that are concerned with a user's willingness to scroll. Here are links to a couple if you want deeper info:

http://www.boxesandarrows.com/view/blasting-the-myth-of

http://blog.clicktale.com/?p=19

The conclusion for the research I read was this: 78% of the users were actually scrolling down a page automatically, (as long as the content was exactly what they where looking for) because they are so into reading what is new or what this website can do for them.

SQL to grab latest records based on date field

2009-10-23T13:02:00.001-07:00

This example will show you how to grab the latest records from a table (based on an effective-date field) using only one query.

For this example, let's say we have the following products table:

item      effectiveAt    price
----      -----------    -----
shoes     2009-10-07     1.29
shoes     2009-08-03     1.59
shoes     2009-04-06     2.29
shirt     2009-07-13     1.47
shirt     2009-08-28     1.59
tie       2009-04-28     2.57
tie       2009-07-09     3.19
tie       2009-09-05     3.57

The most common way people address this is to use a subquery such as:

SELECT *
FROM products AS p1
WHERE effectiveAt = (
 SELECT MAX(effectiveAt)
 FROM products AS p2
 WHERE p1.item = p2.item
)

The problem with this query is that for every unique item, a subquery is run. That can be a lot of queries and it's generally a good idea to avoid subqueries if at all possible.

Here is how to get the same results with only one query:

SELECT * FROM products AS p1
LEFT JOIN products AS p2 ON (
p1.item = p2.item AND
p1.effectiveAt < p2.effectiveAt) 
WHERE p2.effectiveAt IS NULL

This query selects each item and left joins another record with a greater effective date. The one actually returned is where p2.effectiveAt IS NULL, which means p1.effectiveAt is the greatest effectiveAt for that product. To get a better idea of how it actually works, just run the query without the 'WHERE p2.effectiveAt IS NULL'.

Joomla Partials

2009-10-23T11:53:00.001-07:00

HOW THE HECK DO YOU DO PARTIALS IN JOOMLA!

This is one of the most frustrating issues I have seen in Joomla. There is no method of reusing code in Joomla.

Joomla is broken down into 3 major parts.

Components
Modules
Views of Components or Modules

Joomla does allow the reuse of modules. However, how often do you want to include multiple modules on the same site? Cool ....

If you include a twitter module feed. It is possible that you might want to include different twitter feeds, but more likely than not you will just have one site feed.

However.. It is MORE likely that you will want to reuse the Views of Components or Modules. It is possible that you want to create one form and use it multiple places. It is possible that you have repeated code in different view directories.

If this is the case you do NOT want to have the same code repeated whatsoever, complex or not. Joomla does not provide a method for this. Instead they offer the newbie solution of...


$html = '';
$html .= '<div>';
$html .= 'Your Content';
$html .= '</div>';

echo $html;

BUT I have a solution... Thanks Robbie!

For now, I included this in my custom components helper file, but once I have access to custom created site-wide functions (e.g. JSite, JRequest). I will refactor it so that I can pass in components and call the function from anywhere to anywhere.


  /**
   * Helper Functions
   *
   */
  function include_partial($name, $params = array()) {
      $partial_file = JPATH_COMPONENT.DS.'helpers'.DS.'_'.$name.'.php';
   
      if (!is_file($partial_file)) {
          JError::raiseError(500, JText::_($partial_file.' not found!') );
      }
   
      ob_start();
   
      extract($params);
      require($partial_file);
   
      $partial = ob_get_clean();
      echo $partial;
  }

This function allows you to pass in the name of the partial file and an array of parameters. These parameters will be available to the partial file. You can then write actual html/php without being retarded. No more concatinating html into strings.

This is a replicated symfony slot, but easy enough to use as a get_partial.

Xie Xie. Arigato. Gracias. Thank you.

Joomla Exceptions

2009-10-23T11:34:00.000-07:00

Joomla Exceptions! Another piece of the puzzle.

Starting Joomla can be frustrating.. Especially when you may be use to another framework or style of coding... Thoughts run through your mind, "What is Joomla's equivalent method!".

If you are trying to prevent people from accessing pages that you create. One of the ways to do that is by throwing an exception. Most of the time you want to prevent people from entering in bad urls.

Yoursite.com/index.php?my_content_id=apple

Most likely you do not want to accept apple as an id parameter. This can be solved by throwing an exception if valid content is not retrieved from "my_content_id=apple"

So how do you handle Exceptions?

Using or customizing the exception page under templates/system/error.php
Throwing an exception: JError::raiseError( 404, JText::_('ALERTNOTAUTH') );

You can be fancy and create a function to throw dynamic errors from this if you wish. It would be best if you put this function in some sort of static method site-wide, but I still haven't found a way to create site-wide custom classes or libraries (e.g. JSite, JRequest, etc)

FireWorks CS 4 XHTML & CSS Export Function - It Works! Here's how...

2009-10-22T17:25:00.000-07:00

Well, actually, it's been many years since I seriously tried FireWorks for prototyping websites and generating template code... I gave up on earlier versions, then just sliced and coded by hand.

But the advent of FireWorks CS4 promises the ability to export in valid XHTML 1.0 Transitional code and quality CSS... so I thought, what the hay, let's give it another shot.

And it's pretty good!

Yes, you DO have to follow a set of best practices to get optimal results, but they're all reasonable, understandable, and relatively easy to implement.

Here's the beauty... FireWorks CS4 now recognizes blocks of text as just that, blocks of text. You no longer have to slice them out as graphics and then replace the guts later. In fact, FireWorks CS4 now does a great job of recognizing "space" in general.

Essentially now, you just slice your graphics, but leave your text and your blank spaces, and FireWorks CS4 will export quality XHTML with relative positioned CSS if you follow some best practices like:

Do not allow your element containers to overlap (otherwise FireWorks CS4 will output absolute positioned div's).
Let your element container widths be consistent. For instance, if you're prototyping a left column with a content area of 150px with 5px margins, make sure all the text blocks you put in the column are 140px wide, even if it's just a one word header. This is so that when you export, the container div that gets created will have the proper width value, and if you happen to turn that one-word header into two, the second word won't break to the next line.
Use Guides. The guides let you set up the margins and paddings of your document, and give you consistent lines to snap to when placing elements.
Use the FireWorks naming conventions. Now, you can take any text object, and preface it's name with a tag, and when exported, that text element will be wrapped with that tag, with the ID being the text object's given name. Only certain tags work, the default tag is a P tag.
Use background containers. If you draw a rectangle, and then put text & symbols on top of it. FireWorks CS4 generates a container div the size of that rectangle, and all elements within it become subordinates of that container. (Hmm, I haven't tried if elements are only partially within the rectangle.)

Those just a few general hints to get your documents looking right. Adobe provides some great, more in-depth tutorials. Here is an excellent tutorial that can take you step by step if the above isn't clear.

Watch your fonts

2009-10-22T10:54:00.000-07:00

I’m writing this, so that in the future we may all be on the same page when it comes to choosing the right font for the body text of new developments. I’m sure I don’t need to go into why you need to use web safe fonts… so I want to focus on which are the best of the web safe fonts and why to use them.

Let’s start of with a list of basic web safe fonts:

Serif fonts:

Times New Roman
Times New Roman is a Windows serif font that looks like something you'd see in a newspaper or magazine. It's specially hinted for easier reading on-screen, but Microsoft's free font Georgia is really a better typeface for the screen.

Times
Times is a Mac serif font. It's similar to Times New Roman, but it isn't as carefully designed for on-screen viewing.

Georgia
Georgia is a Microsoft's serif face. It's much easier to read on screen than other serif Web fonts because it was designed for the screen.
Georgia has excellent italics that are easy to read and attractive.

Sans Serif fonts:

Arial
Arial is a Windows sans serif font that has a streamlined, more modern look, but isn't easy to read on screen because it's narrow and can look very light in smaller sizes.

Helvetica
Helvetica is a Mac sans serif font similar to Arial.

Verdana
Verdana is an extremely easy-to-read sans serif font that's included with the Internet Explorer. It was designed for the screen.

Tahoma
Tahoma comes with Microsoft Office and is almost identical to Verdana.

Conclusion:
If you are working on an existing site that has an established font that the user is used to, go with it. But for all new developments, please use Verdana as the first choice (default) for all body text. Should the client request a serif font, please use Georgia. The reason is simple… as you can see above both fonts where designed for on screen and therefore deliver better readability.

Ideal page width for new web design projects

2009-10-15T16:13:00.000-07:00

Ideal page width for new web design projects

I started doing some research about this topic after a recent discussion with Will, when he told me that he had to reduce the page width of ACFA to 960 instead of a 1000 pixels.

After reading some forums / blogs on the topic, it seems like 960px is the assumed optimum width for the majority of web designers. This takes in consideration space for scroll bars. Several other design / development companies have published articles (like this one: http://www.diginovations.com/web-video-expert/2009/9/9/its-time-to-re-think-the-ideal-page-width-for-web-design.html) based on raw data extracted mostly form Google Analytics, to make 960 pixels the new minimum page width.

It is important to consider, that there is still a small but resistant group of “conservative” designers, which believes 775px is the ideal width. Their argument: some people don’t browse with their browser maximized, plus they might have their favorites / history open or other plug-ins such as toolbars taking up additional space.

Interesting….

d

use your webserver to process php cronjob scripts

2009-10-15T10:13:00.000-07:00

Ever wish you could run php cronjobs on a webserver without running the script in cgi mode? cgi mode adds another layer of complexity. If your webserver itself runs the script, then you know as long as your server is up, it's processing your cronjobs. Or perhaps maybe you're on a server that doesn't have php cgi available.
If your server is running on Linux, you can use 'wget' to call your scripts and redirect the output to null. Here's an example:
0 12 * * * wget -O /dev/null http://localhost/myscript.php

And if needed, you can also pass query string args as usual:
0 12 * * * wget -O /dev/null http://localhost/myscript.php?hour=12
0 18 * * * wget -O /dev/null http://localhost/myscript.php?hour=18

Symfony MIME Type File Validation

2009-10-14T22:37:00.001-07:00

http://www.symfony-project.org/book/1_2/10-Forms#chapter_10_sub_standard_symfony_validators

(scroll down to sfFileValidator)

Be careful relying on the MIME type validation of the File Validator in Symfony. The MIME type of the file actually comes from the browser (aka user) as part of the upload.

As I'm sure you already know, this (and any) user data should not be entirely trusted! Just because it passed the "image/jpeg" MIME test does not mean it's a JPEG image.

In addition, if you're planning on validating uploads like FLV files then you'll be in for a headache with the different MIME types that browsers will throw at you. Further muddling this mess is that if the browser can't determine what it is then it gets sent as a generic "application/octet-stream".

Your easiest bet for these two problems is to detect the file's MIME type yourself. This way you're reasonably sure it is what it should be and you now have a consistent (hopefully single) MIME type to test against. (View notice below!)

One such solution could be PHP's Fileinfo extension.

BEWARE: Your validation routine is not complete!

The reason why is that the MIME detection is only a method of detection, it is not a method of validation. The issue is that the MIME detection only tests particular parts of a file, such as the first few bytes, to determine its type. This is commonly done via a magic file.

This does not test that the file is a legitimate GIF image for instance. It could very well be corrupt, and even downright malicious (CVE-2004-0200 comes to mind).

For a more thorough image validation, you can attempt to open the image with GD and see if it chokes. Also, ImageMagick's identify application is available.

However, once again, this will probably not test for a malicious image. This level of security checking requires specialized detection routines that need to be kept up to date. They can be installed with special applications (Snort), Perl scripts, or even PHP functions that simply detect a special byte ordering.

Protect your website and your users.

Note: Although I focused on Symfony and images for the examples, the same level of security should be applied to all user uploads.

Note: Symfony 1.2 attempts to locate finfo_open and mime_content_type, and the file tool in sfValidatorFile validator. However if none are found, then it still uses the browser-provided MIME.

The Cloud: Brief Overview

2009-10-13T17:30:00.000-07:00

The term "cloud" is used to portray an image of abstraction from the internal infrastructure. From there you attach a word to describe your services such as "cloud computing" or "cloud hosting".

Clouds have roughly three levels of offerings:

Application
Platform
Infrastructure

Depending on your degree of control determines which level you're offered.

Application level provides only the application for you to use. It's appearance could be just a website, the fact that it's running on "the cloud" is almost completely unknown to the consumer.

Platform level provides you an area to run your programs that sometimes have to be made "cloud aware" or "cloud compatible" with that service provider. Offerings such as Rackspace Cloud Sites will just appear as simple shared hosting.

Infrastructure level provides a logical server or other "hardware" devices such as a managed switch. You can control the actual server services and configuration thus allowing you to install applications like gearmand.

Examples:

Application - Salesforce.com
Platform - Google App Engine
Infrastructure - Amazon EC2

Currently with non-cloud hosting such as Ender's Los Angeles shared hosting, only bandwidth was an important measurement. However, with cloud offerings an additional important measurement is "compute cycles" (essentially CPU usage) used by the applications making application optimization an important consideration beyond just Quality of Service.

Google translator needs a human touch

2009-10-12T17:15:00.000-07:00

Soon I will be translating Ender Technology's web site to Spanish. There are some technical terms that I need to get familiar with so I have started doing my research and started playing with Google's translator. Google translator can be very useful when translating a word and even phrases but I am not sure about full sentences. For instance, our home page reads: ' Welcome, Ender Technology is a Los Angeles web development company providing a broad range of technology services from simple web design to the planning and development of complex web applications.' I used the google translator to translate this text to Spanish and I got: 'Bienvenido, Ender La tecnología es un desarrollo de Los Angeles web de la compañía proporciona una amplia gama de servicios de tecnología de diseño de páginas web sencillas a la planificación y desarrollo de aplicaciones web complejas.' Uh huh?? If you speak Spanish, you might be able to understand the message after reading it ten times but is anyone going to spend that extra energy on trying to understand the text in a website? I really doubt it.

This tool is definitely useful to find out how to say/write a word in another language but if you are going to implement it in your site I would reconsider it. Translating is not the same as interpretating and in order to interpret a message there needs to be a human touch.

The Best Open Source Live Chat for Websites

2009-10-11T02:18:00.000-07:00

I just discovered Crafty Syntax this past week in looking for a Live Chat solution for our corporate site.

I checked out a few others, Volusion's Joomla plugin, as well as the Chatango Joomla plugin... but:

Volusion's solution is really just a loss leader to get you to buy their commercial version. The free version is woefully under - featured.
Chatango is simply janky.

Both solutions require desktop software to be installed, which I'm not too fond of.

Crafty Syntax on the otherhand, is totally free (though I did donate), and completely web based. It has an automated setup process, and lots of configuration options. It basically provides everything the commercial version of Volusion does... albeit, in a not-quite-so-sexy interface.

My favorite features include:

Referrer reporting (where did your user get to your site from? what keyword?)
Page tracking (what path did the user take through your site?)
Chat invite & auto-invite

And of course, all design / graphical aspects are totally customizable so you can make it suit your brand.

I'm not sure how effective a website live chat will be to boost sales with a boutique web development shop like ours, but I'll give it a try and we'll see.

Google Results Depend on Whether You're Logged In

2009-10-11T02:07:00.000-07:00

Just an FYI for those of you who haven't already noticed, but Google results seem to differ depending on whether you're logged in to Google or not.

For instance, when I search "los angeles web design" and I'm logged in, my site, EnderTechnology.com, comes up #1!!! But, if I log out and do that same search, we're like on page 3.

Super annoying, because now I can't trust the results if I'm trying to do a search engine analysis. I have to open a window with a separate logged out session. I wonder why / how they are doing this?

Perhaps it occurs only if you have the Google Toolbar installed, because clearly they're tracking which websites I've visited most, and giving some kind of preferences to those in their rankings.

Don't like it...

Google Earth DIV Overlay via Iframe Shim

2009-10-09T11:49:00.000-07:00

Need to create a div overlay on top of a Google Earth browser plugin rendering?

Use an iframe shim and load it AFTER the GE has rendered!

Use a timeout or render it when via a GE event such as a balloon click.

Unfortunately, it appears that transparency does not work with the overlay.

New template engine for Symfony?

2009-10-07T16:34:00.000-07:00

http://fabien.potencier.org/article/34/templating-engines-in-php

Will Twig be integrated with Symfony 2.0? Maybe a plugin for 1.0+?

The Symfony templating component has already been released:

http://components.symfony-project.org/templating/

Twig will probably will remain as its own project like Propel and Doctrine.

I wonder if Fabien's Smarty benchmarks used the compiled templates.

Edit 2009-10-12: Fixed my ramblings with descriptions

Symfony 1.2 Pagination Max Per Page Selection

2009-10-06T16:37:00.000-07:00

With symfony 1.2, it is fairly easy to paginate a list of results with the sfPager object, but not every nicety is given.

Here is a simple PaginationMaxPerPageForm class using the symfony 1.2 form framework:


<?php

class PaginationMaxPerPageForm extends sfForm
{
 protected
   $user = null;

 /**
  * Constructor.
  *
  * @param sfUser A sfUser instance
  * @param array  An array of options
  * @param string A CSRF secret (false to disable CSRF protection, null to use the global CSRF secret)
  *
  * @see sfForm
  */
 public function __construct(sfUser $user, $options = array(), $CSRFSecret = null)
 {
   $this->user = $user;

   parent::__construct(array(), $options, $CSRFSecret);

   // the defaults depend on the options, so set them after construction
   $this->setDefaults(array('max_per_page' => $user->getAttribute(self::getMaxPerPageName(), self::getMaxPerPageValue())));
 }

 /**
  * Changes the current max_per_page attribute.
  */
 public function save()
 {
   $this->user->setAttribute(self::getMaxPerPageName(), $this->getValue('max_per_page'));
 }

 /**
  * Processes the current request.
  *
  * @param  sfRequest A sfRequest instance
  *
  * @return Boolean   true if the form is valid, false otherwise
  */
 public function process(sfRequest $request)
 {
   $data = array('max_per_page' => $request->getParameter('max_per_page', $this->user->getAttribute(self::getMaxPerPageName(), $this->getDefault('max_per_page'))));
   if ($request->hasParameter(self::$CSRFFieldName))
   {
     $data[self::$CSRFFieldName] = $request->getParameter(self::$CSRFFieldName);
   }

   $this->bind($data);

   if ($isValid = $this->isValid())
   {
     $this->save();
   }

   return $isValid;
 }

 public function configure()
 {
   $this->setWidgets(array(
     'max_per_page' => new sfWidgetFormSelect(array(
       'choices' => self::getMaxPerPageChoices(),
     )),
   ));

   $this->setValidators(array(
     'max_per_page' => new sfValidatorChoice(array(
       'choices' => array_keys(self::getMaxPerPageChoices()),
       'required' => false,
     )),
   ));
 }

 protected function getMaxPerPageChoices()
 {
   if (
     isset($this->options['max_per_page_choices'])
     &&
     is_array($this->options['max_per_page_choices'])
     &&
     !empty($this->options['max_per_page_choices'])
   )
   {
     return $this->options['max_per_page_choices'];
   }

   return array(
     5 => 5,
     10 => 10,
     25 => 25,
     50 => 50,
   );
 }

 protected function getMaxPerPageName()
 {
   if (
     isset($this->options['max_per_page_name'])
     &&
     is_string($this->options['max_per_page_name'])
     &&
     !empty($this->options['max_per_page_name'])
   )
   {
     return $this->options['max_per_page_name'];
   }

   return 'pager_max_per_page';
 }

 protected function getMaxPerPageValue()
 {
   if (
     isset($this->options['max_per_page_value'])
     &&
     is_numeric($this->options['max_per_page_value'])
     &&
     !empty($this->options['max_per_page_value'])
   )
   {
     return intval($this->options['max_per_page_value']);
   }

   return 10;
 }
}

With this code in your project's lib/form/PaginationMaxPerPageForm.class.php file, you can create the following in a template:


<?php
$form = new PaginationMaxPerPageForm($sf_user, array(), false);
$form->process($sf_request);

echo $form->renderFormTag(url_for($sf_request->getUri()), array(
  'method' => 'get',
));
?>
<div class="max_per_page">
<?php
echo $form->renderGlobalErrors();
echo $form->renderHiddenFields();

if ($sf_request->getParameter('page')) : ?>
<input type="hidden" name="<?php echo 'page' ?>" value="<?php echo $sf_request->getParameter('page') ?>" />
<?php endif;

echo 'Show';

echo $form['max_per_page']->render(array('onchange' => 'this.form.submit()'));
echo $form['max_per_page']->renderLabel('items per page');
echo $form['max_per_page']->renderError();
?>
<noscript><input type="submit" value="Submit" /></noscript>
</div>
</form>

NOTE: the above url_for() value in the renderFormTag() assumes you will be submitting the form to the current page URL.

In the related template's action class (which should have a pager object), you can check for the max per page value with something like the following:


<?php
$max_per_page = $request->getParameter('max_per_page', $this->getUser()->getAttribute('pager_max_per_page', 10));
?>

Don't forget, you can customize some of the values used by the PaginationMaxPerPageForm by setting some options in the second parameter array, like:


<?php
$form = new PaginationMaxPerPageForm($sf_user, array(
  'max_per_page_name' => 'myMaxPageVar',
  'max_per_page_value' => 100,
  'max_per_page_choices' => array(
    10 => 10,
    50 => 50,
    100 => 100,
    150 => 150,
  ),
), false);
?>

So simple.

Visually stunning portfolios

2009-10-06T15:07:00.000-07:00

I figured I would cover visual topics…
So let’s kick it off with some visually stunning pieces. They are all flash driven photographers’ portfolios. Not only is the design of the flash player very clean, minimalist with an eye to avoid clutter, but also the content is visually appealing. So besides highlighting a current trend in flash design, these sites showcase another important hurdle we often face in the visual component of web design: the content has to be good for the site to rock! Let’s enjoy the following, even though we may have to deal with a bit of load time…

http://www.jamesdayphoto.com/

http://www.steffenjahn.com/publish/html/index.html

http://www.shuakashi.com/

Check out the Gordon Ramsey picture under Fame
http://www.johnwrightphoto.com/

Foomla Jail?

2009-10-06T14:05:00.000-07:00

I spent my last two Monday nights, till the wee hours mind you, trying to finish a custom Joomla Component. I am creating a Flash game component for my site oghead.com. The component will allow images and flash files to be uploaded and will be categorically displayed on the corresponding frontend page. Easy right?

Wrong... Creating a component for Joomla is the most taxing thing that you can do in Joomla. Joomla offers terrible documentation and guidance for creating components. Since Joomla is so extension driven, it is a shame to see such poor implementation practices.

In my last two programming sessions, I was trying to install a system for uploading flash files and their images. The files had to be uploaded to specific directories that may or may not have to be created. The directory structure I created was a file name based system. However, this posed some problems. If you try and rename a file to one in which was already taken, you would either A, not be able to do it, or B, overwrite what was already there.

I had to come up with another alternative. Now... each directory is based off of a file name structure consisting of id_name. This fixed the problem posed earlier. To supplement this change I had to implement a routine to move the directory based on the changing files names.

...All fine and dandy, but this was barely a dent into the component itself. I was expecting for libraries to handle most of this... So.. moving onto the code...

Joomla does have some basic file and folder libraries, however, they do not work properly, or did not in my instance. Since the libraries were spitting out incorrect values, I was left to hand coding an implementation. I ended up having to use base php functions such as @mkdir, move_uploaded_files, file_exists, and rename. This wasn't really a problem, but having to hand code something that should already exist is a pain (time squambler - yes I said squambler).

So after spending so many hours just trying to get file uploading working correctly. That was just a small segment of what a component is. To summarize, a component is a large file structure, of possibly useless functions, to house segmented functionality. Each method Joomla has to offer has to be recoded by hand. There isn't any predefined methods or functions for handling these measures. That means I am left with hand coding or copying/altering code to finish off the implementation. This also poses problems because when implementing all of those ambiguous methods, warnings and notices end up creeping in... and you're like...why?!?

Anyways... There are some means of creating cookiecutter components, but they are still lacking a gross amount of implementation. See here.. http://www.notwebdesign.com/joomla-component-creator/index.php

This still is a poor option. Creating a component file structure is terrible enough, but having to implement 40 functions, AGAIN, that should already be defined is just wrong.

Walking through a custom component yourself (creating a complicated one from scratch) is definitely good practice in learning Joomla, I won't argue, but there is definitely a need for autogenerated code. There are so many functions and methods which seem useless to me. But with poor documentation, you cannot even begin to fathom Joomla's, perhaps pointless, architectural decisions.

This leaves us with a bit of a dilemma. Right now my framework options are either Joomla or Symfony. Use Joomla, have a wide range of extensions (majority of which are subpar and written subpar-ly), and have super slow custom development. Or use Symfony, have a quicker custom development, but have limited access to extensions....

If I can harness and create autogenerating components, I think the roughest edges of Joomla should be softened, at least a bit. So Joomla, you win for now.

Gearman and Poison Messages/Jobs

2009-10-06T13:50:00.000-07:00

"Gearman provides a generic application framework to farm out work to other machines or processes that are better suited to do the work." (gearman.org)

gearmand, the server, includes an important feature that should be enabled for all installations:

-j, --job-retries

This feature protects against poison messages/jobs which is not enabled by default.

Poison Message: A message in the queue that has met the threshold of allowable retries.

For Gearman, this would be when a worker receives a job but disconnects from the server without advising the job's completion whether success or failure.

A disconnect can occur due to an error in the worker's processing of a job such as "out of memory", an error in the processing such as a "Segmentation fault" when resizing an image, or even an error in the Gearman PHP extension.

Now why is this important?

Let's run a scenario:

Worker connects to gearmand
Worker receives job
Worker begins processing job
Worker dies due to an error and disconnects
Gearmand sesnses disconnect
Gearmand redispatches job to another worker
Lather, rinse, repeat until all workers crash

If you set --job-retries=3 then as soon as the above scenario happens three times, gearmand will delete the message and log the error saving the rest of your workers from similar fate.

What if I want to handle that deleted job?
(for example, emailing the client that their job failed)

Unfortunately, currently, you will have to scan the log looking for deleted jobs and manually handle them.

However, aligned with Ender Tech's open source commitment, I have submitted a feature request to have the message redispatched to a "Dead Letter Queue" or "Poison Queue":

https://bugs.launchpad.net/gearmand/+bug/442539

This way you can assign a worker to handle the dead jobs without scanning the log thereby allowing a fast, robust, and consistent job handling infrastructure.

Joomla Email Cloak and Email Form Fields Don't Mix

2009-10-03T23:37:00.000-07:00

Just discovered this slight annoyance with Joomla.

I was integrating a custom form into a Joomla module using this cool plugin.

If my form validation logic invalidated the form and re-routed back to the form, where there should be the user's email address echo'd back to him, would be a JavaScript tag that echoes an email address so as to mask it from email harvesting bots.

Turns out this is on account of the default Joomla 1.5 plugin "Content - Email Cloaking". Disabilng the plugin solves the problem, but then you lose the masking. It would be nice if somehow the plugin could distinguish the context, and if the email addresses is quoted inside a "value" attribute, leave it alone.

Quick Tweet with PHP - How to integrate Twitter with 20 lines of code

2009-10-03T22:56:00.000-07:00

I found this tidbit on php.net. Super quick and easy tweet from within PHP courtesy of Fabien Potencier.

<?php
function tweet($message, $username, $password)
{
  $context = stream_context_create(array(
    'http' => array(
      'method'  => 'POST',
      'header'  => sprintf("Authorization: Basic %s\r\n", base64_encode($username.':'.$password)).
                   "Content-type: application/x-www-form-urlencoded\r\n",
      'content' => http_build_query(array('status' => $message)),
      'timeout' => 5,
    ),
  ));
  $ret = file_get_contents('http://twitter.com/statuses/update.xml', false, $context);
 
  return false !== $ret;
}

?>

Pretty easy, no? Using the tweet() function is of course a piece of cake:

<?php
tweet('From PHP, yeah...', 'fabpot', 'Pa$$');
?>

Symfony PHP Framework - The Experts Choice

2009-10-02T00:33:00.000-07:00

I've been building PHP web apps since 1998, and I've built them in all sorts of ways.

Of course my initial attempts were amateurish, the type of procedural code you see from newbies, but as my experience grew, my techniques became more sophisticated, and my understanding of why advanced techniques were necessary became more clear.

Before Symfony, I took several shots at my own framework. First, with PHP4, just a loose set of common functions organized around common needs like content management, shopping carts, form processing, etc.

I quickly discovered the need to automate the creation of forms and form processing logic from database schema, and whipped together a procedural system for that.

As my company grew, i.e. I started hiring employees, I realized that this loose methodology didn't scale and extend very well. It worked if you built it and you knew how everything worked, but when you had to teach it to others, there just wasn't enough standardization. We needed a better way to collaborate on projects.

As I was coming to this realization, PHP5 was maturing, and PHP really became an object oriented language.

Now, I wasn't a programmer by education (my major was Philosophy), but rather a geek by love and by nature, you know, one of those junior high and high school kids who were infatuated by computers and the burgeoning Internet in the 90's. So object oriented programming, design patterns, MVC, and the elements of this paradigm were foreign to me to start.

Fortunately, an excellent book was recommended to me that really opened my eyes about PHP and object oriented programming, "PHP 5 Objects, Patterns, and Practice" by Matt Zandstra. I highly recommend this book to anyone looking to understand the "what's", "why's", and "how's" of PHP 5 OOP programming.

After reading that, I was inspired to build my own PHP 5 object oriented framework for web applications. At the time, (say, early 2004?), there wasn't anything very mature out there. CakePHP was still really a PHP 4 system, and I hadn't even heard of Symfony (probably in early Alpha at the time) at the time.

My team and I worked on our own framework for a year or two, building some pretty sophisticated apps for small and mid-sized businesses, and it was a definite improvement over our PHP 4 methods. We learned a lot, and many of those websites are still running today on our framework.

However, over time, we found our home built framework lacking in features, and discovered Symfony in late 2007. Symfony was an open source project with a small army of super talented developrs, and was light years ahead of our framework, so we decided to jump on the bandwagon... and thank goodness we did!

Symfony solved many common challenges the same way we did, for instance automatic form and code generation based upon database schema... but did so more thoroughly, and added a whole bunch on top, not the least of which is fantastic documentation and an awesome beginners guide entitled, "The Definitive Guide to Symfony" by Fabien Potencier.

Now, with 10 web developers collaborating on projects for dozens of clients, we finally had a robust methodology to standardize our company's practices on. Everyone could easily learn the Symfony framework, as there was great documentation available, and tons of utilities and plugins readily available to incorporate into our projects.

The result has been phenomenal, we are producing the highest quality websites, with the most robust code, very rapidly. Our systems are easily extended by virtue of the clear MVC pattern and code organization practices Symfony enforces. Projects can be collaborated on and exchanged between developers due to the commonality of file organization and standard plugins for common purposes like authentication, permissions, and more.

As the leader of a boutique web development agency, with over a decade of web development experience under my belt, I highly recommend Symfony as the de facto standard for enterprise class PHP applications.

References:
The Symfony Project
The Definitive Guide to Symfony
PHP 5 Objects, Patterns, and Practice