Drupal Web Development

Install the Quickstart Development Environment

This tutorial covers building your own development server with a fresh Ubuntu install and the shell scripts from the Quickstart Drupal Development Environment. See: http://drupal.org/project/quickstart

The Quickstart project provides a pre-made virtual box which you can download with links from the project’s page. If you just want to start with the virtual box, then you can skip to the next section which covers adding the Hudson server.

Let's get started.

On your Ubuntu machine (or virtual machine) open a command line (Applications->Accessories->Terminal) and check out the Quickstart files from CVS. Type or paste the following commands:

Read More »

The following are notes to my developers which I share now for the benefit of clients and dev teams who are curious how an enterprise build system may be set up.

Understanding the repo structure

The Mercurial repos in Kiln are structured as follows:

Project->Environment Stack [repo group] -> Repos (code, dbsnaps, [others])

Project

Projects are usually titled with the project number preceded by the letter “p”. The letter “p” is short for “project” and is only there because Kiln (and other repos) will not accept just numbers for a project title.

In a few cases, like for the RRA website, the project will have a string (text) name instead of a project number. This is because multiple concurrent projects are running for the RRA web site and it makes sense to keep the work under one project. In this case the project is titled, “prra”. Note the letters are all lower case.

Environment Stack

The default stack is called, “main”. This is the dev/integration environment on the cloud dev server.

Other stacks may include qa, loadtesting, production, or anything else needed. These stacks may exist on the dev server, or other servers, including the production server.

Repos

Change Note: This section notes our change from the practice of separating Drupal code into three repos. We have found the benefits of managing all code in one repo outweighs longer initial pull or push times. The older version of this section remains at the end of this doc for historical purposes.

code

This repo combines the previous three repos (drupalcore, sites, and sitesall) into one. This repo contains all code to run the client site, as well as special files build scripts which can modify the build process at runtime.

dbsnaps

This repo may contain multiple database snapshots (sql files). The one file titled, “db.sql” is the primary source file for site builds.

Here is a typical scenario:

1. We start a new dev project (new or existing site)

a. We receive an sql file which is a full sql dump of the site database (hopefully with the cache and watchdog tables truncated).

b. This file goes into the dbsnaps repo and will be titled, “db.sql”.

c. We should all build off this version controlled file.

d. Auto builds will also take place off this file.

2. We may decide to test some changes to the branch we are working on.

a. These changes could involve changes to the data structure.

b. To help us roll back quickly, or to make a diff of the changes we made, we can use the backup_and_migrate module to output a snapshot before the changes and then output a snapshot after these changes.

c. These files may be checked into the repo for safe keeping and may be used to create a diff file/patch to look for exactly what changes were made to what tables.

Remember, these sql files are not the primary source for the builds, and db.sql will overwrite the db on the next build.

3. Actual db changes should be made in code via views export, features, chaos tools, and sql statements in hook_update functions.

How to trigger a build

To trigger a build, push a change to the repo.

That’s it.

If you’d like to know what happens after that, read on.

The Kiln/HG web hook will trigger a call to the dispatcher script on our dev server.

1. The call to the dispatcher includes a token to authenticate its source

2. and a json payload containing information about the check in.

The dispatcher will then:

1. authenticate the source of the call and record the call in its log file

2. upon authentication prepare the secure call to Hudson to trigger the build with parameters parsed and passed from the json payload

3. ignore a call to build on the production server as these are manually controlled via Hudson

4. perform miscellaneous tasks such as emailing notifications and adding a record of the dispatch to our intranet

Once Hudson receives the call to start a build, the following happens:

1. The repo is initially cloned if it does not exist on the dev server, or pulled from then updated with a hg update --clean.

2. A build ini file is searched for in the updated repo and if found parsed for variables.

3. A prebuild sh script is searched for in the updated repo and if found ran.

4. The project is then build with an import of the dbsnap and a symlink from the web root to the code repo.

5. A build sh script is searched for in the updated repo and if found ran.

6. A vhost file is added to /etc/apache2/sites-available/ in Apache, enabled, and Apache is reloaded.

7. A postbuild sh script is searched for in the updated repo and if found ran.

8. The following Drush and shell commands are ran in downstream jobs and the results recorded in the Hudson console and actions are triggered by results.

Miscellaneous cleanup and logging

a. drush --yes pm-enable dblog

b. drush --yes watchdog-delete all

c. drush --yes cache-clear all

d. drush --yes updatedb

e. drush --yes cron

f. drush --yes pm-refresh

g. drush --yes status

h. drush --yes watchdog-show

i. drush --yes pm-list

Run Coder Module from Drush

j. drush --yes dl coder-6.x-2.x-dev

k. drush --yes pm-enable coder

l. drush --yes coder contrib critical

Patch and run Simpletests

m. drush --yes dl simpletest-6.x-2.x-dev

n. patch -f -p0 < ${WWWROOT}${STACK}${PROJECT}/sites/all/modules/simpletest/D6-core-simpletest.patch

o. wget http://drupal.org/files/issues/simpletest_run_on_a...

p. patch -f -p0 < ${WWWROOT}${STACK}${PROJECT}/sites/all/modules/simpletest/simpletest_run_on_any_profile_6.16_0.patch

q. cd ${WWWROOT}${STACK}${PROJECT}

r. drush --yes pm-enable simpletest

s. mv ${WWWROOT}${STACK}${PROJECT}/sites/all/modules/simpletest/run-tests.sh ${WWWROOT}${STACK}${PROJECT}/scripts

t. rm -rf ${WWWROOT}${STACK}${PROJECT}/scripts/tests/*.xml

u. /usr/bin/php ${WWWROOT}${STACK}${PROJECT}/scripts/run-tests.sh --url http://${DEVURL}/${STACK}${PROJECT} --all --color

Tools

1. Drush: http://drupal.org/project/drush

2. Coder: http://drupal.org/project/coder

3. Simpletest: http://drupal.org/simpletest

Deprecated Section on Repos

We use four repos to break code into sets that are managed differently and/or updated at different frequencies. This separation of code into sets speeds up your most frequent pulls and pushes and our most frequent auto builds.

drupalcore

The “drupalcore” repo is updated least often. It should contain only core Drupal code and should be unchanged from the version in which it was downloaded from Drupal.org. For example: the Drupal 6.16 zip file. Sometimes we may receive client code which has Drupal core code modified. We should try to bring the client’s code back to core by moving any hacks on core into a custom module or theme function.

sites

The “sites” repo contains everything in the sites folder found in a normal Drupal setup, except the “sites/all” directory. Additionally, ff the web site uses “sites/default” as its Drupal site folder then this directory will contain the site’s assets.

For the auto build process, the “sites” directory will have an extra site folder to be used on the dev server. For example, if the project number is 957, then the sites directory will contain a folder called, “main957.seanbuscay.net”. In “main957.seanbuscay.net” there will be a settings.php file with the config info needed to run on the development server.

This repo is rarely changed. We may, from time to time, add files to it from the production server so we may see the latest images and assets while testing on the dev server. No primary code changes should take place here.

sitesall

The structure of this repo is a little different from the others. It starts off with just one immediate child directory called, “all”. This folder corresponds to the typical Drupal setup path of “[Drupal root]/sites/all”. Inside this “all” folder are all modules (custom or contrib.), themes, libraries, and features.

Typically, this is where we do most of our coding. Adding modules or writing custom modules. Customizing themes and creating new ones. This repo is updated most often.

Please note: In some cases, a module or theme should only be exposed to one site instance in a Drupal multisite setup. If this is the case, we can either do our dev work in this repo and then later move the modules or code to the site instance folder, or start off in that site instance folder in the “sites” repo. In this case the “sites” repo will become our most frequently updated repo.

dbsnaps

This repo may contain multiple database snapshots (sql files). The one file titled, “db.sql” is the primary source file for site builds.

I’m looking for several freelance Drupal developers.

Currently I’d like to find one to two very good module developers who also use Views and CCK quite a bit. I’m also interested in a good themer.

I’d prefer U.S. based or near-shore based developers (such as developers in Costa Rica). I manage all projects directly and coordinate work via an online project management and ticketing system. If you are comfortable working this way, and can work with other developers using modern version control systems, please contact me.

You may contact me by emailing developers [at] seanbuscay.com.

Please include:

1. information about your Drupal experience

2. your freelance rate

3. your preferred number of hours per week

Includes Upgrading Drupal Modules, Themes, and Site

I am a fan of check lists. They help simplify and improve the effectiveness of repetitive processes over time. The following is a recent check list I shared with a client as a summary of the process used to upgrade from Drupal 5 to Drupal 6. Please feel free to use it and add your own notes.

1. Together with client identify the list of key content and expected behaviors/displays. Includes any publishing/business logic.
• Should be approximately 30 or so check list items, with accompanying screenshots where appropriate.
• This will form the QA test script to confirm the quality of the upgrade and will be the list for acceptance testing.
2. Duplicate the site, code, db, and files into an upgrade branch on dev server.
3. Version control all code, db, view exports, ect. throughout the process
4. Confirm there are no custom hacks on Drupal core.
5. Run the upgrade process for Drupal Core.
6. Test
• All steps titled test include some automated testing as well as use of the coder module.
• I also just came across the deadwood module.
• Test steps also include running through the QA test script when appropriate.
• See links below.
7. Run the upgrade process on views, converting views to views 2
8. Test
9. Identify any custom hacks on contrib. modules.
10. Upgrade contrib. modules which have D6 versions
11. Test
12. Identify upgrade paths for non standard modules and those which do not have a d6 version
13. Code the upgrades and run them
14. Test (full developer review)
15. Client QA and site review/Bug fixing Cycle
16. Prepare deploy plan
17. Deploy the upgrade on production server
18. Client QA and site review/Server Configuration and Bug fixing Cycle
19. Client Acceptance testing (walking through the script), reporting and fixing issues cycle

Links About Upgrading Drupal

Converting 5.x modules to 6.x - http://drupal.org/node/114774

Updating Drupal Modules to D6 in Three Easy Steps - http://www.webmasterworld.com/content_management/3...

Coder Module - http://drupal.org/project/coder/

Deadwood - http://drupal.org/project/deadwood - http://boombatower.com/tools/deadwood

While contracting for the Tree House Agency last fall, I am very happy to have architected, project managed, and lead the development of the New House Democratic Website (http://dems.gov)

From: http://www.dems.gov/press/house-democratic-caucus-launches-new-website

The site features what Democrats are saying each day, by issue and about major legislation. For the first time, visitors will be able to easily find quotes from Members on the topics and legislation they care about most. The site also features major legislation passed by Democrats and summaries of upcoming bills. In addition, the public can find House Democrats’ official websites, YouTube channels, and Facebook pages using an interactive directory.

The new site allows the caucus to:

1. Aggregate content from congressional members (video, text, feeds, and other forms of media)

2. Share content with the public and reporters in a way that it easy to navigate

3. Provide multiple views for visitors to browse and search content by facets of interest

Of course we built it in Drupal for the public facing website because of its ability to present content to visitors in a variety of ways; with all the social networking tools of today’s websites and Drupal on the administrative end was the clear choice when it came to meeting the administrative and security needs of the Caucus.

Nice post on Smashing Magazine reminding us not to forget the small stuff in the new year: http://www.smashingmagazine.com/2010/01/23/dont-forget-the-small-stuff-this-year/ .  For example, did you change the copyright date on all your clients’ sites?

Here are two interesting client issues I had in after the New Year.

Expired Security Credentials

A client called to ask for help getting access to an old Content Management System they had for about four years.  The site just stopped allowing them to log in. Turned out to be the site was on a Windows Server which authenticated via the Windows User Groups.  After four years, the server reset the user account privileges. 

The lesson: Watch server configs and security accounts in the new year.  They often (correctly) expire after a certain period and this often occurs at the start of a new year.

CCK Date Fields Stopped Working

Here’s one for us Drupal CCK Date field users.

One of my clients has a content type which includes a publication date using a CCK Date field.  Suddenly, they could no longer save changes to nodes in this content type. 

This was the error message:

An illegal choice has been detected. Please contact the site administrator.

Here’s what happened:

The CCK Date field has a setting called, “Years back and forward.” 

Read More »

My new favorite jquery tool for data table sorting and searching.

Excellent tool with as little or as much configuration as you'd need for most any project. Very slick.

I've been both developer and manager for years on both large and small projects with various sized teams. During the week to week management of these teams I tend to send little notes to help guide our work process. This week I have been breaking in a new team.

I thought it might be useful to other technical project managers and team leaders if I shared some of this week’s raw notes. I hope it’s useful for other managers to have a glimpse into the day to day notes of another in their same place.

Time Sheets

Please enter your time regularly into Fresh Books. Smaller more frequent time entries are preferred. Try to break them down by tickets and tasks in those tickets. Please always include the ticket number you were working on.

Please keep your entries up to date daily while on a project.

SVN

I believe in update and commit often.

Make sure you update and resolve conflicts before committing.

Enter in notes about what you are committing and when possible, enter the relevant ticket number.

Commits will auto deploy to the server within a few minutes. If you break something, you can roll it back but email the rest of the team to let us know so we can make sure to match our revision to the latest version on the dev server.

To roll back, log into [Spring Loops], choose the project, and then deploy the previous version number.

Remember that other devs are working in SVN too. So, update often, typically before starting a new change set. And commit often. I can help with any merging issues if you need it.

[Info here on how to contact me for an urgent need.]

Further Notes on SVN

The following is just a brief reminder about steps to take to not over write others work.

Read More »

If your are not a jQuery dev, don't worry, there are plenty of excellent resources to get you started using jQuery to enhance your site.

Star here

http://drupal.org/project/Modules/category/101 ( Drupal Modules: JavaScript Utilities )

Then simply:

1. Browse
2. Download
3. Install & Configure

Some module which I recommend checking out

UI Improvements

• jTooltips: http://drupal.org/project/jtooltips - adds in context user feedback
  
• JQuery Interface: http://drupal.org/project/jquery_interface  - might not be D6 ready
  
• jQuery UI: http://drupal.org/project/jquery_ui - a full library of effects
  
• Username originality AJAX check: http://drupal.org/project/username_check - This is the kind of enhancements jQuery is perfect for.

Content Browsing

• jCarousel: http://drupal.org/project/jcarousel - nice for creating a recent post/image browser, or a headlines browser.
  • Also see @ http://drupal.org/project/jcarousel_block
    
• Views carousel: http://drupal.org/project/viewscarousel

Photos

• Thickbox: http://drupal.org/project/thickbox or Lightbox2: http://drupal.org/project/lightbox2 
  • @see Comparison of “lightbox” type modules: http://drupal.org/node/266126
  • @see Lightbox2 demo http://www.stellapower.net/lightbox2
• Slideshow Creator: http://drupal.org/project/slideshow_creator
  • @see Cool demo here: http://www.malsup.com/jquery/cycle/

Theming and Display

• jQuery Eye Candy: http://drupal.org/project/jq_eyecandy  - if you really must use jQuery rather than CSS
  
• JavaScript Theming: http://drupal.org/project/js_theming
  
• Page Style: http://drupal.org/project/pagestyle - a style changer in a block and in the browser for better Web Accessibility

jQuery Plug-in Management

• jQuery plugins: http://drupal.org/project/jquery_plugin - call jQuery plugins from any module or theme by calling the jquery_plugin_add($plugin) function
  
• jQuery Plugin Handler (JQP): http://drupal.org/project/jqp

• jQ: http://drupal.org/project/jq

Read More »

In D6, you can put a .js file in your base theme directory and it will automatically be loaded by Drupal.

You can also include a js file via an entry in your theme’s .info file:

scripts[] = your_javascript.js

However, I recommend the following:

Consider your JavaScript theme-able like HTML.

1. It should be able to be themed
2. Put JavaScript in .js file
3. Use drupal_add_js() in a theme function
4. If in a module, call it with hook_init()

As HTML in a theme function / tpl file, calling your JS with a theme function allows theme developers to modify or override your JS in the theme layer.

Here it is in a module:

<?php

/**

 * Implementation of hook_init()

 */

function trickoutmysite_init() {

 theme('trickoutmysite_javascript');

}

/**

 * Implementation of hook_theme()

 */

function trickoutmysite_theme($existing, $type, $theme, $path) {

  return array(

    'trickoutmysite_javascript' => array(

      'arguments' => array(),

    ),

  );

}

/**

 * A themable function which uses drupal_add_js to include the js file

 */

function theme_trickoutmysite_javascript() {

  drupal_add_js (drupal_get_path('module', 'trickoutmysite') . '/trickoutmysite.js');

}

?>

Read More »

Use jQuery to Enhance Site UI, Display, or Interaction

jQuery is an excellent tool for improving your site’s interface. It’s also helpful to target and change specific CSS and HTML elements quickly. [1]

I tend to use it mostly for UI (user interface) enhancements or when there is a semi-dynamic event in the display, after the page has loaded.

Examples uses include:

• User feedback in messages
• Tool tips (in-context user guidance)
• Enhanced image/gallery display
• Tabs for quick access to hidden information
• As a style sheet switcher (helpful with accessibility)
• Form enhancements and client-side feedback
• Menu enhancements
• Data entry such as a date picker
• Table and data sorting
• Media players
• Content flagging
• Tagging (better input methods)
• Loading additional content without a page refresh
• Dynamic Charts
• Content Carousels (albums, photos, headline news stories)

Do Not Use jQuery as the Primary Driver for Site UI, Display, or Interaction

jQuery should not be used as the primary driver of your site’s display and not as a cornerstone of any primary functionality for your site.

In other words, sure, enhance your web sites e-commerce process with jQuery, but have it:

1. degrade gracefully,
2. making sure people may still go through the e-commerce process,
3. even if jQuery fails completely.

Simple uses can have big impact

When consider the rendering of display effects such as rounded corners and color gradients, consider CSS instead.

Read More »

With Drupal being an open source software and having such a great set of features right out-of-the-box, people may wonder what they pay for when going to an expert Drupal developer or shop for their Drupal install.

A good Drupal site may cost thousands of dollars. For an enterprise level site the budget may be in the hundreds of thousands of dollars. What’s that all about? I thought Drupal was “free”. The following is a bit of explanation.

Most web site features are already built into Drupal and ready to be configured. With professional Drupal development you are not paying for features, but rather expertise. What expertise?

Professional Drupal developers and shops offer expertise in Drupal site:

1. branding and visual design
2. theming (to be defined below),
3. feature integration,
4. architecture,
5. scaling,
6. optimization,
7. and security.

What are professional Drupal development budgets based on?

Not Just Features

As Drupal’s base install already has a large feature set which will meet many site’s needs, it is important to note that there are actually very little features lost or gained at budget levels past a certain point (the cost of install, setup, and deployment).  What budget levels do affect, that is, what is lost or gained are levels of feature integration (interaction) and design implementation (theming).

Theming

To meet real business branding and marketing objectives, companies need their professional visual design developed into Drupal in a way that effectively and successfully communicates their brand.

Read More »

I'm often asked what's involved in migrating a site to Drupal. The following is my typical answer (when responding in writing) .  I hope this helps both clients and Drupal service providers in their next planned migration to Drupal. 

In general migrating a site and upgrading to Drupal involves the same process as any other migration to a modern system.

Things to consider and come to your developers with include:

1. A well defined vision for the new site (ideally agreed upon by all key decision makers)
2. A set of business goals to meet
3. An overall idea of desired traffic levels (which helps dev teams evaluate and plan the Drupal system set-up that will best meet your needs)

The following things are usually identified and developed along the way to building the web site.  Clients generally come to me with some to none of these completed.

Read More »

I’m looking for Drupal developers interested in occasional freelance work.

As a (thankfully) busy freelancer myself, I’d like a small pool of good developers to help augment those weeks when the work is more than one developer can do.

I manage all the projects directly and coordinate the work via an online project management and ticketing system.  If you are comfortable working this way, and can use SVN and other repos to code in a distributed environment, please contact me.

Various skill levels and expertise welcome (designers, themers, coders, Drupal lovers).

You may contact me by emailing developers [at] seanbuscay.com.

Please include information about your Drupal experience and your freelance rate. 

I look forward to hearing from you.

Just read, “Everything You Know About CSS Is Wrong!” http://www.sitepoint.com/books/csswrong1/