Install the Quickstart Development Environment

This tutorial covers building your own development server with a fresh Ubuntu install and the shell scripts from the Quickstart Drupal Development Environment. See: http://drupal.org/project/quickstart

The Quickstart project provides a pre-made virtual box which you can download with links from the project’s page. If you just want to start with the virtual box, then you can skip to the next section which covers adding the Hudson server.

Let's get started.

On your Ubuntu machine (or virtual machine) open a command line (Applications->Accessories->Terminal) and check out the Quickstart files from CVS. Type or paste the following commands:

Read More »

The following are notes to my developers which I share now for the benefit of clients and dev teams who are curious how an enterprise build system may be set up.

Understanding the repo structure

The Mercurial repos in Kiln are structured as follows:

Project->Environment Stack [repo group] -> Repos (code, dbsnaps, [others])

Project

Projects are usually titled with the project number preceded by the letter “p”. The letter “p” is short for “project” and is only there because Kiln (and other repos) will not accept just numbers for a project title.

In a few cases, like for the RRA website, the project will have a string (text) name instead of a project number. This is because multiple concurrent projects are running for the RRA web site and it makes sense to keep the work under one project. In this case the project is titled, “prra”. Note the letters are all lower case.

Environment Stack

The default stack is called, “main”. This is the dev/integration environment on the cloud dev server.

Other stacks may include qa, loadtesting, production, or anything else needed. These stacks may exist on the dev server, or other servers, including the production server.

Repos

Change Note: This section notes our change from the practice of separating Drupal code into three repos. We have found the benefits of managing all code in one repo outweighs longer initial pull or push times. The older version of this section remains at the end of this doc for historical purposes.

code

This repo combines the previous three repos (drupalcore, sites, and sitesall) into one. This repo contains all code to run the client site, as well as special files build scripts which can modify the build process at runtime.

dbsnaps

This repo may contain multiple database snapshots (sql files). The one file titled, “db.sql” is the primary source file for site builds.

Here is a typical scenario:

1. We start a new dev project (new or existing site)

a. We receive an sql file which is a full sql dump of the site database (hopefully with the cache and watchdog tables truncated).

b. This file goes into the dbsnaps repo and will be titled, “db.sql”.

c. We should all build off this version controlled file.

d. Auto builds will also take place off this file.

2. We may decide to test some changes to the branch we are working on.

a. These changes could involve changes to the data structure.

b. To help us roll back quickly, or to make a diff of the changes we made, we can use the backup_and_migrate module to output a snapshot before the changes and then output a snapshot after these changes.

c. These files may be checked into the repo for safe keeping and may be used to create a diff file/patch to look for exactly what changes were made to what tables.

Remember, these sql files are not the primary source for the builds, and db.sql will overwrite the db on the next build.

3. Actual db changes should be made in code via views export, features, chaos tools, and sql statements in hook_update functions.

How to trigger a build

To trigger a build, push a change to the repo.

That’s it.

If you’d like to know what happens after that, read on.

The Kiln/HG web hook will trigger a call to the dispatcher script on our dev server.

1. The call to the dispatcher includes a token to authenticate its source

2. and a json payload containing information about the check in.

The dispatcher will then:

1. authenticate the source of the call and record the call in its log file

2. upon authentication prepare the secure call to Hudson to trigger the build with parameters parsed and passed from the json payload

3. ignore a call to build on the production server as these are manually controlled via Hudson

4. perform miscellaneous tasks such as emailing notifications and adding a record of the dispatch to our intranet

Once Hudson receives the call to start a build, the following happens:

1. The repo is initially cloned if it does not exist on the dev server, or pulled from then updated with a hg update --clean.

2. A build ini file is searched for in the updated repo and if found parsed for variables.

3. A prebuild sh script is searched for in the updated repo and if found ran.

4. The project is then build with an import of the dbsnap and a symlink from the web root to the code repo.

5. A build sh script is searched for in the updated repo and if found ran.

6. A vhost file is added to /etc/apache2/sites-available/ in Apache, enabled, and Apache is reloaded.

7. A postbuild sh script is searched for in the updated repo and if found ran.

8. The following Drush and shell commands are ran in downstream jobs and the results recorded in the Hudson console and actions are triggered by results.

Miscellaneous cleanup and logging

a. drush --yes pm-enable dblog

b. drush --yes watchdog-delete all

c. drush --yes cache-clear all

d. drush --yes updatedb

e. drush --yes cron

f. drush --yes pm-refresh

g. drush --yes status

h. drush --yes watchdog-show

i. drush --yes pm-list

Run Coder Module from Drush

j. drush --yes dl coder-6.x-2.x-dev

k. drush --yes pm-enable coder

l. drush --yes coder contrib critical

Patch and run Simpletests

m. drush --yes dl simpletest-6.x-2.x-dev

n. patch -f -p0 < ${WWWROOT}${STACK}${PROJECT}/sites/all/modules/simpletest/D6-core-simpletest.patch

o. wget http://drupal.org/files/issues/simpletest_run_on_a...

p. patch -f -p0 < ${WWWROOT}${STACK}${PROJECT}/sites/all/modules/simpletest/simpletest_run_on_any_profile_6.16_0.patch

q. cd ${WWWROOT}${STACK}${PROJECT}

r. drush --yes pm-enable simpletest

s. mv ${WWWROOT}${STACK}${PROJECT}/sites/all/modules/simpletest/run-tests.sh ${WWWROOT}${STACK}${PROJECT}/scripts

t. rm -rf ${WWWROOT}${STACK}${PROJECT}/scripts/tests/*.xml

u. /usr/bin/php ${WWWROOT}${STACK}${PROJECT}/scripts/run-tests.sh --url http://${DEVURL}/${STACK}${PROJECT} --all --color

Tools

1. Drush: http://drupal.org/project/drush

2. Coder: http://drupal.org/project/coder

3. Simpletest: http://drupal.org/simpletest

Deprecated Section on Repos

We use four repos to break code into sets that are managed differently and/or updated at different frequencies. This separation of code into sets speeds up your most frequent pulls and pushes and our most frequent auto builds.

drupalcore

The “drupalcore” repo is updated least often. It should contain only core Drupal code and should be unchanged from the version in which it was downloaded from Drupal.org. For example: the Drupal 6.16 zip file. Sometimes we may receive client code which has Drupal core code modified. We should try to bring the client’s code back to core by moving any hacks on core into a custom module or theme function.

sites

The “sites” repo contains everything in the sites folder found in a normal Drupal setup, except the “sites/all” directory. Additionally, ff the web site uses “sites/default” as its Drupal site folder then this directory will contain the site’s assets.

For the auto build process, the “sites” directory will have an extra site folder to be used on the dev server. For example, if the project number is 957, then the sites directory will contain a folder called, “main957.seanbuscay.net”. In “main957.seanbuscay.net” there will be a settings.php file with the config info needed to run on the development server.

This repo is rarely changed. We may, from time to time, add files to it from the production server so we may see the latest images and assets while testing on the dev server. No primary code changes should take place here.

sitesall

The structure of this repo is a little different from the others. It starts off with just one immediate child directory called, “all”. This folder corresponds to the typical Drupal setup path of “[Drupal root]/sites/all”. Inside this “all” folder are all modules (custom or contrib.), themes, libraries, and features.

Typically, this is where we do most of our coding. Adding modules or writing custom modules. Customizing themes and creating new ones. This repo is updated most often.

Please note: In some cases, a module or theme should only be exposed to one site instance in a Drupal multisite setup. If this is the case, we can either do our dev work in this repo and then later move the modules or code to the site instance folder, or start off in that site instance folder in the “sites” repo. In this case the “sites” repo will become our most frequently updated repo.

dbsnaps

This repo may contain multiple database snapshots (sql files). The one file titled, “db.sql” is the primary source file for site builds.

Includes Upgrading Drupal Modules, Themes, and Site

I am a fan of check lists. They help simplify and improve the effectiveness of repetitive processes over time. The following is a recent check list I shared with a client as a summary of the process used to upgrade from Drupal 5 to Drupal 6. Please feel free to use it and add your own notes.

1. Together with client identify the list of key content and expected behaviors/displays. Includes any publishing/business logic.
• Should be approximately 30 or so check list items, with accompanying screenshots where appropriate.
• This will form the QA test script to confirm the quality of the upgrade and will be the list for acceptance testing.
2. Duplicate the site, code, db, and files into an upgrade branch on dev server.
3. Version control all code, db, view exports, ect. throughout the process
4. Confirm there are no custom hacks on Drupal core.
5. Run the upgrade process for Drupal Core.
6. Test
• All steps titled test include some automated testing as well as use of the coder module.
• I also just came across the deadwood module.
• Test steps also include running through the QA test script when appropriate.
• See links below.
7. Run the upgrade process on views, converting views to views 2
8. Test
9. Identify any custom hacks on contrib. modules.
10. Upgrade contrib. modules which have D6 versions
11. Test
12. Identify upgrade paths for non standard modules and those which do not have a d6 version
13. Code the upgrades and run them
14. Test (full developer review)
15. Client QA and site review/Bug fixing Cycle
16. Prepare deploy plan
17. Deploy the upgrade on production server
18. Client QA and site review/Server Configuration and Bug fixing Cycle
19. Client Acceptance testing (walking through the script), reporting and fixing issues cycle

Links About Upgrading Drupal

Converting 5.x modules to 6.x - http://drupal.org/node/114774

Updating Drupal Modules to D6 in Three Easy Steps - http://www.webmasterworld.com/content_management/3...

Coder Module - http://drupal.org/project/coder/

Deadwood - http://drupal.org/project/deadwood - http://boombatower.com/tools/deadwood

This post from 37 signals provides a nice understanding of estimates.

While contracting for the Tree House Agency last fall, I am very happy to have architected, project managed, and lead the development of the New House Democratic Website (http://dems.gov)

From: http://www.dems.gov/press/house-democratic-caucus-launches-new-website

The site features what Democrats are saying each day, by issue and about major legislation. For the first time, visitors will be able to easily find quotes from Members on the topics and legislation they care about most. The site also features major legislation passed by Democrats and summaries of upcoming bills. In addition, the public can find House Democrats’ official websites, YouTube channels, and Facebook pages using an interactive directory.

The new site allows the caucus to:

1. Aggregate content from congressional members (video, text, feeds, and other forms of media)

2. Share content with the public and reporters in a way that it easy to navigate

3. Provide multiple views for visitors to browse and search content by facets of interest

Of course we built it in Drupal for the public facing website because of its ability to present content to visitors in a variety of ways; with all the social networking tools of today’s websites and Drupal on the administrative end was the clear choice when it came to meeting the administrative and security needs of the Caucus.

Nice post on Smashing Magazine reminding us not to forget the small stuff in the new year: http://www.smashingmagazine.com/2010/01/23/dont-forget-the-small-stuff-this-year/ .  For example, did you change the copyright date on all your clients’ sites?

Here are two interesting client issues I had in after the New Year.

Expired Security Credentials

A client called to ask for help getting access to an old Content Management System they had for about four years.  The site just stopped allowing them to log in. Turned out to be the site was on a Windows Server which authenticated via the Windows User Groups.  After four years, the server reset the user account privileges. 

The lesson: Watch server configs and security accounts in the new year.  They often (correctly) expire after a certain period and this often occurs at the start of a new year.

CCK Date Fields Stopped Working

Here’s one for us Drupal CCK Date field users.

One of my clients has a content type which includes a publication date using a CCK Date field.  Suddenly, they could no longer save changes to nodes in this content type. 

This was the error message:

An illegal choice has been detected. Please contact the site administrator.

Here’s what happened:

The CCK Date field has a setting called, “Years back and forward.” 

Read More »

A friend just emailed me the following question:

As a Business to Business Company is it better to have your company blog hosted on the company website or should it be more indirect and hosted on wordpress or something like that. 

I am getting conflicting answers from my people and was wondering if you had any advice.

The following is my response back.  Hope this helps you as well.

In answer to your question about whether a B2B company should have their blog hosted on their website or on something external I'd look at the importance of the following:

1. Branding
2. Maintaining the technology
3. Lock-in
4. Reasons to not be on your main company site

Branding

To me, branding is the most important.  If your blog is a key part of your communications strategy then you should have full control over all the elements that show up on the blog:

This is not to say you can't use a service hosted off your current servers.  Just that the logo, design, content, domain, comments system, auto-emails, and even functionality should all reflect your brand.  I recommend the blog be on your company's main domain.  If it must be hosted somewhere else, then a sub domain can work.  The key is that you do not dilute your brand across multiple domains. 

Additionally there should be no third-party content that could show up on your blog without your direct control, such as advertising automatically shown by the blog network you may be hosted on.

Read More »

Totally loving this tool to not only have my info backed up in DropBox but to also have it synced between my Laptop and Desktop.

Right now I have junctions (and thus synced directories, data, and profiles) for:

1. Skype
2. Firefox
3. Chrome
4. Aptana (Eclipse)
5. Topstyle
6. CuteFTP
7. My sticky notes
8. My local web server, sites, and databases
9. Pidgin

My new favorite jquery tool for data table sorting and searching.

Excellent tool with as little or as much configuration as you'd need for most any project. Very slick.

Used this to commit horrible usability crimes at the client's request. Shame.

After reading the book version of this article, Big Macs vs. The Naked Chef by Joel Spolsky I wrote down a few thoughts on the difference between knowledge management and creating a procedures/rules book.  

I often examine processes to improve productivity. On my teams I push for documenting solutions to record and share with other team members. However, it’s important to understand that documenting process and procedure alone does not equal the spread of knowledge or an increase in expertise within your team or organization.

As any phone call to Dell tech support will show you, the act of following a tech manual step by step will usually not get your computer fixed. Procedures are nothing without very smart and experienced people working for you to find a solution. It’s also the very smart and experienced who don’t really use a procedures manual.

As Joel Spolsky writes in his “Big Macs vs. The Naked Chef” post:

1. Some things need talent to do really well.

2. It's hard to scale talent.

3. One way people try to scale talent is by having the talent create rules for the untalented to follow.

4. The quality of the resulting product is very low.

So what do we use in our teams and organizations instead of rules and regulations?

1. We hire very smart and experienced people
2. We implement a knowledge management system

Knowledge management helps when you have a team of experts or hire really smart people, instead of just hiring people to follow a rules book.

Knowledge management means that geniuses within your company may apply different and varied solutions instead of a single cookie cutter one, as long as:

Read More »

I noticed two nice design changes in low-end laptops when I purchased a new disposable laptop this past weekend.

Wider (though) shorter screens and wider keyboards.

For cheap laptops screen sizes remain small. However, the trend now is to keep the height short  while adding to the width across.

Wider screens provide a nice usability enhancement in that they allow for more application windows to be open across the width of the screen while the total screen area remains smaller (shorter in height) to keep the price down.

Since the laptops have wider screens, they can have a wider keyboard. I looked over a number of nice sized keyboards with an additional number pad as a standard. This enhancement is really great if you find yourself working with numbers and data often.

These simple but thoughtful changes provide some nice usability enhancements to the less expensive laptops while allowing their prices to remain less expensive.

Kudos to industry for these changes.

By the way I bought the HP - Pavilion Laptop with Intel® Core™2 Duo Processor for $550 from Best Buy.

• Dimensions: 1.6 x 16 x 10.2 inches 
• 15.6" high-definition widescreen display With LED BrightView technology and 1366 x 768 resolution

I've been both developer and manager for years on both large and small projects with various sized teams. During the week to week management of these teams I tend to send little notes to help guide our work process. This week I have been breaking in a new team.

I thought it might be useful to other technical project managers and team leaders if I shared some of this week’s raw notes. I hope it’s useful for other managers to have a glimpse into the day to day notes of another in their same place.

Time Sheets

Please enter your time regularly into Fresh Books. Smaller more frequent time entries are preferred. Try to break them down by tickets and tasks in those tickets. Please always include the ticket number you were working on.

Please keep your entries up to date daily while on a project.

SVN

I believe in update and commit often.

Make sure you update and resolve conflicts before committing.

Enter in notes about what you are committing and when possible, enter the relevant ticket number.

Commits will auto deploy to the server within a few minutes. If you break something, you can roll it back but email the rest of the team to let us know so we can make sure to match our revision to the latest version on the dev server.

To roll back, log into [Spring Loops], choose the project, and then deploy the previous version number.

Remember that other devs are working in SVN too. So, update often, typically before starting a new change set. And commit often. I can help with any merging issues if you need it.

[Info here on how to contact me for an urgent need.]

Further Notes on SVN

The following is just a brief reminder about steps to take to not over write others work.

Read More »

If your are not a jQuery dev, don't worry, there are plenty of excellent resources to get you started using jQuery to enhance your site.

Star here

http://drupal.org/project/Modules/category/101 ( Drupal Modules: JavaScript Utilities )

Then simply:

1. Browse
2. Download
3. Install & Configure

Some module which I recommend checking out

UI Improvements

• jTooltips: http://drupal.org/project/jtooltips - adds in context user feedback
  
• JQuery Interface: http://drupal.org/project/jquery_interface  - might not be D6 ready
  
• jQuery UI: http://drupal.org/project/jquery_ui - a full library of effects
  
• Username originality AJAX check: http://drupal.org/project/username_check - This is the kind of enhancements jQuery is perfect for.

Content Browsing

• jCarousel: http://drupal.org/project/jcarousel - nice for creating a recent post/image browser, or a headlines browser.
  • Also see @ http://drupal.org/project/jcarousel_block
    
• Views carousel: http://drupal.org/project/viewscarousel

Photos

• Thickbox: http://drupal.org/project/thickbox or Lightbox2: http://drupal.org/project/lightbox2 
  • @see Comparison of “lightbox” type modules: http://drupal.org/node/266126
  • @see Lightbox2 demo http://www.stellapower.net/lightbox2
• Slideshow Creator: http://drupal.org/project/slideshow_creator
  • @see Cool demo here: http://www.malsup.com/jquery/cycle/

Theming and Display

• jQuery Eye Candy: http://drupal.org/project/jq_eyecandy  - if you really must use jQuery rather than CSS
  
• JavaScript Theming: http://drupal.org/project/js_theming
  
• Page Style: http://drupal.org/project/pagestyle - a style changer in a block and in the browser for better Web Accessibility

jQuery Plug-in Management

• jQuery plugins: http://drupal.org/project/jquery_plugin - call jQuery plugins from any module or theme by calling the jquery_plugin_add($plugin) function
  
• jQuery Plugin Handler (JQP): http://drupal.org/project/jqp

• jQ: http://drupal.org/project/jq

Read More »

In D6, you can put a .js file in your base theme directory and it will automatically be loaded by Drupal.

You can also include a js file via an entry in your theme’s .info file:

scripts[] = your_javascript.js

However, I recommend the following:

Consider your JavaScript theme-able like HTML.

1. It should be able to be themed
2. Put JavaScript in .js file
3. Use drupal_add_js() in a theme function
4. If in a module, call it with hook_init()

As HTML in a theme function / tpl file, calling your JS with a theme function allows theme developers to modify or override your JS in the theme layer.

Here it is in a module:

<?php

/**

 * Implementation of hook_init()

 */

function trickoutmysite_init() {

 theme('trickoutmysite_javascript');

}

/**

 * Implementation of hook_theme()

 */

function trickoutmysite_theme($existing, $type, $theme, $path) {

  return array(

    'trickoutmysite_javascript' => array(

      'arguments' => array(),

    ),

  );

}

/**

 * A themable function which uses drupal_add_js to include the js file

 */

function theme_trickoutmysite_javascript() {

  drupal_add_js (drupal_get_path('module', 'trickoutmysite') . '/trickoutmysite.js');

}

?>

Read More »