So, amongst all the discussion of import methods lately I just wanted to flag another possible approach - creating a CiviCRM hook module for the Drupal migrate module

There are a bunch of great blogs out there on how to use the table wizard module with the migrate module to import data from various mysql tables or views into Drupal nodes / users / taxonomies / content types - for example:

http://www.lullabot.com/articles/drupal-data-imports-migrate-and-table-wizard

The migrate module has a bunch of hooks to allow you to use it for other forms of migrations. I rattled up the module / code pasted at the end of this blog in a couple of hours as a proof of content for using this approach to CiviCRM imports. The code I threw together just offers up civicrm_contact table fields but I think there must be some clever ways to use existing import tools rather than this rudimentary approach.

The really nice thing about using this approach is that it constructs an array or object (in this example the $params object) based on the front-end configured mappings and then additional hooks have an opportunity to re-factor this $params object (e.g. re-parsing address fields) before the $params array is passed to the contact create api I think there is lots of potential here - especially since migrate module:

- already interacts with drush

- allows you to specify how many contacts to import at a time

- non-developers can make changes without needing the code to change (this can be a problem with a scripted solution if you have both developers & non-developers involved)

- allows migrations to be reversed or updated as you tweak it

- provides error reporting

- potentially allows you to use related tables as your source data - ie. it should be possible from what I understand to create an import that imports contacts with associated contributions from multiple tables - http://drupal.org/node/591776#comment-2107050 - I haven't worked through this yet

If we wrote a really good migrate hook module then all we would need to do to customise our imports is write hooks to massage aspects of the data. Obviously this is drupal centic approach but I think most of the people looking at big scripted migrations are doing it in drupal.

To get this code working you will need the modules:

- table wizard (tw)
- schema
- migrate
- views
- view ui (recommended)

You will also need to have a source mysql table to use with a primary key. You need to add this table in table wizard and analyse it before you can go to migrate module & create a content set. The blog by lullabot or the docmentation on migrate should help here

** - NB - I struggled to find a useful place to post this as a zip as I didn't seem to be able to add a file to the wiki page******

Code for module civicrm_migrate

migrate.info

****************************
; $Id$
name = Fuzion Migrate CiviCRM
description = Add on for Migrate module to migrate into CiviCRM
core = 6.x
package = AA Fuzion
version = 0.0
dependencies[] = "migrate"
dependencies[] = "civicrm"

************************************
migrate.module
**************************************

<?php 

function civicrm_migrate_migrate_types() {
  $types = array('contact' => t('CiviCRM Contact'), 'contribution' => t('CiviCRM Contribution'));
  return $types;
}

function civicrm_migrate_migrate_import_contact($tblinfo, $row) {


  civicrm_initialize( );      
  require_once( 'api/v2/Contact.php' ) ;

  //section copied from example
  $params = array();
  // Initially populate the new object according to the mappings
  // this is a standard bit of code from the example
  foreach ($tblinfo->fields as $destfield => $values) {
      if ($values['srcfield'] && $row->$values['srcfield']) {
      $params[$destfield] = $row->$values['srcfield'];
    }
    else {
      $params[$destfield]  = $values['default_value'];
    }
  }

  // Give other modules a shot at manipulating the object
  $errors = migrate_destination_invoke_all('prepare_contact', $params, $tblinfo, $row);

  $success = TRUE;
  foreach ($errors as $error) {
    if ($error['level'] != MIGRATE_MESSAGE_INFORMATIONAL) {
      $success = FALSE;
      break;
    }
  }


        
      if ($success) {
        $result  = giantrobot_civicrm_contact_add($params);  

   // $newid = example_sub_save($sub);
    // Call completion hooks, for any processing which needs to be done after node_save
    $errors = migrate_destination_invoke_all('complete_contact', $params, $tblinfo, $row);

    $sourcekey = $tblinfo->sourcekey;
    migrate_add_mapping($tblinfo->mcsid, $row->$sourcekey, $newid);
  }
  return $errors;
  
        
}

function giantrobot_civicrm_contact_add($params) {
  $params['dupe_check'] = TRUE ;

 
  $contact = civicrm_contact_add($params) ;
 
  if ( !civicrm_error( $contact ) ) {
    // for clarity
       return $contact;
  }
 
  else {
     return;
    // let's see if we have multiple matches
    if ( stristr($contact['error_message'],'Found matching contacts') ) {
      // if so, we'll get the lowest contact ID and update them
      $contact_ids = explode(',',$contact['error_data']) ;
      sort($contact_ids) ;
      $contact_id = array_shift($contact_ids) ;
      if ( (int)$contact_id > 0 ) {
        $params['contact_id'] = $contact_id ;
        $params['dupe_check'] = FALSE ;
        $contact = civicrm_contact_add($params);
        // for clarity
        return $contact ;
      }
      else {
        // some unlikely civicrm_error which gave us a non-numeric
        // contact_id
      }
    }
    else {
      // not multiple duplicates - some other civicrm_error
    }
  }
  // we didn't handle update to first dupe; this is either a
  // successful add of a non-dupe, or a civicrm_error

  return $contact ;
}





function civicrm_migrate_migrate_fields_contact($type) {

  $sql = " SHOW COLUMNS FROM civicrm_contact ";
  $contactFields = db_query($sql);

  while ( $field = db_fetch_array( $contactFields) ) {
   $fields[$field['Field']] =  $field['Field'] ;
  }  

  
  return $fields;
}