A couple month ago I raised the question about CiviCRM importing scalability, and received mixed answers.
CiviCRM should be mostly used for data import, not for data cleaning. Most importing scalability issues stem from people's reliance on the system to perform both data clean up and import. CiviCRM has a great data cleaning process that should be taken advantage of when doing a large import.Blogs
Keep up-to-date with blogs from the core team, working groups, developers, users and champions worldwide. Subscribe to our newsletter to receive regular updates by email. We also have an RSS feed.
June 8, 2010
By Dave Greenberg Filed under CiviCRMFolks at the Electronic Frontier Foundation (EFF) are currently evaluating options for replacing their current suite of online tools. Their requirements include membership management, fundraising, email blasts and activism - and they would like to use open source solutions if possible (and they are already using Drupal for parts of their web presence). Their team has spent some time exploring CiviCRM's feature set, and Lobo and I met with them last week to discuss their findings, answer questions and evaluate fit and gaps.
June 8, 2010
By dhruvahuja Filed under Architecture, CiviCRMI am writing this post to take community feedback on porting CiviCRM to PostgreSQL, the best way to do it, and to team up with any possible members willing to contribute to this effort. Here are my efforts until now. I have not been able to make it 'all' work on PostgreSQL, but certainly to a degree where I can see light at the end of the long tunnel. I am linking 3 PostgreSQL compatible files here:
structure.sql - this has the columns, primary keys, indexes, unique constraints data.sql - this is the sample data included with CiviCRM
June 5, 2010
By Eileen Filed under CiviCRM, DrupalSo, amongst all the discussion of import methods lately I just wanted to flag another possible approach - creating a CiviCRM hook module for the Drupal migrate module
There are a bunch of great blogs out there on how to use the table wizard module with the migrate module to import data from various mysql tables or views into Drupal nodes / users / taxonomies / content types - for example:
http://www.lullabot.com/articles/drupal-data-imports-migrate-and-table-wizard
The migrate module has a bunch of hooks to allow you to use it for other forms of migrations. I rattled up the module / code pasted at the end of this blog in a couple of hours as a proof of content for using this approach to CiviCRM imports. The code I threw together just offers up civicrm_contact table fields but I think there must be some clever ways to use existing import tools rather than this rudimentary approach.
June 3, 2010
By yashodha Filed under CiviCRMThe team is excited to announce that the first public ALPHA release of version 3.2 is now available for download. You can also try it out on our sandbox site. Please remember this is an ALPHA release and it should NOT be used on production sites.
June 2, 2010
By AllenShaw Filed under CiviCRMAs of now (version 3.1.5), CiviCRM limits finding and merging of duplicate records to users with the "Administer CiviCRM" permission. A recent thread on the forums points out that some organizations will want to allow that privilege to non-administrative users. Having a need for this myself, I'm looking for the best way to do it. If a reasonable solution can be found, I'm hoping the changes will make it into core at some point in the future.
May 29, 2010
By Eileen Filed under CiviCRMA while ago I mentioned that I was finding the backup and migrate module a better option to syncing Civi installs that the Civi drush command and Lobo asked if I would do a write-up on it.
I'm going to keep this pretty brief but I think it is worth telling people about as I find it really quick & easy.
http://drupal.org/project/backup_migrate (AKA BAM)
May 28, 2010
By Anonymous Filed under CiviCRM, SchoolsThe next phase of the CiviSchool project is collecting and maintaining all parent / student information online. This avoids the annual filling out forms work by the parents, and also the stuffing of envelopes by school staff during the summer and reentering all the information in the SIS when school starts.
May 22, 2010
By charlesc Filed under MeetupsWe(NET Tuesday Taiwan) have a CiviCRM workshop since last August and had meetups for about 6-7 times. After Jimmy Huang came from CiviCon/Translation Sprint, we had a review meetup at 5/15 and gathered about 40 people from local NGOs.
What's the next? Due to limited resource, I think we need have 2~3 successful local case studies and empower ourselves to better support local NGOS.
presentation
May 20, 2010
By lobo Filed under CiviCRMAt CiviCon SF 2010, we met the folks from Emotive LLC (chang and matthew b) who expressed an interest in helping scale CiviCRM and in specific help with building an import system that can handle large data sets.
I started discussing this with chang on the forums and IRC. You can read our forum thread here: http://forum.civicrm.org/index.php/topic,13630.0.html
