Moving CiviCRM's user and administrator guide to gitbook or readthedocs

For a while we have been thinking about some new infrastructure for CiviCRM's user and administrator documentation.  We'd like to move the guide somewhere that has better support for more of the features that we want, which include

• simple user interface that makes it easy for people to contribute
• produces clean source code
• produces a 'good looking' html version of the book
• ability to export to different format (including ebook and pdf and output suitable for traditional publishing)
• version control so that people can work on different 'branches' of the documentation at the same time (that we can merge independently when ready for publication)
• translation infrastructure, so that our language communities can easily create different language versions of the manual, and can easily keep these up to date when the manual is updated
• can handle growing documentation needs (including end user documentation for extensions)
• uses widely used standards that will outlive any particular platform choices we make

Following a few different conversations at different sprints over the years we grew fond of the idea of markdown stored in a git repository as a solid base that will help us meet many of the above requirements and started to look for a set of open source tools that would allow us to do this.  Gitbook "Modern book format and toolchain using Git and Markdown" seemed to tick a lot of boxes and so (while waiting for my plane at Denver airport) I thought I would take advantage of the free wifi and give it a test drive. I wrote a simple script https://github.com/michaelmcandrew/booki2gitbook that takes output from booki (our current tool) and turns it into something suitable for gitbook.  You can see the result here: http://gitbook.civicrm.org/ (this link will likely stop working once we find it a proper home). It is an alpha - lots to neaten up and lots of details to work out, but a good starting point for discussion.

You can see the source code that creates this book here: https://github.com/civicrm/civicrm-user-guide. Each part of the book is a directory and each of the chapters in that part are files in that directory.  All images are stored at /images.

Some areas we need to think through some more, with some initial thoughts:

How do we handle translations?

Symfony also store their documentation in github and have parallel repositories for each language e.g. https://github.com/symfony/symfony-docs and https://github.com/symfony-fr/symfony-docs-fr. I suspect that they have a fairly well developed procedure for handling the translation and updates of their documentation that we could learn from. Anyone have experience of similar documentation projects and how they do their translations?

What about extension documentation?

I think the obvious approach is for each extension to get its own repository which can then be registered somehow on civicrm.org so we can create a library of extension documentation.

What about the user friendly editor interface?

Or more generally, how can we make it as easy as possible for people to contribute to documentation?  I think we have two broad types of editors that we want to encourage.

1. drive by edits from people who want to quickly make a correction or add some missing info - these people want a simple login to a web based interface.
2. large and ongoing edits by power editors. These people will probably want to take advantage of desktop editors and so on, and might be branching and merging in git.

For better or worse, the vast majority of people in our current documentation community fit into this second category and I think they will be well served by this move since they will have a lot of options to choose from (both desktop and web-based). But I am not sure yet what the best option is for a simple UI for the drive by editors.  Right now we could ask them to use the gitbook interface

1. sign up for a gitbook account
2. import the the repository
3. make the at gitbook.com using their nice web-based gitbook interface
4. submit a pull request back to the main repo

Is this too onerous? It wil definitley discourage some drive by edits I suspect, and it would be great to see if we can put together a simpler process.

Why gitbook? what about readthedocs?

Just to clear it up, although gitbook run a hosted service, "The GitBook toolchain is open source and completely free, the source code of the tool is available on GitHub." (from http://help.gitbook.com/index.html) so it seems like an appropriate tool for us. And since we are using git and markdown as our underlying infrastructure we can complement it with other open source tools if we want to, and migrate away in the future if a better platform surfaces.  The other project / community that I looked at a bit is http://readthedocs.org/ and the associated http://www.mkdocs.org project.  They offer a fairly similar set of tools and it is probably worthwhile investigating what a readthedocs set up would look like and how they compare. Anyone have experience with readthedocs? The export script could be fairly easily updated to output to readthedocs, I think.

What next?

How does this sound to people? What have I forgotten? It would be great to hear your ideas on how we could move forward on this, and also great to get a few people together who can help get it ready for the sprints we have coming up in September in DC, and October in London.  Some hints for getting stuck in: either sign up for a gitbook account and import the repo https://github.com/civicrm/civicrm-user-guide or download the desktop editor, clone the repo locally and open it in the editor. And let me know what you think!