Drupal 6: bi-directional user synchronization between various Drupal sites

Introduction: what are you talking about ?

Ok so let me explain this long title a little bit more. We have various Drupal sites: let's call one the hub and the other ones the clients. The requirements are the following:
  • Users shall be able to login using the same username and password on all sites (hub and clients)
  • When a user updates his profile on a client, changes shall be reflected on the hub and on the other clients
  • When a user updates his profile on the hub, changes shall be reflected on the clients
These three requirements, even though they are quite simple to express, are really hard to realize, at least if you want to use industry standards and make it easily extensible to new clients Before we go on, please note that the solution presented in this article is still experimental !

How ?

I found some similar use cases on the Internet, two of which were really interesting: Solving problems through collaboration and Simple Sign-On with OpenID. Thanks to Palantir.net and Developmentseed for sharing these use cases. Just like the guys at Palantir.net, we looked at Deploy and found out, unfortunately after some development had already happened, that it wasn't a good solution for our use case. We therefore drifted towards a solution based on OpenID for user login and PubSubHubBub for content synchronization. However, unlike the guys at developmentseed, we needed to synchronize not only user accounts but also their profiles, and we needed bi-directional synchronization.

OpenID Simple Sign On

We implemented the OpenID simple sign on using the process described in the article Simple sign on with OpenID.
  1. On the hub, download and install the openid_provider_sso module. Note that the module you will find on my github repository is the same as the one you will find in DevelopmentSeed's article, the only difference being a small message explaining the login process to the user.
  2. On the client, download and install the OpenID SSO module. Again, this module is the same as the one you will find on DevelopmentSeed's article.
  3. Following the screencast in DevelopmentSeed's article, add a relying party on the hub and add the hub's address on the client
  4. Depending on how your Drupal installation is set up, you might want to apply the patch of this issue to your OpenID module
You should now have a working installation for a simple sign on with OpenID.

User account synchronization

This solution assumes you are using Content profile, and therefore that a user account is basically a username, an email address and an openid. Also, please note that one limitation of this solution is that it limits the number of OpenIDs per user to 1.
  1. Download and install the following modules:
    • On both the hub and the client: Keyauth. Again, this module comes from DevelopmentSeed's article, however, for now, URL key authentication is not used in this solution, and will be one of the improvements needed to this solution.
    • On the hub: PuSH user. Again, this module comes from DevelopmentSeed, with a few small modifications.
    • On the client: Sync User. This module comes from DevelopmentSeed as well, but this time, I did quite a lot of modifications on it to allow for user profile synchronization.
  2. Once these modules are installed, you should not be able to change your email and/or username on the client, and a message should appear saying that you need to change these on the hub. Synchronization of user accounts is only one way (from the hub to the client), however don't worry, user profile synchronization is two-way.
  3. Try to change your email address on the hub: the client shall be notified and the email address shall be changed on the client as well. Make sure this works fine before going to the next step.

User profile synchronization

Now this is where most of the added value of this solution comes in. Up until now, it was basically a repeat of DevelopmentSeed's article.
  1. Download and install the following modules on both the hub and the client:
    • Views Atom: this module allows you to display Drupal nodes in Atom format using Views. User profiles will be pushed to the hub and clients in Atom feeds.
    • Feeds Atom: this module parses Atom feeds for insertion using Feeds. Note that both of these modules (Views Atom and Feeds Atom) were provided by Palantir.net for this use case I talked about at the beginning of this article.
    • PubSubHubbub pusher: this module allows you to configure which content types you want to push, when and how.
    • Sync Nodes: this module adds a Processor to Feeds which allows Node synchronization.
  2. Apply some patches on the Views Atom and Feeds Atom modules. The patch for Views Atom can be found here: it changes the way the guid is generated (in order to keep the same guid on the hub and the clients, and therefore allow synchronization) and adds a new view style called "RDF Nodes (Custom)" which allows you to define your own mapping of RDF properties. The patch for Feeds Atom can be found here: it adds a parser to Feeds which allows you to define your own mapping sources using a text field.
  3. I'll refer you to the screencast you will find at the bottom of this article for details on how you should configure your views and feed importers. The following is just an outline:
    1. Configure the view of your user profiles on the client and the hub. On the hub, you can choose "RDF (Nodes)" as the row style, but on the client, you need to choose "RDF (Nodes) Custom" and define the label of each field you want to show to be the same name as the label of the hub.
    2. Configure the feed importers on both the client and the hub. Look at the screencast really carefully here because there are some small tricks that will need to be fixed in the future.
    3. Subscribe the profile feed importer of the client to the hub doing an import: again, this will need to be fixed in the future...
  4. It should be done. If you followed the screencast really carefully, you should have a working bi-directional user synchronization.

Screencast

[blip.tv ?posts_id=4435112&dest=-1]

Part 1 - Setting up OpenID

[blip.tv ?posts_id=4435329&dest=-1]

Part 2 - Setting up user account synchronization

[blip.tv ?posts_id=4435280&dest=-1]

Part 3 - Setting up user profile synchronization on the hub

[blip.tv ?posts_id=4435399&dest=-1]

Part 4 - Setting up user profile synchronization on the client

[blip.tv ?posts_id=4435444&dest=-1]

Part 5 - It works !

Limitations and improvements

This solution is still experimental, obviously, and has various limitations and needs improvements:
  • User deletion: this solution only handles user inserts and updates. It should also handle user deletions.
  • Signed URLs: even though DevelopmentSeed's article uses keyauth in order to sign each URL, I'm deactivating it here, because for some reason, it didn't work when importing user profiles... It will need to be reactivated in the future.
  • As Drupal is going towards RDF, it would probably be better to use RDF instead of Atom feeds, and therefore use the RDF module to generate the views instead of views atom and feeds atom...
  • Support various OpenIDs: for now, this solution assumes that users only connect to the hub through OpenID. They can not connect through any other OpenID provider...

Conclusion: lots of advantages...

I believe using industry standards such as OpenID, Pubsubhubbub and Atom is a great way to synchronize Drupal sites, and can obviously be applied to any type of node. There are probably other ways to do this, but I believe this one is the closest one to Drupal's philosophy of adopting more and more RDF...

Update

The second part of this implementation can be seen on Guillaume's site: http://www.viguierjust.com/en/2011/02/06/drupal-6-complex-node-synchronization-with-pubsubhubbub/

Comments

Nice writeup. So basically you have implemented a StackOverflow style synchronization using Drupal. Very nice if it works. Will have to test it out sometime. I wonder if it could work similarly using Shibboleth instead of OpenID...

In reply to by YW

Permalink

Hi K,
Good question. Technically, there's no reason why it wouldn't, you just need to replace the authentication layer...

In reply to by YW

Permalink

Thanks Alex ! I thought about using OpenID AX but I needed the solution to be extensible to other types of nodes (not only content profile nodes). The pubsubhubbub pusher I wrote will allow me to easily extend this to other content types :)

After installing and configuring the two sites, when I logging into the hub and then to the client I get an error message on the client "OpenID login failed." Therefore the client fails to create a user account.

And I can't seem to find an error log reports from either site.

Has anyone else experienced this issue?

Thanks

In reply to by YW

Permalink

Hi Russom,

Try emptying manually the tables openid_association and openid_nonce on the client and openid_provider_association on the server...

In reply to by YW

Permalink

Hi Eric,

Unfortunately, our client wanted to use his client website on a different server, with a different database... So I don't believe the Domain set of modules would have worked, as it means one single shared database...