skip to navigation
skip to content

Not Logged In

django-phpBB3 0.3.0

django database models of phpBB3

-------------
Django-phpBB3
-------------

Django-phpBB3 provides the database models of a existing phpBB3 installation. This is usefull for all people who like to access the phpbb forum database from a django application.
It's not a phpBB clone or a forum!

Django-phpBB3 has also a script for migrate a phpBB3 forum into a `DjangoBB forum <http://djangobb.org/>`_. See below.

Notes:

* There are a few mismatches between the origin phpBB3 Database schema and the ``models.py`` form Django-phpBB3. Because of this: Creating the phpBB3 tables via ``syncdb`` will not work with phpBB3.

    * phpBB3 used often ``0`` for a not set value, but we need a real ``null`` value in db (to get ``None`` back)

* phpBB3 Models with composite keys are unsupported, but stored in `django_phpBB3/unsupported_models.py <https://github.com/jedie/django-phpBB3/blob/master/django_phpBB3/unsupported_models.py>`_

Warning:
Write access to the original phpBB database is not tested, yet. So it's not guaranteed that changes trough the django admin will work with phpBB3!
So, in the current state, you should only access 'read-only' to the origin phpBB3 data ;)

Patches are welcome! Please send pull requests.

----------------
example projects
----------------

We add two example projects:

+-------------------------------+-----------------------------------------------------------------------+
| **`django_phpBB3_project`_**  | minimal project without extra depencies                               |
+-------------------------------+-----------------------------------------------------------------------+
| **`phpBB2DjangoBB_project`_** | DjangoBB example project for use the phpBB2DjangoBB migration command |
+-------------------------------+-----------------------------------------------------------------------+

.. _django_phpBB3_project: https://github.com/jedie/django-phpBB3/tree/master/django_phpBB3_project
.. _phpBB2DjangoBB_project: https://github.com/jedie/django-phpBB3/tree/master/phpBB2DjangoBB_project

-----
setup
-----

::

    ~$ virtualenv DjangoBB_env
    ~$ cd DjangoBB_env/
    ~/DjangoBB_env$ source bin/activate
    (DjangoBB_env)~/DjangoBB_env$ pip install django-phpBB3 # Will also install Django

(If default python interpreter is <2.7: Use: ``virtualenv -p python2.7 DjangoBB_env`` - see below)

Or if you will use the most recent git version:

::

    # install git read-only clone (Will also install Django):
    (DjangoBB_env)~/DjangoBB_env$ pip install -e git+git://github.com/jedie/django-phpBB3.git#egg=django-phpBB3

If you would like to test/use the DjangoBB migration install this:

::

    (DjangoBB_env)~/DjangoBB_env$ pip install -e git+git://github.com/slav0nic/DjangoBB.git#egg=DjangoBB
    (DjangoBB_env)~/DjangoBB_env$ pip install Whoosh

Maybe you want my **dev** branch of DjangoBB (More info in `My DjangoBB github wiki page <https://github.com/jedie/DjangoBB/wiki>`_):

::

    (DjangoBB_env)~/DjangoBB_env$ pip install -e git+git://github.com/jedie/DjangoBB.git@dev#egg=DjangoBB
    (DjangoBB_env)~/DjangoBB_env$ pip install Whoosh

local_settings.py example
=========================

::

    # database table prefix (default is: "phpbb3_")
    #PHPBB_TABLE_PREFIX = "phpbb_"

    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
            'NAME': 'phpbb3', # Or path to database file if using sqlite3.
            'USER': 'phpbb3', # Not used with sqlite3.
            'PASSWORD': 'foo bar', # Not used with sqlite3.
            'HOST': '', # Set to empty string for localhost. Not used with sqlite3.
            'PORT': '', # Set to empty string for default. Not used with sqlite3.
        }
    }

    # settings for phpBB3 migration:

    # filesystem path to the /files/ sub directory of the phpBB installation:
    # Set to None, if your phpBB3 forum has no attachments
    PHPBB_ATTACHMENT_PATH = "/path/to/phpBB/files/"

init database
=============

If you will test the two example projects, you need to initialises the database, e.g.:

::

    ~$ cd DjangoBB_env/
    ~/DjangoBB_env$ source bin/activate
    (DjangoBB_env)~/DjangoBB_env$ cd src/django-phpbb3/phpBB2DjangoBB_project
    (DjangoBB_env).../phpBB2DjangoBB_project$ # create local_settings.py here, see above!
    (DjangoBB_env).../phpBB2DjangoBB_project$ ./manage.py syncdb --all
    (DjangoBB_env).../phpBB2DjangoBB_project$ ./manage.py migrate --fake

------------------
test django-phpBB3
------------------

We added a test project for easy start in: `/django_phpBB3_project/ <https://github.com/jedie/django-phpBB3/tree/master/django_phpBB3_project>`_

Create a ``local_settings.py`` into ``/django_phpBB3_project/`` see example above.

You must initialises the database (see above).

You can startup a test server with, e.g:

::

    ./manage.py runserver --traceback --insecure

-----------------------------------------
migrate a phpBB3 installation to DjangoBB
-----------------------------------------

We add a example project for a phpBB3 migration in: `/phpBB2DjangoBB_project/ <https://github.com/jedie/django-phpBB3/tree/master/phpBB2DjangoBB_project>`_

The migration from phpBB3 to DjangoBB is implement as a migration command here: `/django_phpBB3/management/commands/phpbb2djangobb.py <https://github.com/jedie/django-phpBB3/blob/master/django_phpBB3/management/commands/phpbb2djangobb.py>`_

To run the migration you can use the minimal ``phpBB2DjangoBB`` example project example.
This project includes the DjangoBB forum app and django-phpBB3 app. Before start, you must install the dependencies!

You must create a ``local_settings.py`` into ``/phpBB2DjangoBB_project/`` see example above.

**Before you start migration:** You should check if database encoding settings are ok.
To check, login and check in *admin / Django_phpBB3 / **Posts*** if non-ascii characters are ok.

Typical problem are: e.g. in default ubuntu/debian setup a mysql server will use ``latin-1`` !
So before you start, you should correct this. e.g.:

::

    ALTER DATABASE test DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci

**Make a SQL Backup before doing anything!!!**

Run the migration with ``./manage.py phpbb2djangobb``, e.g:

::

    ~$ cd DjangoBB_env/
    ~/DjangoBB_env$ source bin/activate
    (DjangoBB_env)~/DjangoBB_env$ cd src/django-phpbb3/phpBB2DjangoBB_project
    (DjangoBB_env).../phpBB2DjangoBB_project$ ./manage.py phpbb2djangobb

after migration
===============

**Important:** No phpBB forum permissions would be migrated, yet. So: Before you make a migrated Forum available to public, you must manually *hide* any *internal* forums, if exists!
To limit a forum in DjangoBB, e.g.:

* Create a new user group in django admin panel

* Create a seperate djangobb category and add the creates user group

* Move *internal* forums to the new category

* To give a user the permissions to the *internal* forums: Edit the user and add the created user group.

You must update the haystack search index with e.g.:

::

    (DjangoBB_env).../phpBB2DjangoBB_project$ ./manage.py update_index --verbosity=2

more information: `http://django-haystack.readthedocs.org/en/latest/management_commands.html <http://django-haystack.readthedocs.org/en/latest/management_commands.html>`_

Depend on the used database you must reset sequences.

::

    Sequences are indexes used by some database engines to track the next available number for automatically incremented fields.

more info: `https://docs.djangoproject.com/en/dev/ref/django-admin/#sqlsequencereset-appname-appname <https://docs.djangoproject.com/en/dev/ref/django-admin/#sqlsequencereset-appname-appname>`_

options
=======

cleanup_users
-------------

Add ``--cleanup_users=x`` to setup which users are migrate, where **x** is a number:

+-------+-----------------------------------------------------------+
| **0** | all users                                                 |
+-------+-----------------------------------------------------------+
| **1** | users with email addresses (skip bots)                    |
+-------+-----------------------------------------------------------+
| **2** | users with email and has a 'lastvisit' date **(default)** |
+-------+-----------------------------------------------------------+
| **3** | email + lastvisit and has created at least one post       |
+-------+-----------------------------------------------------------+

migration limitations
=====================

Things that would not be transfered, yet:

* Forum moderators

* Category groups

* sort of Category / Forum would not be converted to position number

Handling missing features in DjangoBB:

* We transfer phpBB Users who are in the user group ``ADMINISTRATORS`` or ``GLOBAL_MODERATORS`` to all forum moderators.

* *global posts* and *announcement* converted to normal sticky posts

* sub forums would be flatten

* *ghost moved posts* are ignored (only the moved post would be created)

* ForumWatch are ignored

* Guest posts would be mapped to a new "Anonymous" user (has unusable password and no email address). Note: guest posts are planed in DjangoBB, see: `http://support.djangobb.org/topic/98/ <http://support.djangobb.org/topic/98/>`_

TODO:

* Handle forum permissions: Now all forums will be public, after migration!

* Bugfix for user post count migration - `https://github.com/jedie/django-phpBB3/issues/6 <https://github.com/jedie/django-phpBB3/issues/6>`_

* migrate user avatar - `https://github.com/jedie/django-phpBB3/issues/7 <https://github.com/jedie/django-phpBB3/issues/7>`_

* add a separate command for migrate phpBB private messages to django-messages (very low priority, patches are welcome)

migration - notes
=================

username
--------

The migration script copied the phpBB username 1:1 and doesn't cleanup anything.
The django forms for creating/change a user will only limit with: ``Letters, digits and @/./+/-/_ only``
But the model fields is a simple ``CharField``.
A nick will be accepted in login depend on the login form you use in your project.

password
--------

We migrate the phpBB3 passwords and add a own password hasher.
`Django upgrade the old password <https://docs.djangoproject.com/en/1.4/topics/auth/#password-upgrading>`_ with the default hasher (PBKDF2) on the first login.
For this, it's needed to add the phpBB3 password hasher to your settings, e.g.:

::

    PASSWORD_HASHERS = (
        'django.contrib.auth.hashers.PBKDF2PasswordHasher',
        'django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher',
        'django.contrib.auth.hashers.BCryptPasswordHasher',
        'django.contrib.auth.hashers.SHA1PasswordHasher',
        'django.contrib.auth.hashers.MD5PasswordHasher',
        'django.contrib.auth.hashers.CryptPasswordHasher',
        'django_phpBB3.hashers.PhpBB3PasswordHasher',
    )

more info: `https://docs.djangoproject.com/en/1.4/topics/auth/#how-django-stores-passwords <https://docs.djangoproject.com/en/1.4/topics/auth/#how-django-stores-passwords>`_

migration - open questions
==========================

* Are the phpBB times in ``phpbb_posts`` in UTC oder in local time from user?

troubleshooting
===============

HTMLParseError while post migration
-----------------------------------

Maybe you used a older Python Version than 2.7.3 In older versions the ``HTMLParser`` is not so robust as in 2.7.3
Maybe default Python interpreter is 2.6, but 2.7 is installed to, than try:

::

    # Display the exsiting Python version:
    ~$ python --version # Default python
    Python 2.6.6
    ~$ python2.7 --version # Try if 2.7 is available
    Python 2.7.3

    # Create env with 2.7:
    ~$ virtualenv -p python2.7 DjangoBB_env

Work-a-round if you can't create a virtualenv with a newer Python interpreter:
set: ``settings.DJANGOBB_SMILES_SUPPORT = False``
More Information here: `https://github.com/jedie/django-phpBB3/issues/14 <https://github.com/jedie/django-phpBB3/issues/14>`_

UnicodeEncodeError in the terminal while running migration
----------------------------------------------------------

Solution: You must set either your locale, or the ​PYTHONIOENCODING environment variable, according to your terminal's encoding — in most cases, utf-8.
More info can be found `in the comment of issues 9 <https://github.com/jedie/django-phpBB3/issues/9#issuecomment-8494830>`_

How to debug migration
----------------------

We use python built-in pdb module for debug a migration traceback.
`pudb <http://pypi.python.org/pypi/pudb>`_ is used, if installed. To install it, run e.g.:

::

    ~$ cd DjangoBB_env/
    ~/DjangoBB_env$ source bin/activate
    (DjangoBB_env)~/DjangoBB_env$ pip install pudb

------------
phpBB3 links
------------

Here some links for phpBB documentations:

* `http://wiki.phpbb.com/Tables <http://wiki.phpbb.com/Tables>`_

* `http://www.phpbbdoctor.com/doc_tables.php <http://www.phpbbdoctor.com/doc_tables.php>`_

The SQL schema files from phpBB can be found here:

* `https://github.com/phpbb/phpbb3/tree/develop/phpBB/install/schemas <https://github.com/phpbb/phpbb3/tree/develop/phpBB/install/schemas>`_

-------
history
-------

* v0.3.0 - 25.09.2012

    * migrate phpBB3 passwords, too and add a PASSWORD_HASHERS for this, see below.

* v0.2.1 - 21.09.2012

    * Add "is_staff" flag to all "global moderators"

    * Set "is_active" flag depent on "user_inactive_time"

    * Add ``--only_users`` for only migrate/update users.

    * Use |{get_or_create| for getting the pseudo "Anonymous" user account

* v0.2.0 - 21.09.2012

    * Speedup migration by disable haystack search index creation.

    * Bugfix: Handle not existing phpBB users -> Map to Anonymous user.

    * Add support for pudb debugger (optional) and fallback to pdb

    * Bugfix in migration: clean user signature, too.

    * Don't check PHPBB_ATTACHMENT_PATH if no attachments exists in phpBB3 forum.

    * `Avoid UnicodeEncodeError in migration output <https://github.com/jedie/django-phpBB3/issues/9#issuecomment-8494830>`_

    * Speedup user migration by stats only output

    * Add ``--flush_djangobb`` and ``--max_entries`` in migration command for testing.

* v0.1.6 - 29.08.2012

    * Add more information in README

    * Fix unicode errors

* v0.1.5 - 31.07.2012

    * `migrate user language, too <https://github.com/jedie/django-phpBB3/commit/170f035b076fb5e7b54a1453b298679653c2719b>`_.

* v0.1.4 - 30.07.2012

    * `many bugfixes in models <https://github.com/jedie/django-phpBB3/compare/de68647c86bf...6d3bfb623b24>`_.

* v0.1.3 - 27.07.2012

    * catch error if a post can't be migrated and continue migration

    * remove django_authopenid and django_messages from phpbb2djangobb project settings

    * nicer output while migrating with eta time

* v0.1.2 - 27.07.2012

    * remove date from version string

    * add this histroy list ;)

* v0.1.0 - 24.07.2012

    * migrate post subscriptions, too.

* 23.07.2012

    * migrate phpBB3 file attachments, too.

    * bugfixes in bbcode cleanup

    * handle html entities in cleanup

* 20.07.2012

    * bbcode cleanup contributed by nitely

* 19.07.2012

    * start with phpBB3 migration command

* 16.07.2012

    * project starts by Jens Diemer

.. |{get_or_create| image:: {get_or_create

-------------
Project links
-------------

+----------+-----------------------------------------------+
| Homepage | `https://github.com/jedie/django-phpBB3`_     |
+----------+-----------------------------------------------+
| PyPi     | `http://pypi.python.org/pypi/django-phpBB3/`_ |
+----------+-----------------------------------------------+

.. _https://github.com/jedie/django-phpBB3: https://github.com/jedie/django-phpBB3
.. _http://pypi.python.org/pypi/django-phpBB3/: http://pypi.python.org/pypi/django-phpBB3/

donation
========

* `Flattr this! <http://flattr.com/thing/03352794dbef7081256c14e349fb58fc>`_

* Send `Bitcoins <http://www.bitcoin.org/>`_ to ``1YJk2JTMyDrZ8oTHkMhTaQtRWqSgrHPiS``

contact
=======

Come into the conversation, besides the github communication features:

+---------+--------------------------------------------------------+
| Forum   | `http://www.pylucid.org/en/forum/11/`_                 |
+---------+--------------------------------------------------------+
| IRC     | #pylucid on freenode.net (Yes, the PyLucid channel...) |
+---------+--------------------------------------------------------+
| webchat | `http://webchat.freenode.net/?channels=pylucid`_       |
+---------+--------------------------------------------------------+

.. _http://www.pylucid.org/en/forum/11/: http://www.pylucid.org/en/forum/11/
.. _http://webchat.freenode.net/?channels=pylucid: http://webchat.freenode.net/?channels=pylucid
 
File Type Py Version Uploaded on Size
django-phpBB3-0.3.0.tar.gz (md5) Source 2012-09-25 47KB
  • Downloads (All Versions):
  • 88 downloads in the last day
  • 176 downloads in the last week
  • 797 downloads in the last month