django – Parerga und Paralipomena http://www.michelepasin.org/blog At the core of all well-founded belief lies belief that is unfounded - Wittgenstein Mon, 15 Dec 2014 10:50:50 +0000 en-US hourly 1 https://wordpress.org/?v=5.2.11 13825966 Setting up the new ‘staticfiles’ app on Django 1.3 http://www.michelepasin.org/blog/2012/01/24/setting-up-the-new-staticfiles-app-on-django-1-3/ Tue, 24 Jan 2012 10:26:09 +0000 http://www.michelepasin.org/blog/?p=1734 Even if I’ve been using Django 1.3 for a while now, I’ve been holding off on some of it new features, such as the new way to handle static files via a new app called (guess what) staticfiles . Essentially, what the new static app does is allowing you to leave all static files within the /static directory of each of the django apps you’re using. This is for development; when you’re deploying and (most likely) want these files to be served via a (faster) separate server process, the staticfiles app helps you gather all of them into a predefined directory by using a handy shell command.

That’s the gist of it. I finally took a look at django.contrib.staticfiles last week, so here’s a brief report on how to get it to work (p.s. a couple of related threads on stack overflow can be found here and here).

1. Set up the static and media paths settings

My previous settings looked like this:

# the site root is one level up from where settings.py is
SITE_ROOT = os.path.dirname(os.path.realpath(__file__)).rsplit('/', 1)[0]
MEDIA_ROOT = os.path.join(SITE_ROOT, 'static')
MEDIA_URL = '/static/'
ADMIN_MEDIA_PREFIX = '/static/adminmedia1.3/'

The media root contained all of my static files, some of which I had to copy in there manually, each time I added a new django app to my project. On the production server, the MEDIA_URL is mapped to a local path (in the apache conf settings) that is essentially a duplicate of the /static directory we have in the development server. The only difference, the static stuff is delivered directly by Apache, bypassing django (=so to make it faster).

The new way of declaring these variables is this instead:

MEDIA_URL = '/media/uploads/'
STATIC_URL = '/media/static/'
ADMIN_MEDIA_PREFIX = '/media/static/admin/'

# Absolute path to the directory that holds media uploaded
# I keep the uploads folder at the project-root level server
MEDIA_ROOT = os.path.join(SITE_ROOT, 'uploads') 

# physical location of extra static files in development server
STATICFILES_DIRS = (
    os.path.join(SITE_ROOT, 'myproject/static'),
)
# path used with "python manage.py collectstatic"
# I normally put this at the project-root level that contains also the wsgi files for apache
STATIC_ROOT = os.path.join(SITE_ROOT, 'apache/static')

Obviously on a production server, you will have to set up the required aliases in the apache conf file, so that MEDIA_URL and STATIC_URL are pointing at the right physical locations:

Alias /media/uploads/ /path/to/mysite.com/uploads/
Alias /media/static/ /path/to/mysite.com/apache/static/

The django docs explain the new approach with these words:

In previous versions of Django, it was common to place static assets in MEDIA_ROOT along with user-uploaded files, and serve them both at MEDIA_URL. Part of the purpose of introducing the staticfiles app is to make it easier to keep static files separate from user-uploaded files.

For this reason, you need to make your MEDIA_ROOT and MEDIA_URL different from your STATIC_ROOT and STATIC_URL.

The STATIC_ROOT directory is not necessary in the development server: in fact is only used if you call the collectstatic manangement command. It’s not needed to add the directory to the STATICFILES_DIRS setting to serve your static files!

Furthemore, the STATICFILES_DIRS variable tells Django of the location of static files which are not within specific applications. Mind that during the ‘collection’ operations also these files will be copied into the STATIC_ROOT directory.

 

2. Add context processors and the app

Add the required context processor in setting.py:

TEMPLATE_CONTEXT_PROCESSORS += 'django.core.context_processors.static'

Add also the new static app to your installed apps:

INSTALLED_APPS = (
....    
    'django.contrib.staticfiles',    
.....
)

Keep in mind that it’s very likely that in your templates you will have to manually change all references to MEDIA_URL into STATIC_URL!

 

3. Url configuration

On your development server, this is what you would do:

from django.contrib.staticfiles.urls import staticfiles_urlpatterns
# ... the rest of your URLconf goes here ...
urlpatterns += staticfiles_urlpatterns()

This will inspect your STATIC_URL setting and wire up the view to serve static files accordingly. Don’t forget to set the STATICFILES_DIRS setting appropriately to let django.contrib.staticfiles know where to look for files additionally to files in app directories.

WARNING: the staticfiles_urlpatterns helper function will only work if DEBUG is True and your STATIC_URL setting is neither empty nor a full URL such as http://static.example.com/ (more info here).

Finally, mind that in this new approach you will need to arrange for serving of files in MEDIA_ROOT yourself; staticfiles does not deal with user-uploaded files at all. You can, however, use django.views.static.serve() view for serving MEDIA_ROOT in development (for more info see Serving other directories).

if settings.LOCAL_SERVER:     # ===> static files on local machine    
    urlpatterns += patterns('', 
        (r'^media/uploads/(?P<path>.*)$', 'django.views.static.serve', 
            {'document_root': settings.MEDIA_ROOT, 'show_indexes': True}),
        )

In the end, I conflated the two things into this code (ps I’ve added a variable called LOCAL_SERVER to quickly see which platform I’m on):

if settings.LOCAL_SERVER:     # ===> static files on local machine
    from django.contrib.staticfiles.urls import staticfiles_urlpatterns
    urlpatterns += staticfiles_urlpatterns()    
    urlpatterns += patterns('', 
        (r'^media/uploads/(?P<path>.*)$', 'django.views.static.serve', 
            {'document_root': settings.MEDIA_ROOT, 'show_indexes': True}),
        )

 

4. On your production server

Easy: in your urlconf there’s no need to do anything, as static urls are usually handled by Apache directly. However for that to happen what you have to do is collect all static files into the directory that apache is looking into, that is, the one specified with the STATIC_ROOT setting. This is how you do it:

python manage.py collectstatic

This shell command will
a) look in the /static/ directory of each of the apps of yours INSTALLED_APPS setting.
b) look in directories you specify in the STATICFILES_DIRS setting.

…and copy whatever it finds into the STATIC_ROOT directory, as defined in your settings (ps: it’ll preserve the original directory structure).

That’s all folks!

 

]]>
1734
Django B_project available on gitHub http://www.michelepasin.org/blog/2011/10/12/django-b_project-available-on-github/ http://www.michelepasin.org/blog/2011/10/12/django-b_project-available-on-github/#comments Wed, 12 Oct 2011 09:34:00 +0000 http://www.michelepasin.org/blog/?p=1593 I’ve put together and shared on gitHub all the files needed to get started with a Django project – essentially this is a clean file structure usable to get going with a Django project nice and easy. The idea is that after downloading this, you just have to change a couple of names (see the README file for more details) and there you go you have a complete Django project ready to be ran and customized.

The code is here: bitbucket.org/magicrebirth/django_bproject. Feel free to download or fork it as you like!

Currently the B_project is set up to work with Django 1.3 and it has a few ‘batteries’ included (= commonly used apps):

  • MPTT: Utilities for implementing a modified pre-order traversal tree in django
  • REGISTRATION: A user-registration application for Django.
  • DJANGO EXTENSIONS: global custom management extensions for the Django Framework.
  • PICKLEFIELD: an implementation of a pickled object field.
  • Django B_project structure

    The other things included in the framework are:

  • a settings.py file that calls localized settings specifications depending on parameters (eg dev server or live one)
  • a bunch of generic python utilities
  • a few ‘enhanced’ model classes
  • all that is needed for working with trees in the admin (as discussed here: Django admin and MPTT)
  • a javascript autocomplete framework, integrated in the admin (as discussed here: Autocomplete in Django)
  • some basic templates and templatetags
  • Enjoy!

     

    ]]>
    http://www.michelepasin.org/blog/2011/10/12/django-b_project-available-on-github/feed/ 2 1593
    DJFacet: a faceted browser for Django http://www.michelepasin.org/blog/2011/06/09/djfacet-a-django-faceted-browser/ http://www.michelepasin.org/blog/2011/06/09/djfacet-a-django-faceted-browser/#comments Thu, 09 Jun 2011 09:26:00 +0000 http://www.michelepasin.org/blog/?p=1439 DJFacet is a pluggable module for the Django web application framework that allows you to navigate the data in your webapp using an approach based on ‘facets’. DJFacet relies entirely on the django models you’ve already defined within your project and on a configuration file where you can create the facets and assign them behaviour. This makes it very easy to integrate within your Django application.

    I’ve been working on DJFacet on and off for more than a year now, so I’m really happy to finally release a stable version of it. The software is still under active development, so be certain that in the coming months new features and bug fixes will be released!

     
    djfacet screenshot

     
    In a nutshell, the main features of DJFacet are:

  • Rapid installation and integration with existing Django projects
  • It’s back-end agnostic (as it rests on Django’s Database API)
  • Has a minimal and customisable look and feel, based on template override
  • It follows a REST architecture: urls of a search are stable and bookmarkable
  • It supports pivoting (the type of objects being searched for can be changed dynamically)
  • It provides a dedicated caching system (useful for apps with many facets/zoom points)
  • Find out more about it using these links:

  • Source code on Bitbucket: bitbucket.org/magicrebirth/djfacet
  • Documentation: michelepasin.org/support/djfacet/docs/
  • Demo installation: demos.michelepasin.org/djfacet/
  • Project page: www.michelepasin.org/artifacts/software/djfacet/
  •  

    ]]>
    http://www.michelepasin.org/blog/2011/06/09/djfacet-a-django-faceted-browser/feed/ 1 1439
    Python links (and more) 7/2/11 http://www.michelepasin.org/blog/2011/02/03/python-links-and-more-7211/ Thu, 03 Feb 2011 15:23:21 +0000 http://www.michelepasin.org/blog/?p=1075 This post contains just a collection of various interesting things I ran into in the last couple of weeks… they’re organized into three categories: pythonic links, events and conferences, and new online tools. Hope you’ll find something of interest!

    Pythonic stuff:

  • Epidoc
    Epydoc is a handy tool for generating API documentation for Python modules, based on their docstrings. For an example of epydoc’s output, see the API documentation for epydoc itself (html, pdf).
  • PyEnchant
    PyEnchant is a spellchecking library for Python, based on the excellent Enchant library.
  • Dexml
    The dexml module takes the mapping between XML tags and Python objects and lets you capture that as cleanly as possible. Loosely inspired by Django’s ORM, you write simple class definitions to define the expected structure of your XML document.
  • SpecGen
    SpecGen v5, ontology specification generator tool. It’s written in Python using Redland RDF library and licensed under the MIT license.
  • PyCloud
    Leverage the power of the cloud with only 3 lines of python code. Run long processes on the cloud directly from your shell!
  • commandlinefu.com
    This is not really pythonic – but nonetheless useful to pythonists: a community-based repository of useful unix shell scripts!
  • Events and Conferences:

  • Digital Resources in the Humanities and Arts Conference 2011
    University of Nottingham Ningbo, China. The DRHA 2011 conference theme this year is “Connected Communities: global or local2local?”
  • Narrative and Hypertext Workshop at the ACM Hypertext 2011 conference in Eindhoven.
  • Culture Hack Day, London, January 2011
    This event aimed at bringing cultural organisations together with software developers and creative technologists to make interesting new things.
  • History Hack Day, London, January 2011
    A bunch of hackers with a passion for history getting together and doing experimental stuff
  • Conference.archimuse.com
    The ‘online space for cultural informatics‘: lots of useful info here, about publications, jobs, people etc.
  • Agora project: Scholarly Open Access Research in European Philosophy
    Project looking at building an infrastructure for the semantic interlinking of European philosophy datasets
  • Online tools:

  • FactForge
    A web application aiming at showcasing a ‘practical approach for reasoning with the web of linked data’.
  • Semantic Overflow
    A clone of Stack Overflow (collaboratively edited question and answer site for programmers) for questions ‘about semantic web techniques and technologies’.
  • Google Refine
    A tool for “working with messy data, cleaning it up, transforming it from one format into another, extending it with web services, and linking it to databases”.
  • Google Scribe
    A text editor with embedded autocomplete suggestions as you type
  • Books Ngram Viewer
    Tool that displays statistical information regarding the use of user-selected sentences in a corpus of books (e.g., “British English”, “English Fiction”, “French”) over the selected years
  • ]]>
    1075
    Setting up Django Registration http://www.michelepasin.org/blog/2011/01/14/setting-up-django-registration/ http://www.michelepasin.org/blog/2011/01/14/setting-up-django-registration/#comments Fri, 14 Jan 2011 14:16:06 +0000 http://www.michelepasin.org/blog/?p=1057 Django’s admin framework includes the basic functionalities for logging in and out of the admin site, but if you’re building a so-called ‘social’ application and want people to be able to sign-up, log in and thus benefit from additional site functionalities then you need to use a more generic ‘registration’ application. The good news is: most of the work has already been done, so it’s just a matter of putting things together correctly.

    I found a nicely written blog post on this topic, so in what follows I’m just doing a re-cap of the main steps involved and adding some other useful info.

    Before we start: what follows has been tested on Django 1.1 Django 1.3, Python 2.6 and MySQL (note: on 13/8/12 I revised this post so that the code would work on django 1.3 – many thanks to the folks who left comments and pointed out what needed to be updated!)

    1) Install

    Download the package from bitbucket and add it to your application directory (or put it somewhere else and then modify your python path as needed). Then add the ‘registration’ app to the installed_apps tuple in your settings.py:

    INSTALLED_APPS = (
        'django.contrib.auth',
        'django.contrib.humanize',
        'django.contrib.contenttypes',
        'django.contrib.sessions',
        'django.contrib.sites',
        'django.contrib.admin',
        # etc..
        'registration',
    )
    

    2) Syncdb

    Do a quick syncdb from the console, which will result in one new table being created in the DB, ‘RegistrationProfile’. All it contains is its primary key, the ‘user’ key, and the ‘activation code’ fields. Now we’ve got the data structures needed for setting up the registration application!

    > python manage.py syncdb
    

    3) Setting.py

    Create some registration settings in settings.py:

    ACCOUNT_ACTIVATION_DAYS=7
    EMAIL_HOST='localhost'
    EMAIL_PORT=1023
    EMAIL_HOST_USER='username'
    EMAIL_HOST_PASSWORD='password'
    

    Obviously you’ve got to change the settings above depending on your specific server setup (more information is available here or here). Also, see point 7 below if you’re just testing things out without a SMTP server available.

    4) Urls.py

    Include the required registration urls to your urls.py:

    urlpatterns = patterns(",
      (r'^admin/', include('django.contrib.admin.urls')),
      (r'^accounts/', include('registration.urls')),
    )
    

    5) Templates

    Add the registration templates to your django-templates directory. The django-registration package we downloaded earlier on doesn’t include any templates, but you can easily find some of them online (for example here).
    Anyways, no need to do that: I took those templates and added some other ones too so to create the minimal working package that’ll get you going (that doesn’t include any fancy css styling but all the basic html stuff is there). You can download the templates here, expand the zip file and put it in templates/registration.

    6) Done! Let’s recap..

    We’re now ready to go. Let’s pause for a moment and recap what we achieved: we installed and activated django-registration, which sets up a whole bunch of new urls. These are divided into two groups:

    a) /login, /logout, the two-step password change at password/change/ and password/change/done/; the four-step password reset at password/reset/, password/reset/confirm/, password/reset/complete/ and password/reset/done/.
    This is the original URL specification source code (you can see it on bitbucket too) :

    # from django-registration / registration / auth_urls.py
    urlpatterns = patterns('',
                           url(r'^login/$',
                               auth_views.login,
                               {'template_name': 'registration/login.html'},
                               name='auth_login'),
                           url(r'^logout/$',
                               auth_views.logout,
                               {'template_name': 'registration/logout.html'},
                               name='auth_logout'),
                           url(r'^password/change/$',
                               auth_views.password_change,
                                {'template_name': 'registration/password_change_form.html'},
                               name='auth_password_change'),
                           url(r'^password/change/done/$',
                               auth_views.password_change_done,
                                {'template_name': 'registration/password_change_done.html'},
                               name='auth_password_change_done'),
                           url(r'^password/reset/$',
                               auth_views.password_reset,
                                {'template_name': 'registration/password_reset_form.html'},
                               name='auth_password_reset'),
                           url(r'^password/reset/confirm/(?P<uidb36>[0-9A-Za-z]+)-(?P<token>.+)/$',
                               auth_views.password_reset_confirm,
                                {'template_name': 'registration/password_reset_confirm.html'},
                               name='auth_password_reset_confirm'),
                           url(r'^password/reset/complete/$',
                               auth_views.password_reset_complete,
                                {'template_name': 'registration/password_reset_complete.html'},
                               name='auth_password_reset_complete'),
                           url(r'^password/reset/done/$',
                               auth_views.password_reset_done,
                                {'template_name': 'registration/password_reset_done.html'},
                               name='auth_password_reset_done'),
    )
    

    b) the second group of urls are /activate and /register:
    This is the original URL specification source (see it on bitbucket) :

    # django-registration / registration / backends / default / urls.py 
    urlpatterns = patterns('',
                           url(r'^activate/complete/$',
                               direct_to_template,
                               { 'template': 'registration/activation_complete.html' },
                               name='registration_activation_complete'),
                           # Activation keys get matched by w+ instead of the more specific
                           # [a-fA-F0-9]{40} because a bad activation key should still get to the view;
                           # that way it can return a sensible "invalid key" message instead of a
                           # confusing 404.
                           url(r'^activate/(?P<activation_key>w+)/$',
                               activate,
                               { 'backend': 'registration.backends.default.DefaultBackend' },
                               name='registration_activate'),
                           url(r'^register/$',
                               register,
                               { 'backend': 'registration.backends.default.DefaultBackend' },
                               name='registration_register'),
                           url(r'^register/complete/$',
                               direct_to_template,
                               { 'template': 'registration/registration_complete.html' },
                               name='registration_complete'),
                           url(r'^register/closed/$',  # UNUSED
                               direct_to_template,
                               { 'template': 'registration/registration_closed.html' },
                               name='registration_disallowed'),
                            # this is the default django login module 
                           (r'', include('registration.auth_urls')),
                           )
    

    7) Testing

    If you’re testing things on a development server you might not have access to an SMTP server (needed to test the email-based registration process). In such a case you can still try out your application using a workaround. In your settings.py file change the registration settings with the following ones:

    EMAIL_HOST = 'localhost'
    EMAIL_PORT = 1025
    EMAIL_HOST_USER = "
    EMAIL_HOST_PASSWORD = "
    EMAIL_USE_TLS = False
    DEFAULT_FROM_EMAIL = 'testing@example.com'
    

    Then open up another console window and run a temporary ‘dummy’ SMTP server with python:

    bash-3.2$ python -m smtpd -n -c DebuggingServer localhost:1025
    

    This local SMTP server remains there waiting for incoming messages. If now you go to /accounts/register/, fill out the form and hit ‘send’ you’ll see the registration email printed out in the console. Basically what happened is this: the ‘dummy’ python SMTP server we’ve just set up picked up django’s email-sending request and consequently printed out the email contents. If this is indeed what happened, it means that your code is working properly… and that you can use the url provided in the email to test the activation functionality too.

    For example, here is what you would see in the console after registering:

    
    ---------- MESSAGE FOLLOWS ----------
    Content-Type: text/plain; charset="utf-8"
    MIME-Version: 1.0
    Content-Transfer-Encoding: quoted-printable
    Subject: Activate your djangoproject.com account - you have 2 days!
    From: testing@example.com
    To: michele.pasin@hotmail.com
    Date: Wed, 12 Jan 2011 16:49:59 -0000
    Message-ID: <20110112164959.3366.35638@mymac.local>
    X-Peer: 127.0.0.1
    
    
    Someone, hopefully you, signed up for a new account at djangoproject.com using this email address. If it was you, and you'd like to activate and use your
    account, click the link below or copy and paste it into your web browser's address bar:
    
    http://127.0.0.1:8000/accounts/activate/6342fca5ffd430a820be6d98acde6e59a4c2d29c/
    
    If you didn't request this, you don't need to do anything; you won't receive any more email from us, and the account will expire automatically in two days.
    
    ------------ END MESSAGE ------------

    Pasting the ‘activation’ url in your browser should allow you to complete the registration process and check the rest of your code.

    Finally, keep in mind also that the django-registration application sends out emails that contain your site’s URL for the activation link, and that URL is dynamically determined using the ‘sites’ application (normally added via settings.py). By default, your domain name is listed as ‘example.com’, and the easiest way to change this is to log into the admin application and click on the ‘Sites’ link on the admin home page to get to the relevant entry.

    ===== ++ =====

    Some other possibly useful links:

    Fromt the Django website:

  • User authentication in django
  • Sending emails
  • From the Django book:

  • Chapter 12: Sessions, users, and registration
  •  

    ]]>
    http://www.michelepasin.org/blog/2011/01/14/setting-up-django-registration/feed/ 23 1057
    Preloading stuff in django’s interactive shell http://www.michelepasin.org/blog/2010/11/17/preloading-stuff-in-djangos-interactive-shell/ http://www.michelepasin.org/blog/2010/11/17/preloading-stuff-in-djangos-interactive-shell/#comments Wed, 17 Nov 2010 18:41:07 +0000 http://www.michelepasin.org/blog/?p=1012 Django’s shell is a fantastic way to interact with all the components of your django application, eg when testing new functionalities or debugging a nasty error. Sometimes though you end up loading the same variables or importing the same modules every time you run the shell , for example because you are trying out a large function that needs being refined through a trial and error process.

    Opening up the shell and reloading all the components you need to have handy can thus become a bit tedious; here’s an easy way to go around this problem.
    In doing this, I’ve just been inspired some code found in the handy django-extensions app. The extensions ship with a number of command modules, that is, functions that you can run from the unix shell using the usual python manage.py some_command syntax.

    The module I’m talking about here is called shell_plus – it’s an enhanced version of the Django shell. It will autoload all your models making it easy to work with the ORM right away. Here’s the implementation:

    # django_extensions/management/commands/shell_plus.py

    import os
    from django.core.management.base import NoArgsCommand
    from optparse import make_option

    class Command(NoArgsCommand):
        option_list = NoArgsCommand.option_list + (
            make_option('--plain', action='store_true', dest='plain',
                help='Tells Django to use plain Python, not IPython.'),
            make_option('--no-pythonrc', action='store_true', dest='no_pythonrc',
                help='Tells Django to use plain Python, not IPython.'),
        )
        help = "Like the 'shell' command but autoloads the models of all installed Django apps."

        requires_model_validation = True

        def handle_noargs(self, **options):
            # XXX: (Temporary) workaround for ticket #1796: force early loading of all
            # models from installed apps. (this is fixed by now, but leaving it here
            # for people using 0.96 or older trunk (pre [5919]) versions.
            from django.db.models.loading import get_models, get_apps
            loaded_models = get_models()

            use_plain = options.get('plain', False)
            use_pythonrc = not options.get('no_pythonrc', True)

            # Set up a dictionary to serve as the environment for the shell, so
            # that tab completion works on objects that are imported at runtime.
            # See ticket 5082.
            from django.conf import settings
            imported_objects = {'settings': settings}
            for app_mod in get_apps():
                app_models = get_models(app_mod)
                if not app_models:
                    continue
                model_labels = ", ".join([model.__name__ for model in app_models])
                print self.style.SQL_COLTYPE("From '%s' autoload: %s" % (app_mod.__name__.split('.')[-2], model_labels))
                for model in app_models:
                    try:
                        imported_objects[model.__name__] = getattr(__import__(app_mod.__name__, {}, {}, model.__name__), model.__name__)
                    except AttributeError, e:
                        print self.style.ERROR_OUTPUT("Failed to import '%s' from '%s' reason: %s" % (model.__name__, app_mod.__name__.split('.')[-2], str(e)))
                        continue
            try:
                if use_plain:
                    # Don't bother loading IPython, because the user wants plain Python.
                    raise ImportError
                import IPython
                # Explicitly pass an empty list as arguments, because otherwise IPython
                # would use sys.argv from this script.
                shell = IPython.Shell.IPShell(argv=[], user_ns=imported_objects)
                shell.mainloop()
            except ImportError:
                # Using normal Python shell
                import code
                try: # Try activating rlcompleter, because it's handy.
                    import readline
                except ImportError:
                    pass
                else:
                    # We don't have to wrap the following import in a 'try', because
                    # we already know 'readline' was imported successfully.
                    import rlcompleter
                    readline.set_completer(rlcompleter.Completer(imported_objects).complete)
                    readline.parse_and_bind("tab:complete")

                # We want to honor both $PYTHONSTARTUP and .pythonrc.py, so follow system
                # conventions and get $PYTHONSTARTUP first then import user.
                if use_pythonrc:
                    pythonrc = os.environ.get("PYTHONSTARTUP")
                    if pythonrc and os.path.isfile(pythonrc):
                        try:
                            execfile(pythonrc)
                        except NameError:
                            pass
                    # This will import .pythonrc.py as a side-effect
                    import user
                code.interact(local=imported_objects)

    Essentially, this is what’s going on here:

    a. A subclass of NoArgsCommand is created following the standard approach stated in Django’s docs for creating custom management commands .

    b. All the applications in your django project and the related models get loaded, and their references are added to the imported_objects dictionary, eg when the code reads:

    
    imported_objects = {'settings': settings}

    c. The code tries to load the best python shell available: iPython if present, otherwise the normal python shell with the necessary libraries are called.

    That’s all. The key line to look at is therefore the last one, that is:

    
    # .......
    code.interact(local=imported_objects)

    That’s what launches the interpreter and initializes it with the imported_objects dictionary; this dictionary contains all the (extra) symbols that we want to make available through the new python interpreter instance. So in order to have more stuff there all we have to do is add more elements to that dictionary, eg:

    
    # .......
    alist = range(1000)
    imported_objects['alist'] = alist
    code.interact(local=imported_objects)

    Insert these two lines right before the last line of the script above, save it using a new name (eg. my_fancyshell.py )in the /management/commands/ directory of one of your applications (it needs to be there so that django interprets it as a custom command), and the game is done. Now you can invoke

    
    bash-3.2$ python manage.py my_fancyshell.py

    from the unix command line, and the ‘alist’ symbol will be available.
    Obviously in a real world situation you might end up loading and adding to the imported_objects dictionary various many things too, but the principle will remain the same!

    ]]>
    http://www.michelepasin.org/blog/2010/11/17/preloading-stuff-in-djangos-interactive-shell/feed/ 3 1012
    Trying out Django-cms http://www.michelepasin.org/blog/2010/08/03/trying-out-django-cms/ Tue, 03 Aug 2010 13:33:52 +0000 http://www.michelepasin.org/blog/?p=792 I tried installing django-cms, an open-source project from Switzerland’s company Divio. It is a content management system based on the web framework Django and is written in Python.

    Among the interesting features of DjangoCms (besides the fact that it is based on django, which I happen to have fun with in most of my programming work):

  • Flexible Plugin Architecture. Build flexible pages with a wide range of plugins.
  • SEO Optimization. The structure of the pages is optimized for SEO.
  • Editorial workflow. Workflows for publishing and approval.
  • Permission Management. Set specific rights to different users.
  • Versioning. Each modification of the page will be saved. You can restore any state you wish.
  • Multisites. Administrate multiple websites over the same admin interface.
  • Multilanguage. Support for different languages (i.e. arabic, chinese or russian)
  • Applications (Apps). Add apps to different pages of the CMS.
  • Media Asset Manager (MAM). MAM allows you to manage all kind of assets (pictures, PDFs, videos and other documents).
  • It’s quite interesting to check out the other alternatives too, if you want a django-based CMS. This comparison webpage has all the info you need.

    —————————————–

    Getting to the meaty part: here are the steps I followed during installation:

    1. went to gitHub and tried to install the latest version available (2.1.0.beta2). Tried to run the example app within that package, but it was throwing too many errors so I decided to go for the 2.0 stable release (django-cms-2.0.0.final.tar.gz).

    2. This version worked fine, I just had to set the MEDIA directory giving using the full path on my machine, and remove all references to ‘South‘ and ‘Reversion‘ django apps (in ‘installed apps’) cause I don’t use them.

    3. fill out the DB specs as needed (‘DATABASE_ENGINE = ‘sqlite3’), update the DB using ‘python manage.py syncdb‘, and then start the server using ‘python manage.py runserver

    4. That’s it. Go to /admin, set up some pages, add content and see the result.

    Screen shot 2010-08-03 at 14.19.23.png

    The templates that come by default are good at giving you an idea of how the information is structured and retrieve, but they are a bit messy in my opinion. So I created my own template and added it to the default ones (all the info on how to do this on the official docs)

    Screen shot 2010-08-03 at 14.22.04.png

    The end result: not a masterwork, but a decent starting point for certain!

    Screen shot 2010-08-03 at 14.26.29.png

    ———————————

    Other places online where you might find useful django-cms tips:

    – django-cms mailing list on google groups
    – another mailing list on gmame.org

    That’s it for now. I like very much the plugin architecture (plugins or extensions can be downloaded here), maybe I’ll be posting some more about that in the future…

    ]]>
    792
    The power of django’s Q objects http://www.michelepasin.org/blog/2010/07/20/the-power-of-djangos-q-objects/ http://www.michelepasin.org/blog/2010/07/20/the-power-of-djangos-q-objects/#comments Tue, 20 Jul 2010 00:46:44 +0000 http://www.michelepasin.org/blog/?p=755 I’ve been spending some time today doing research on how to best use django’s Q objects. Then I did a bit of testing myself and decided to put this whole thing together for future’s sake, you know, for the time when I’ll find myself lost and in need some coding recipe.

    First things first: the official documentation.

  • Django doc: Complex lookups with Q objects
    The main intro
  • Django doc: OR lookups
    More in-depth examples
  • Now… let’s get started by resurrecting the simple model used in the django Tutorial, Polls, and adding a few instances to play with:

    # ==> in models.py
    from django.db import models
    import datetime
    
    class Poll(models.Model):
            question = models.CharField(max_length=200)
            pub_date = models.DateTimeField('date published')
    
    
    # ==> add some instances from the shell
    >>> p = Poll(question='what shall I make for dinner', pub_date=datetime.date.today())
    >>> p.save()
    >>> p = Poll(question='what is your favourite meal?', pub_date=datetime.datetime(2009, 7, 19, 0, 0)) #one year ago
    >>> p.save()
    >>> p = Poll(question='how do you make omelettes?', pub_date=datetime.date.today())
    >>> p.save()
    
    

    What are Q Objects Handy for?

    In a nutshell, the main reason why you might want to use them is because you need to do complex queries, for example involving OR and AND statements a the same time.

    Imagine that you want to retrieve all polls whose question contains the word ‘dinner’ OR ‘meal’. The usual ‘filter’ option lets you do the following:

    >>> Poll.objects.filter(question__contains='dinner').filter(question__contains='meal')
    []
    
    

    But this is not really what we want, is it? We just got a concatenation of AND statements (= all constraints need to be satisfied), while we wanted an OR (=at least one of the constraints needs to be satisfied).
    This is where Q objects become useful:

    >>> from django.db.models import Q
    # a simple query
    >>> Poll.objects.filter(Q(question__contains='dinner'))
    [<Poll: what shall I make for dinner>]
    # two constraints linked by AND
    >>> Poll.objects.filter(Q(question__contains='dinner') & Q(question__contains='meal'))
    []
    # two constraints linked by OR - that's it!
    >>> Poll.objects.filter(Q(question__contains='dinner') | Q(question__contains='meal'))
    [<Poll: what shall I make for dinner>, <Poll: what is your favourite meal?>]

    Note that if you don’t specify any logical connector the Q sequence is implicitly interpreted as AND:

    # no logical connector is interpreted as AND
    >>> Poll.objects.filter(Q(question__contains='dinner'), Q(question__contains='meal'))
    []
    

    Things start getting more interesting when creating complex queries:

    # eg (A OR B) AND C:
    >>> Poll.objects.filter((Q(question__contains='dinner') | Q(question__contains='meal')) & Q(pub_date=datetime.date.today()))
    [<Poll: what shall I make for dinner>]
    

    Dynamic Query Building

    Now, it is likely that you want to build up queries like the ones above dynamically. This is another place where Q objects can save lots of time….for example, when constructing search engines or faceted browsers where the interface lets a user accumulate search filters in an incremental manner.

    One way to handle this situation is by creating lists of Q objects, and then combining them together using python’s operator and reduce methods:

    >>> import operator
    # create a list of Q objects
    >>> mylist = [Q(question__contains='dinner'), Q(question__contains='meal')]
    # OR
    >>> Poll.objects.filter(reduce(operator.or_, mylist))
    [<Poll: what shall I make for dinner>, <Poll: what is your favourite meal?>]
    # AND
    >>> Poll.objects.filter(reduce(operator.and_, mylist))
    []
    

    Now, if you’re building the query dynamically you probably won’t know in advance which are the filters you have to use. It is likely instead that you find yourself having to generate a list of Q objects programatically from a list of strings representing queries to your models:

    # string representation of our queries
    >>> predicates = [('question__contains', 'dinner'), ('question__contains', 'meal')]
    # create the list of Q objects and run the queries as above..
    >>> q_list = [Q(x) for x in predicates]
    >>> Poll.objects.filter(reduce(operator.or_, q_list))
    [<Poll: what shall I make for dinner>, <Poll: what is your favourite meal?>]
    >>> Poll.objects.filter(reduce(operator.and_, q_list))
    []
    
    # now let's append another filter to the query-strings..
    >>> predicates.append(('pub_date', datetime.date.today()))
    >>> predicates
    [('question__contains', 'dinner'), ('question__contains', 'meal'), ('pub_date', datetime.date(2010, 7, 19))]
    # .. and the results will change too
    >>> q_list = [Q(x) for x in predicates]
    >>> Poll.objects.filter(reduce(operator.or_, q_list))
    [<Poll: what shall I make for dinner>, <Poll: what is your favourite meal?>, <Poll: how do you make omelettes?>]
    >>> Poll.objects.filter(reduce(operator.and_, q_list))
    []
    

    Query Expansion and Q Objects

    Using query expansion syntax you could normally do things like this:

    >>> mydict = {'question__contains': 'omelette', 'pub_date' : datetime.date.today()}
    >>> Poll.objects.filter(**mydict)
    [<Poll: how do you make omelettes?>]
    

    Here you are basically sequencing AND statements which are built dynamically from some string values.

    A really cool feature of django ORM is that you can keep doing this even while using Q objects. In other words, you can delegate all the ‘normal’ stuff to the query expansion mechanism, while instead resorting to Q objects whenever you need more complex queries, such an OR statement.

    For example (remember to put the dictionary always in second position):

    # OR plus query expansion
    >>> Poll.objects.filter(reduce(operator.or_, q_list), **mydict)
    [<Poll: how do you make omelettes?>]
    # AND plus query expansion
    >>> Poll.objects.filter(reduce(operator.and_, q_list), **mydict)
    []
    

    In the first case, we have one result only because, although the OR constraint on q_list is would match more stuff by itself, **mydict is exploded as a series of constraints connected by AND, thus making [Poll: how do you make omelettes?] the only matchable object.
    In the second case instead there are no results at all simply because we are asking for a Poll instance that satisfies simultaneously all the constraints (which doesn’t exist)!

    Other Resources

    That’s all for now! As I said, I found quite a few very good posts about the subject online and I strongly advise you to check them out too in order to have a clearer picture of how Q objects work.

  • Adding Q objects in Django
    Shows another method to chain Q objects together using ‘add’
  • A Tip on Complex Django Queries
    Discusses how queries need to be written in order to produce the expected mySQL code
  • The power of Q
    Really nice article that overviews all the major features of Q objects (and, or, negation, picking)
  • Dynamic Django queries (or why kwargs is your friend)
    Overview of dynamic querying with django
  •  

    ]]>
    http://www.michelepasin.org/blog/2010/07/20/the-power-of-djangos-q-objects/feed/ 18 755
    DjangoCon Europe 2010 http://www.michelepasin.org/blog/2010/06/30/djangocon-europe-2010/ Wed, 30 Jun 2010 11:58:45 +0000 http://www.michelepasin.org/blog/?p=725 >>]]> Picture 1.png

    DjangoCon returned for its second year in Europe; the first as a wholly community-supported conference. The conference website contains a lot of info and interesting links (even an iPhone app), but in my opinion the coolest thing is the freely available videos of the conference!

    >>>

    ]]>
    725
    Django debug toolbar http://www.michelepasin.org/blog/2010/05/24/django-debug-toolbar/ http://www.michelepasin.org/blog/2010/05/24/django-debug-toolbar/#comments Mon, 24 May 2010 10:46:53 +0000 http://www.michelepasin.org/blog/?p=690

    Django Debug Toolbar 0.8 from Idan Gazit on Vimeo.

    This is just an amazingly useful application you might want to add to your django development tools.

    Installation is a breeze:

    
    $ easy_install django-debug-toolbar
    

    Then you need to add some parameters to your settings.py, as explained here.
    It’s likely that you don’t want the debug-toolbar to show on your production site as well, but only where you’re doing development. So a more portable alternative to the usual way to add the debug-toolbar parameters is the following:

    
    # set to True if you're on the production server
    LIVESERVER = False
    
    # ..... other settings....
    
    if not LIVESERVER:
    	# stuff for the debug toolbar
    	INSTALLED_APPS += 	('debug_toolbar',)
    	INTERNAL_IPS = ('127.0.0.1',)
    	MIDDLEWARE_CLASSES += ( 'debug_toolbar.middleware.DebugToolbarMiddleware', )
    
    

    The LIVESERVER variable is the only thing you’ve got to manually change when copying your code to the production server…

    ]]>
    http://www.michelepasin.org/blog/2010/05/24/django-debug-toolbar/feed/ 1 690