You have a many-to-many relationship on a model — i.e. Video has many Tags — and you want to be able to use just a text input to manage that relationship.

I recently came across this problem while working on a Video site that called for tagging of uploaded Media. Django’s default way of dealing with ManyToMany fields (checkboxes) did not seem appropriate since the UX would degrade very quickly as the number of tags grow. The UI paradigm that everyone is familiar with is obviously to enter the tags in a comma-delimited format, like how most popular blog sites have it. This post will walk you through how to implement this in a ModelForm that can be used in your admin. This tutorial is for Django version 1.1.1. In retrospect, I guess a custom widget may have been the option with best reusability, but if for nothing else, this post could give you a pretty good idea of how flexible Django is to extend and modify.

Models

First off, we create the VideoTag and Video class. Notice how the tags attribute of the Video model has editable=False - since we do not want to bother with the select boxes in our form.

from django.utils.translation import ugettext as _
from django.db import models
from django.template.defaultfilters import slugify

# models.py
class VideoTag(models.Model):
	slug = models.SlugField(max_length=30, editable=False, blank=True, unique=True)
	name = models.CharField(_('name'), max_length=30)
	parent = models.ForeignKey('VideoTag', verbose_name=_('parent'), related_name='children', blank=True, null=True)

	def __str__(self):
		return '%s' % (self.name)

	def __unicode__(self):
		return u'%s' % (self.name)

	def save(self, force_insert=False, force_update=False):
		self.slug = slugify(self.name)
		super(VideoTag, self).save(force_insert, force_update)

class Video(models.Model):
	title = models.CharField(_('title'), max_length=32)
	summary = models.TextField(_('summary'), max_length=300)
	tags = models.ManyToManyField(VideoTag, verbose_name=_('tags'), related_name='videos', blank=True, editable=False)

Forms

Now that we have the models, we can run go straight to the forms part, where all the relevant code is. As you can see, there are two steps to this: the first part is to override the initialization of the form to set the initial value of the tags_text field to read the tags M2M fields; the second part is to capture the input and save it as related objects.

#forms.py
from django.utils.translation import ugettext as _
from django import forms
from django.template.defaultfilters import slugify

from your_app.models import Video, VideoTag

class VideoForm(forms.ModelForm):
	tags_text = forms.CharField(label=_("video tags"), required=False, help_text=_("A comma-delimited list of the tags that you would like to tag this with"),)

	def __init__(self, *args, **kwargs):
		super(VideoForm, self).__init__(*args, **kwargs)

		# args.__len__ is greater than zero if it is a postback
		if len(args) == 0 and self.instance != None and self.instance.id != None:
			tags = list(self.instance.tags.all().order_by('name'))
			if len(tags) > 0:
				initial_text = reduce(lambda x, y: u'%s, %s' % (x, y.name), tags)
				self.fields["tags_text"].initial = initial_text

	def save(self, commit=True):
		instance = super(VideoForm, self).save(commit=True)
		tags = self.cleaned_data["tags_text"]
		instance.tags.clear()
		if tags:
			for tag in tags.split(','):
				if tag.lower().strip() != "":
					video_tag = None
					tag = tag.strip()
					try:
						video_tag = VideoTag.objects.get(slug=slugify(tag))
					except VideoTag.DoesNotExist:
						video_tag = VideoTag()
						video_tag.name = tag
						video_tag.save()
					instance.tags.add(video_tag)
		return instance

	def save_m2m(self):
		pass

	class Meta:
		model = Video

Admin

Next, you may want to use this form in your admin, so let’s set the admin to use the correct form:

#admin.py
from django.contrib import admin
from your_app.models import Video
from your_app.forms import VideoForm

class VideoAdmin(admin.ModelAdmin):
	form = VideoForm
admin.site.register(Video, VideoAdmin)

Please be aware that this is a hack. It is not an optimal solution. Stuff like form.as_* will NOT work with the solution below. All this does is add a property called ordered_fields to your ModelForm which implements the correct behavior.

# models.py
class Bar(models.Model):
	pass

class Foo(models.Model):
        # ordinarily, Django 1.0 would list the regular_field first and all M2Ms last
	many_to_many = models.ManyToManyField('Bar', related_name='foos')
	regular_field = models.CharField(max_length=10)

# forms.py
class FooForm(forms.ModelForm):
	def ordered_fields(self):
		list = []
		for field_name in self._meta.fields:
			list.append(self[field_name])
		return list

	class Meta:
		model = Foo
		fields = ('many_to_many',
			     'regular_field',)

#Template:

<form action="{{ app_path }}" method="post" id="foo-form">
	<fieldset class="module aligned ()">
	{% for field in form.ordered_fields %}
		<div class="form-row thin wide ()">

			{{ field.label_tag }}
			{{ field }}
			{{ field.errors }}
			{% if field.help_text %}
			<p class="help">{{ field.help_text }}</p>
			{% endif %}
		</div>
	{% endfor %}
	</fieldset>
	<div class="submit-row aligned ()">
		<input class="default" type="submit" value="Send" />
	</div>
</form>

    #ProfileForm is a ModelForm
    form = ProfileForm(request.POST, request.FILES, instance=profile)

There were two libraries that I found for parsing RSS; RSS.py by Mark Nottingham which unfortunately didn’t support RSS 2.0, and Mark Pilgrim’s feedparser.py. I settled on the latter since it contains more features, has easier syntax and has better unit-test coverage.

For what I’m doing, a template tag works out best for rendering the RSS items; let’s fast-forward to the code:

foo.html:

{% load rss_feed %}
{% rss_feed %}

[application]/templatetags.py:

import feedparser
from datetime import datetime
from django.template import Library, Node
register = Library()

@register.simple_tag
def rss_feed(itemCount=5):
	channel = feedparser.parse('http://blog.dev360.com/feed/')

	html = []
	html.append('<div id="rss-feed">')
	html.append('	<h1>Latest blog posts</h1>')
	html.append('	<ul>')
	for entry in channel.entries[:itemCount]:
		url = entry.id
		title = entry.title
		date = datetime.strptime(entry.date, "%a, %d %b %Y %H:%M:%S +0000");
		html.append('<li><p><strong><a href="%s">%s</a></strong>' % (url, title));
		html.append('%s</p></li>' % (date.strftime("%B %d, %Y")))

	html.append('	</ul>')
	html.append('</div>')
	return "\n".join(html)

On many occassions in the past, I have found myself writing roll-over code that looks something like this:

<ul class="navigation">
  <li onmouseover="this.className='selected';" onmouseout="this.className='';" class="selected"><a href="#">Link 1</a></li>
  <li onmouseover="this.className='selected';" onmouseout="this.className='';"><a href="#">Link 2</a></li>
  <li onmouseover="this.className='selected';" onmouseout="this.className='';"><a href="#">Link 3</a></li>
</ul>

As you can see, the onmouseover/onmouseout makes the code less readable and takes up a lot of unnecessary space. One way to get rid of it is obviously to take advantage of the :hover pseudo class on the a element but if you want the hovered item to be the only item marked with the selected class then obviously that solution doesn’t cut it.

jQuery’s event wiring, along with its selector syntax allows you to remove the onmouse**** tags and earn some well deserved HTML karma.

JS

$lastActiveItem = null;
jQuery(document).ready(function(){
	$('.navigation li').mouseover(function(){
		$lastActiveItem = $('.navigation li.selected');
		$('.navigation li').removeClass('selected');
		$(this).addClass('selected');
	});

	$('.navigation li').mouseout(function(){
		$('.navigation li').removeClass('selected')
		$lastActiveItem.addClass('selected')
	});
});

C:\Users\Administrator\twill\other_packages\_mechanize_dist\_auth.py:14: DeprecationWarning: the md5 module is deprecated; use hashlib instead import re, base64, urlparse, posixpath, md5, sha, sys, copy
C:\Users\Administrator\twill\other_packages\_mechanize_dist\_auth.py:14: DeprecationWarning: the sha module is deprecated; use the hashlib module instead import re, base64, urlparse, posixpath, md5, sha, sys, copy

The solution is trivial; to supress deprecation warnings in python, just add an ingore filter before the line where you import Twill (or any other offending library). The alert reader may notice that the second part of the filterwarnings argument (’.*’) is a regular expression; this would allow you to tailor exactly which deprecation warnings you wish to propagate to the command-line.

import warnings
warnings.filterwarnings('ignore', '.*')

from twill.commands import *
import twill

Let me start by saying that I can’t say that I’m very knowledgeable or up to date when it comes to the academic research on the subject - apart from following a few AI and Semantic Web blogs. Now, the thing that immediately raised my eyebrows was a little time line of the history of the Internet which listed a handful of milestones - with the most recent item reading “2009 Dr Stephen Wolfram launches Wolfram Alpha.”, listed right after the traditional stuff that you would expect to see (ARPANET, first PC etc). For a site that is not even public (as of Monday May 4, 2009), it seems awfully rash to have it on a list like that.

Reading the fine print confirmed more and more of my skepticism. For instance, Dr. Wolfram admits that information in the engine is first “curated” or assessed by experts AND it also requires approximately 1,000 people for maintaining the site’s databases and keeping current with new information. All this begs to question how exactly is this an AI or anything revolutionary? And to top that off, the end of the article mentions that Google already is working on something similar which is already live. Wow! - Revolutionary - let’s credit Dr. Wolfram and say a loud Amen.

A person’s past is perhaps the best way to look for clues and Dr. Wolfram is no exception. This is where Dr. Dubya appears to have a solid track record as a self-indulgent hype machine. Consider for instance the overwhelmingly negative feedback and concerns about originality for his supposedly epic ‘10 years in the making’ book ironically titled “A New Kind of Science”. A New Kind of Search-Engine? - No, I doubt so.

The only way I was able to connect to the server was with SCP and the Windows client WinSCP actually somehow managed to give me SSH access. For some reason I post-poned this fix for the longest time and just used WinSCP’s SSH access, but 3 days ago I decided to do something about it. First, I backed up my /etc/fstab file and used WinSCP to edit it to say:

/dev/sda1               /                       ext3    defaults 1 1
/dev/sda2               /mnt                    ext3    defaults 0 0
/dev/sda3               swap                    swap    defaults 0 0
none                    /dev/pts                devpts  gid=5,mode=620 0 0
none                    /dev/shm                tmpfs   defaults 0 0
none                    /proc                   proc    defaults 0 0
none                    /sys                    sysfs   defaults 0 0

After that, just reboot your instance and you will be able to log in as usual again.

Mod_python is one of those mods - and it also happens to be something that I rely on for my development environment on the Mac. The following is the command to change from 64-bit to 32-bit:

sudo mv /usr/sbin/httpd /usr/sbin/httpd.bkp
sudo lipo -thin i386 /usr/sbin/httpd.bkp -output /usr/sbin/httpd.i386
sudo ln -s /usr/sbin/httpd.i386 /usr/sbin/httpd

Unfortunately, I have noticed that Apache goes back to 64-bit mode each time I run a system update. It probably restores the original httpd file. It might go away if you use the actual file and not the sym-link, but I just run the command again.

Use this command if you want to revert back to 64-bit:

sudo mv /usr/sbin/httpd.bkp /usr/sbin/httpd

So what exactly do I mean by the term code generation? Code generation is the practice whereby a tool or a script generates source code from templates as opposed to typing the code by hand. One example would be of a script that examines the columns in a table and creates a stored procedure. A more extreme case would be an MDA/MCD tool like Visual Paradigm.

Learning the hard way

A few months ago I was tasked with developing a large application for one of my clients that made use of extensive model taxonomies that also required a management interface.

With a good ORM and some thought behind the persistence mechanism, we were able to leverage a lot of reusable code that gave us a unified, DRY approach to persistence, validation, search queries, caching, lazy row-fetching and transactional operations. Only one daunting task remained and that was the UI. Again, we did our due diligence and came up with some useful generic base classes for Save/Update/List views but even so, the amount of fields on each Save/Update view was staggering and it was incredibly time-consuming to add them all. At some point when I had done about 10% of the work, I started realizing how genuinely tedious the task was. Integers would always be rendered as asp:TextBox accompanied by an asp:RegularExpressionValidator, one-to-many relationships would use a select-box or auto-complete widget and so forth and so on.

Knowing that my models’ members were lush with juicy meta data defined via attributes, I decided to use introspection to generate the code for the views. It took me about 2 weeks to write an IronPython script that could make sense of all the various types, validation attributes and implicit rules in my DLL. Once I had that, the rest was pretty easy. A single command would spit out thousands of lines of HTML and C#, but it still required modifications here and there since there were spots of non-standard behavior.

We now we have some 60 views that contain a lot of tedious code. As soon as something changes in our model, we still have to update the templates to reflect new validation rules or object definitions. It’s not as simple as re-generating the views because that would overwrite changes that have been made. I’m not ashamed of the quality of the code - with the abstractions that are already there its far exceeding the level of quality that you would expect from our average .NET developer. What bothers me is the sheer size and complexity which threatens to turn this code into an unmaintainable mess and a recipe for failure in the long run.

What went wrong? For starters, the problem wasn’t how to create hundreds of lines of HTML in each view – the appropriate question was why. To use a common anology – I was curing the symptom, not the disease. The real problem I was trying to solve was the severe GUI/Model impedance mismatch in my web-framework, ASP.NET. Had I been brave enough then I could just have used those two weeks wrangling with ViewState to come up with a well thought out, Reflection-based Form abstraction like the one found in Django.

Conclusion

In many instances code generation is an appropriate solution to a repetitive task, but first you have to ask yourself why you want to use it. If it’s solving a time-consuming, reoccurring task of low complexity then go ahead and use it, but implore yourself to look beyond the immediate problem at hand. You will be surprised to find out that frequently, the problems you are trying to solve are far more systemic and can be solved more elegantly by adopting a useful abstraction instead.

Code-generation can be a divisive and destructive approach to programming when applied systematically in a development environment or development methodology. Programming should not be viewed as a necessary evil as this fosters the wrong mind-set. It tends to put developers in two camps – one that knows how to solve problems and one that just knows a tool - most people who interview and hire programmers can attest to this.