Activity Not Available

News

  Analyzed 4 months ago based on code collected 4 months ago.
 
Posted almost 4 years ago by Caktus Consulting Group
I am excited to announce that The Buddha site won two Webby Awards! It was built in partnership with Sonnet Media as a companion to The Buddha documentary directed by David Grubin that aired on PBS in April 2010. The site was built using Django to create a multi layered interactive experience to further explore ...
Posted almost 4 years ago by Ken Cochrane
This post is the fourth in my series about the new Django hosting services that just recently hit the market. Previously I have reviewed ep.io , gondor.io , and dotCloud.com .This post reviews AppHosted.com , another similar service that is currently ... [More] in beta. I was lucky enough to get a sneak peak of this service before it was released to the general public, and here are my notes and impressions for the short time I had to play with the new service.

Overview
AppHosted.com uses a similar concept as the other django hosting services, they provide you with a command line client called metro that you use to interact with the service. Like the other command line clients, it is pretty much just a nice wrapper around their API, so anything you can do with the metro client you can do with their API. The client is used to configure your service and push your files up into their servers. Once the application is created and your code has been uploaded to their server, you can use their web based control panel to configure or manage your application.

So that we can compare apples to apples I will go through the process of installing my blog onto the service so that we can see how it works with apphosted compared to the other services. It will also hopefully provide a very simple how to guide on how to install your application onto AppHosted.com.

First Steps
Like all new projects the first thing that I like to do is create a new project directory and a new virtual environment for the project.

# move into my projects directory
$ cd ~projects/

# make my virtual environment apphosted
$ mkvirtualenv apphosted

# clone my blog
$ hg clone https://bitbucket.org/kencochrane/kencochranenet kencochrane_apphosted

# move into the new project directory
$ cd kencochrane_apphosted

Now that I have a local working copy of my blog code and a new virtual environment I'm ready to go.

Command Line Interface
AppHosted has a command line client called Metro, it uses this client to configure and upload your application onto the apphosted servers. Metro only has three dependencies: httplib2, paramiko, and pycrypto, if you don't have them installed pip should handle that for you.

# install the metro client
$ pip install metroclient

Downloading/unpacking metroclient
Running setup.py egg_info for package metroclient
Requirement already satisfied (use --upgrade to upgrade): httplib2 in /Users/ken/.virtualenvs/apphosted/lib/python2.6/site-packages (from metroclient)
Requirement already satisfied (use --upgrade to upgrade): paramiko in /Library/Python/2.6/site-packages (from metroclient)
Requirement already satisfied (use --upgrade to upgrade): pycrypto>=1.9 in /Library/Python/2.6/site-packages (from paramiko->metroclient)
Installing collected packages: metroclient
Running setup.py install for metroclient
changing mode of build/scripts-2.6/metro from 644 to 755
changing mode of /Users/ken/.virtualenvs/apphosted/bin/metro to 755
Successfully installed metroclient

Initial Data & Django admin user
At the moment Apphosted.com doesn't provide an easy way to create a django admin user, in order to create this user in your appHosted.com database you need to create it on a local database and then export that data into a fixture called initial_data.json.

# locally I run this command to dump the data, so that I can load on app hosted.
$ python manage.py dumpdata > initial_data.json

There is another way to create the django admin user, which I'll describe a little later. This would also be the same way that you would import any application data that you need in order for your application to run correctly.

Metro Client
One of the goals for appHosted.com was that they didn't want to require that you make lots of changes to your project directory structure in order to use their service. In order to do this they need to do a lot more work to figure out what it is you want to do, and if they can't figure it out, they will ask you some questions.

Now that we have our project directory and our initial_data.json file all ready, lets go ahead and deploy the application. Activate the virtual environment and go into your project directory. Kick off the metro client and answer the questions that pop up.

# run the metro client.
$ metro

Metro Application Manager 1.21
Copyright (c) Lumentica, http://www.lumentica.com

Application directory (i.e. path to project): ./
Application name: kencochraneblog
Application version: 0.1
Application description: Ken's AppHosted Blog
Application modules (extra; comma-separated): South==0.7.3, django-activitysync==0.2.2, django-debug-toolbar==0.8.4, django-disqus==0.3.4, django-memcache-status==1.0.1, django-tagging==0.3.1, django-xmlrpc==0.1.2, feedparser==4.1, httplib2==0.6.0, oauth2==1.2.0, python-memcached==1.47, python-twitter==0.8.1, simplejson==2.1.2, wsgiref==0.1.2, yolk==0.4.1, docutils==0.6, Pygments==1.3.1
WSGI module (optional):
Paste config (optional):
Fixtures (optional; comma-separated):
Sync database and load fixtures? (y/n): y
Use SSL? (y/n): n
Force SSL: (y/n): n
Build complete.
Upload to AppHosted? (y/n): y
AppHosted Username: kencochrane
AppHosted Password:
Deploying...
:: Configuring application environment...
:: Configuring application permissions...
:: Configuring application server...
:: Application deployed to server successfully...
:: Application deployment complete...

What did this do for us? It takes our application, and uploads it to their server, and then it installs all of our dependencies that we listed above. Then it configures it according to our answers, and tries to start it up, if we told it to, it will sync our database and load our fixtures. It also takes all of this information and stores it in a file called metro.config, so that we don't need to type that in every time if nothing has changed.

You might have noticed that I needed to manually enter in all of my dependencies when prompted, when I asked about this, I was told that if I didn't enter all of the dependencies it would scan the project for a pip style requirements.txt file, and use that to load all of your dependencies. I haven't had a chance to test that out yet, since I already entered my requirements by hand, and once you do it, you don't need to do it again.

My blog uses south for migration management, and at the time of this writing there is a bug, and it isn't correctly kicking off the migrate command during the database sync process. They are aware of the issue and they are working on a fix, but there is a work around in the meantime. I'll describe the work around shortly.

Once your application is deployed it will almost immediately be available at http://<your-app-name>.apphosted.com.

Log Files
In order to view the log files, you need to log into the web control panel and from there you can download the logs that you are interested in. At the moment they only support downloading the log files, which is a little inconvenient, it would be nice to have the ability to tail the logs, or see the last 50 or 100 lines in the control panel. I asked about this, and they agreed and said it is on their list of things to work on.

Shell Access
Metro gives you the ability to shell into your application directory, so that you can kick off django management commands, and other cool things. One important thing to note is that when you login to the shell, it is a jailed shell, and you only have access to your own project, and with limited command access. This is done for security reasons. It is nice to know that your application runs in it's own jailed area, and no one else's applications can access your information.

Setting up the shell access requires a few steps. Follow these directions to get the full details: http://docs.apphosted.com/apphosted/apphosted_shell.html , but basically it requires the following.

create a ssh key if you don't already have one.
Login to the web console and add your public key.
In the web console browse over to your application Settings->Tools page and click on the “Update Public Keys” button. (Don't forget about this step, or you won't be able to login.
Use the metro client to open the shell.

Note: It would be real cool if we didn't have to do step 3, why can't it automatically update the public keys for all apps? I'm guessing it is a security reason. The first time, I didn't hit the update public keys button and I couldn't figure out why it wasn't working, and luckily one quick email to the support line, and they let me know what I did wrong, and I was up and running in no time.

This is how you can kick off the shell, there are a bunch of command line args that you can pass as well, just do metro --help and it will show them all to you.

$ metro -s

Note: There is currently a bug that is preventing me from logging into the shell from my MacBook Pro OS X 10.6.7 using Python 2.6.1 . When I do it gives me this error. I have informed appHosted, and they said it looks like an issue with paramiko and python 2.6.1 on the Mac, not sure if it affects other python versions on other operating systems. They are working on the issue.

$ metro -s
Metro Application Manager 1.21
Copyright (c) Lumentica, http://www.lumentica.com

Application name: kencochraneblog
AppHosted Username: kencochrane
AppHosted Password:
Traceback (most recent call last):
File "/Users/ken/.virtualenvs/apphosted/bin/metro", line 75, in run_shell
ssh.connect(host, username=app_name, port=port)
File "/Library/Python/2.6/site-packages/paramiko/client.py", line 278, in connect
for (family, socktype, proto, canonname, sockaddr) in socket.getaddrinfo(hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM):
gaierror: [Errno 8] nodename nor servname provided, or not known
Error: [Errno 8] nodename nor servname provided, or not known

Work Arounds
Now that you know how to shell into your application, you can also kick off django management commands. I mentioned earlier that there was another way to create the django admin user, well here is how you do it. Login to the metro shell and then kick off the createsuperuser django management command. Logging into the shell and finding things isn't easy do to the added security so to save you some trouble this is what I did. Python isn't in your path so you need to put the fully qualified path to the virtual env's python. Not sure if the jailed shell allows you to edit your .profile if so, it might be nice to configure this so that you don't always have to do the full path.

# Create superuser (django admin)
$ /ve/kencochraneblog/bin/python /app/kencochraneblog/mysite/manage.py createsuperuser

# kicking off south migrations by hand.
$ /ve/kencochraneblog/bin/python /app/kencochraneblog/mysite/manage.py migrate

My application's name is kencochraneblog, so make sure you replace my application name with yours, it should be something like this. replace app_name,django_project_dir and command with your variables.

$ /ve/<app_name>/bin/python /app/<app_name>/<django_project_dir>/manage.py <command>

Background Tasks
AppHosted also supports background tasks via celery, they don't have the documentation created yet, but it should be available soon. They are also planning on supporting a cron like service, not sure if it will be cron, or something similar, but it should do, what you need it to do.

API
AppHosted provides a pretty nice API that allows you to do anything that the metro client does. This will allow you to write your own automation and management scripts to interact with your application. If you practice continuous deployment, it will allow you to automatically deploy the latest version of code on a post commit hook, if all of your tests pass.

Services
Right now they only support memcache for caching, and PostgreSQL for databases, but they plan on adding more services as they move forward.

Conclusion
I have been playing with AppHosted for a few weeks now, and I'm pretty happy with what they have so far. They still have a few issues to work out and they are still a work in progress, but I have been seeing improvements all of the time, so it will only be a matter of time before they are ready for the general public. Every time I came across an issue, they were quick to reply to my emails and solve my issues, or tell me where I was going wrong. If you have any more questions I recommend checking out their documentation @ http://docs.apphosted.com/index.html . Check them out, and make sure you let me know what you think.

UPDATE:

They are now open to the general public. [Less]
Posted almost 4 years ago by Ken Cochrane
I work for a company called CashStar, most people have never heard of us, but a lot of people have used our products without even knowing it. CashStar is the man behind the curtain so to speak. We created an e-gift card platform that is used by some ... [More] of the largest companies in the United States. Here is a short list of some of our customers: The Home Depot, Gap, Old Navy, Banana Republic, Staples, Coach, Starbucks, CVS, Chilis, Regal Cinemas, Pottery Barn, The CheeseCake Factory, Dell, Pizza Hut, and many more. To see the full list follow this link: http://www.cashstar.com/all-egift-card-clients/

Another thing that most people don't know is that we power that platform using the awesome python framework called Django. Using Python and Django, our small engineering team in Portland, Maine has taken our startup which didn't exist 3 years ago and turned it into a leader in the electronic gift card market. Due to this phenomenal growth, we need to expand our engineering team. We are currently looking for developers as well as QA folks to join our team. You will need to work on site in Portland Maine, and be legally allowed to work in the United States.

If you are interested in joining a fun, fast paced startup, please send the following to kencochrane@gmail.com with the subject line "CashStar.com Engineering Openings". Failure to follow these simple directions will automatically disqualify you from these positions.

Resume
Links to any projects you worked on recently
Links to your github, bitbucket, stackoverflow, etc. accounts
Availability (when you could start)
How and when is best to contact you.

More about CashStar.com:

All engineers receive a top of the line MacBook Pro and a 24" monitor.

Our technology stack:

Python
Django
Celery
MySQL
Linux
Mercurial
nginx
gunicorn
memcached
Redis
jQuery
virtualenv + pip
Selenium
Hudson

We have a fully stocked kitchen with all kinds of snacks and all of the soda and coffee you can drink, and at least one free company lunch per week.

We are located in the Old Port section of Portland Maine. If you have never visited Portland Maine before, you are missing out, it was ranked #1 on forbes.com's America's Most Livable Cities in 2009, and it was also ranked America's Foodiest Small Town by Bon Appetit Magazine in 2009. Located right on the coast, and only a 90 minute drive to Boston, it is a great place to live and an even better place to raise a family.

CashStar is privately held and has received venture funds from a multi-billion dollar global investment firm.

If you have any questions feel free to contact me

email: KenCochrane@gmail.com
twitter: @KenCochrane
linkedin: http://www.linkedin.com/in/kencochrane [Less]
Posted almost 4 years ago by DjangoCon EU Blog
Conference schedule posted
It has been a while and a lot has happened. We’re past due for an update!

Talks
The conference schedule has been finalized; we are proud to present an interesting program on a wide variety of Django ... [More] subjects.

You can now also submit entries for the lightning talks on the lightning talks page

Waiting list
The waiting list is full. Starting May 18th we will invite people who have subscribed to the waiting list on a first-come, first-served basis.

We’re also pleased to announce that the higher “Night Owl” rate has been cancelled; the waitlist tickets will be sold at the standard conference rate.

Sprints
The Sprints are free for anyone to join. If you weren’t able to purchase a ticket but would love to spend some time with your fellow Djangonauts, the sprints are an excellent way of doing just that.

If you plan on taking part in the sprints, please let us know via the sprint subscription page.

If you’ve purchased tickets for the conference, you must still indicate that you’re staying for sprints by registering.

Space at the sprint venue is limited, and we’d like to accomodate the maximum number of participants. Registering helps us figure out how many sprinters are coming (and helps us to plan accordingly.)

See you soon! [Less]
Posted almost 4 years ago by Reinout van Rees' weblog
I recently switched one of our django sites over from apache+mod_wsgi to
apache(+mod_proxy)+gunicorn. The advantage of gunicorn is that every site runs in its own process instead of
everything running within one apache. When a site barfs, you ... [More] can at least see
the culprit when you type "top". Oh, and the speed seemed higher. (For an
introduction on gunicorn, see my summary of the gunicorn talk at
last year's djangocon.eu).

But today a colleague using IE8 couldn't load the PNG images. And in my log
I'd get "connection reset by peer" errors out of gunicorn. I had no problems
in firefox.

Assumption: perhaps django's gzip middleware messes things up. I've seen
gzip-related errors before. Never had a problem with it in django, but worth a
try. So I disabled it. IE8 failed to work as before, but now also firefox
didn't load the images. They came in as zero-length responses!

The advantage: now I could debug it myself.

I did a wget -S on the server itself to take a look at the headers as
close to the site as possible. First a request to gunicorn running on port
10003 (some lines omitted as they're not relevant):

Connecting to localhost|127.0.0.1|:10003... connected.
HTTP request sent, awaiting response...
HTTP/1.0 200 OK
Server: gunicorn/0.12.1
Connection: close
Cache-Control: max-age=0
Content-Type: image/png
Length: unspecified [image/png]
Saving to: `image.png'

[ <=> ] 17,961 --.-K/s in 0s

2011-05-13 11:28:54 (160 MB/s) - `image.png' saved [17961]

And now the same request to apache on port 80:

Connecting to localhost|127.0.0.1|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Server: gunicorn/0.12.1
Cache-Control: max-age=0
Content-Type: image/png
Content-Length: 0
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Length: 0 [image/png]
Saving to: `image.png'

[ <=> ] 0 --.-K/s in 0s

2011-05-13 11:29:32 (0.00 B/s) - `image.png' saved [0/0]

What? Yes, adding apache into the mix results in a zero-length
response. Something is wrong. The error log showed this traceback:

Error processing request.
Traceback (most recent call last):
File "/srv/mysite/eggs/gunicorn-0.12.1-py2.5.egg/gunicorn/workers/sync.py", line 70, in handle
self.handle_request(req, client, addr)
File "/srv/mysite/eggs/gunicorn-0.12.1-py2.5.egg/gunicorn/workers/sync.py", line 98, in handle_request
resp.write(item)
File "/srv/mysite/eggs/gunicorn-0.12.1-py2.5.egg/gunicorn/http/wsgi.py", line 227, in write
util.write(self.sock, arg, self.chunked)
File "/srv/mysite/eggs/gunicorn-0.12.1-py2.5.egg/gunicorn/util.py", line 174, in write
return write_chunk(sock, data)
File "/srv/mysite/eggs/gunicorn-0.12.1-py2.5.egg/gunicorn/util.py", line 170, in write_chunk
sock.sendall(chunk)
File "<string>", line 1, in sendall
error: (104, 'Connection reset by peer')

Chunked? Oh, the response is being served in chunks instead of as one big
response. In the gunicorn documentation on deployment, there's a warning about needing to use
one of the "async worker classes" instead of the default "sync" if you don't
use ngnix's proxy buffering.

So I started googling for an apache config switch that would tell it to just
grab gunicon's response in one go. Turns out you can set a variable that does
just that by enabling only http 1.0 for the proxy requests. See the apache
documentation for mod_proxy environment settings. My apache
config now looks like this:

<VirtualHost *:80>
ServerName mysite.whatever

# Force http 1.0 for proxying: needed for gunicorn!
SetEnv force-proxy-request-1.0 1

... lots of settings ...

# Static files are hosted by apache itself.
# User-uploaded media: MEDIA_URL = '/media/'
Alias /media/ /srv/mysite/var/media/
# django-staticfiles: STATIC_URL = '/static_media/'
Alias /static_media/ /srv/mysite/var/static/

RewriteEngine on
ProxyPreserveHost On
# Don't rewrite /media and /static_media
RewriteRule ^/media/.* - [L]
RewriteRule ^/static_media/.* - [L]
# Django is run via gunicorn. So proxy the rest.
RewriteRule ^(.*) http://localhost:10003$1 [P]

</VirtualHost>

In the end, the two different headers I looked at with wget contained an extra
hint: one has HTTP/1.1, the other HTTP/1.0...

Summary: if you use gunicorn with a regular "sync" worker in combination
with apache, add SetEnv force-proxy-request-1.0 1. Or use one of the async
workers (eventlet, gevent, etc).

[Less]
Posted almost 4 years ago by Antonio Cangiano
Note: This is the Python version of the Ruby guide I just published. In this brief tutorial I’ll show you how to create a complete Python and Django setup for DB2 on Ubuntu. By following my step-by-step instructions, you’ll be able to install the ... [More] following components: Python easy_install Django DB2 Express-C 9.7.4 The official Python [...]

Possibly related posts:Enabling support for DB2 and Python/Django/SQLAlchemy on Mac OS X Snow Leopard
Installing Ruby on Rails and DB2 on Ubuntu 11.04
Installing Django with PostgreSQL on Ubuntu [Less]
Posted almost 4 years ago by Reinout van Rees' weblog
I wanted to create a south migration for
one of my django apps. An app with geospatial field types. Oops:

$> bin/django schemamigration lizard_opendap --initial
! Cannot freeze field 'lizard_opendap.point.geom'
! (this field ... [More] has class django.contrib.gis.db.models.fields.PointField)
! Cannot freeze field 'lizard_opendap.line.geom'
! (this field has class django.contrib.gis.db.models.fields.LineStringField)

! South cannot introspect some fields; this is probably because they are custom
! fields. If they worked in 0.6 or below, this is because we have removed the
! models parser (it often broke things).
! To fix this, read http://south.aeracode.org/wiki/MyFieldsDontWork

Weird. I googled a bit and found out that support for those geom fields was
added some years ago. So it should work.

The solution: I forgot to add 'django.contrib.gis' to
INSTALLED_APPS in settings.py. I thought that entry wasn't needed
anymore, so I tend to leave it out. (Iirc, it used to be necessary because of
geospatial databases, but those databases work just fine without adding
django.contrib.gis to INSTALLED_APPS now).

Yep, now it works:

$> bin/django schemamigration lizard_opendap --initial
+ Added model lizard_opendap.Setting
+ Added model lizard_opendap.SourceModel
+ Added model lizard_opendap.Scenario
+ Added model lizard_opendap.ModelResult
+ Added model lizard_opendap.Point
+ Added model lizard_opendap.PointValue
+ Added model lizard_opendap.Line
+ Added model lizard_opendap.LineValue
Created 0001_initial.py. You can now apply this migration with: ./manage.py migrate lizard_opendap

[Less]
Posted almost 4 years ago by unfoldthat
This post is about Chef. Chef is a big odd but very powerful piece of software. You'll probably find it extremely useful for your deploy tasks - as soon as you can understand it.
Posted almost 4 years ago by Reinout van Rees' weblog
Big news for us: if everything goes right, we'll be moving in one or two
months!

I never expected it to go this fast. We wanted a slightly bigger house in
order to give both our kids (boy + girl) a separate room. So mostly the ... [More] same
size, but with one extra room. And a slightly bigger garden to house our bikes
wouldn't hurt.

Two months ago we started looking at houses for rent. In the Netherlands
every region has a system of doling out houses from the so-called "social
housing rental corporations". Based on the amount of time you wait. You can
sign in on three houses per two weeks and the longest waiting time
wins. That's how we got our current house.
Aforementioned housing system only is allowed up until a certain
income. We're above that margin, so we had to look for the more expensive
rental houses that are handled outside the social rent system. The
advantage: they're available earlier. The disadvantage: they're more
expensive.
Three weeks ago we subscribed ourselves at two housing agencies for rental
houses. And I booked an appointment at a mortgage agent just in case it
was possible for us to buy a house.

In the Netherlands, houses are very expensive. Since about 15 years, banks can
legally lend you much more money than was possible earlier. The advantage: you
can buy a more expensive house. Drawback: everyone can do it, so the price
of houses expands to fill the available credit. Houses get more expensive
without them being intrisically more valuable than before.

In the Netherlands, lots of young married people both work. Two incomes. So
the available credit goes up. So the housing prices go up to match the
available credit. So, as a one-income family we're competing against
dual-income-based housing prices. Yes, my wife is still looking for a job!
In practice, this made renting the only option I could see.

The recent fiscal crisis brought some changes. Lots of mortgages turned out to
be rope the people could hang themselves with. Fiscally unsound mortgages. So
the credit you could get got less. So the housing prices finally went down a
bit.

A few days before meeting the mortgage agent we got some extra money from
sources I won't disclose here.
Suddenly, we had the financial buffer needed to get me to comtemplate buying
a house.
A couple of houses actually seemed to be reasonably priced, being ex-rental
houses.
After the meeting with the mortgage agent we made the appointment to visit
two houses...

If nothing untowards happens we'll sign the mortgage next week and get our
house somewhere near the start of June. That'll be fun with a djangocon.eu
conference on 6-8 June. Busy times :-)

Some data on the house for those interested:

In Nieuwegein, some 1km from our current home.
The kids can stay at the same school.
One room (a pretty big attic room) extra, just what we wanted. For the rest
it is mostly the same size we have now, which is fine.
The garden is twice as big as our current smallish one, so that's great,
too.
A bus stop is in front of our neighbours' house, so that's the closest I've
ever lived to public transport. I thought I was close now at 30m from a
tram stop :-) (Public transport is important as we don't own a car).
Build in 1982, so structurally sound without big problems. One attic window
needs replacing, that's the biggest item on the list.

One drawback: it is pretty hard to keep myself focused on my work (and on a
couple of other tasks). Luckily today I had a tricky problem,
that helped in focusing me :-)

I'm happy!

[Less]
Posted almost 4 years ago by Reinout van Rees' weblog
We've got a django-cms site that started out on sqlite. Now we're adding GIS
shapefiles so we wanted to move to postgres/postgis. Ok, switch over the
DATABASES definition in settings.py, make a fixture and load it,
right? Ahem ... [More] , no.

There were a couple of problems, partially with our code, partially
django-cms, partially django. Some solutions below might not be needed in
every case, but I'm just going to list everything in the hope that it helps
someone.

South migration dependencies
Lots of our code uses south for database
migrations. But when creating our database from scratch, "migrate" would
complain that a certain table of application A didn't exist yet when migrating
application B. Uh oh, a dependency.

Turns out that's easily solved. In the failing migration step, just add a
dependency on the missing application's relevant step:

...
class Migration(SchemaMigration):

depends_on = (
("application_a", "0001_initial"),
)

def forwards(self, orm):
...

After that, all migrations run fine.

Small sqldiff limitation
We've got django-extensions in our projects. One of the
handy extra management commands it
provides is "sqldiff" (so manage.py sqldiff or, as we use buildout,
bin/django sqldiff). (The best extension is "graph_models" which creates
a picture of your model structure).

We got some complaints when importing a fixture. In the end it turned out to
be that someone somehow copied an old and wrong fixture instead of a fresh
one. But we suspected that some older migration had mis-fired. "sqldiff"
will tell you the difference between the current database schema inside the
actual database and what it should be according to django.

We got a bunch of missing fields in some custom user model inside django-cms!
What! Faulty south migrations? What?

In the end, it turned out to be a false alarm. Django-cms's user model
inherits from django's own user model and adds one field. In the database this
ends up as a small table with a pointer to a corresponding django user item's
ID and the one single extra field. But sqldiff complains about all the regular
django user model fields being missing :-)

Note: I don't fault django-extensions very much here. They note it is an
experimental feature and we just happened to hit a corner case. Most of the
time it is a real helper!

IntegrityError: duplicate key value violates unique constraint
See for instance a stackoverflow question
and especially Mark van Lent's recent explanation of what can happen.

There are two basic ways, I think, such an IntegrityError can crop up when
loading a fixture:

You're loading them out of order or with gaps in the primary key IDs. This
can mess up the automatic ID generation.
You've really already got an item with that ID in the database and the
complaint is right.

IntegrityError solution 1: natural keys
See django ticket #7052. Solved. Solved, that is, when
the model that gives you a problem supports so-called natural
keys. Basically, your fixture doesn't point at a user with primary key "54",
but at a user with username "pietje", whatever the ID.

This takes away a too-hard dependency on primary key ID numbers. To get it
going in your fixture, add the --natural flag:

$> bin/django dumpdata --indent=2 --natural > all.json

IntegrityError solution 2: make sure there are no duplicates
One of our problems was that django's content_type table was already
pre-filled with several items after the regular "syncdb". Quite sensibly. But
it conflicted when loading our fixture. (It might be that the natural key
solution mentioned above would have solved this, but we tried it in a
different order).

The solution: remove those content types from the table before loading the
fixture. The fixture is a full fixture of everything in the database, so
that's OK.

In your manage.py dbshell or bin/django dbshell, run:

truncate django_content_type cascade;

IntegrityError solution 3: split the fixture
Django-cms's cms.placeholder items continued to give grief no matter
what. Perhaps they ought to start to support the natural key solution? Not
sure if that would help, but I guess so.

The solution? Grab the whole exported fixture and split that all.json into
two json files, one with only "cms.placeholder" items and one without
them. (Make sure the last dict-like item in the json should be without a
closing comma!

all_only_placeholders.json:

[
{
"pk": 1,
"model": "cms.placeholder",
"fields": {
"slot": "leftbar",
"default_width": null
}
},
{
"pk": 2,
"model": "cms.placeholder",
"fields": {
"slot": "content",
"default_width": null
}
},
...
}
]

all_without_placeholders.json:

[
# Right, everything apart from cms.placeholder :-)
...
]

After that, load the data:

$> bin/django loaddata all_only_placeholders
$> bin/django loaddata all_without_placeholders

Closing comment
In the end, we got it running. I hope someone gets a good tip out of this
entry when faced with a similar problem.

One django advantage I noticed: everything is visible. You can always
inspect the source code. "What exactly is in that one model that's giving me
grief?" "Which migration steps are available?" Etc.

The advantage of open source!

[Less]