Django reusable apps

Posted October 31st, 2012 in Python by Florentin

Warning: this article is still a draft

Explaining the Django reusable apps in an agile way

  • as a developer
    • i want to easily plug the app into the project so that i can improve the productivity
    • i want to customize and extend existing Django apps so that they fit the requirements
  • some of the things that need customization:
    • models
      • e.g. make the app work with a different model class than the ones provided by the app
      • e.g. create a custom model based on the one supplied by the app and add extra fields or modify the existing field definition
    • forms
      • e.g. the app should use a custom form
      • e.g. a custom form which adds extra fields or change the definition of the existing ones
    • template names
    • success or redirect urls
    • template variables (extra_context)

1 James Bennett’s approach – the custom objects are passed as arguments to the views

 

(APP/views.py)
def register(request, success_url=None,
form_class=RegistrationForm
template_name=’registration/registration_form.html’,
extra_context=None):

2 Use the settings to define the custom objects

 

  • in settings.py, define the values for models/forms/etc which will be later used by the app
  • it’s useful to prefix all the names required by a specific app, i.e. BLOGAPP_MODEL, BLOGAPP_FORM etc
  • example

(settings.py)
MY_APP_MODEL = MyAppModel

(APP/views.py)
from django.conf import settings
from .models import AppModel

def register(request):

my_model = getattr(settings, ‘MY_APP_MODEL’, AppModel)

What the current approaches don’t offer

  • an easy way to modify or extend the urls. The urlpatterns is just a list, it’s difficult to delete/edit/insert url definitions.
  • models placed in models.py are used by Django in different db operations (i.e. syncdb).

One might want to ignore those models and let Django use customized/child Models.

  • the Model and Form fields are not easy to alter

Ideas for a new solution

  • make everything a class. Forms and Models are classes but the urls (or maybe the views) are not.
  • move the Model definitions from models.py and allow the developer to use it’s own Models (i.e. though getters or factories)
    • instead one can play with abstract models but without any concrete Models the app won’t work out of the box
  • define the Form and Model fields inside a method so that the derived classes can modify the definitions

Resources

Django project skeleton

Posted October 24th, 2011 in Python by Florentin

If you need a Django project skeleton to base your work upon, please check this one:

https://github.com/florentin/django-project-skeleton

And now a Django app skeleton as well

https://github.com/florentin/django-app-skeleton

Django settings

Posted October 24th, 2011 in Python by Florentin

I’m going to talk here about various techniques regarding the Django’s settings.

Let’s assume the following project layout:
.
├── __init__.py
├── manage.py
├── settings
└── urls.py

How do I use custom settings or overwrite existing ones on my development machine ?

Answer 1

Create your custom settings file under the same parent as the existing settings.py. Let’s name the new file settings_local.py
At the bottom of settings.py, add the following:

try:
    from settings_local import *
except ImportError:
    pass

Pros

  • no need to change the manage.py or the wsgi file in order to apply the new settings

Cons

  • hard/impossible to use different settings for different environment

 Answer 2 (preferred)

Create a new directory called “settings” and move the existing setting file there. Then make a different file for each type of environment.
.
├── __init__.py
├── manage.py
├── settings
│   ├── __init__.py
│   ├── development.py
│   ├── production.py
│   └── settings.py
└── urls.py

settings.py will include the default Django settings, probably the file created by django-admin.py
development.py will hold settings/overwrites needed for the development environment.
The extra settings files (production.py, development.py, etc) will extend the existing settings.py (or another parent file which in turn extends settings.py) and add their own settings customizations.
This could be your development.py file:

from .production import *
DEBUG = True
TEMPLATE_DEBUG = True
DJANGO_SERVE_PUBLIC = True
PREPEND_WWW = False
SEND_BROKEN_LINK_EMAILS = False
# APP: debug_toolbar
MIDDLEWARE_CLASSES += (
"debug_toolbar.middleware.DebugToolbarMiddleware",
)
INSTALLED_APPS += (
"debug_toolbar",
)
DEBUG_TOOLBAR_CONFIG = {
'INTERCEPT_REDIRECTS': False,
}
TEMPLATE_CONTEXT_PROCESSORS += [
'django.core.context_processors.debug'
]

Where production.py is:
from .settings import *
import os.path
PROJECT_ROOT = os.path.join(os.path.abspath(os.path.dirname(__file__)), '../')
DEBUG = False
TEMPLATE_DEBUG = False

Once your settings setup is in place, all you have to do is change manage.py and your WSGI file.
The manage.py file could now look like this:

#!/usr/bin/env python
from django.core.management import execute_manager
import sys, os
PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))
sys.path.insert(0, PROJECT_ROOT)
try:
import settings.development # Assumed to be in the same directory.
except ImportError, e:
import sys
sys.stderr.write("Error: Can't find the file 'settings.py' in the directory containing %r. It appears you've customized things. You'll have to run django-admin.py, passing it your settings module. (If the file settings.py does indeed exist, it's causing an ImportError somehow.) " % __file__)
sys.exit(1)
if __name__ == "__main__":
execute_manager(settings.development)

In the same time, your WSGI file would use settings.production:

os.environ[“DJANGO_SETTINGS_MODULE”] = “settings.production”
Pros

  • easy to create settings for each of your environments (production, development, etc)
  • it’s a great way to keep your settings organized, easy to find and edit.
  • easier to reuse your settings for other projects. For example, we could use the same development.py file (as shown above) with other production.py settings.

Cons

  • you have to change the manage.py and the WSGI file, which might not be possible on your client server.

 Other Django settings tips

  • do not use your project name in the settings, i.e. use ROOT_URLCONF = ‘urls’ and not ROOT_URLCONF = ‘myproject.urls’, use INSTALLED_APPS = (“appname”,) and not INSTALLED_APPS = (“myproject.appname”, ). You will then be able to easily move settings and applications between one project to another
  • use calculated paths for TEMPLATE_DIR, MEDIA_ROOT, STATIC_ROOT etc, i.e. PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__)) and then MEDIA_ROOT = os.path.join(PROJECT_ROOT, “media”)

reCaptcha for Django’s comments framework

Posted July 26th, 2011 in Python by Florentin

This is how you add reCaptcha (the captcha system built by Google) to the Django’s comments framework.

Requirements

For this task you will need to build a custom module which depends on django-recaptcha-works. recaptcha-works provides a special Form field named “RecaptchaField”
and a decorator function “fix_recaptcha_remote_ip” which inserts the request’s REMOTE_ADDR (IP) into the request.POST data.

django-recaptcha-works can be downloaded from http://pypi.python.org/pypi/django-recaptcha-works/ or install it with:
pip install django-recaptcha-works

You also need a pair of reCaptcha keys from this address:

https://www.google.com/recaptcha/admin/create

Build a custom comment module

Create a new comment module, add it to INSTALLED_APPS and set a special settings COMMENTS_APP. The process is described on this page:

https://docs.djangoproject.com/en/dev/ref/contrib/comments/custom/

cd /path/to/your/project
mkdir my_comment_app

Make these changes to your settings file (usually settings.py):

INSTALLED_APPS = [
...
'my_comment_app',
...
]
COMMENTS_APP = 'my_comment_app'
RECAPTCHA_PUBLIC_KEY = 'public key'
RECAPTCHA_PRIVATE_KEY = 'private key'

recaptcha-works provides more settings, please check the settings.py from inside the recaptcha-works module.
Here are some interesting settings:

RECAPTCHA_USE_SSL = False
RECAPTCHA_OPTIONS = {
        'theme': 'white',
        'lang': 'en',
        'tabindex': 0,
}
RECAPTCHA_VALIDATION_OVERRIDE = False

Inside your new module “my_comment_app”, create these files:

__init__.py

from .forms import CustomCommentForm
def get_form():
return CustomCommentForm

forms.py

from django.contrib.comments.forms import CommentForm
from django.contrib.comments.models import Comment
from recaptcha_works.fields import RecaptchaField

class CustomCommentForm(CommentForm):
    recaptcha = RecaptchaField(label='Human test', required=True)
    def get_comment_model(self):
        return Comment

models.py

from django.db import models
from django.contrib.comments.models import Comment

class CustomComment(Comment):
    pass

urls.py

from django.conf.urls.defaults import patterns, include, url
from . import views

urlpatterns = patterns("",
    url(r'^post/$', views.custom_post_comment, name='comments-post-comment'),
    url(r'', include('django.contrib.comments.urls')),
)

views.py

from django.contrib.comments.views.comments import post_comment
from recaptcha_works.decorators import fix_recaptcha_remote_ip

def custom_post_comment(request, next=None, using=None):
    return fix_recaptcha_remote_ip(post_comment)(request, next, using)

You can download a sample of the module here.

Searching for Django help and inspiration

Posted July 18th, 2011 in Python by Florentin

docs.djangoproject.com

The most important resource which will answer most of your questions is the manual.
1. You can either use the built in search:
https://docs.djangoproject.com/search/?q=forms&release=5
2. Or search the documentation though Google (my preference):
http://www.google.com/search?hl=en&tbo=1&q=forms%20site%3Ahttp%3A%2F%2Fdocs.djangoproject.com/en/dev%2F

stackoverflow.com

This is the second biggest resource of information regarding Django development.
There are several ways to search for answers:
1. Use the search form and look for the terms "django" and the topic you are interested in (e.g. "forms"):
http://stackoverflow.com/search?q=django%20forms
2. Search for the tag "django" and your topic of interest (e.g. "forms")
http://stackoverflow.com/search?q=[django]+forms

The first form yields many more results than the second, but the relevance of the second type of search is higher.
Furthermore, you can sort your results by "relevance", "newest", "votes" and "active". I recommend using "votes" when searching for a high level topic like "forms".

Google

If the first 2 methods don’t help, you may need to ask the big boss:
http://www.google.com/search?hl=en&tbo=1&q=django%20forms

You may have to remember to use "+" or "*" inside your queries if you are looking for something very specific.

djangosnippets.org

Small fragments of code that you can use or get inspiration from.

pypi.python.org

Search for Python modules. Try your search with and without the "django" term added to the query.
Example: http://pypi.python.org/pypi?%3Aaction=search&term=django%20forms
A nicer list of Django packages with details about usage, contributions and others is http://djangopackages.com/

others

If you are used to IRC channels, mailing lists or forums this page gives some indications:
https://docs.djangoproject.com/en/dev/faq/help/

search with firefox shortcuts

One last trick for Firefox users, "Add a keyword for this search" improves your query rate a lot. Some of my settings:

Location: http://stackoverflow.com/search?q=%s
Keyword: so

Location: http://www.google.com/search?hl=en&tbo=1&q=%s%20site%3Ahttp%3A%2F%2Fdocs.djangoproject.com/en/dev%2F
Keyword: dj

Location: http://www.google.com/search?hl=en&q=%s%20site%3Adjangosnippets.org
Keyword: djsnip

Location: http://pypi.python.org/pypi?%3Aaction=search&term=%s
Keyword: pip

NiftyUrls – A Django powered Popurls script

Posted March 21st, 2011 in Python by Florentin

Hello and welcome

Niftyurls is a popurls-style script built entirely with Django.

Get NiftyUrls from GitHub

View some examples

www.JavascriptNews.com
www.PythonDaddy.com
From the popurls site:

… is the dashboard for the latest web-buzz, a single page that encapsulates up-to-the-minute headlines from the most popular sites on the internet.

 

Unique features

  • Powered by Django
  • No tooltips by default, there are better ways of reading the text
  • Read the news in a Facebook-style lightbox (“pop” layout)
  • Read the news in a clean page (“page” layout)
  • Easy editable feed settings, titles, urls, positions in the page
  • Javascript assisted by jQuery
  • Grid templates provided by Yui3 Grids
  • Fast text replacement with Cufon

 

Upcoming features

  • remember visitor’s last viewed links, mark new/old links
  • show new links only
  • videos support
  • admin interface for feed configuration, more config options
  • multiple domain support
  • site search with database support
  • usability improvements
  • user accounts, openid support
  • code comments, svn support

Installation (linux, localhost)

    • 1. Make sure you have the following Python packages available: Django – http://pypi.python.org/pypi/Django/ Pil – http://pypi.python.org/pypi/PIL/ Feedparser – http://pypi.python.org/pypi/feedparser/ You may install these with the “pip” tool (http://pypi.python.org/pypi/pip/) $ pip install “django>=1.3″ $ pip install pil $ pip install feedparser
    • 2. Add “niftyurls” to the INSTALLED_APPS tuple. The Niftyurls application depends on the following Django packages: ‘django.contrib.staticfiles’ ‘django.contrib.admin’ To make sure every app is enabled, add the following line to your project’s “settings.py” file: INSTALLED_APPS += (‘django.contrib.staticfiles’, ‘django.contrib.admin’, ‘niftyurls’, )
    • 3. Synchronize the database $ python manage.py syncdb
    • 4. To add the “niftyurls” in your templates, use the following:

{% load niftyurls_tags %} {% niftyurls_media “js,css” %} {% niftyurls_content %}

  • 5. Please check the available Niftyurls settings in niftyurls/settings.py You may add custom values to NIFTYURLS_SETTINGS (please see niftyurls/settings.py) and retrive them inside your templates with: {% niftyurls_settings title %} {% niftyurls_settings h1 %}
  • 6. Add some feeds in the admin interface http://127.0.0.1:8000/admin/niftyurls/feed/add/ Here are some feed urls examples: – http://feeds.delicious.com/v2/rss/popular/python?count=15 – http://www.reddit.com/.rss
  • 7. Run the following command so that fresh entries are added to the database. $ python manage.py niftyurls_feeds
  • 8. Niftyurls templatetags depend on the existing of the “request” inside the templates, in case of errors verify that you have passed the “RequestContext” to the templates. http://docs.djangoproject.com/en/dev/ref/templates/api/#subclassing-context-requestcontext Make sure the TEMPLATE_CONTEXT_PROCESSORS contains the following: (‘django.core.context_processors.request’, ‘django.contrib.auth.context_processors.auth’,)

NiftyUrls’s page on GitHub

NiftyArticles – extract articles from any webpage with Python, Webkit and Readability

Posted September 2nd, 2010 in Python by Florentin

What is NiftyArticles

NiftyArticles is a small Python script which lets you identify and save articles from inside webpages. It is based on Readability.

Update 10.02.2011

I have made 2 versions of the script, one is using QT4 and the other is based on GTK.
The QT4 version is slightly better.

Installation

Each version depends on readability.js, please download that file and place it in the same directory as the niftyarticle script.

The installation details for each version is located inside the script file.

Usage (QT4 version)

To get a full list of options, run:
python niftyarticles-qt.py –help
OR if you made it executable,
/path/to/niftyarticles-qt.py –help

# save an article from ubuntu.com
python niftyarticles-qt.py -u http://www.ubuntu.com/project/derivatives

Help for the QT4 version

The script has been developed and tested on Python 2.6.6 / Ubuntu Maverick

Install the required packages: python-qt4 libqt4-webkit
Ubuntu or Debian
sudo apt-get install python2.6 python-qt4 libqt4-webkit

(Optional) Mark the script as executable
chmod +x niftyarticles-qt.py

Usage Examples:
python niftyarticles-qt.py --help

# save an article from ubuntu.com
python niftyarticles-qt.py -u http://www.ubuntu.com/project/derivatives

# show the browser window, load images, enable java applets
python niftyarticles-qt.py -b -i -v -u http://www.ubuntu.com/project/canonical-and-ubuntu

# to run the script on a server with no GUI available, you must install xvfb
sudo apt-get install xvfb
# then you can do
xvfb-run -a python niftyarticles-qt.py -u http://www.ubuntu.com/project/derivatives

Webkit Reference

http://webkitgtk.org/reference/index.html

Download the NiftyArticles from GitHub

Easy deployment with Python and Fabric

Posted August 26th, 2010 in Python by Florentin

Intro

This article is about writing tools in Python to easily deploy your web projects.
A well know deployment library in Python is Fabric. While it’s not really mature (version 0.9.1), it is easy to understand and work with it. The best place to start is the Fabric tutorial.

The Challenge

You develop and maintain many projects. You backup and restore databases several times a day during the development phase or you need to upload or download files from the hosting server often. How would you improve these processes instead using Ftp, mysql/mysqldump or phpmyadmin ?

Quick Fabric Introduction

Fabric gives you instruments to run commands on different hosts and copy/download files to/from remote hosts. In order to do that, you have to prepare a file of available commands named fabfile.py which will be placed in the project’s directory. Having some methods in that file allows you to run the commands like that:

fab hello

where hello is one function defined in fabfile.py
Fabric installation process.
sudo easy_install fabric
OR
sudo pip install fabric

The Solution

Having a lot of projects to maintain and deploy makes difficult to write one fabfile.py for each of them.
Also using remote hosts or writing database queries require you to put sensitive information in the fabfile.py (or pass them as parameters but that’s a different talk)
My thought is to have one location available where I place a fabfile.py and configuration files of all the sensitive information required for deployments, i.e. remote host addresses and maybe passwords, database connection information, etc. If you care more about security than speed of development, don’t store sensitive information there.
I have created a directory /work/fabric with the following structure:
├── fabfile.py
├── projects
│   ├── __init__.py
│   ├── ajaxdaddy.py
│ ├── default.py

fabfile.py contains the deployment code based on Fabric
local.py store configuration details of the localhost database
projects/* store configuration details of specific projects
The project name is the same with the file name used for the configuration.

Instead of placing a fabfile.py inside the project’s directory, we will instruct Fabric where to find the fabfile.
Create a file ~/.fabricrc and write the following line:
fabfile = /path/to/your/fabfile.py
I have mine placed in /work/fabric/fabfile.py

There are more details on fabfile discovery here.

Now we can run “fab” in any location. Here are some examples:
fab c:localhost c:localdb e:db_name=test q:clean
fab e:db_name=test_elgg e:db_file=elgg.dump.sql q:export

Commands Supported

The fabfile.py available for download in the end of this article gives you the power to run several commands useful in deploying projects or databases. Many commands require some parameters to exist, for example a database “clean” command which deletes all the tables from one database would require some connection details i.e. host,user,pass,db_name

Several commands may use the same parameters. To make these parameters available they are loaded from configuration files or passed to the “fab” command. All parameters are stored inside fabfile.py in a dictionary called “env” (environment). The environment is a big dictionary like variable (associative array for the Php users) which may store connection parameters for remote hosts, database connection properties or other information.

Most of the time you will need several params available for one command, e.g. a database command cannot function with only host name and no user provided. To make things easy, configuration files can be written where you define groups of parameters. Such a configuration file is /work/fabric/projects/default.py which is loaded by default. Configurations are simple Python dictionaries defined in Python files under the “projects” directory. It’s up to you to choose the names and the purpose of those settings, i.e., some might refer to database settings and other to remote hosting details. If you want to enable a set of parameters from a config file, first you need to load the file then enable the params.

Let’s take for example:
fab c:localhost c:localdb e:db_name=test q:clean

q:clean is the command and it means: delete all tables from the database define before
e:db_name defines the database
c:localdb loads the localhost database configuration, i.e. host, user, password, etc
c:localhost loads the localhost host details, i.e. host name, port, etc

Example of /work/fabric/projects/default.py file:
config = {
‘localdb': { # configuration name
‘db_user': ‘root’,
‘db_pass': ‘betterprogramming rules’,
‘db_host': ‘localhost’,
‘db_name': ‘test’, # default db_name if none is givven
‘db_file': ‘dump.sql’, # default dump file
},
}

“l” loads a configuration file located under “projects” directory (see the tree structure above)

fab l:ajaxdaddy

# not needed because default is loaded
fab l:default

“c” enables a configuration

# “localhost” is hardcoded into the fabfile.py because it’s mandatory
fab c:localhost

# enables a database configuration
fab c:localdb
fab l:default c:localdb

It’s possible to enable one configuration for a command then enable a different config for the next command.

# the second configuration “secondlocaldb” would overwrite the first one, “localdb”
fab l:default c:localdb c:secondlocalddb

After you enable custom configuration, it is possible to change specific parameters by using the “e” command.

# after i load and enable ajaxdaddy’s parameters, i.e. db_user, db_pass, db_host, db_name i can change the previously stored db_name value by using “e”
fab l:ajaxdaddy c:ajaxdaddydb e:db_name=db2_ajaxdaddy e:db_user=ajaxuser

In conclusion, there are 3 ways to set the parameters which will be used by your commands:
– by writing configuration files and place them under “projects” directory, load the config file (e.g. “l:ajaxdaddy”) and enable a specific group (e.g. “c:ajaxdaddydb”)
– by passing single parameters to the “fab” command (e.g. “e:db_user=gigi”)
– by passing parameters to the command itself (you will see how it works later)

Datase Commands

Database commands starts with “q” and are followed by an action.
The available actions are:
clean: delete all the tables inside the “db_name” database
export: export the database “db_name” to the file “db_file” (db_file may be provided or created programmatically by the fabfile.py from db_name)
import: import “db_file” into “db_name”
show: list all the databases where the “db_user” has access
create: create a new database called “db_name”
grant: grant all on “db_name_new” to “db_user_new” identified by “db_pass_new”

You may define your own actions in the configuration files. You just have to create a class named “DbAction”+actionname , e.g. class DbActionCreate(DbAction) or class DbActionClean(DbAction)
When loading a configuration file, your action class will also load.

# enables localhost (it’s enabled by default anyway), enables localdb so that “q” knows where to operate, sets the environment variable “db_name” to “test” then execute command
fab c:localhost c:localdb e:db_name=test q:clean
same with
fab c:localdb e:db_name=test q:clean

# selects the “test” db and export that db into “dump.sql” file
fab c:localhost c:localdb e:db_name=test e:db_file=dump.sql q:export

# create a new database “test_2″ and grant access to the user defined in “localdbnew”
fab c:localhost c:localdb c:localdbnew e:db_name=test_2 q:create q:grant

# delete all tables from “test” db and import the “test_elgg.sql” file
fab c:localdb e:db_name=test e:db_file=test_elgg.sql q:clean q:import

The list of all parameters used by the database commands:
db_user
db_pass
db_host
db_name
db_file
db_user_new
db_pass_new
db_host_new
db_name_new

File commands

# zip the temp directory, temp must be located under the current directory, it will create “temp.zip”
fab zp:temp

# zip the “temp” directory and create the archive named “mytemp.zip”
fab e:file=temp,dest=mytemp.zip zp

# zip “temp” directory’s content, i.e. do not include the “temp” parent directory in the archive
fab zpi:temp

# unzip “filename.zip” into the “dest” directory
fab uzp:filename.zip,dest
# same with
fab e:file=filename uzp:,dest
# same with
fab e:file=filename,dest=dest uzp

# load the “temp” config file and enable dreamhost parameters (“c:dreamhost”) then upload fabric.localhost.zip into the “stuff” directory located on the remote host inside the base dir. The base directory is defined in “base_dir” parameter, if base_dir=/var/www then the uploaded file will be located in /var/www/stuff/fabric.localhost.zip
fab l:temp c:dreamhost up:fabric.localhost.zip,stuff

# download /var/www/remotefile.zip to the current directory if base_dir=/var/www
fab l:temp c:dreamhost dl:remotefile.zip

List of file commands:
zp: zip a “file” (file or directory) to “dest”
zpi: zip a directory’s contents, exclude the directory itself from the archive
uzp: unzip a “file” to “dest”
up: upload a “source” (local host) to “dest” (remote host)
dl: download a “source” (remote host) to “dest” (local host)
cl: delete a “file”

Complex Commands

# exports “test_elgg” database into “dump.sql” and upload it to “dreamhost”. “up:e.db_file” means that the value of “db_file” located in the environment should be uploaded.
fab c:localhost c:localdb e:db_name=test_elgg e:db_file=dump.sql q:export l:dreamhost c:dreamhost up:e.db_file
# same with
fab c:localdb e:db_name=test_elgg e:db_file=dump.sql q:export l:dreamhost c:dreamhost up:dump.sql
# similar but different dump name “test_elgg.sql” created by default from “db_name” + “.sql”
fab c:localdb e:db_name=test_elgg q:export l:dreamhost c:dreamhost up:test_elgg.sql

# same as above, but upload a zip file which will be called by default “dump.sql.localhost.zip”
fab c:localdb e:db_name=test_elgg e:db_file=dump.sql q:export zp:dump.sql l:dreamhost c:dreamhost up

# same but unzip on destination
fab c:localdb e:db_name=test_elgg e:db_file=dump.sql q:export zp:dump.sql l:dreamhost c:dreamhost up uzp

# same but delete “cl” the dump.sql from localhost and delete “cl” dump.sql from the remote host
fab c:localdb e:db_name=test_elgg e:db_file=dump.sql q:export zp:dump.sql cl:dump.sql l:dreamhost c:dreamhost up uzp cl:dump.sql

Download fabfile.py and a few examples

Filter your video collection with Python and IMDbPY

Posted August 23rd, 2010 in Python by Florentin

If you have a large collection of movies (documentaries, dvd rips, etc) and you would like to decide on what worth watching next, this script may come in handy.
It’s a small Python script which scans your movie collection, collects data about each one from imdb.com and allows you to filter the movies by genres, actors or publishing studio.

The movies are collected from subdirectories (of indefinite depth) of a single parent directory which is either the default path (defined in the DEFAULT_MOVIES_PATH variable inside the script) or specified by the “-p” (or –path) parameter.
movies.py is looking for directories which contain at least one movie file (a file having the extension .avi, .mkv or .mpeg) The name of the movie is extracted from the directory name, not the file name.

Examples:
├── temp
│   ├── Capitalism.A.Love.Story
│   │   └── cd1
│   │   └── movie.avi
│   ├── Home.tt1014762
│   │   ├── imdb.html
│   │   ├── imdb.info
│   │   └── movie.dvdrip.avi
│   ├── Religuous.tt0815241
│   │   ├── imdb.html
│   │   ├── imdb.info
│   │   └── moviefile.avi
│   ├── stuff
│   │   └── nothinghere.txt
│   └── Tapped.tt1344784
│   ├── file.avi
│   ├── imdb.html
│   └── imdb.info

movies.py -p temp will identify the following directories as movies: “Home.tt1014762″, “Religuous.tt0815241″, Tapped.tt1344784 and “Cd1″
“Cd1″ is only a subdirectory but is seen as a movie because there is one movie file inside it.
You will have to move the movie.avi file from inside “Cd1″ to it’s parent, so that “Capitalism.A.Love.Story” will be recognized as a movie name.
If your movie is not properly identified on imdb, you may rename the directory and add the imdb movie id to it, e.g., “Capitalism.A.Love.Story.tt1232207″ or “Capitalism.A.Love.Story.1232207″

In order to use movies.py, you will need a couple of things:
1. Python 2.6 on a Linux machine (I have Ubuntu 10.04)
2. imdbpy library available
sudo pip install imdbpy
OR
sudo easy_install imdbpy
3. prettytable library
sudo pip install prettytable
OR
sudo easy_install prettytable

Download the script on your machine and execute:
# A) this will make your script executable
chmod +x yourusername
# B1) use an alias to run the ‘movies’ script from anywhere
alias movies=”/your/path/to/movies.py”
# B2) alternatively, you can make use of symbolic links
sudo ln -s /your/path/to/movies.py /usr/local/bin/movies.py

Examples of how to use the script, in case you have put movies.py somewhere in your PATH (method B2)

scan and refresh movies from inside the path “/home/elmo/Movies”. In case you don’t use “-p”, the default path is stored in the script variable DEFAULT_MOVIES_PATH (defined inside the script code)
movies.py -p “/home/elmo/Movies” -s -r

show movies with “Andy Serkis” but not with “Brad Pitt”
movies.py -l -a “+andy serkis,-brad pitt”

movies with at least one of these actors playing
movies.py -l -a “?andy serkis,?angelina jolie”

movies with at least one favorite actor playing
movies.py -l -o actor

comedy movies but no dramas
movies.py -l -g “+comedy,-drama”

movies made by “lionsgate” production company
movies.py -c “lionsgate”

movies made by any of the favorite production companies
movies.py -o company

There are a couple of ‘predefined’ filters.
For example, in the script’s code you will see a large list of (good) actors. If you want to find movies where at least one favorite actor is playing, run:
movies.py -o actor
The same is true for production companies
movies.py -o company

Example of output:
movies.py -l -g “+documentary,-drama”
+———+———–+——————–+——+———————+
| Imdb ID | Movie | Genres | Rate | Basename |
+———+———–+——————–+——+———————+
| 1014762 | Home | documentary | 8.4 | Home.tt1014762 |
| 0815241 | Religuous | documentary,comedy | 7.8 | Religuous.tt0815241 |
| 1344784 | Tapped | documentary | 6.8 | Tapped.tt1344784 |
+———+———–+——————–+——+———————+

View the whole list of options using:
movies.py -h
OR
movies.py –help

I hope you will enjoy the script and stay tuned for updates :)

Download here