Pro Django

Marty Alchin

Pro Django

Copyright © 2009 by Marty Alchin

All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher.

ISBN-13 (pbk): 978-1-4302-1047-4

ISBN-13 (electronic): 978-1-4302-1048-1

Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1

Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark.

Java™ and all Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc., in the US and other countries. Apress, Inc., is not affiliated with Sun Microsystems, Inc., and this book was written without endorsement from Sun Microsystems, Inc.

Lead Editor: Steve Anglin

Technical Reviewers: Jacob Kaplan-Moss, George Vilches

Editorial Board: Clay Andres, Steve Anglin, Mark Beckner, Ewan Buckingham, Tony Campbell, Gary Cornell, Jonathan Gennick, Jonathan Hassell, Michelle Lowman, Matthew Moodie, Duncan Parkes, Jeffrey Pepper, Frank Pohlmann, Ben Renow-Clarke, Dominic Shakeshaft, Matt Wade, Tom Welsh Project Manager: Richard Dal Porto

Copy Editors: Liz Welch, Ami Knox

Associate Production Director: Kari Brooks-Copony

Production Editor: Laura Cheu

Compositor: Kinetic Publishing Services, LLC

Proofreader: April Eddy

Indexer: Julie Grady

Cover Designer: Kurt Krames

Manufacturing Director: Tom Debolski

Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax 201-348-4505, e-mail

, or

visit

.

For information on translations, please contact Apress directly at 2855 Telegraph Avenue, Suite 600, Berkeley, CA 94705. Phone 510-549-5930, fax 510-549-5939, e-mail

.

Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use.

eBook versions and licenses are also available for most titles. For more information, reference our Special Bulk Sales–eBook Licensing web page at http://www.apress.com/info/bulksales.

The information in this book is distributed on an “as is” basis, without warranty. Although every precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work.

The source code for this book is available to readers at

. You will need to answer

questions pertaining to this book in order to successfully download the code.

Contents at a Glance

About the Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix CHAPTER 1 Understanding

Django . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

CHAPTER 2

Django Is Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

CHAPTER 3 Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

CHAPTER 4

URLs and Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

CHAPTER 5 Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

CHAPTER 6 Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

CHAPTER 7 Handling

HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

CHAPTER 8 Backend

Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

CHAPTER 9 Common

Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

CHAPTER 10 Coordinating

Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

CHAPTER 11 Enhancing

Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

APPENDIX

Contributing to Django . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

iii

Contents

About the Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix CHAPTER 1 Understanding

Django . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Philosophy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Django’s Interpretation of the MVC Pattern . . . . . . . . . . . . . . . . . . . . . . 2

Loose Coupling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Don’t Repeat Yourself (DRY) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

A Focus on Readability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Failing Loudly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Community . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Management of the Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

News and Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Reusable Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Getting Help . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

CHAPTER 2

Django Is Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

How Python Builds Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Building a Class Programmatically . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Metaclasses Change It Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Using a Base Class with a Metaclass . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Declarative Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Common Duck-Typing Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Callables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Iterables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

v

vi

C O N T E N T S

Augmenting Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Excess Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Decorators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Descriptors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

__get__(self, instance, owner) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

__set__(self, instance, value) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Keeping Track of Instance Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Introspection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Common Class and Function Attributes . . . . . . . . . . . . . . . . . . . . . . . . 37

Identifying Object Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Function Signatures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Docstrings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Applied Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Tracking Subclasses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

A Simple Plugin Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

CHAPTER 3 Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

How Django Processes Model Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Setting Attributes on Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Getting Information About Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Class Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Field Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

Primary Key Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

Configuration Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

Accessing the Model Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Using Model Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Common Field Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Common Field Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Subclassing Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

Deciding Whether to Invent or Extend . . . . . . . . . . . . . . . . . . . . . . . . . 62

Performing Actions During Model Registration . . . . . . . . . . . . . . . . . . 63

Altering Data Behavior. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

Controlling Database Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Dealing with Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

get_directory_name(self) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

get_filename(self, filename). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

generate_filename(self, instance, filename) . . . . . . . . . . . . . . . . . . . . 72

C O N T E N T S

vii

save_form_data(self, instance, data) . . . . . . . . . . . . . . . . . . . . . . . . . . 73

delete_file(self, instance, sender) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

attr_class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

Customizing the File Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

class_prepared . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

pre_init and post_init . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

pre_save and post_save . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

pre_delete and post_delete . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

post_syncdb . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Applied Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

Loading Attributes on Demand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

Creating Models Dynamically at Runtime . . . . . . . . . . . . . . . . . . . . . . 87

Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

CHAPTER 4

URLs and Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

URLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Standard URL Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Resolving URLs to Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Resolving Views to URLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Templates Break It Up a Bit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Anatomy of a View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Writing Views to Be Generic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

View Decorators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

Using an Object As a View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Applied Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Dual-Format Decorator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

CHAPTER 5 Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

Declaring and Identifying Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

Binding to User Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

Validating Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Custom Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Controlling Widgets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

viii

C O N T E N T S

Defining HTML Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Custom Widgets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Customizing Form Markup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Accessing Individual Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

Customizing the Display of Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

Applied Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Pending and Resuming Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

CHAPTER 6 Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

What Makes a Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

The Process at Large . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

Content Tokens . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

Parsing Tokens into Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

Template Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

Rendering Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

Simple Variable Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

Complex Variable Lookup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

Including Aspects of the Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

Retrieving Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

django.template.loader.get_template(template_name) . . . . . . . . . 141

django.template.loader.select_template(template_name_list) . . . 142

Shortcuts to Load and Render Templates . . . . . . . . . . . . . . . . . . . . . 142

Adding Features for Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

Setting Up the Package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

Variable Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

Template Tags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

Adding Features to All Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

Applied Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

Embedding Another Template Engine . . . . . . . . . . . . . . . . . . . . . . . . 148

Enabling User- Submitted Themes . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

CHAPTER 7 Handling

HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Requests and Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

HttpRequest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

HttpResponse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

C O N T E N T S

ix

Writing HTTP Middleware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

MiddlewareClass.process_request(self, request) . . . . . . . . . . . . . . . 174

MiddlewareClass.process_view(self, request, view,

args, kwargs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

MiddlewareClass.process_response(self, request, response) . . . . 175

MiddlewareClass.process_exception(self, request, exception) . . . 175

Deciding Between Middleware and View Decorators . . . . . . . . . . . 176

HTTP-Related Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

django.core.signals.request_started . . . . . . . . . . . . . . . . . . . . . . . . . 178

django.core.signals.request_finished . . . . . . . . . . . . . . . . . . . . . . . . . 178

django.core.signals.got_request_exception . . . . . . . . . . . . . . . . . . . 178

Applied Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

Signing and Validating Cookies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

CHAPTER 8 Backend

Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

Database Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

django.db.backends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

Creation of New Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

Introspection of Existing Structures . . . . . . . . . . . . . . . . . . . . . . . . . . 191

DatabaseClient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

DatabaseError and IntegrityError . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

get_user(user_id) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

authenticate(**credentials) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

Storing User Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

The Base File Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

Handling Uploads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

Storing Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

Session Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

Caching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Specifying a Backend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Using the Cache Manually . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

Template Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

load_template_source(template_name,

template_dirs=None) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

load_template_source.is_usable . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

Context Processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

x

C O N T E N T S

Applied Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

Loading Templates Using a Different Engine . . . . . . . . . . . . . . . . . . 205

Scanning Incoming Files for Viruses . . . . . . . . . . . . . . . . . . . . . . . . . . 210

Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

CHAPTER 9 Common

Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

Core Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

django.core.exceptions.ImproperlyConfigured . . . . . . . . . . . . . . . . . 213

django.core.exceptions.MiddlewareNotUsed . . . . . . . . . . . . . . . . . . 214

django.core.exceptions.MultipleObjectsReturned . . . . . . . . . . . . . . 214

django.core.exceptions.ObjectDoesNotExist . . . . . . . . . . . . . . . . . . . 215

django.core.exceptions.PermissionDenied . . . . . . . . . . . . . . . . . . . . 215

django.core.exceptions.SuspiciousOperation . . . . . . . . . . . . . . . . . . 216

django.core.exceptions.ViewDoesNotExist . . . . . . . . . . . . . . . . . . . . 216

Text Modification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

get_text_list(items, last_word='or') . . . . . . . . . . . . . . . . . . . . . . . . . . 217

javascript_quote(s, quote_double_quotes=False) . . . . . . . . . . . . . . 217

normalize_newlines(text) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

phone2numeric(phone) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

recapitalize(text) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

smart_split(text) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

truncate_words(s, num) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

truncate_html_words(s, num) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

wrap(text, width) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

django.utils.datastructures.MergeDict . . . . . . . . . . . . . . . . . . . . . . . . 220

django.utils.datastructures.MultiValueDict . . . . . . . . . . . . . . . . . . . . 221

django.utils.datastructures.SortedDict . . . . . . . . . . . . . . . . . . . . . . . . 222

Functional Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

django.utils.functional.curry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

django.utils.functional.memoize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

django.utils.functional.wraps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

Defining a Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

Sending a Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

Capturing Return Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

Defining a Listener . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

C O N T E N T S

xi

Registering Listeners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

Forcing Strong References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

CHAPTER 10 Coordinating

Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

Contacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

contacts.models.Contact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

contacts.forms.UserEditorForm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

contacts.forms.ContactEditorForm . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

contacts.views.edit_contact. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235

Admin Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

URL Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

Real Estate Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

properties.models.Property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

properties.models.Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

properties.models.PropertyFeature . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

properties.models.InterestedParty . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

Admin Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

URL Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

CHAPTER 11 Enhancing

Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

Recording the Current User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

The Thread- Local Approach—Useful but Dangerous . . . . . . . . . . . 254

The Admin Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256

Introducing the CurrentUserField . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

Keeping Track of CurrentUserField Instances . . . . . . . . . . . . . . . . . . 257

The CurrentUserMiddleware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

Performance Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

Keeping Historical Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

Intended Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264

Overview of the Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266

Step 1: Copy the Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

Step 2: Register Signal Handlers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272

Step 3: Assign a Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272

Now What? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278

xii

C O N T E N T S

APPENDIX

Contributing to Django . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

Reporting a Ticket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

Supplying a Patch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

Writing Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280

Writing Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280

Development Sprints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

Publishing Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

Releasing an Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

About the Author

MARTY ALCHIN is a professional programmer with a passion for the Web. Over the past two and a half years, he has developed and released a few Django applications and a significant improvement to Django’s file storage handling.

Raised in the wild by a pack of mechanical engineers, Marty learned at a young age the importance of knowing how things work and how to improve them. When not coding for work, he goes by the name Gulopine and codes for fun. He keeps a blog at

,

where much of this code is announced and described.

xiii

Acknowledgments

I can’t imagine anyone taking on a project like this alone. In the year and a half since I first considered putting my thoughts on paper, no one has been more supportive than my beautiful wife, Angel. Without her, I’d be lost and confused, mumbling incoherently about declarative metaclass implementations. There are no words to express how much help she’s been throughout the process.

I’d also like to thank George Vilches for stepping up to take on a book he hadn’t been involved with from the beginning. He’s been an amazing asset, going well beyond what was required of him to make sure this book is as good as we could possibly make it.

Of course, the Lawrence Journal-World and its Internet division are to thank for the Django Web framework’s existence and for its release to the public, which made all of this possible in the first place. I don’t expect they had any idea how far it would go when designing and releasing it. I have a feeling this is far from the end.

In fact, the entire community that surrounds Django has fueled me in more ways than I can explain. It’s because of people like you that I chose to take on this challenge, and it’s the thought of a greater community that keeps me going. Thank you.

xv

Preface

Programming has always been equal parts art and science. It’s easy to see the science in teaching computers how to do things, but once that’s out of the way, we often try to embrace the artistic side. We spend our first few years learning to make code functional and the rest of our careers trying to make it beautiful.

Django started its life in much the same way, serving the day-to-day needs of a local news organization. In the years since its first public release, Django itself has grown more elegant and has helped its adopters to write more elegant code for their own applications.

This focus on beauty isn’t unique to Django. Most Python applications strive for a notion of being “Pythonic”—an unwritten ideal that embodies the nature and spirit of the Python language itself. Having a vague goal like that may seem problematic; after all, how do you know when you’ve succeeded? Ironically, that’s the point: there is no finish line. There’s not even a measuring stick to tell you how close you are to achieving your goal.

The true goal is the journey itself, the lessons learned along the way, the discoveries that open your eyes to new ideas. Python includes a number of tools that make this process quite interesting, especially for those programmers coming from other languages. Django builds on that toolset, adding its own techniques for easing the burden on other programmers, making it easy to produce more beautiful code all around.

I first got started with Django shortly after it completed its “magic removal” phase, which was a long process of making the framework more Pythonic overall. I was new to Python at the time, and reading about the process and the ideals that encouraged it caused me to dig deeper into what made Django work. I was fascinated by the richness of the toolset at my disposal and quickly began my own journey of discovery.

What fascinated me most was how few people knew about some of the tricks that can be used to encourage Pythonic code for programmers using the framework. Every time I showed a new trick to someone, I joked that I could write a book about what I’ve learned so far. After several months of doing so—and several people encouraging me to drop the joke and do it for real—I finally took the plunge and contacted Apress.

I’m not interested in making a fortune with this book. My goal has always been to help more people understand the many tools available with Python and Django, in hopes that they too can have enriching journeys of their own. I hope this book will help bring Django to new people and new places, where it might have been previously considered inappropriate.

Those of us working with Django are often called Djangonauts with good reason. The

“-naut” suffix has been used historically to represent sailors and is the same concept as in the word “nautical.” More generally, it often refers to those who sail into the unknown, such as astronauts and cosmonauts. It represents explorers and adventurers, those people brave enough to challenge what they knew before and dare to discover new things and new places.

I am a Djangonaut. What follows is my journey thus far.

xvii

Introduction

P ro Django represents two and a half years of accumulated knowledge in Python and Django, designed to educate readers who are already familiar with both topics and would like to take them further than they had previously done. You will learn a wide range of advanced techniques available in both Python and Django, along with tips on how to use them to achieve advanced functionality.

This book is designed to be both a narrative to be read from start to finish and a general reference to be searched for specific information. Since you may not know what to look for or where to find it yet, feel free to read through the book first, then keep it handy for refreshing your memory as necessary.

What This Book Is Not

There are plenty of resources available for learning Python and Django, so this book does not strive to teach the basics. For readers new to Python, I highly recommend Dive Into Python by Mark Pilgrim (Apress, 2004). For learning Django, I’d recommend The Definitive Guide to Django: Web Development Done Right by Adrian Holovaty and Jacob Kaplan-Moss (Apress, 2006). Additionally, Practical Django Projects by James Bennett (Apress, 2008) is an excellent resource for general application development.

Who This Book Is For

Because Pro Django doesn’t dwell on introductory details, readers will be expected to have experience with both Python and Django. If you’re new to either subject, please consider one of the books mentioned in the previous section before trying to tackle this book.

Even if you’ve only experimented on your own without launching a full site yet, a basic familiarity should be sufficient. You don’t need to be an expert to start reading Pro Django, but you might be by the time you finish.

Interpreting Code Samples

Pro Django uses a simple format, interleaving explanations of Python’s and Django’s available features with code that demonstrates their use in the real world. There are two types of code samples used, which differ in how they should be executed.

Python’s interactive interpreter is a great way to test out small pieces of code and see how it works in a variety of situations. Lines of code intended for use in that environment will always be prefixed with three characters: three greater-than signs (

) or three periods (

).

Lines with greater-than signs are the outermost block of code, while the period-prefixed lines are indented at least one level. The three initial characters are also followed by a space. These xix

xx

I N T R O D U C T I O N

first four characters are not typed into the interactive interpreter directly; they simply mimic what the interpreter itself looks like by reproducing its output.

A line started with three periods but containing no other text indicates that you should simply press Enter on a blank line in the interpreter. This completes any open code blocks, bringing you back to the

prompt. Any lines that don’t begin with either

or

repre-

sent the output of the code or the result of the previous expression.

The first line of an interactive example will always begin with

; everything else is code

that should be written in a file and executed as part of a running Django application. The surrounding text will indicate what file the code should be placed in and how it will execute.

Prerequisites

Pro Django is written for Django 1.0, which was released on September 3, 2008. That release or a more recent checkout from the Django code repository is required for the code samples to work properly. Since Django in turn relies on Python, these examples also assume a working Python environment of version 2.3 or higher.

C H A P T E R 1

Understanding Django

Code alone isn’t enough. Sure, it’s what the computer runs, but code has to come from somewhere. A programmer has to sit down and decide what features to include, how they should be implemented, what other software to utilize and how to provide hooks for future enhancements to be added. It’s easy to skip straight to code, ignoring the cognitive process that produces it, but great programmers always have reasons for the decisions they make.

With a framework, like Django, many such decisions have already been made, and the tools provided are shaped by these decisions and by the programmers who made them. By adopting these philosophies in your own code, not only will you be consistent with Django and other applications, but you may even be amazed at what you’re able to accomplish.

Beneath even the most fundamental code is the thought process that went into its creation. Decisions were made about what it should do and how it should do it. This thought process is a step often overlooked in books and manuals, leading to an army of technicians slaving away, writing code that manages to accomplish the task at hand but without a vision for its future.

While the rest of this book will explain in detail the many basic building blocks Django provides for even the most complicated of projects, this chapter will focus on these even more fundamental aspects of the framework. For those readers coming from other backgrounds, the ideas presented in this chapter may seem considerably foreign, but that doesn’t make them any less important. All programmers working with Python and Django would do well to have a solid understanding of the reasons Django works the way it does, and how those principles can be applied to other projects.

You may want to read this chapter more than once, and perhaps refer to it often as you work with Django. Many of the topics covered in this chapter are common knowledge in the Django community, so reading this chapter carefully is essential if you plan to interact with other programmers.

Philosophy

Django relies heavily on philosophy, both in how its code is written and how decisions are made about what goes into the framework. This isn’t unique in programming, but it’s something newcomers often have trouble with. It is essential to maintain both consistency and quality, and having a set of common philosophies to refer to when making decisions helps maintain both. Since these concepts are also important to individual applications, and even collections of applications, a firm grasp on these philosophies will yield similar benefits.

1

2

C H A P T E R 1

฀ UNDERSTANDING DJANGO

Perhaps the best- known and most- quoted passage of Python philosophy comes from Tim Peters, a longtime Python guru who wrote down many of the principles that guide Python’s own development process. The 19 lines he came up with have been so influential to Python programmers over time that they are immortalized as Python Enhancement Proposal (PEP) 201 and in the Python distribution itself, as an “easter egg” module called

.

While some of this is clearly intended for humor, the majority is a good summation of many Python philosophies. The remainder of this chapter highlights some specific principles that are often cited within the Django community, but all professional Python programmers should keep this text in mind and reference it often.

One important thing to keep in mind is that many of the lines in the Zen of Python are subjective. For example, while “beautiful” may be better than “ugly,” definitions of “beautiful” are plentiful and can vary as much as the people who provide them. Similarly, consider notions of simplicity and complexity, practicality and purity; each developer will have a different opinion on which side of the line a particular piece of code should be placed.

Django’s Interpretation of the MVC Pattern

One of the most common application architectures—as adopted by hobbyists and corporations alike—is the Model-View- Controller (MVC) pattern, as it provides clean separation of tasks and responsibilities among the prominent aspects of an application. Django only loosely follows this model. A proper discussion should kick off with a quick overview of its components.

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

1

C H A P T E R 1

฀ UNDERSTANDING DJANGO

3

While this pattern has proven very effective in many domains, Django’s authors weren’t looking to conform to any form of pattern at the outset. They were simply interested in finding the most effective way to develop software for the Web. After all, Django was built for the daily needs of a working newspaper, where things have to happen very quickly if they’re to happen at all. Ultimately, the separation of tasks into discrete groups serves a few different purposes.

฀

฀

฀

฀ ฀

฀ ฀ ฀

฀

฀ ฀

฀ ฀

฀

฀

฀

฀ ฀

doesn’t need to make assumptions about completely unrelated parts of the application.

฀

฀

฀

฀ ฀

฀

฀

฀ ฀

฀

฀

-

ferent view and controller layers may connect to a single model layer. This enables a variety of applications to share the same business logic and data, presenting it and interacting with it in different ways, for different audiences.

฀

฀

฀

฀

฀ ฀

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀ ฀

฀

work being performed. This specialization helps to curb frustration and fatigue, while fostering creativity and excellence within each developer’s domain of specialty.

There are certainly other smaller benefits, but these are generally the main goals achieved with the use of MVC. It’s interesting to note, however, that the only part of those benefits that applies to any specific division in the MVC pattern is the ability to plug multiple applications into a single model layer. The rest is just an arbitrary division based on common development plans.

Django’s developers sought these same benefits, but with an emphasis on rapid development, without worrying about creating a development pattern. After getting a set of tools that made sense for their workflow, they ended up with what some have called a Model-Template- View (MTV) pattern. However, there are really four primary code divisions in a Django application, which are outlined next.

Model

Given the benefit of keeping models apart from the rest of the application, Django follows that part of MVC to the letter. Django models provide easy access to an underlying data storage mechanism, and can also encapsulate any core business logic, which must always remain in effect, regardless of which application is using it.

Models exist independent of the rest of the system, and are designed to be used by any application that has access to them. In fact, the database manipulation methods that are available on model instances can be utilized even from the interactive interpreter, without loading a Web server or any application- specific logic.

Chapter 3 covers Django models in more detail, including how they’re defined and utilized, how to include your own business logic and much more.

View

While they share a name with the original MVC definition, Django views have little else in common with the traditional paradigm. Instead, they combine some of the traditional view’s responsibility with the entirety of the controller’s tasks. A view accepts user input, including simple requests for information; behaves according to the application’s interaction logic; and returns a display that is suitable for users to access the data represented by models.

4

C H A P T E R 1

฀ UNDERSTANDING DJANGO

Views are normally defined as standard Python functions that are called when a user requests a specific URL. In terms of the Web, even a simple request for information is considered an action, so views are intended to handle that alongside data modifications and other submissions. They can access the models, retrieving and updating information as necessary to accomplish the task requested by the user.

Since views are simply called as functions, without requiring any specific structure, they can be specified a number of ways. In addition to a simple function, a view could take the form of any Python callable, including instance methods, callable objects and curried or decorated functions.

Template

While views are technically responsible for presenting data to the user, the task of how that data is presented is generally relegated to templates, which are an important enough part of Django development to be considered a separate layer entirely. Many have drawn a parallel between Django templates and the traditional view layer, since templates handle all the presentational details the user will see.

Django provides a simple template language for this purpose, to ensure that template designers don’t need to learn Python just to work with templates. Django’s template language is also not dependent on any particular presentation language. It’s primarily used for HTML

but can be used to generate any text- based format.

Keep in mind, however, that this template engine is just one tool that views can use to render a display for a user. Many views may use HTTP redirects to other URLs, third- party Portable Document Format (PDF) libraries or anything else to generate their output.

URL Configuration

By combining this architectural philosophy with its nature as a framework for the Web, Django provides a separate layer of glue to make views available to the outside world via specific URLs.

By supplying a regular expression as the URL component, a single declaration can accommodate a wide variety of specific URLs, in a highly readable and highly maintainable manner.

This configuration is defined separately from views themselves to allow a view to be configured at more than one URL, possibly with different options at each location. In fact, one of the core features of Django is the concept of generic views. These are views intended for common needs, with configuration options that allow them to be used in any application, requiring only a URL configuration to enable them.

Perhaps most important of all, having URLs as a separate part of the process encourages developers to think of URLs as part of an application’s overall design. Since they must be used in bookmarks, blog posts and marketing campaigns, URLs are sometimes more visible than your application. After all, users who are paying attention while browsing the Web will see your URL before they even decide to visit your site. URLs get even more important when using print media for advertising campaigns.

Chapter 4 covers URL configurations in more detail, including some guidelines on proper URL design.

C H A P T E R 1

฀ UNDERSTANDING DJANGO

5

Loose Coupling

One key feature of the MVC architecture, and of Django’s slightly modified form, is the notion that sections of code that perform significantly different functions shouldn’t know how the others operate. This is called loose coupling. Contrast this with tight coupling, where modules often rely heavily on the internal details of other modules’ implementations.

Tight coupling causes a whole host of problems with long- term code maintenance, as significant changes to one section will invariably affect others. This creates a mountain of extra work for the programmer, having to change code that has little—if anything—to do with the work that needs to be done. This extra work is unfortunate for not only the programmer; it’s often quite costly for the employers as well.

It may seem like this principle would advocate that no code should ever know anything about any other code, but that’s hardly the case, as a program written like that couldn’t actually do anything. Some sections of code will always need to reference others; that’s unavoidable.

The key is to make sure that the details of one feature are hidden from the others.

In Python, loose coupling is typically provided in a number of ways, some of which are shown in the following list. There are countless others, which could fill a book on their own, but the techniques shown here are described in detail in Chapter 2.

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

Don’t Repeat Yourself (DRY)

If you’ve been around the block a few times, you know all too well how easy it is to write

“boilerplate” code. You code once for one purpose, then again for another, and again, and again and again. After a while, you realize how much code has been duplicated, and if you’re lucky, you have the time, energy and presence of mind to look at what’s common and move those pieces into a common location.

This process is one of the primary reasons for a framework to exist. Frameworks provide much of this common code, while attempting to make it easier to avoid duplicating your own code in the future. This combines to represent a common programming practice: Don’t Repeat Yourself.

Often abbreviated DRY, this term comes up quite often in conversations, and can be used as

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀ ฀

฀

฀

฀ ฀

The basic idea is that you should only write something once. It should be as reusable as possible, and if other code needs to know something about what you’ve already written, it should be able to get the necessary information automatically using Python, without requiring the programmer to repeat any of that information.

6

C H A P T E R 1

฀ UNDERSTANDING DJANGO

To facilitate this, Python provides a wealth of resources for peeking inside your code, a process called introspection. Many of these resources, covered in Chapter 2, are incredibly useful when supporting DRY in your code.

A Focus on Readability

“Readability counts.” It’s mentioned specifically in the Zen of Python, as noted earlier, and is perhaps one of the most important features of Python. Indeed, many Python programmers take pride in the readability of both the language and the code they write. The idea is that code is read far more often than it’s written, especially in the world of open source.

To this end, Python as a language provides a number of features designed to improve readability. For instance, its minimal use of punctuation and forced indentation allow the language itself to help maintain the readability of your code. When you’re working with code in the real world, however, there’s far more to consider.

For real life, the Python community has developed a set of guidelines for writing code, intended to improve readability. Set forth in PEP- 8,2 these guidelines are designed to maintain not only readability of an individual program, but also consistency across multiple programs.

Once you get the feel for one well- written program, you’ll be able to easily understand others.

The exact details of PEP- 8 are too numerous to list here, so be sure to read it thoroughly to get a good idea for how to write good code. Also, note that if you read Django’s own source code, some of the rules set forth in PEP- 8 aren’t followed. Ironically, this is still in the interest of readability, as certain situations would suffer unnecessarily if every last rule was followed.

After all, to quote the Zen of Python again, “Practicality beats purity.” The examples in this book will follow the style used by Django’s own source code.

Failing Loudly

“Errors should never pass silently. / Unless explicitly silenced.” This may seem like a simple sentiment, but at two lines, it comprises over 10 percent of the Zen of Python, and there’s something to be said for that. Dealing with exceptions is an important part of programming, and this is especially true in Python. While all programming languages can generate errors, and most have a way to handle them gracefully, each language has its own best practices for dealing with them.

One key to keep in mind is that, while the names of most Python exceptions end in

,

the base class is called

. To understand how they should be used and handled, it’s useful to start by learning why that particular word was used. Looking at some of the dictionary definitions for the word “exception,” it’s easy to see variations on a theme.

฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀ ฀

฀

฀

฀

฀

฀ ฀ ฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀ ฀

฀ ฀

Rather than an error, which describes a situation where a problem occurred, an exception is simply when something unexpected occurred. This may seem like a subtle distinction, but some philosophies treat exceptions as errors, reserving them solely for unrecoverable 2.

C H A P T E R 1

฀ UNDERSTANDING DJANGO

7

problems like corrupted files or network failure. This is reinforced by the fact that, in some languages, raising exceptions is extremely expensive, so to prevent performance problems, exceptions are avoided whenever possible.

In Python, however, exceptions are no more expensive than simple return values, allowing them to be more accurate to their dictionary definition. If we define an exception as a violation of a rule, it stands to reason that we must first define a rule.

Defining Rules

Since this is the most important aspect of understanding exceptions, it’s necessary to be perfectly clear: there’s no Python syntax for defining rules. It’s simply not a feature of the language. Some other languages explicitly support design by contract,3 and many can support it through framework- level code, but Python doesn’t support any form of it natively.

Instead, rules are defined by programmers in what they intend their code to do. That may seem like an oversimplification, but it’s really not. A piece of code does exactly what its author intends it to do, and nothing more. Anything outside the intentions of the programmer can—

and should—be considered an exception. To illustrate this, here are some of the rules used by Python and Django:

฀

฀

฀

฀

฀ ฀ ฀

฀

฀

฀

฀

฀

) returns the item at the

specified position.

฀

฀ ฀

฀

method makes sure that a specified item is no longer a member of the set.

฀

฀ ฀

฀

method returns exactly one object that matches the arguments provided.

Examples like these are important because even though these rules are simple, they accurately describe how the given features will behave in various situations. To further illustrate, consider the following scenarios and how the rule impacts behavior.

฀

฀ ฀

฀

฀

฀ ฀ ฀

฀ ฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀

be returned. If it doesn’t, an exception (

) is raised. If the value used as an

index isn’t an integer, a different exception (

) is raised.

฀

฀ ฀

฀

฀

฀removed from a set using

is already a member of the set, it’s

simply removed. If it wasn’t a member of the set,

returns without raising an

exception, since discard only tries to make sure that the item is not in the set.

฀

฀ ฀

฀

฀

฀ ฀ ฀

฀

method match one record in the database,

that record is returned as an instance of the appropriate model. If no records match, an exception (

) is raised, but if more than one record matches, a different exception (

) is raised. Finally, if the arguments can’t be used

to query the database (due to incorrect types, unknown attribute names or a variety of other conditions), still another exception (

) is raised.

Clearly, even simple rules can have profound effects, as long as they’re defined explicitly.

While the only requirement is that to be defined in the mind of the author, rules are of little 3.

8

C H A P T E R 1

฀ UNDERSTANDING DJANGO

use if not conveyed to anyone else. This becomes especially important in the case of a framework such as Django, built for distribution to the masses.

Documenting Rules

There are a number of appropriate ways to document the specific rules a piece of code was written to follow. It’s even quite useful to specify them in more than one way, and in varying levels of complexity. There are four main places where people look for this information, so providing it in any or all of these locations would serve the purpose quite well.

฀

฀

฀

฀

฀ ฀

฀

฀

฀ ฀

฀

฀

฀

application, it stands to reason that these rules would be included.

฀

฀

฀ ฀฀

฀

฀

฀

฀will often peek at

the code itself to see how it works. Docstrings allow you to provide plain- text explanations of these rules right alongside the code that implements them.

฀

฀

฀

฀ ฀

฀explanations of these rules for humans to understand,

it’s a great idea to provide them in a way that Python can understand. This allows your rule to be verified on a regular basis. In addition, by writing doctests—tests embedded inside docstrings—the tests are also human- readable, and both purposes can be served at once.

฀

฀

฀ ฀

฀

฀ ฀complicated enough that a broad over-

view, such as might be found in full documentation or even the docstring, doesn’t give sufficient information about what a particular chunk of code is expected to do.

Python’s emphasis on readability makes this fairly infrequent, but it does still happen.

When it does, comments can be a useful way of explaining to others what the code is intended for, and thus what should be considered an exception.

Regardless of how you choose to describe your rules, there’s one lesson that must always take precedence: be explicit. Remember, anything not laid out in your rule should be considered an exception, so defining the rule explicitly will help you decide how the code should behave in different situations, including when to raise exceptions.

Also, be consistent. Many classes and functions will look similar in name or interface, and where at all possible, they should behave similarly. Programmers who are accustomed to a particular behavior will expect similar behavior from similar components, and it’s best to meet those expectations. This is especially true when writing code that mimics types provided by Python or Django, as they’re already well- documented and well- understood by many programmers.

Community

Since being released to the public in 2005, Django has achieved great success, both technically and culturally. It has amassed a tremendous following throughout the world of Python Web development, among hobbyists and professionals alike. This community is one of the greatest assets to the framework and its users, and it’s most certainly worth discussing in some detail.

C H A P T E R 1

฀ UNDERSTANDING DJANGO

9

AN EVOLVING COMMUNITY

It’s important to realize that like any social structure, the Django community will evolve and change over time. As such, the information in this section may not always accurately reflect current practices and expectations.

There’s no reason to let that deter you, though. The one thing I don’t expect to change is the community’s willingness to embrace new members. You’ll always be able to get in touch with a variety of people, if you’re willing to put yourself out there.

Management of the Framework

One of the first things to understand about development of Django—and about Python in general—is that, while the code for the framework is available for anyone to view and manipulate (it is open source, after all), the overall management of the core distribution is overseen by a small group of people. These “core developers” consist of those with access to update the main code repository.

WHAT IS “CORE”?

Because Django is open source, any user may make changes to Django’s code and distribute those modified copies. Many developers have done so, adding significant features and enhancements and providing their work for others to use. Advanced users can make considerable alterations to the central code without impacting those who don’t need the features provided by the copy.

In addition, developers are allowed—and encouraged—to make their applications generic and distribute them to others. These sometimes become so ubiquitous that many developers include them by default in any new project they start.

In contrast, Django’s core is simply the code that is distributed through the main Django Web site, either as an official release or as the main trunk development code. So when a discussion includes a debate about whether something should be “in core,” the dilemma is whether it should go into the official distribution or in some third- party format, such as a branch or a distributed application.

This structure helps ensure that those with the most experience with the framework and its history are responsible for looking over, and often tweaking, all patches before they are committed to the repository. They also regularly discuss issues concerning recent develop-ments in the framework, major overhauls that need to be done, significant improvements that can be made and so on.

There is still someone at the top of the management chain. This position is called the Benevolent Dictator for Life, often abbreviated BDFL, and is reserved for those who have ultimate authority over all decisions, should they need to break a tie or override a majority decision. Thankfully, they are truly benevolent dictators, a distinction not taken lightly.

In fact, the idea of a BDFL is more humorous than anything else. Though they do hold ultimate authority, this power is rarely exercised, as they tend to favor group opinion. When they do need to step in and arbitrate a decision, their ruling is based on years of experience in knowing what’s best for the framework and its audience. In fact, they will often raise their own 10

C H A P T E R 1

฀ UNDERSTANDING DJANGO

ideas to the group at large for discussion, possibly even deferring to the group if suitable coun-terarguments are raised.

The concept of a BDFL may seem foreign to those readers coming from corporate backgrounds, where design decisions are often made by committees, where majority rules and changes need to go through exhaustive bureaucratic processes. Instead, less direct oversight often leads to small groups of experts in different areas, who are quite capable of acting independently, producing high- quality code. This simple structure allows the process to run more quickly when it needs to, and, more importantly, helps maintain greater consistency within the framework.

In the Python world, Guido van Rossum, creator of Python itself, holds the position of BDFL. For Django, it’s held by two people, each with the official title of co- BDFL: Adrian Holovaty, co- creator of the framework, and Jacob Kaplan- Moss, lead developer of the current work being done with Django. The principles and philosophies found throughout this chapter are generally reflections of the opinions and ideals of the BDFLs.

News and Resources

With a community as passionate and dynamic as Django’s, it’s important to keep up to date on what others are doing, what solutions they’re finding to common problems, new applications that are available and many other things. Given the community’s size and diversity, keeping up may seem like a daunting task, but it’s really quite simple.

The first thing to keep an eye on is the Django weblog4—the official news outlet—which contains news and updates about the framework itself, its development and its use in major endeavors. For example, the Django weblog announces new releases, upcoming development sprints, updates to the project’s Web site and more.

Perhaps more important is the Django community news aggregator,5 which gathers articles from developers around the world, displaying them all in one place. The variety of information available here is much more diverse, as it’s generated by community members, making it an extremely valuable resource. Example content could include new and updated applications, tips and tricks for solving common problems, new Django- powered Web sites and much more.

Reusable Applications

One of the most valuable aspects of Django is its focus on application- based development.

Rather than building each site from scratch, developers are encouraged to write applications for specific purposes, and then combine them to build a site. This philosophy encourages many community members to release their applications to the public, as open source, so that others can benefit from their features and be a part of its success.

While developers are free to host their applications anywhere they wish, many choose Google Code,6 the open source hosting service from Google. It uses Subversion, the same repository software as Django itself, which makes it easy to work with, and incorporates its 4.

5.

6.

C H A P T E R 1

฀ UNDERSTANDING DJANGO

11

own issue tracking system. Many applications7 are hosted there, so it’s definitely a good idea to spend a few minutes looking around to see if someone has already written something you need.

After all, that’s one of the primary goals of open source software: a larger community can produce better, cleaner, more functional code than a smaller group of dedicated programmers.

The Django community both exhibits this behavior and encourages others to take advantage of it.

Getting Help

Despite all the knowledge contained in this and other books, it would be foolish to pretend that every potential situation can be documented ahead of time. What’s more, the documentation that is available isn’t always easy to find or to understand. In any of these cases, you may well find yourself needing to pose your situation to live people, with real- world experience, in hopes that someone can identify the problem and propose a solution.

The first thing to know is that this isn’t a problem. Anyone can run into an unexpected situation, and even the best and brightest of us can get confounded by the simplest of syntax errors. If this happens to you, know that Django’s community is very gentle, and you should definitely ask for help when you need it.

Read the Documentation

The first step when trying to resolve any problem is always to read the official documentation.

It’s quite thorough and updated regularly, as new features are added and existing behaviors are changed. When running into an error, the documentation will help ensure that you’re using Django the way it’s intended.

Once your code matches what the documentation shows to be appropriate, it’s time to look at other common problems.

Check Your Version

As mentioned previously, the official documentation keeps up with Django’s trunk development, so there’s a definite possibility that the documented features don’t match the features available in the code you’re using. While this is more likely to occur if you’re using an official release, it can still happen if you’re tracking trunk, depending on how often you update your local copy.

When you’re tracking trunk, the article on backwards- incompatible8 changes should be considered an essential part of the official documentation. If you run into problems after updating, make sure that none of the features you’re using have changed.

Frequently Asked Questions (FAQ)

After a few years of answering questions using the methods that follow, the Django community has heard a variety of questions that come up on a regular basis. To help answer these 7.

8.

12

C H A P T E R 1

฀ UNDERSTANDING DJANGO

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

9 includes many questions

not related to troubleshooting problems, there are still several common issues listed there.

The Internet Relay Chat (IRC) channel has its own set of questions and answers, and has its

฀

10

Mailing Lists

One of the easiest ways to get help is to ask your question on the django- users mailing list.11

Because it operates over standard email, it’s accessible to everyone, without requiring any special software. Simply join the list and you’ll be able to post your questions for thousands of other users to look at. There are no guarantees, but most questions get answered quickly.

One key advantage of the mailing list is that all conversations are archived for future refer-

฀ ฀

฀ ฀

฀

฀

฀

฀

฀฀

฀

฀

฀

฀

฀

฀ ฀

฀

invaluable resource when you’re trying to track down a problem that might have occurred to someone before.

IRC

If you need answers more quickly, the best option is the Django IRC channel,12 where many knowledgeable members of the Django community are available for direct conversation. It’s a very helpful environment, but you should be prepared to provide specific details about the problem. This may include the exact error traceback, snippets of the models, views and other code that might be involved with the problem.

This code is most often shared using an online “pastebin”—a place to put some code temporarily, for others to look at. Code can be pasted onto a public Web site for a limited time, allowing it to be shared with others. The Django community has its own pastebin called dpaste,13 which is the recommended tool for sharing code with users on IRC.

Now What?

Of course, learning about philosophy and community doesn’t get any code written. It helps to know how to put tools to good use, but that’s nothing without a set of tools to work with.

The next chapter outlines many of the less commonly used tools that Python itself has to offer, while the remaining chapters explore much of Django’s own toolset.

9.

10.

11.

12.

13.

C H A P T E R 2

Django Is Python

Django, like other frameworks, is built on an underlying programming language—in this case, Python—to do its work. Many people who are new to Django are also new to Python, and Python’s natural- feeling syntax combined with Django’s energy- saving features can make Django seem like it uses some kind of meta- language, which isn’t the case.

A proper understanding of what can be done in Django must begin with the knowledge that Django is simply Python, as are all of your applications. Anything that can be done in Python can be done in Django, which makes the possibilities nearly limitless.

This also means that Django applications have access not only to the entire Python standard library, but also to an immense collection of third- party libraries and utilities. Interfaces to some of these are provided along with Django itself, so for many cases, the existing code and documentation will be sufficient to quickly get an application up and running.

Later in this book, some additional utilities are covered, along with some tips on how to integrate them into a Django application. The possibilities aren’t limited to the options outlined in this book, so feel free to look around for Python utilities that will help support your business plan, and use the techniques listed in this book to integrate them into your application.

Though learning Python is beyond the scope of this book, Django uses some of its advanced features. In this chapter, I’ll discuss many of those features to help you understand how Python can contribute to the goal of making things easier for everyone.

How Python Builds Classes

Some of the most advanced Python techniques that Django relies on are related to how Python constructs its classes. This process is often taken for granted by most developers—as well it should be—but since it’s at the heart of Django, it forms the basis of this exploration.

When the Python interpreter encounters a class definition, it reads its contents just as it would any other code. Python then creates a new namespace for the class and executes all the code within it, writing any variable assignments to that new namespace. Class definitions generally contain variables, methods and other classes, all of which are basically assignments to the namespace for the class. However, nearly any valid code is allowed here, including print-ing to console output, writing to log files or even triggering GUI interaction.

Once the contents have finished executing, Python will have a class object that is ordinarily placed in the namespace where it was defined (usually the global namespace for the module), where it is then passed around or called to create instances of that class.

13

14

C H A P T E R 2

฀ DJANGO IS PYTHON

As you can see, code executes within the class definition, with any assigned variables showing up as class attributes once the class is ready.

Building a Class Programmatically

The process described in the preceding section is used for any source- declared class, but the way Python goes about it offers the possibility of something far more interesting. Behind the scenes, details about the class declaration are sent off to the built- in object, which takes care of

creating an appropriate Python object for the class. This happens automatically, for every class, immediately when it finishes parsing the contents of the class declaration.

The constructor for

accepts three arguments, which represent the entire class declaration.

฀

฀

—The name provided for the class, as a string

฀

฀

—A tuple—possibly empty—of classes in the inheritance chain of the class

฀

฀

—A dictionary of the class namespace

NEW-STYLE VS. OLD- STYLE CLASSES

The process described in this section is true for new- style Python classes, a distinction introduced in Python 2.2.1

Old- style classes are no longer recommended for general use and will be going away entirely in Python 3.0, so this section will focus solely on new- style classes. To force a new- style class, simply make sure that the class inherits from the built- in

type somewhere in its inheritance chain.

All the classes Django provides to be subclassed will already derive from

, so any further

derivatives will automatically be new- style classes, without any extra effort on your part. Still, it’s important to keep the difference in mind, so that any custom classes your application may need will exhibit the behaviors outlined in this chapter.

Like any Python object, a new

can be instantiated at any time, from any block of

code. This means that your code can construct a new class based on data collected at runtime.

The following code demonstrates a way to declare a class at runtime, which is functionally equivalent to the example provided in the previous section.

1.

C H A P T E R 2

฀ DJANGO IS PYTHON

15

A WARNING ABOUT TYPE()

Using

manually makes it easy to create classes with duplicate names, and even the module location can be customized by providing a

key in the dictionary in the

argument. Although

these features can be useful, as will be demonstrated later in this book, they can lead to problems with introspection.

You could reasonably have two different classes with the same name and module, but your code won’t be able to tell the difference between them. This may not be a problem in some situations, but it’s something to be aware of.

Metaclasses Change It Up

is actually a “metaclass”—a class that creates other classes—and what we’ve been engag-ing in is called “metaprogramming.”2 In essence, metaprogramming creates or modifies code at runtime rather than at programming time. Python allows you to customize this process by allowing a class to define a different metaclass to perform its work.

If a class definition includes a separate class for its

attribute, that metaclass

will be called to create the class, rather than the built- in object. This allows your code to

read, modify or even completely replace the declared class to further customize its functionality. The

attribute could technically be given any valid Python callable, but most metaclasses are subclasses of

. The metaclass receives the new class as its first argument and provides access to the class object along with the details regarding its declaration.

To help illustrate how the metaclass arguments are derived from a class definition, take the following code as an example.

2.

16

C H A P T E R 2

฀ DJANGO IS PYTHON

Notice that the class wasn’t instantiated at any time; the simple act of creating the class triggered execution of the metaclass. Notice

in the list of attributes: this attribute is

a standard part of all Python classes.

While this example uses the

method to perform special processing on the newly

created class, there is another, somewhat more powerful method called

, with the

potential for a different set of possibilities. As described in later chapters, Django uses when configuring many of its classes.

Using a Base Class with a Metaclass

Metaclasses can be quite useful, but the

variable is an implementation detail,

which shouldn’t need to be part of the process when defining classes. Another problem is that while each class gets processed by the metaclass, they don’t inherit from any concrete class.

This means that any additional functionality, such as common methods or attributes, would have to be provided during metaclass processing in order to be of any use.

With a bit of care, a concrete Python class can use a metaclass to solve both of these problems. Since subclasses inherit attributes from their parents, the variable

is automatically provided for all subclasses of a class that defines it. This is a simple, effective way to provide metaclass processing for arbitrary classes, without requiring that each class define the

attribute. Following the example from the previous section, look what happens when we subclass

.

Notice how the subclass here doesn’t have to worry about the fact that there’s a metaclass in use behind the scenes. By just specifying a base class, it inherits all the benefits. Django uses this behavior to implement one of its most prominent features, described in the next section.

C H A P T E R 2

฀ DJANGO IS PYTHON

17

Declarative Syntax

Some of Django’s more prominent tools feature a “declarative syntax” that is simple to read, write and understand. This syntax is designed to minimize “boilerplate” repetitive syntax and provide elegant, readable code. For example, here’s what a typical Django model and more might look like:

This declarative syntax has become an identifying feature of Django code, so many third- party applications that supply additional frameworks are written to use a syntax similar to that of Django itself. This helps developers easily understand and utilize new code by making it all feel more cohesive. Once you understand how to create a class using declarative syntax, you’ll easily be able to create classes using many Django features, both official and community provided.

Looking at declarative syntax on its own will demonstrate how easy it is to create an entirely new framework for Django that fits with this pattern. Using declarative syntax in your own code will help you and your colleagues more easily adapt to the code, ensuring greater productivity.

After all, developer efficiency is a primary goal of Django and of Python itself.

While the next few sections describe declarative syntax in general, the examples shown are for Django’s object- relational mapper (ORM), detailed in Chapter 3.

Centralized Access

Typically, a package will supply a single module from which applications can access all the necessary utilities. This module may pull the individual classes and functions from elsewhere in its tree, but they will all be collected in one central location.

Once imported, this module provides one class intended as the base class for subclasses based on the framework. Any remaining classes are intended to be used as attributes of the new subclass. Together, these objects will combine to control how the new class will work.

The Base Class

Each feature starts with at least one base class. There may be more, depending on the needs of the framework, but at least one will always be required in order to make this syntax possible.

Without it, every class you ask your users to define will have to include a attribute

explicitly, which is an implementation detail most users shouldn’t need to know about.

In addition to inspecting the defined attributes, this base class will provide a set of methods and attributes that the subclass will automatically inherit. Like any other class, it can be as simple or complex as necessary to provide whatever features the framework requires.

18

C H A P T E R 2

฀ DJANGO IS PYTHON

Attribute Classes

The module supplying the base class will also provide a set of classes to be instantiated, often with optional arguments to customize their behavior and assigned as attributes of a new class.

The features these objects provide will vary greatly across frameworks, and some may behave quite differently from a standard attribute. Often they will combine with the metaclass to provide some additional, behind-the- scenes functionality beyond simply assigning an attribute. Options to these attribute classes are usually read by the metaclass when creating this extra functionality.

For example, Django’s

uses the names and options of field attributes to describe an underlying database table, which can then be created automatically in the database itself.

Field names are used to access individual columns in that table, while the attribute class and options convert native Python data types to the appropriate database values automatically.

More information on how Django handles model classes and fields is available in the next chapter.

Ordering Class Attributes

One potential point of confusion when using declarative syntax is that Python dictionaries are unordered, rather than respecting the order in which their values were assigned. Ordinarily this wouldn’t be a problem, but when inspecting a namespace dictionary it’s impossible to determine the order in which the keys were declared. If a framework needs to iterate through its special attributes, or display them to a user or programmer, it’s often useful to access these attributes in the same order they were defined. This gives the programmer final control over the order of the attributes, rather than some arbitrary ordering decided by the programming language.

A simple solution to this is to have the attributes themselves keep track of the instantiation sequence; the metaclass can then order them accordingly. This process works by having all attribute classes inherit from a particular base class, which can count how many times the class is instantiated and assign a number to each instance.

Object instances have a different namespace than classes, so all instances of this class will have a

, which can be used to sort the objects according to the order in which they were instantiated. This is how Django sorts fields for both models and forms.

Class Declaration

With all of these classes in a module, creating an application class is as simple as defining a subclass and some attributes. Different frameworks will have different names for the attribute C H A P T E R 2

฀ DJANGO IS PYTHON

19

classes, and will have different requirements as to which classes are required or the combinations in which they may be applied. They may even have reserved names that will cause conflicts if you define an attribute with that name, but such problems are rare, and reserving names should generally be discouraged when developing new frameworks for use with this syntax. The general rule is to allow developers to be as flexible as they’d need to be, without the framework getting in the way.

This simple code alone is enough to allow the framework to imbue the new class with a wealth of additional functionality, without requiring the programmer to deal with that process manually. Also note how all the attribute classes are provided from that same base module and are instantiated when assigned to the model.

A class declaration is never limited to only those features provided by the framework.

Since any valid Python code is allowed, your classes may contain a variety of methods and other attributes, intermingled with a framework’s provided features.

Common Duck-Typing Protocols

You’ve probably heard the old adage, “If it walks like a duck, and talks like a duck, it’s a duck.”

Shakespeare played on this idea a bit more romantically when he wrote in Romeo and Juliet,

“that which we call a rose by any other name would smell as sweet.” The recurring theme here is that the name given to an object has no bearing on its true nature. The idea is that, regardless of labels, you can be reasonably sure what something is just by looking at its behavior.

In Python, and in some other languages, this concept is extended to refer to object types.

Rather than relying on some base class or interface to define what an object can do, it simply implements the attributes and methods necessary to behave as expected. A common example of this in Python is a “file- like object,” which is any object that implements at least some of the same methods as a Python file object. In this way, many libraries may return their own objects that can be passed to other functions that expect a file object while retaining special abilities, such as being read- only, compressed, encrypted, pulled from an Internet- connected source or any number of other possibilities.

Also, like interfaces in other languages, Python objects can be more than one type of duck at a time. It’s not uncommon, for instance, to have an object that can behave as a dictionary in some respects, while behaving like a list in others. Django’s object exhibits both

of these behaviors, as well as mimicking an open file object.

In Django, many features utilize duck-typing by not providing a particular base class.

Instead, each feature defines a protocol of sorts, a set of methods and attributes that an object must provide in order to function properly. Many of these protocols are presented in the official Django documentation and this book will cover many more. You will also see some of the special abilities that can be provided by using this technique.

20

C H A P T E R 2

฀ DJANGO IS PYTHON

The following sections describe a few common Python protocols that you’ll see throughout Django, and indeed throughout any large Python library.

Callables

Python allows code to be executed from a number of sources, and anything that can be executed in the same manner as a typical function is designated as callable. All functions, classes and methods are automatically callable, as would be expected, but instances of arbitrary object classes can be designated as callable as well, by providing a single method.

__call__(self[, ...])

This method will be executed when the instantiated object is called as a function. It works just like any other member function, differing only in the manner in which it’s called.

Python also provides a built- in function to assist in the identification of callable objects.

The

function takes a single argument, returning

or

, indicating whether

the object can be called as a function.

C H A P T E R 2

฀ DJANGO IS PYTHON

21

Dictionaries

A dictionary is a mapping between keys and values within a single object. Most programming languages have dictionaries in some form; other languages call them “hashes,” “maps” or

“associative arrays.” In addition to simple access to values by specifying a key, dictionaries in Python provide a number of methods for more fine- grained manipulation of the underlying mapping. To behave even more like a true dictionary, an object may provide other methods, documented in the Python Library Reference.3

__contains__(self, key)

Used by the

operator, this returns

if the specified key is present in the underlying map-

ping, and returns

otherwise. This should never raise an exception.

__getitem__(self, key)

This returns the value referenced by the specified key, if it exists. If the key is not present in the underlying mapping, it should raise a

.

__setitem__(self, key, value)

This stores the specified value to be referenced later by the specified key. This should overwrite any existing value referenced by the same key, if such a mapping is already present.

Dictionaries are also expected to be iterable, with the list of keys used when code loops over a dictionary’s contents. Refer to the upcoming “Iterables” section for more information.

3.

22

C H A P T E R 2

฀ DJANGO IS PYTHON

Files

As mentioned previously, files are a common way to access information, and many Python libraries provide file- like objects for use with other file- related functions. A file- like object doesn’t need to supply all of the following methods, just those that are necessary to function properly. In the case of the file protocol, objects are free to implement read access, write access or both. Not all methods are listed here, but only the most common. A full list of file methods is available in the Python standard library documentation, so be sure to check there for more details.4

read(self, [size])

This retrieves data from the object or its source of information. The optional argument

contains the number of bytes to be retrieved; if omitted, the method should return as many bytes as possible (often the entire file, if available, or perhaps all the bytes available on a network interface).

write(self, str)

This writes the specified

to the object or its source of information.

close(self)

This closes the file so it can no longer be accessed. This can be used to free any memory resources that had been allocated, to commit the object’s contents to disk or simply to satisfy the protocol. Even if this method provides no special functionality, it should be provided to avoid unnecessary errors.

A VERY LOOSE PROTOCOL

File-like objects come in many varieties, because this protocol is one of the loosest defined in all of Python.

There are quite a few features, from buffering output to allowing random access to data, that are inappropriate in some situations, so objects designed for those situations will typically just not implement the corresponding methods. For example, Django’s

object, described in Chapter 7, only allows writes in

sequence, so it doesn’t implement

,

or

, causing errors when used with certain

file- manipulation libraries.

The common approach in situations like this is to simply leave any inappropriate methods unimplemented so that trying to access them raises an

. In other cases, a programmer may

decide it’s more useful to implement them but simply raise a to display a more

descriptive message. Just make sure to always document how much of the protocol your object obeys, so users aren’t surprised if these errors occur while trying to use them as standard files, especially in third- party libraries.

4.

C H A P T E R 2

฀ DJANGO IS PYTHON

23

Iterables

An object is considered iterable if passing it to the built- in returns an iterator.

is often called implicitly, as in a

loop. All lists, tuples and dictionaries are iterable, and any new- style class can be made iterable by defining the following method.

__iter__(self)

This method is called implicitly by

and is responsible for returning an iterator that

Python can use to retrieve items from the object. The iterator returned is often implied by defining this method as a generator function, described in the upcoming “Generators” section.

Iterators

When

is called with an object, it’s expected to return an iterator, which can then be used to retrieve items for that object in sequence. Iterators are a simple method of one- way travel through the available items, returning just one at a time until there are no more to use.

For large collections, accessing items one by one is much more efficient than first gathering them all into a list.

next(self)

The only method required for an iterator, this returns a single item. How that item is retrieved will depend on what the iterator is designed for, but it must return just one item. After that item has been processed by whatever code called the iterator,

will be called again to retrieve

the next item.

24

C H A P T E R 2

฀ DJANGO IS PYTHON

Once there are no more items to be returned,

is also responsible for telling Python to

stop using the iterator and to move on after the loop. This is done by raising the exception. Python will continue calling

until an exception is raised, causing an infinite

loop. Either

should be used to stop the loop gracefully or another exception should be used to indicate a more serious problem.

Note that iterators don’t explicitly need to define

in order to be used properly,

but including that method allows the iterator to be used directly in loops.

Generators

As illustrated in the Fibonacci examples, generators are a convenient shortcut to create simple iterators without having to define a separate class. Python uses the presence of the statement to identify a function as a generator, which makes it behave a bit differently from other functions.

When calling a generator function, Python doesn’t execute any of its code immediately.

Instead, it returns an iterator whose

method will then call the body of the function, up

to the point where the first

statement occurs. The expression given to the

statement

is used as the

method’s return value, allowing whatever code called the generator to get a value to work with.

C H A P T E R 2

฀ DJANGO IS PYTHON

25

The next time

is called on the iterator, Python continues executing the generator function right where it left off, with all of its variables intact. This repeats as long as Python encounters

statements, typically with the function using a loop to keep yielding values.

Whenever the function finishes without yielding a value, the iterator automatically raises to indicate that the loop should be ended and the rest of the code can continue.

Sequences

While iterables simply describe an object that retrieves one value at a time, these values are often all known in advance and collected on a single object. This is a sequence. The most common types are lists and tuples. As iterables, sequences also use the method to

return their values one by one, but since these values are also known in advance, some extra features are available.

__len__(self)

With all the values available, sequences have a specific length, which can be determined using the built- in

function. Behind the scenes,

checks to see if the object it’s

given has a

method and uses that to get the length of the sequence. To accomplish this,

should return an integer containing the number of items in the sequence.

Technically,

doesn’t require that all the values be known in advance, as long as it’s at least known how many values there will be. And since there can’t be partial items—an item either exists or it doesn’t—

should always return an integer. If it doesn’t,

will coerce it to an integer anyway.

__getitem__() and __setitem__()

All the values in a sequence are already ordered as well, so it’s possible to access individual values by their index within the sequence. Since the syntax used for this type of access is identical to that of dictionary keys, Python reuses the same two methods that were previously described for dictionaries. This allows a sequence to customize how individual values are accessed or perhaps restrict setting new values to the sequence, making it read- only.

Augmenting Functions

In addition to standard declarations and calls, Python provides options that allow you to invoke functions in interesting ways. Django uses these techniques to help with efficient code reuse.

You can use these same techniques in your applications as well; they are standard parts of Python.

26

C H A P T E R 2

฀ DJANGO IS PYTHON

Excess Arguments

It’s not always possible to know what arguments will be provided to a function at runtime.

This is often the case in Django, where class methods are defined in source even before a subclass itself is customized appropriately. Another common situation is a function that can act on any number of objects. In still other cases, the function call itself can be made into a sort of API for other applications to utilize.

For these situations, Python provides two special ways to define function arguments, which allow the function to accept excess arguments not handled by the explicitly declared arguments. These “extra” arguments are explained next.

Note that the names

and

are merely Python conventions. As with any function

argument, you may name them whatever you like, but consistency with standard Python idioms makes your code more accessible to other programmers.

Positional Arguments

Using a single asterisk before an argument name allows the function to accept any number of positional arguments.

Python collects the arguments into a tuple, which is then accessible as the variable

. If

no positional arguments are provided beyond those explicitly declared, this argument will be populated with an empty tuple.

Keyword Arguments

Python uses two asterisks before the argument name to support arbitrary keyword arguments.

Notice that

is a normal Python dictionary containing the argument names and values. If no extra keyword arguments are provided,

will be an empty dictionary.

C H A P T E R 2

฀ DJANGO IS PYTHON

27

Mixing Argument Types

Arbitrary positional and keyword arguments may be used with other standard argument declarations. Mixing them requires some care, as their order is important to Python. Arguments can be classified into four categories, and while not all categories are required, they must be defined in the following order, skipping any that are unused.

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

This order is required because

and

only receive those values that couldn’t

be placed in any other arguments. Without this order, when you call a function with positional arguments, Python would be unable to determine which values are intended for the declared arguments and which should be treated as an excess positional argument.

Also note that, while functions can accept any number of required and optional arguments, they may only define one of each of the excess argument types.

Passing Argument Collections

In addition to functions being able to receive arbitrary collections of arguments, Python code may call functions with any number of arguments, using the asterisk notation previously described. Arguments passed in this way are expanded by Python into a normal list of arguments, so that the function being called doesn’t need to plan for excess arguments in order to be called like this. Any Python callable may be called using this notation, and it may be combined with standard arguments using the same ordering rules.

28

C H A P T E R 2

฀ DJANGO IS PYTHON

As illustrated in the final lines of this example, take special care if explicitly passing any keyword arguments while also passing a tuple as excess positional arguments. Since Python will expand the excess arguments using the ordering rules, the positional arguments would come first. In the example, the last two calls are identical, and Python can’t determine which value to use for .

Decorators

Another common way to alter the way a function behaves is to “decorate” it with another function. This is also often called “wrapping” a function, as decorators are designed to execute additional code before or after the original function gets called.

The key principle behind decorators is that they accept callables and return new callables.

The function returned by the decorator is the one that will be executed when the decorated function is called later. Care must be taken to make sure that the original function isn’t lost in the process, as there wouldn’t be any way to get it back without reloading the module.

Decorators can be applied in a number of ways, either to a function you’re defining directly or to a function that was defined elsewhere. As of Python 2.4, decorators on newly defined functions can use a special syntax. In previous versions of Python, a slightly different syntax is necessary, but the same code can be used in both cases; the only difference is the syntax used to apply the decorator to the intended function.

C H A P T E R 2

฀ DJANGO IS PYTHON

29

The older syntax in this example is another technique for decorating functions, which can be used in situations where the syntax isn’t available. Consider a function that’s been declared elsewhere but would benefit from being decorated. Such a function can be passed to a decorator, which then returns a new function with everything all wrapped up. Using this technique, any callable, regardless of where it comes from or what it does, can be wrapped in any decorator.

Decorating with Extra Arguments

Sometimes, a decorator needs additional information to determine what it should do with the function it receives. Using the older decorator syntax, or when decorating arbitrary functions, this task is fairly easy to perform. Simply declare the decorator to accept additional arguments for the required information so they can be supplied along with the function to be wrapped.

However, the Python 2.4 decorator syntax complicates things. When using this new syntax, the decorator always receives just one argument: the function to be wrapped. There is a way to get extra arguments into decorators, but first we’ll need to digress a bit and talk about

“partials.”

Partial Application of Functions

Typically, functions are called with all the necessary arguments at the time the function should be executed. Sometimes, however, arguments may be known in advance, long before the function will be called. In these cases, a function can have one or more of its arguments applied beforehand so that the function can be called with fewer arguments.

For this purpose, Python 2.5 includes the

object as part of its

module. It

accepts a callable along with any number of additional arguments and returns a new callable, which will behave just like the original, only without having to specify those preloaded arguments at a later point.

30

C H A P T E R 2

฀ DJANGO IS PYTHON

For versions of Python older than 2.5, Django provides its own implementation of in the

function, which lives in

. This function works on Python

2.3 and greater.

Back to the Decorator Problem

As mentioned previously, decorators using the Python 2.4 syntax present a problem if they accept additional arguments, since that syntax only provides a single argument on its own.

Using the partial application technique, it’s possible to preload arguments even on a decorator.

Given the decorator described earlier, the following example uses (described in Chapter 9)

to provide arguments for decorators using the newer Python 2.4 syntax.

This is still rather inconvenient, since the function needs to be run through every

time it’s used to decorate another function. A better way would be to supply this functionality directly in the decorator itself. This requires some extra code on the part of the decorator, but including that code makes it easier to use.

The trick is to define the decorator inside another function, which will accept the arguments. This new outer function then returns the decorator, which is then used by Python’s standard decorator handling. The decorator, in turn, returns a function that will be used by the rest of the program after the decoration process is complete.

As this is all fairly abstract, consider the following, which provides the same functionality as in previous examples but without relying on

, making it easier to deal with.

C H A P T E R 2

฀ DJANGO IS PYTHON

31

This technique makes the most sense in situations where arguments are expected. If the decorator is applied without any arguments, parentheses are still required in order for it to work at all properly.

The second example fails because we didn’t first call

. Thus, all subsequent calls

to

send their arguments to

instead of

. Since this is a mismatch, Python

throws an error. This situation can be a bit difficult to debug because the exact exception that will be raised will depend on the function being wrapped.

32

C H A P T E R 2

฀ DJANGO IS PYTHON

A Decorator With or Without Arguments

One other option for decorators is to provide a single decorator that can function in both of the previous situations: with arguments and without. This is more complex but worth exploring.

The goal is to allow the decorator to be called with or without arguments so it’s safe to assume that all arguments are optional; any decorator with required arguments can’t use this technique. With that in mind, the basic idea is to add an extra optional argument at the beginning of the list, which will receive the function to be decorated. Then, the decorator structure includes the necessary logic to determine whether it’s being called to add arguments or to decorate the target function.

This requires that all arguments passed to the decorator be passed as keyword arguments, which generally makes for more readable code. One downside is how much boilerplate would have to be repeated for each decorator that uses this approach.

Thankfully, like most boilerplate in Python, it’s possible to factor it out into a reusable form, so new decorators can be defined more easily, using yet another decorator. The following function can be used to decorate other functions, providing all the functionality necessary to accept arguments or it can be used without them.

C H A P T E R 2

฀ DJANGO IS PYTHON

33

This makes the definition of individual decorators much simpler and more straightforward. The resulting decorator behaves exactly like the one in the previous example, but it is able to be used with or without arguments. The most notable change that this new technique requires is that the real decorator being defined will receive the following three values:

฀

฀

—The function that was decorated using the newly generated decorator

฀

฀

—A tuple containing positional arguments that were passed to the function

฀

฀

—A dictionary containing keyword arguments that were passed to the function An important thing to realize, however, is that the

and

that the decorator

receives are passed as positional arguments, without the usual asterisk notation. Then, when passing them on to the wrapped function, the asterisk notation must be used to make sure the function receives them without having to know about how the decorator works.

34

C H A P T E R 2

฀ DJANGO IS PYTHON

Descriptors

Ordinarily, referencing an attribute on an object accesses the attribute’s value directly, without any complications. Getting and setting attributes directly affects the value in the object’s instance namespace. Sometimes, additional work has to be done when accessing these values.

฀

฀

฀

฀

฀ ฀

฀

฀

฀ ฀ ฀

฀ ฀

฀

฀

฀

฀ ฀

฀

฀ ฀ ฀

฀

฀ ฀

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀ ฀ ฀฀

฀

฀

฀

฀

฀ ฀ ฀

In some programming languages, this type of behavior is made possible by creating extra instance methods for accessing those attributes that need it. While functional, this approach leads to a few problems. For starters, these behaviors are typically more associated with the type of data stored in the attribute than some aspect of the instance it’s attached to. By requiring that the object supply additional methods for accessing this data, every object that contains this behavior will have to provide the necessary code in its instance methods.

One other significant issue is what happens when an attribute that used to be simple sud-denly needs this more advanced behavior. When changing from a simple attribute to a method, all references to that attribute also need to be changed. To avoid this, programmers in these languages have adopted a standard practice of always creating methods for attribute access so that any changes to the underlying implementation won’t affect any existing code.

It’s never fun to touch that much of your code for a change to how one attribute is accessed, so Python provides a different approach to the problem. Rather than requiring the object to be responsible for special access to its attributes, the attributes themselves can provide this behavior. Descriptors are a special type of object that, when attached to a class, can intervene when the attribute is accessed, providing any necessary additional behavior.

C H A P T E R 2

฀ DJANGO IS PYTHON

35

Creating a descriptor is as simple as creating a standard new- style class (by inheriting from

), and specifying at least one of the following methods. The descriptor class can include any other attributes or methods as necessary to perform the tasks it’s responsible for, while the following methods constitute a kind of protocol that enables this special behavior.

__get__(self, instance, owner)

When retrieving the value of an attribute (

), this method will be called

instead, allowing the descriptor to do some extra work before returning the value. In addition to the usual

representing the descriptor object, this getter method receives two arguments.

฀

฀

—The instance object containing the attribute that was referenced. If the attribute was referenced as an attribute of a class rather than an instance, this will be

.

฀

฀

—The class where the descriptor was assigned. This will always be a class object.

The

argument can be used to determine whether the descriptor was accessed from an object or its class. If

is

, the attribute was accessed from the class rather

than an instance. This can be used to raise an exception if the descriptor is being accessed in a way that it shouldn’t.

Also, by defining this method, you make the descriptor responsible for retrieving and returning a value to the code that requested it. Failing to do so will force Python to return its default return value of

.

Note that, by default, descriptors don’t know what name they were given when declared as attributes. Django models provide a way to get around this, which is described in Chapter 3, but apart from that, descriptors only know about their data, not their names.

__set__(self, instance, value)

When setting a value to a descriptor (

), this method is called so that a more

specialized process can take place. Like

, this method receives two arguments in addi-

tion to the standard

.

฀

฀

—The instance object containing the attribute that was referenced. This will never be

.

฀

฀

—The value being assigned.

Also note that the

method of descriptors will only be called when the attribute is assigned on an object and will never be called when assigning the attribute on the class where the descriptor was first assigned. This behavior is by design, and prohibits the descriptor from taking complete control over its access. External code can still replace the descriptor by assigning a value to the class where it was first assigned.

Also note that the return value from

is irrelevant. The method itself is solely respon-

sible for storing the supplied value appropriately.

36

C H A P T E R 2

฀ DJANGO IS PYTHON

Keeping Track of Instance Data

Since descriptors short- circuit attribute access, you need to take care when setting values on the attached object. You can’t simply set the value on the object using

; attempting to

do so will call the descriptor again, resulting in infinite recursion.

Python provides another way to access an object’s namespace: the attribute.

Available on all Python objects,

is a dictionary representing all values in the object’s

namespace. Accessing this dictionary directly bypasses all of Python’s standard handling with regard to attributes, including descriptors. Using this, a descriptor can set a value on an object without triggering itself. Consider the following example.

Unfortunately, this technique requires giving the attribute’s name to the descriptor explicitly. You can work around this with some metaclass tricks; Django’s model system (discussed in Chapter 3) shows one possible workaround.

Introspection

Many Python objects carry metadata beyond the code they execute. This information can be quite useful when working with a framework or writing your own.

Python’s introspection tools can help greatly when trying to develop reusable applications, as they allow Python code to retrieve information about what a programmer wrote without requiring the programmer to write it all over again.

Some of the features described in this section rely on a powerful standard library module,

. The

module provides convenient functions to perform advanced introspection.

Only some of

’s many uses will be detailed here, as they hold the most value to applications written using Django. For full details of the many other options available in this module, consult the Python standard library documentation.5

5.

C H A P T E R 2

฀ DJANGO IS PYTHON

37

MORE ON OLD- STYLE CLASSES

The examples shown in this section are all for new- style classes, which, as described earlier in this chapter, will behave differently from old- style classes, especially with regards to introspection. The exact differences are beyond the scope of this book, since the usual recommendation is to simply use new- style classes.

If any of your code seems to behave differently than what’s described here, make sure that all your classes inherit from

, which will make them into proper new- style classes.

Common Class and Function Attributes

All classes and functions provide a few common attributes that can be used to identify them.

฀

฀

—The name that was used to declare the class or function

฀

฀

—The docstring that was declared for the function

฀

฀

—The import path of the module where the class or function was declared In addition, all objects contain a special attribute,

, which is the actual class

object used to create the object. This attribute can be used for a variety of purposes, such as testing to see whether the class provided a particular attribute or if it was set on the object itself.

Identifying Object Types

Since Python uses dynamic typing, any variable could be an object of any available type. While the common principle of duck-typing recommends that objects simply be tested for support of a particular protocol, it’s often useful to identify what type of object you’re dealing with.

There are a few ways to handle this.

Getting Arbitrary Object Types

It’s easy to determine the type of any Python object using the built- in described earlier.

Calling

with a single argument will return a

object, often a class, which was instanti-

ated to produce the object.

38

C H A P T E R 2

฀ DJANGO IS PYTHON

This approach usually isn’t the best way to determine the type of an object, particularly if you’re trying to decide what branch of execution to follow based on an object’s type. It only tells you the one specific class that’s being used, even though subclasses should likely be considered for the same branch of execution. Instead, this approach should be used in situations where the object’s type isn’t necessary for a decision but rather is being output somewhere, perhaps to the user or a log file.

For example, when reporting exceptions, it’s quite useful to include the exception’s type along with its value. In these situations,

can be used to return the class object, and its

attribute can then be included in the log, easily identifying the exception’s type.

Checking for Specific Types

More often, you’ll need to check for the influence of a particular type, whether a class descends from it or whether an object is an instance of it. This is a much more robust solution than using

, as it takes class inheritance into account when determining success or failure.

Python provides two built- in functions for this purpose.

฀

฀

—Returns

if

and

are the same, or if

inherits

from

somewhere in its ancestry

฀

฀

—Tests if the object is an instance of

or any of its ancestors

C H A P T E R 2

฀ DJANGO IS PYTHON

39

There’s a clear relationship between

and

:

is equivalent to

.

Function Signatures

As described earlier in this chapter, Python functions can be declared in a number of ways, and it can be quite useful to have access to information about their declarations directly inside your code.

Of particular importance when inspecting functions is

, a function

that returns information about what arguments a function accepts. It accepts a single argument, the function object to be inspected, and returns a tuple of the following values:

฀

฀

—A list of all argument names specified for the function. If the function doesn’t accept any arguments, this will be an empty list.

฀

฀

—The name of the variable used for excess positional arguments, as described previously. If the function doesn’t accept excess positional arguments, this will be

.

฀

฀

—The name of the variable used for excess keyword arguments, as described previously. If the function doesn’t accept excess keyword arguments, this will be

.

฀

฀

—A tuple of all default values specified for the function’s arguments. If none of the arguments specify a default value, this will be

rather than an empty tuple.

Together, these values represent everything necessary to know how to call the function in any way possible. This can be useful when receiving a function and calling it with just the arguments that are appropriate for it.

Handling Default Values

As the previous example illustrates, default values are returned in a separate list from argument names, so it may not seem obvious how to tell which arguments specify which defaults.

However, there’s a relatively simple way to handle this situation, based on a minor detail from the earlier discussion of excess arguments: required arguments must always be declared before any optional arguments.

This is key because it means the arguments and their defaults are specified in the order they were declared in the function. So in the previous example, the fact that there are two default values means that the last two arguments are optional, and the defaults line up with them in order. The following code could be used to create a dictionary mapping the optional argument names to the default values declared for them.

40

C H A P T E R 2

฀ DJANGO IS PYTHON

Docstrings

As mentioned previously, classes and functions all have a special attribute, which contains the actual string specified as the code’s docstring. Unfortunately, this is formatted exactly as it was in the original source file, including extra line breaks and unnecessary indentation.

To format docstrings in a more readable manner, Python’s

module provides

another useful function,

. It removes unnecessary line breaks, as well as any extra indentation that was a side effect of where the docstring was written.

The removal of indentation merits a bit of explanation. Essentially, finds the

leftmost non- whitespace character in the string, counts up all the whitespace between that character and the start of the line it’s in, and removes that amount of whitespace from all the other lines in the docstring. This way, the resulting string is left- justified but retains any additional indents that exist for the sake of formatting the documentation.

In situations where docstrings should be displayed to users, such as automated documentation or help systems,

provides a useful alternative to the raw docstring.

C H A P T E R 2

฀ DJANGO IS PYTHON

41

Applied Techniques

There are innumerable combinations of Python features that can be used to accomplish a vast multitude of tasks, so the few shown here should by no means be considered an exhaustive list of what can be done by combining the many features of Python. However, these are useful tactics in terms of Django, and serve as a solid basis for the other techniques listed throughout this book.

Tracking Subclasses

Consider an application that must, at any given time, have access to a list of all subclasses of a particular class. Metaclasses are a terrific way to go about this, but they have one problem.

Remember, each class with a

attribute will be processed, including this new

base class, which doesn’t need to be registered (only its subclasses should be registered).

Extra handling is required to make this possible, but doing so is fairly straightforward and will provide great benefits:

The metaclass performs two functions. First, the

block makes sure that the parent

class,

, has already been defined. If it hasn’t been, a

is raised, indicating

that the metaclass is currently processing

itself. Here, more processing could be

done for

, but the example simply ignores it, allowing it to bypass the registration.

In addition, the

clause makes sure that another class hasn’t specified

explicitly as its

attribute. The application only wants to register subclasses of

, not other classes that might not fit the proper requirements for the application.

42

C H A P T E R 2

฀ DJANGO IS PYTHON

Any application author who wants to use a declarative syntax similar to Django’s could use this technique to provide a common base class, from which specific classes can be created.

Django uses this process for both its models and its forms so that its declarative syntax can be fairly consistent throughout the framework.

If Python makes it through those tests without bailing out early, the class is added to the registry, where all subclasses of

can be retrieved at any time. Any subclasses of

will show up in this registry, regardless of where the subclass is defined. Executing the class definition will be sufficient to register it; that way, the application can import any modules that might have the necessary classes and the metaclass does the rest.

Though its registry provides many more features than a simple list, Django uses an extension of this technique to register models, since they must each extend a common base class.

A Simple Plugin Architecture

In reusable applications, it’s usually desirable to have a well- defined core set of features, combined with the ability to extend those features through the use of plugins. While this may seem like a tall order that might require extensive plugin architecture libraries, it can be done quite simply and entirely in your own code. After all, a successful, loosely- coupled plugin architecture comes down to providing just three things:

฀

฀ ฀

฀

฀

฀ ฀

฀ ฀

฀

฀

฀ ฀

฀ ฀

฀

฀

฀ ฀

฀

฀

฀ ฀

฀

฀ ฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀ ฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

the plugins should be registered and accessed

Armed with this simple list of requirements and a healthy understanding of what Python has to offer, a few simple lines of code can combine to fulfill these requirements.

That’s all it takes to get the whole thing working, keeping track of registered plugins, storing them in a list on the

attribute. All that’s left is to work out how to achieve each of the points listed earlier. For the following examples, we’ll create an application for validating the strength of a user’s password.

The first step will be the neutral access point, which I’ll call a mount point, from which each side of the equation can access the other. As mentioned before, this relies on metaclasses, so that’s a good place to start.

C H A P T E R 2

฀ DJANGO IS PYTHON

43

You could add more to this if you want, but what’s here is the only part that’s essential to get the process working properly. When looking to add more to it, just know that individual plugins will subclass it and will thus inherit anything else you define on this class. It’s a handy way of providing additional attributes or helper methods that would be useful for all the plugins to have available. Individual plugins can override them anyway, so nothing would be set in stone.

Also note that the plugin mount point should contain documentation relating to how plugins will be expected to behave. While this isn’t expressly required, it’s a good practice to get into, as doing so will make it easier for others to implement plugins. The system only works if all the registered plugins conform to a specified protocol; make sure it’s specified.

Next, set up your code to access any plugins that were registered, using them in whatever way makes sense for the application. Since the mount point already maintains its own list of known plugins, all it takes is to cycle through the plugins and use whatever attributes or methods are appropriate for the task at hand.

44

C H A P T E R 2

฀ DJANGO IS PYTHON

These examples are a bit more complicated than most, since they require error handling, but it’s still a very simple process. Simply iterating over the list will provide each of the plugins for use. All that’s left is to build some plugins to provide this validation behavior.

Yes, it really is that easy! Here’s how these plugins would look in practice.

Now What?

With a solid understanding of what Python has to offer, you’re ready to dive into some of the ways Django uses these tools for many of its features and how you can apply the same techniques in your own code. Forming the foundation of most Django applications, models make use of many of these advanced Python features.

C H A P T E R 3

Models

Data is at the center of most modern Web applications, and Django aims to provide support for a variety of data structures and persistence options. Models are the primary aspect of the traditional MVC model that Django maintains intact. Models are an essential part of any application that needs to persist data across multiple requests, sessions or even server instances.

Django models are defined as standard Python classes, with a wealth of additional features added in automatically. Behind the scenes, an object- relational mapper (ORM) allows these classes and their instances access to databases. Without this ORM, developers would be required to deal with the database directly, using Structured Query Language (SQL), the standard way to access content in databases.

The primary goal of SQL is to describe and access the relationships that are stored in a relational database. SQL does not generally provide high- level relationships for applications, so most applications include handwritten SQL for data activities. This is definitely possible, but it tends to lead toward lots of repetition, which in and of itself violates the DRY principle outlined in Chapter 1.

These bits of SQL littered throughout an application’s code quickly become unmanageable, especially since the programmers who have to manage the code aren’t typically experts in relational databases. That also means that these databases are quite prone to bugs, which are often troublesome to track down and fix.

That still doesn’t factor in the biggest issue of all: security. SQL injection1 attacks are a common way for malicious attackers to access or even modify data they shouldn’t have access to.

This occurs when handwritten SQL doesn’t take appropriate precautions with regard to the values that are passed into the database. The more SQL statements that are written by hand, the more likely they are to be susceptible to this type of attack.

All of these problems are extremely common in Web development, regardless of language, and ORMs are a common way for frameworks to mitigate them. There are other ways to avoid some of these problems, such as SQL injection, but Django’s ORM was written with these concerns in mind and handles much of it behind the scenes. By accessing data using standard Python objects, the amount of SQL is minimized, reducing the opportunity for problems to crop up.

1.

45

46

C H A P T E R 3

฀ MODELS

How Django Processes Model Classes

Described in Chapter 2, one of Django’s most recognizable features is its declarative syntax for model definitions. With this, model definitions can be simple and concise, while still providing a vast array of functionality. The basic process of using metaclasses for declarative syntax is described in detail in Chapter 2, but there are more specific steps taken when handling models, which deserve some extra attention.

The metaclass responsible for processing model definitions is

, living at

. This provides a few key features, listed here in the order in which the actions are performed.

1. A new class is generated to be used for the actual model, preserving the module location where the original model was defined.

2.

options are pulled out of the model and placed in a special object, which

is described in more detail later in this chapter.

3. Two special exception classes,

and

, are created

and customized for the new model.

4. If a custom

wasn’t provided for the model, it’s determined based on the module where it was declared.

5. If the model was already defined—which can happen because of differences in how the module was imported at different stages—the existing model is retrieved from the application cache and returned, making sure that the same class object is always used.

6. Attributes and methods defined on the original model are added to the newly- created model class.

7. Settings from inherited parent models are set on the new model.

8. The new model is registered with the application cache for future reference.

9. The newly- created model is returned to be used in place of the class that was defined in the source file.

Abstract models and inherited models are special cases, where not all of these actions occur. Specific differences for these cases are covered later in this chapter.

Setting Attributes on Models

Python provides useful tools for getting and setting attributes on objects without knowing the name in advance, but while

and

represent the standard way of accessing

attributes on objects, one of Django’s hooks for model fields requires some additional handling. Django provides a class method,

, on all of its models, which should be

used as a substitute for

.

The syntax and semantics of

are slightly different than the traditional func-

tions. It’s actually a class method, rather than a built- in or even module- level function, which means the class is provided implicitly, rather than being an explicit first argument. This method checks the provided value for the presence of a

method, and calls it

if it exists. Otherwise, the standard

function is used to add the value to the model.

These behaviors are mutually exclusive; only one will happen in a given call.

C H A P T E R 3

฀ MODELS

47

It’s important to realize that this isn’t just for Django’s own internal code. If an application has need to add arbitrary objects as attributes to models, they must call

. This way,

developers working with the application can pass any object in, and be assured that it will be handled the same as if it had been applied directly on the model’s class definition.

This whole process changes what the classes look like when using the introspection techniques described in Chapter 2. In order to determine the declared fields, the database table being used or the display name for the model, some additional knowledge is required.

Getting Information About Models

Once a model has been processed by Python, along with Django’s metaclass, its

original structure can still be determined by using an attribute that exists on every Django model and its instances called

.

There are a number of attributes available on

, which combine to describe the model,

how it was defined and what values were provided to customize its behavior. These can also be classified into two separate groups: attributes that are determined by looking at the actual structure of the original class and those that are specified directly as part of a class defined

inside the model.

REGARDING THE STABILITY OF _META

Names beginning with underscores typically refer to private attributes that shouldn’t be used directly.

They’re often used internally by functions and methods that are more public in nature, and are generally accompanied by warnings about likely changes and undocumented behavior. In most cases, these warnings are valid; programmers usually write tools for their own use, and find little need in documenting their behavior or securing their longevity.

However,

is a bit of an exception to the rule. While it is indeed part of a private API, which isn’t necessary for the vast majority of situations, it shares something with many tools described in this book; it can prove extremely useful if understood and used properly. In fact, goes one better, by being quite stable and highly

unlikely to change without considerable effort to keep it backwards- compatible. It’s the foundation of much of Django’s own internal code, and is already being accessed directly by many third- party applications as well.

So, while names beginning with underscores do generally spell danger, potential incompatibilities and lack of support, you can rely on

quite safely. Just make sure to keep up with Django’s list of backwards- incompatible changes. Anything new that would break will be listed there.

Class Information

While most of the basic introspection techniques covered in Chapter 2 apply to Django models, there are a number of details that are also made available on the attribute. Most of

this is information Django itself needs in order to properly deal with models, but as with many other features, it can be quite useful for other applications as well.

One important distinction to make with models is whether they’re “installed” or not. This means checking whether the application that contains them is listed in the site’s setting. Many Django features, such as

and the built- in admin interface, require an appli-

cation to be listed in

in order to be located and used.

48

C H A P T E R 3

฀ MODELS

If an application is designed to accept any Django model directly, rather than iterating through

, it will often need some way to determine whether the model is properly installed. This is necessary in case the application needs to handle models differently, depending on whether database operations should be performed on the table, for instance.

For this purpose, Django provides the

attribute, which will be

only if the model

belongs to an application listed in

, and

otherwise.

There are two other attributes of model- level information that are commonly useful to application developers. As described in Chapter 2, all Python classes provide an easy way to get the name of the class and the module where it was defined, using the and

attributes, respectively. However, there are some situations where these can be misleading.

Consider a situation where a model may be subclassed without inheriting all the Django-specific model inheritance processing. This requires a bit of tweaking with metaclasses, but can prove useful for solving certain types of problems. When doing this, the and

attributes will refer to the child class, rather than the actual model that sits underneath.

Often, this is the desired behavior, as it’s just how standard Python works, but when attempting to interact with the Django model, or other areas of Django that may need to work with it, it may be necessary to know the details of the model itself, rather than the child class.

One way to go about this would be to use class introspection to get the various parent classes that are in use, checking each to see if it’s a Django model.

This is a fairly unsightly process that takes time to code, time to execute, makes maintenance and readability more difficult and adds boilerplate if it needs to be done often.

Thankfully, Django provides two additional attributes on

to greatly simplify this. The

attribute contains the

attribute from the underlying model, while

pertains to the

attribute of the model.

Field Definitions

A major challenge involved in using and manipulating Django models is the process of locat-ing and using fields that are defined for them. Django uses the technique

described in Chapter 2 to keep track of the order of fields, so they can be placed inside a list for future reference. This list is stored in the

attribute of the model’s

attribute.

As a list, this can be iterated to retrieve all the field objects in order, which is extremely useful when looking to deal with models generically. As described later in this chapter, field objects have attributes containing all the options that were specified for them, so each item in the list can provide a wealth of information.

With this, we can create a custom form or template output, or any other feature that needs to work with fields on an arbitrary model. Consider the following example, which prints out the display names and current values for each field in a given object, without having to know in advance what model is being used.

C H A P T E R 3

฀ MODELS

49

Going about it this way allows the function to ignore the details of the model behind the object. As long as it’s an instance of a proper Django model, the attribute will be

available and all the fields will be accessible in this way. Since Django automatically adds an to any model that doesn’t declare a primary key, the created will also be

included in the

list.

While being able to iterate through a list is great for those situations where all the fields will be taken into account, sometimes only a single field is needed, and the name of that field is known in advance. Since

is a list instead of a dictionary, the only way to get a field by its name would be to loop over the fields, checking each to see if its name matches.

To cater to this need, Django provides a utility method,

. By provid-

ing the field name to the

, it’s easy to retrieve just the specified field. If no

field with that name exists, it will raise a

exception, which lives at

.

To get a better understanding of how these methods work together to identify the fields that were declared on a model, consider the following model declaration.

Then, the model could be inspected to get more information about this declaration, without having to know what it looked like in advance.

Primary Key Fields

Any field can be specified as a primary key, by setting

in the field’s defini-

tion. This means that if code is to handle a model or a model instance without prior knowledge of its definition, it’s often necessary to identify which field was defined as a primary key.

Much like getting a field by name, it would be possible to just iterate over all the fields, looking for one with its

attribute set to

. After all, Django only allows one field

to be specified as a primary key. Unfortunately, this again introduces a fair amount of boilerplate that slows things down and makes it more difficult to maintain.

50

C H A P T E R 3

฀ MODELS

To simplify this task, Django provides another

attribute,

, which contains the field

object that will be used as the primary key for the model. This is also faster than iterating over all the fields, since

is populated once, when the model is first processed. After all, Django needs to determine whether it needs to provide an implicit primary key. The attribute

is also used to enable the

shortcut property on model instances, which returns the primary key value for an instance, regardless of which field is the primary key.

Typically, models don’t need to declare an explicit primary key, and can instead let Django create one automatically. This can be a useful way to avoid repeating such a common declaration, while still allowing it to be overridden if necessary. One potential problem with this, however, is the task of determining whether a model was given an automatic field, and what that field looks like.

It’s possible to make certain assumptions about a model, based on how Django provides this automatic field, and what it would typically look like. However, it’s easy to create a custom field that looks a lot like the implicit field, and it’d be very difficult to tell the difference if your code only looks at its structure and options.

Instead, Django provides two attributes on the

attribute that help with this situation.

The first,

, is

if the model lets Django provide an

field implicitly. If

it’s

, the model has an explicit primary key, so Django didn’t have to intervene.

The second attribute related to the automatic primary key field is

,

which will be the actual field object Django provided for use as the primary key. This will always be an

, and will always be configured the same way for all models that use it.

It’s important to look at this attribute instead of making assumptions about the field’s structure, in case Django makes any changes in the future. It’s an easy way to help make sure your application keeps working properly in the future.

Configuration Options

In addition to providing access to the fields declared on the model, also acts as a container for all the various options that can be set on a model using the inner class. These

options allow a model to control a variety of things, such as what the model is named, what database table it should use, how records should be ordered and a number of others.

These options all have defaults, so that even those attributes that aren’t specified on the model are still available through the

attribute. The following is a list of the many options

that are available in this way, along with their default values and a brief description what the option is intended for.

฀

฀

—A Boolean that indicates whether the model was defined as abstract, a process which is described in more detail in Django’s model inheritance documentation.2

The default value is

.

฀

฀

—A string containing the name Django uses to recognize the application where the model was defined. It’s easiest to understand what this means by looking at the default value, which is the name of the module containing the the model

is specified in. For a model located at

, the

would be

.

2.

C H A P T E R 3

฀ MODELS

51

฀

฀

—The name of the database table that Django will use to store and retrieve data for the model. If not defined explicitly, it’s determined as a function of the model’s name and location. That is, the

for a model called

with an

of

would be

.

฀

฀

—In the case of Oracle, and perhaps other database backends in the future, tables can be placed in different parts of the disk, or different disks entirely. By default, this is simply an empty string, which tells the database to store the table in its default location. This option is ignored for backends that don’t support it.

฀

฀

—The name of a date- based field, such as a

or a

,

which should be used to determine the most recent instance of a model. If not provided, this will be an empty string.

฀

฀

—An instance of a field relating to another model, which is used when ordering instances of this model. This defaults to

, which implies that the

model’s ordering is determined solely by fields within the model itself, rather than any related models.

฀

฀

—A tuple containing the names of fields to be used when ordering instances of the model. By default, this is an empty tuple, which relies on the database to determine the ordering of model instances.

฀

฀

—A sequence of tuples of additional permissions to be added to the model.

Each tuple in the sequence contains two values, the first being the name of the permission to be used in code and in the database, and the second being the text to be displayed in the admin interface when selecting permissions for a user or group.

฀

฀

—A sequence of tuples indicating any groups of fields which must, when combined, be used in only one record in the database. Each tuple in the sequence contains the names of the fields that must be unique together for a particular index.

Multiple tuples don’t have any relation to each other; they each represent a separate index at the database level.

฀

฀

—The display name for a single instance of the model. By default, this is determined by the name of the class itself, by splitting up each capitalized portion into a separate uncapitalized word;

would become

, while

would become

. This will be translated into the correct language by

default, when accessed.

฀

฀

—The display name for multiple instances of the model. By default, this will be simply the

with an “s” at the end.

would be

and

would be

. This will be translated into the correct lan-

guage by default, when accessed.

฀

฀

—The raw, untranslated version of

. Occasionally, it’s

necessary to use the same display name for everyone, regardless of translation. This is particularly useful when storing it away in the cache or database for later access, especially if it’ll be translated at a later point in time.

52

C H A P T E R 3

฀ MODELS

Accessing the Model Cache

Once models have been processed by the

metaclass, they’re placed in a global reg-

istry called

, located at

. This is instantiated automatically,

immediately when the module is imported, and is accessed using the name

. This special

cache provides access to the various models that are known to Django, as well as installs new ones if necessary.

Because

handles registration of new models whenever the class is processed by Python, the models it contains aren’t guaranteed to be part of applications present in the setting. This fact makes it even more important to remember that the attribute on the model contains an

attribute indicating whether the model belongs

to an installed application.

Whenever code accesses one of the features in this section, will automatically

load applications that are listed in

, making sure that whenever some of the

features are accessed, the cache includes all applications and models that should be made available. Without this, the results of these methods would be wildly unpredictable, based solely on which applications were loaded in which order.

As might seem obvious, the application cache can only be fully populated once all the applications have been loaded. Therefore, if an application’s

makes any calls to

as

part of this loading process, it’s possible that the cache might not be fully populated yet.

To protect against this problem,

provides a method to determine whether the

cache itself has been populated and is ready to be accessed. Calling will return

or

depending on whether all of the installed applications have been processed correctly. Using this, applications that could benefit from having their own cache of known models can check if this cache is available for that purpose. If so, it can use this cache directly, while if not, it can manually determine what it needs to know.

Retrieving All Applications

When looking to introspect a site’s contents, it’s also very useful to look at the structure of applications themselves. After all, looking at models is only useful if there are models to look at, and sometimes it’s necessary to just collect all the models currently in use. It’s also useful to have them arranged by the application that declares them. Django already needs to have this information handy, so

is designed to specifically manage this information.

HOW DOES DJANGO SEE APPLICATIONS?

One important thing to keep in mind is that Django needs an object to use as a reference for the application. A Django application is essentially a standard Python package, which is just a collection of modules contained in a single folder. While Python provides an object to use as a reference for individual modules, it doesn’t offer anything to refer to a package.

Because of this, the closest notion Django can have to an application object is the module that Python uses to recognize it as a package. In that case, Django would be using a module object as an application reference.

C H A P T E R 3

฀ MODELS

53

Unfortunately, few projects store anything useful in

, so Django isn’t likely to find any-

thing of interest in it. In order to get at anything really useful, it would have to perform some extra work to traverse the package structure to get a module that contained some pertinent information.

Instead, since Django has to use a module object anyway, it makes more sense to use a module that contains useful information right off the bat. For the majority of applications, the most useful module in a package is

, where all the Django models are defined. Therefore, Django uses this module to recognize an application. Some of the following methods return an application, and in each case, it returns the

module within the application’s package.

The first step in a site- wide introspection is to determine what applications are installed.

Calling

will return such a list, containing the application module for each application in the

setting that contains a

module. That’s not to say that

it only returns applications that have models. It actually checks for the presence of a module, so even an empty

will cause an application to be included in this list.

Take, for example, the following

setting, showing several of Django’s own

contributed applications, as well as some in- house applications and the application described in Chapter 7.

Most of these applications will, by necessity, contain various models. Chapter 7’s

, however, only interacts with the site’s HTTP traffic, so it has no use for the database. Therefore, when looking through the results of

, the

application won’t show up.

54

C H A P T E R 3

฀ MODELS

Retrieving a Single Application

With a list of applications, it’s straightforward to get models from each, so they can be handled appropriately. The next section describes that process in more detail. However, looking at all models isn’t always the best approach; sometimes an application might be given the label of a specific application, so it can deal with just the models in that application.

While it would certainly be possible to just loop through the results from

,

checking the module names against the application module’s attribute, that technique

quickly runs into a few problems. First, the application’s label isn’t the same as its attribute, so trying to compare the two results in a good bit of extra code, most of which is already being done by Django. Also, that code must be tested and maintained, which increases the risk of introducing bugs into the application.

Instead, Django provides a utility for handling this situation. By passing the known label to

, an application can retrieve the application module for just the application matching that particular label. The label referred to here is determined as a specific part of the application’s import path.

Typically referenced as

, an application’s label is usually formed from the last part of the application module’s import path before the

portion. To illustrate a few examples,

consider the following application labels, corresponding to the entries in the setting.

There’s one important note to mention here. As part of the options described in the

official documentation, and briefly touched on earlier in this chapter, any model may override its own

setting to behave as though it was declared inside a different application.

This option does not affect the behavior of

in any way. The

method

simply maps the

to an application module, without regard to what options the modules inside it may have declared.

As demonstrated earlier with

, applications without models are viewed

slightly differently within Django itself than others. By default, will raise an

exception if the application doesn’t contain a

file. Sometimes

it may still be useful to process applications without models, so accepts an

optional second argument to control how such applications are handled.

C H A P T E R 3

฀ MODELS

55

This second argument, called

, takes a Boolean indicating whether the application is

allowed to not contain any models. This defaults to

, which will raise the

exception, but if

is given instead,

will simply return

, allowing the calling

code to continue managing the application.

Dealing with Individual Models

Once an application is known, the next step is to deal with individual models within that application. Once again,

comes through with a few methods to handle this situation.

Retrieving models from the cache typically takes one of two forms, depending on how much is known about the model in advance.

In the first case, consider pure introspection. Remember from the previous section that

provides access to all known applications with a single call to the method, which returns application modules. Since these modules are actually the modules within each application, it may seem easy to just use or iterate over

to get the models that were defined.

Unfortunately, like many uses of simple iteration, that would require the loop to check each individual object in the module to see if it is in fact a model or if it’s something else entirely. After all, Python modules can contain anything, and many models make use of tuples and module- level constants to help do their work, so there’s no guarantee that each item in the module’s namespace is in fact a Django model.

Instead,

retrieves a list of proper Django models that are specific to the given application module. It’s no coincidence that both and

return application modules;

is suitable for use with both of these

methods. That means that a list of models can be retrieved even without an application, but knowing the application in advance reduces the number of models retrieved.

The following code demonstrates how these techniques can be used in combination to retrieve a list of models for each of the known applications in use on the site.

56

C H A P T E R 3

฀ MODELS

As an additional option,

can also be called with no argument, which will

cause it to return all the models that are known to

. This is a useful shortcut to avoid

some of the overhead associated with the extra loop in this example, as a quick way to grab all the models.

There’s a catch, however. When using

directly, with no argument, all regis-

tered models are returned. This may sound like a great idea, and sometimes it is, but remember that

registers all models as they’re encountered, regardless of where they were found.

The full list may include models that aren’t part of an installed application. Contrast that with the

/

combination, which only retrieves models if their applications are found in the

setting.

In practice,

may return different results if called without an argument than if it were called with each of the applications returned from

. Typically, this could

mean that an application may get access to extra models that it might not want to know about.

Sometimes this is indeed the desired behavior, but it’s always important to understand the difference.

One way a model could be in

, but not be installed, is if the application is imported from a separate, installed application, which would cause its model classes to be processed by Django and registered, regardless of whether or not it was in

. Also, if any model

specifies an

on its

class and that application label doesn’t match up with any installed application, the same situation would occur. If an application does wish to access all the models, regardless of whether they’re installed or not, remember that it can use the attribute to identify which models were installed properly.

Sometimes, the name of both the application and the model are provided, perhaps as part of a URL or other configuration. In these cases, it doesn’t make much sense to iterate over all the models for the given application. For this case,

provides another method,

, which retrieves a model class based on an application label and a model name. The application name is case- sensitive, but the model name isn’t.

C H A P T E R 3

฀ MODELS

57

Using Model Fields

One of the most important aspects of models is the set of fields that are available to hold data.

Without fields, a model would just be an empty container with no way to do anything useful.

Fields provide a way to organize a model’s values and validate against specific data types, providing a bridge between the database and native Python data types.

Normally, when accessing a field as an attribute of a model instance, the value will be a standard Python object representing the value found in the database. Previous sections in this chapter have described a variety of ways to get access to the actual field objects themselves, rather than this converted value. There are a variety of useful things that can be done with field objects.

Common Field Attributes

Different field types will have different attributes according to their needs, but there are several attributes that are common across most built- in Django fields. These can be used to generically access various details of fields, and by association, the values and behaviors they’re meant to interface with. Note that there are more attributes used internally than those listed here, but these are the most useful and stable, and will provide the greatest value to applications looking to work with fields.

The descriptions listed here are how Django itself uses these attributes, and how developers will expect them to behave. Other applications will likely find use for them as well, to control certain types of behaviors, so the following descriptions will help illustrate their intended usage.

Some applications may find uses that are slightly different from what Django itself expects to use them for, but the general semantics of the values should remain intact. Remember that developers will build their expectations for these values based on how Django itself behaves, and third- party applications should avoid violating these expectations.

฀

฀

—The name of the attribute on model instances where the database- related value is stored. This is typically the same as the

attribute, for simple cases where

the value from the database is stored directly on the model. In other cases, it’s more appropriate to expose a more complex object, such as another model instance, to other code when the actual field name is accessed. For those cases, and

will be

different, with the attribute referenced by

being the complex object, while the

attribute referenced by

contains the raw data required to create it.

฀

฀

—A Boolean value indicating whether the field must have a value supplied when using a form generated automatically based on the model. This is purely validation- related behavior; the

attribute controls whether a model can actually be saved in the database without a value for the given field.

฀

฀

—A sequence of 2- tuples indicating the valid choices for the field. The first item in each tuple is the actual value that would be stored in the database if selected, while the second item is the text that will be displayed to the user for that value.

58

C H A P T E R 3

฀ MODELS

฀

฀

—The name of the database column that will be used to hold the field’s value.

This will either match

, if the field declared its database column explicitly, or will have been generated automatically, based on the field’s name. Normally, this can be ignored, since Django manages the database interaction directly, but some applications may have need to communicate directly with the database or interface with some other database adapter that will need this information.

฀

฀

—The name explicitly supplied as the database column name for the field’s values. This is different from

in that

refers to what the model itself

declares, rather than what will actually be used. This will only have a value if the model field specified its

argument explicitly; it will be

otherwise.

฀

฀

—A Boolean indicating whether the field was declared to have an index created for it in the database. This only indicates whether the field was configured to instruct Django to create the index. Other indexes may have been added directly in the database itself, which won’t necessarily be reflected in the value of this attribute.

฀

฀

—The tablespace directive indicating where the field’s data will be stored. Currently only supported for the Oracle backend, the format of its contents will depend on which database backend is in place. It will always have a string value, defaulting to the value of the

setting if not set explicitly.

฀

฀

—The default value for the field, to be used if no value has yet been supplied to the field itself. In addition to being inserted into the database in such a case, this value will be used as the field’s initial value for any forms generated based on the model. The type of value stored in this attribute will be whatever native Python data type the field is intended to interact with, such as a string or an integer.

฀

฀

—A Boolean indicating whether the field should be presented to users for editing when generating forms based on the model. This doesn’t make the field itself read- only from within Python so this is far from a guarantee that the field won’t be edited. It’s simply a directive to control the default behavior of forms, though other applications can—and should—use it to control other behaviors as well, if they provide editing capabilities.

฀

฀

—A Boolean indicating whether the field allows an empty string as a possible value. This isn’t an option specified as the configuration of a specific field instance, but is rather defined in the field’s class itself. Many fields, such as and

, treat empty strings separately from

, so this attribute allows back-

ends to decide how to handle empty strings for databases, such as Oracle, that might otherwise lose that distinction.

฀

฀

—The informative text provided in the field definition, to be displayed to users when the field is presented for editing. This will be passed in for forms that are generated based on the model, such as the provided admin interface.

฀

฀

—The maximum length the field’s value can contain. Most string- based fields, such as

and

, use this to limit the length of string content,

both in form fields and the underlying database column. Other field types, such as and

, simply ignore it, as it has no meaning in those cases.

C H A P T E R 3

฀ MODELS

59

฀

฀

—The name of the field, as defined when assigning the field to the model. This is set as part of the

process, to maintain DRY by avoiding having

to type the name twice. This will be the name of the attribute where the field’s native Python value will be assigned and retrieved. Contrast this with

, which stores

the raw data necessary to populate

. Often, the two values will be the same, but the

distinction is important to understand, for cases where they’re different.

฀

฀

—A Boolean indicating whether the field can be committed to the database without a value assigned. This primarily controls how the underlying database column is created, but some applications may find other uses, as long the semantics remain the same.

฀

฀

—A Boolean indicating whether the field should be used as the primary key for the database table. In addition to instructing the database to generate the primary key index, Django uses this indicator to determine which field’s value to use when looking up specific instances, such as related objects through foreign key relationships.

See the section “Primary Key Fields” earlier in this chapter for details on the shortcut for determining which field has this value set to

.

฀

฀

—In the case of fields that relate one model to another, this will be a special object describing the various aspects of that relationship. For all non- relationship field types, this will be set to

.

฀

฀

—A Boolean indicating whether the field should be included when model instances are serialized using the serialization framework.3

฀

฀

—A Boolean indicating the field must be unique among all instances of the model. This is primarily used to create the proper constraints in the database to enforce this condition, but it can also be used by applications. For instance, a content editing application that provides detailed feedback about whether the user- entered values are valid for the model can also take this into account when making that determination.

฀

฀

—The name of a date- related field, such as a

or

,

for which this value should be unique. This is essentially like

, except that the con-

straint is limited to records that occur on the same date, according to the field referenced by this attribute. This can’t be enforced at the database level, so Django manages the constraint manually, as should any other applications that need to provide detailed information about whether a given object can be committed to the database.

฀

฀

—Like

, except that the uniqueness is only required

for objects that occur within the same month, according to the date- related field referenced by the name contained by this attribute.

฀

฀

—Like

, except that the uniqueness is only required

for objects that occur within the same year, according to the date- related field referenced by the name contained by this attribute.

3.

60

C H A P T E R 3

฀ MODELS

฀

฀

—The full name of the field, in plain English, to be displayed to users.

Django’s documentation recommends that this begin with a lowercase letter, so that applications can capitalize it as necessary. If an application needs this value capitalized, be sure to use the

utility method, described in Chapter 9.

Common Field Methods

Like the attributes described in the previous section, these methods are common to most field types, and provide a wealth of functionality that might otherwise be difficult to come by. Not all field types will implement all of these methods, and their exact behavior may change depending on the field type involved, but the general semantics described here will remain the same.

There are more methods that get used even more internally, which aren’t listed here, because they’re primarily responsible for simply populating the attributes described in the previous section. Therefore, it’s generally best to simply reference the generated attributes, rather than attempt to recreate them manually after the fact.

฀

฀

—Configures the field for the class it’s attached to. One of the most important methods on fields, this is called when

is processing the attributes

that were assigned to the model’s class definition. It’s called as

, where

is the model class it was assigned to, and

is the name it

was given when it was assigned there. This allows fields the opportunity to perform any additional setup or configuration, based on this information. It usually doesn’t need to be called directly, but can be a useful way of applying a field to a previously- processed model.

฀

฀

—Returns the database- specific column definition necessary for this field to store its data. Typically, this is only used internally, but as with some of the other attributes listed, if an application needs to access the database directly using some other tool, this can be a useful way to determine what the underlying column looks like.

฀

฀

—Returns a dictionary containing the values necessary to generate the value stored in this field. It’s called as

, where

is a Boolean indicating whether it should follow related objects for data, and is the

model instance the value should be pulled from. The dictionary returned maps names to their values, and usually only contains one such item, using the field’s name as the key. This is used by the serialization framework to obtain a value suitable for output, but could find other usage in applications to obtain a simpler representation of data.

฀

฀

—Returns a form field based on the field’s data type and verbose name, suitable for inclusion on any standard form. It optionally takes one explicit argument,

, which is a form field class to be instantiated, which defaults to whatever form field is most appropriate, as defined by the model field itself. It also accepts any number of additional keyword arguments, which are simply passed through the form field’s constructor before returning the instantiated form field. This is normally called automatically by Django when constructing a form based on a model, but may be used manually as well for other situations. More information can be found in Chapter 5.

฀

฀

—Returns the name that should be used for the

attribute. This is

only called once, while the field is being configured for the class.

C H A P T E R 3

฀ MODELS

61

฀

฀

—Returns a two- item tuple containing the values to be used for the

attribute as well as the

attribute.

฀

฀

—Returns a name suitable for use as a cache for the field, if caching is necessary. This is typically only required for fields that generate complex Python data types, which would suffer significant performance penalties if such a complex object had to be generated on every access, or in cases where it won’t be used. See the applied techniques at the end of this chapter for details on how to use this method in such cases.

฀

฀

—Returns a sequence of 2- tuples that should be used for displaying choices to users looking to enter data into this field. Unlike the attribute, this

may also include an empty option that would indicate no choice has been made. This behavior is controlled by two optional arguments:

, a Boolean indicating

whether it should be included, and

, a list of tuples containing the values

and display text that should be used for the empty options. By default, these arguments are configured so that a single choice of

is included.

฀

฀

—Returns a representation of the supplied value that’s

suitable for comparing against existing values in the database.

฀

฀

—Returns a representation of the supplied value that’s suitable to be stored in the database.

฀

฀

—Returns the default value that would be used for the field. This takes care of all the necessary logic, checking if a default value was provided, executing it if a callable was provided as the default and differentiating between empty strings and

, for database backends needing that behavior.

฀

฀

—Returns a string representing a high- level idea of what type of data the field contains. This is primarily used, along with a mapping provided by each database backend, to determine the actual database column to be used.

฀

฀

—Returns

if the field has a default value associated with it, or

if the default behavior will be left to the database backend.

฀

฀

—Returns a value for the field just prior to being saved in the database. By default, this simply returns the value that is already set on the supplied

, but it could return a value derived from some other field or perhaps completely unrelated to the instance, such as the current time. The argument is

a Boolean indicating whether the provided instance is being added for the first time.

฀

฀

—Stores the supplied data to the appropriate attri-

bute on the supplied instance. This is a shortcut for forms to be able to adequately populate a model instance based on form data.

฀

฀

—Uses the supplied

argument to set the field’s

,

,

and

attributes as necessary. This method defers to

for the

and

values, while

is only set

here if it wasn’t explicitly defined when instantiating the field.

฀

฀

—Coerces the supplied value to a native Python data type that can be used when accessing the field’s value on a model instance. See its description later in this chapter for further details.

62

C H A P T E R 3

฀ MODELS

฀

฀

—Returns without error if the field’s value is appro-

priate for the field’s configuration and other data on a model instance, or raises otherwise.

฀

฀

—Returns a list of errors that were raised when

validating the supplied data according to the field’s configuration.

฀

฀

—Returns the field’s value as it appears on the supplied object.

Subclassing Fields

One of the more useful things that can be done with Django models, particularly with regard to distributed applications, is to tie into a model’s ability to process individual types of fields in a generic fashion. This allows fields themselves to have considerable control over how they interact with the database, what native Python data type is used to access their contents and how they’re applied to the model classes that use them.

The majority of this section assumes that the custom field will need to retain much of the same functionality of existing fields, such as interacting with the database and generated forms. There are many other applications, such as the historical records application described in Chapter 11, that use the hooks described in this section to provide much more functionality than just a simple field.

The term “field” here is used loosely to describe any object that uses some of these techniques to present itself to a Django developer as something resembling a standard Django model field. In reality, such an object could encapsulate complex relationships, such as a tag-ging application, or even control the creation of entire new Django models on the fly, based on the model to which they’re assigned. The possibilities are nearly limitless.

The key to remember is that Django uses duck-typing principles with regard to fields. It simply accesses whatever attributes and methods it expects in each situation, without regard to what those actually do behind the scenes. In fact, there’s not even any requirement that objects be a subclass of

to make use of these hooks. Inherit-

ing from

simply provides an easy way to reuse much of the existing functionality, if that behavior is required.

Deciding Whether to Invent or Extend

One of the first things to consider when writing a new field is whether to try to invent an entire new type of field, starting perhaps from scratch without the aid of at all, or to extend

some existing field type and inherit much of its behavior. There are advantages and disadvantages to each approach, and which is most appropriate depends very much on the demands of the new field being created.

By inheriting from

or one of its subclasses, most of the behaviors in the following sections will be inherited, potentially reducing the amount of new code the custom field must include. If its behavior is similar to an existing field type, this can be a very useful way not only to cut down on new code, which helps reduce bugs, but also to automatically receive any new or updated functionality provided by Django itself in future releases. After all, by relying on Django itself for much of this behavior, updates to that code will automatically be reflected in the behavior of the custom field.

C H A P T E R 3

฀ MODELS

63

On the other hand, if the new field varies considerably from any existing field type, the standard behaviors will need to be rewritten for its own use anyway, negating any value of inheriting from a parent class. If most—or all—of these behaviors have to be written from scratch, inheriting from an existing field will simply create an extra step in the process Python uses to manage the class, even though that extra step offers little or no benefit. In these cases, it’s best, therefore, to simply start from scratch, implementing just those behaviors that make sense for the custom field, and Django will still process it properly, due to its use of duck typing.

Of course, there is some middle ground between the two approaches. For instance, a custom field may interact with a completely unique data type, bearing little resemblance to any existing field types, but it may still store its data in the database like a standard field, and could benefit from reusing many of Django’s more basic field methods, such as assigning names and storing itself in

. In these cases, it’s quite reasonable to inherit from

itself,

rather than a specific subclass, and inherit just this most basic functionality.

Performing Actions During Model Registration

The first step any field goes through is being processed by the metaclass, whenever Python encounters a model class that utilizes the field in question. For standard Python objects, this means simply getting assigned to the model class as normal, with no additional processing. Fields take a different path, however, and each field gets the chance to customize how it’s applied to a model class.

contribute_to_class(self, cls, name)

This is perhaps the most important method a field can contain, as it provides an essential feature: the ability for a field to know what class it was assigned to, and what name it was given.

This may seem like a simple requirement, but Python itself doesn’t normally have a way to facilitate this.

You may recall that descriptors, described in Chapter 2, have a way to identify what class—

and even what instance of that class—was used to access the object, but this is only available at the time the attribute is accessed; there’s still no way to know this information at the time the assignment took place. More importantly, even descriptors don’t provide any way to identify what name was used to access them, which can be a considerable problem when trying to cache information or interact with other features that require the use of a name, such as that of a database column.

Instead, by using a metaclass, Django can intercede at the point where Python is processing the class, and use the presence of a

method to identify objects that

need to be handled differently. If this method exists, it’s called instead of the standard

,

allowing the field to register itself in whatever way is most appropriate for its purpose. When doing so, Django also provides the class itself as an argument, as well as the name it was given, which was discovered while looking through the attributes assigned to the class. Therefore, in addition to the usual

, this method receives two arguments.

฀

฀

—The actual class object of the model the field was assigned to. This can be used to customize the field based on the name or other attributes of the model itself.

฀

฀

—The name, as a string, of the attribute as it was assigned to the model’s class.

Fields will typically store this away as an attribute of the field itself, for future reference.

64

C H A P T E R 3

฀ MODELS

Once these two arguments have been processed in whatever way is appropriate for the field, the method shouldn’t return anything, as its return value is ignored by Django.

CONTRIBUTE_TO_CLASS() VS. SETATTR()

There is one very important thing to keep in mind when dealing with

. It’s

been mentioned a few times already in various places, but it’s so important that it merits driving home very explicitly. If Django identifies an object as having a

method, only that method

will be called.

Normally,

is used to set attributes on an object such as a class, but since model fields don’t get set in the standard namespace, that step is skipped intentionally. Therefore, if a custom field does in fact need to be set as an attribute on the model class itself, doing so is the sole responsibility of the field itself, during the execution of its

method.

Sometimes, fields will instead need to set some other object, such as a descriptor, as the attribute on the class, to provide additional customizations for other types of access. This, too, is the responsibility of the field class, and the only time to do so in a way that will maintain the appearance of a standard field is during the execution of its

method.

In the case of standard Django fields, and perhaps for many types of custom fields and other objects that behave as fields, this avoidance of

is quite intentional. If that behavior is

desired,

should simply avoid setting anything on the model class, and Django’s own behavior will make sure that nothing is assigned to the class itself.

contribute_to_related_class(self, cls, related)

For fields that relate themselves to other models, this is called once the related model is available, so that attributes can be added to that model as well. For example, this is how Django provides a reverse attribute on a related class when a

is applied.

The two arguments it receives are

, the model class the relationship was actually

applied to, and

, the model the relationship points to, where other attributes may yet need to be applied. Like

, this shouldn’t return anything, as it would

simply be ignored anyway.

Altering Data Behavior

Given that most field types exist to interact with specific data types, one of the first things to consider is how to tell Django to handle that data type. This includes how to store it in the database, how to ensure validity of its value and how to represent that value in Python. These are some of the most fundamental aspects of field behavior, and properly altering them can open up a world of possibilities.

get_internal_type(self)

This method returns a string, which helps determine how the database should store values for the field. The string itself isn’t an actual database column type, but instead it’s applied to C H A P T E R 3

฀ MODELS

65

a mapping provided by the database backend to determine what type of column to use. This way, fields can be written without being tied to a specific database backend.

Because the return value for this function gets applied to a known dictionary of types to retrieve the database column name, that value must be a valid entry in that dictionary. Therefore, there’s a finite set of possible return values, which are listed here.

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

validate(self, field_data, all_data)

When a model is being checked for the accuracy of its values, this method is used to determine whether the field’s contents are correct. The arguments it receives are the value of the field itself, and also the values of all the fields on the model. This allows it the option of validating not only the field’s own value, but also that it makes sense in the context of the greater model.

It should be obvious why this would be of use when validating an individual field’s value, but it’s less clear what value lies in using the rest of the model’s values. After all, when writing a field, there’s typically no way to know what other fields will be used alongside it.

Sometimes, however, a field may be written specifically for a particular model, and can therefore know in advance what the entire model will look like. In these cases, the field can, for 66

C H A P T E R 3

฀ MODELS

example, check to see what type of account a person has, because the maximum value for the field depends on that other field.

to_python(self, value)

The value of a field can be stored in a number of different ways, depending on where it’s being stored. In a database, it can be one of a few basic types, such as strings, integers and dates, while when serializing a model, all values will be coerced to strings. That means that often, when instantiating a model, its value has to be forced back into its proper Python representation. This behavior is handled by the

method, though it’s not quite as

straightforward as it may seem on the surface.

The first thing to consider is that the value passed to

could be one of a num-

ber of representations of the data. For instance, it could be whatever format is returned from the database adapter, such as a string, integer or native Python date, but it could also be a string retrieved from a serializer, or if the field manages a more complex custom data type that needs to be initialized, the value could actually be a fully- initialized instance of that type.

To illustrate this, consider the situation of

. Values that get passed into it

could come in a variety of forms, so its

method needs to anticipate this and make

sure that it always returns a Boolean value or throws an exception indicating that the value wasn’t suitable for the field.

As you can see, it has to check for a few different types of values that could all be coerced into Boolean values reliably. In addition to the native

and

, it checks for the string

representations of the same, as well as a couple single- character representations that might turn up in various situations. If it finds something suitable, it simply returns the appropriate native Boolean value, raising the

described in the previous section if a suit-

able value couldn’t be found.

Unfortunately,

is an extra method call that’s not always necessary, so it’s not always called when it seems like it would be. In particular, it’s provided mainly for validating data prior to committing to the database and when retrieving content from serialized data, so when retrieving from the database, it’s assumed that the data has already been validated, and the database backends generally suffice for returning the proper type.

Because of this, Django doesn’t call

when retrieving data from the database.

For the built- in types, and many potential add- on fields, this is sufficient, but for other data types or complex objects, some more work will be done to convert the database value to something appropriate to work with. To support these types of fields, Django provides a special way to force

to be called when populating the field’s value.

Supporting Complex Types with SubfieldBase

Sometimes databases just don’t have the necessary data types to support certain types of applications. For example, most databases don’t have a way to store a length of time and present it to C H A P T E R 3

฀ MODELS

67

Python as a

4 object. PostgreSQL has a column type called

5 for this

purpose, which does map directly to a Python

as it should, but other databases don’t,

which makes this impractical in terms of reusability. It would work suitably for PostgreSQL, but in order to make an application portable, it needs to be usable with more than one database.

Thankfully,

stores its values in days, seconds and microseconds, and can write the entire value based on just a number of seconds passed in as a

. Therefore, it’s pos-

sible for a new

to use a

to store a value in the database, convert it

to a

in Python, then pass it into

for use on the model instance.

This is the type of process that simply can’t be handled without using

, and it

must take place every time the model is instantiated, even when coming from the database.

However, calling an extra method call on every access from the database can get quite expensive, so it’s essential to be able to handle this without penalizing those fields that don’t use it.

As will be shown at the end of this chapter, a descriptor can be used to customize what happens when a field’s value is accessed, which can be an excellent way to control this type of behavior. Of course, descriptors can be tricky if they’re just a means to an end, and the 4.

5.

68

C H A P T E R 3

฀ MODELS

behavior described here is a fairly common need for these complex data types, so Django provides a shortcut to ease the creation of this descriptor.

Located at

, the

metaclass is Django’s

way of easing the creation of model fields whose

method will always be called. By

simply applying this to a model class, it takes care of the rest, setting up a descriptor that calls the first time the field is loaded. Therefore, the

example would use

this in the field definition as follows:

Controlling Database Behavior

Another important aspect of fields is how they interact with the database. This can include how the data itself is stored, how it’s prepared before being sent to the database and how it’s prepared for comparison with values already in the database. This process is already taken by Django itself, with every existing field type providing a few methods to define this behavior.

For custom fields, it’s often necessary to override this behavior, interacting with the database in ways other than how Django itself would expect to do so. The following methods define nearly every aspect of how a field works with the database, so fields have a great deal of control over how the database interaction is handled.

db_type(self)

Rarely overridden by individual fields, this method returns a database- specific string that controls how the column is created for use with the given field. Django internally uses the result of the

method in conjunction with a mapping provided by each individual backend to provide a return value from this method. That functionality is enough for the vast majority of field applications.

The most important thing to remember when considering the use of this method is that its return value is specific to a particular database backend. Overriding this and providing such a database- specific string limits the ability of the field to be distributed to other projects, or to be shared between development and production environments. It would be forever tied to that one database backend, leaving all others out in the cold.

Clearly, it’s most useful to leave this method to Django unless it’s absolutely necessary to override for a particular application. Some databases do support features that can’t be expressed by any value from

, and for internal, proprietary applications, the limits on distribution are irrelevant. Just make sure that the use of is a conscious choice, weigh-ing the costs and benefits of the situation at hand. Don’t use it lightly.

C H A P T E R 3

฀ MODELS

69

get_db_prep_value(self, value)

Both the

and

methods deal with preparing a value

for use with the database. Those two methods are specifically designed for saving and retrieving data, respectively, but both typically share the same code for preparing a value for use in the database at all. The

method is used by both of the following methods

to perform this basic conversion.

In most cases, converting a Python object to some more basic type will suffice to allow a custom field to pass values to the database. By overriding

, the other

database preparation methods can typically use their default implementations without issue.

For example,

requires this type of conversion, since

objects can’t be

passed directly to most databases, which led to using a

to control the column’s

behavior. A custom

method can convert

objects to

values, which can then be passed to the database normally.

get_db_prep_save(self, value)

When preparing to commit a model instance to the database, Django must convert native Python objects to something more suitable for use by a database backend. Since there are as many ways to do this as there are fields to use, it makes sense to have this functionality present as a method on the field itself.

The

method receives just one additional argument: the value to be committed to the database. It’s the responsibility of this method to transform that value into something acceptable by database backends, which can then be stored directly in the database. Exactly how this happens is up to each individual field, with the default implementation deferring to

.

get_db_prep_lookup(self, lookup_type, value)

One other area where fields have to interact with the database is when making comparisons between Python objects and values already stored in the database. This takes place every time a QuerySet’s

method is used, for instance, in order to generate the necessary database query. Since comparisons might require different handling than saving, Django uses the method to manage this task.

When called, this method receives two explicit arguments, detailing how the lookup is expected to take place. The first,

, is the type of comparison that was requested in

70

C H A P T E R 3

฀ MODELS

the

method. The second,

, is the Python object that was provided for compari-

son against database values.

While

is fairly straightforward,

is a little different, because it’s a string con-

taining the requested comparison type. There are several of these available as part of Django’s database API,6 each having its own expectations. This is the full list, including the purpose of each:

฀

฀

and

—The supplied value must match exactly with what’s present in the database, with

being case- insensitive. Django assumes a filter without a lookup type to mean

, which will be passed in to

.

฀

฀

and

—The supplied value must be present in at least part of the value present in the database, with

being case- insensitive.

฀

฀

and

—The database value must compare as greater than the value supplied to the lookup, while

also allows for the values to be equal.

฀

฀

and

—The database value must compare as less than the value supplied to the lookup, while

also allows for the values to be equal.

฀

฀

—The database value must exactly match at least one of the values present in a list supplied as the lookup value.

฀

฀

and

—The database value must begin with the string supplied

as the lookup value, with

being case- insensitive.

฀

฀

and

—The database value must end with the string supplied as the lookup value, with

being case- insensitive.

฀

฀

—The database value must be within the range specified by a 2- tuple of beginning and ending limits supplied as the lookup value.

฀

฀

,

and

—The database value must contain the specified lookup value as its year, month or day portion, depending on which lookup type was used. This is valid for dates only.

฀

฀

—The database value must be equivalent to

in order to be matched.

฀

฀

—The database value must pass a full- text index search. This is valid only for MySQL, and only if the database has been modified to enable the necessary indexing.

฀

฀

and

—The database value must match the format specified by the regular expression supplied as the lookup value, with

being case- insensitive.

Fields that inherit from some existing field can usually avoid overriding this method, as the parent class usually does the right thing. Other times, unfortunately, the child class needs specific handling for certain lookup types, where this can be quite useful. Still other times, it’s necessary to restrict certain types of lookups entirely.

One useful side effect of having Python code executed as part of the lookup process is that it allows exceptions to be thrown for lookups that aren’t valid for that field. This works just like anywhere else, where if you raise an exception, it will bail out of the query early, displaying a message indicating what happened.

6.

C H A P T E R 3

฀ MODELS

71

WHERE’D MY ERROR GO?

Unfortunately, even though it’s possible—and often quite useful—to raise exceptions within

, sometimes you may find that they get suppressed. If this happens, the query will appear to execute, but you’ll likely receive just an empty list as its result, rather than seeing your error.

Due to some of the hoops QuerySets have to jump through internally, certain types of errors—including TypeError, which seems like an obvious choice to use—get caught and suppressed, causing Django to move on with the process in spite of not getting a valid value for that field.

In order to make sure that the error gets raised to its fullest and works as expected, be sure to use instead of

, as it doesn’t get caught in the same trap.

Dealing with Files

Many applications have need to manage content that goes beyond what’s traditionally stored in a database. Beyond the usual numbers and strings, there’s a world of other data formats, from audio and video to print- ready Portable Document Format (PDF) files and plenty more.

Content like this isn’t well suited for being stored directly in the database—though in some cases it’s at least possible—but it’s still useful to tie it to other content that is in the database.

To handle this, Django provides a special

, with extra methods designed to facil-

itate access to files. It also uses many of the hooks described in this chapter to store a reference to the file in the database, as well as provide a special object that can access files in a portable manner. Django also provides an

, which inherits much of its functionality from

, while adding some of its own, specifically tailored for dealing with the special needs of images.

Subclasses of

shouldn’t generally need to override many of its methods, since they’re mostly related to those features of a file that are common to all file types. This includes things like the filename and relative path, which don’t have anything to do with the specifics of a particular type of file. Some, however, such as

, can be overridden to provide

special handling of attributes related to a specific type of file.

get_directory_name(self)

This method simply returns a relative path that will be stored in the database along with the filename. By default, this looks at the

attribute of the field to determine what the directory

should be, and even subclasses should respect this behavior. Exactly how that attribute is used, however, is where subclasses can customize this method to great effect.

Normally, Django creates a directory name using two pieces of information: the string itself and the current date. The date the file was uploaded is applied to the directory name, replacing certain characters with portions of the date. This allows individual fields to more accurately control where their files are stored, which helps keep directories smaller, and can possibly even make better use of disk capacity.

In a subclass, however, it may be more useful to generate the directory name based on some other type of information, such as the current site’s domain name in multisite setups, or the Internet Protocol (IP) address of the machine where the upload was received, in larger production environments where there are multiple Web servers sharing common storage.

72

C H A P T E R 3

฀ MODELS

Essentially, anything’s fair game here, as long as it only requires information that can be determined by only having access to the

instance. The current site or IP address can

be obtained without regard to the current model at all, as can the current time. Other information, however, such as the user who submitted the file, the IP address of his or her remote computer, or the object the file will be attached to, is not accessible from this function, and thus can’t be used.

Of course, there is another option to specify some of this additional information, but doing so bypasses this method entirely. By specifying a callable for

, as described

in Django’s file documentation,7 the directory can be generated based on the object it will be attached to, which may include the

who owns the object.

Note that when using a callable for

, that callable is expected to return the entire

path, including the directory and filename, so

won’t be called at all in

such cases, unless that callable explicitly calls it. Also, the incoming request still isn’t available, even to that callable, so making directory naming decisions based on that information will require a custom view.

get_filename(self, filename)

This works in much the same way as

, except that it’s responsible for

specifying the filename portion of the path instead of the directory. It receives the original filename that was specified with the incoming file, and returns a new filename that will be used in the database, as well as the underlying storage system.

If a

subclass has need to customize the filename that will be used for a particular file, such as stripping out certain characters or altering the file’s extension, this would be the place to do it. That’s also why it receives the original filename as well, so that it has a way to create a filename that’s at least partially related to the one provided by the user.

By default, its output is combined with that of

to form the full path

to be stored in the database and passed to the storage system. Like its counterpart, however, this is only true if the

argument to the field was not a callable. If a callable was specified, it’s responsible for specifying the entire path, including the filename. Therefore, in such cases, this method will only be called if the

callable specifically requests it.

generate_filename(self, instance, filename)

This is the default method used to generate the entire path. It uses the same function signature as a callable

argument, because it plays the exact same role. In fact, internally to

, all references for generating the filename to be used for the file reference this method; if a callable was supplied to

, it’s simply assigned to this same name, replac-

ing the default behavior.

The default behavior is to use

to combine the output of both the

and

methods, ignoring the model instance provided as an

argument. If a

subclass needs the ability to specify the file’s entire path all at once, this method would be the place to do it.

Of course, remember that if a callable was supplied as the argument, this

method will get replaced. This is true regardless of what behavior is supplied by a 7.

C H A P T E R 3

฀ MODELS

73

subclass; the needs of a specific instance always win over the behavior of its class. So, while overriding this behavior can provide a more useful default, it doesn’t remove an individual developer’s ability to replace it entirely.

save_form_data(self, instance, data)

This is a utility method for forms to use as a shortcut for saving a file associated with a model instance. It accepts an instance of the model the field was attached to, as well as the uploaded file data provided by the form. By default, it just extracts the necessary information from the uploaded file object, and passes it through to the standard file saving methods.

The

argument is an instance of the model where the

was defined,

and the

argument is an

object, as described in Chapter 8. The uploaded file

contains a

attribute, which contains the filename and a

method, which is used to

access the file’s contents, so that it can be saved properly.

As this is the primary way files are handled by most areas of Django itself, overriding this field provides an excellent opportunity to tie into extended functionality based on specific field types. For example, Django’s own

uses this as an opportunity to store

the width and height of an image in separate fields, so they can be indexed and searched in the database directly. Other file types could take this same approach, storing certain attributes of the file in other fields for easier access later on.

delete_file(self, instance, sender)

While this may look like simply a way to delete a file, it actually serves a very particular purpose, which is alluded to by the presence of a

argument. The

method of

sets up this method as a listener for the

signal. It’s not

intended to be called individually, but instead it gets called every time a model instance with a

is deleted. As described for

, the

argument is the object that

was just deleted, and the

argument is the model class for that instance.

When triggered, it checks to see if the file referenced by this field on the specified instance should be deleted. After all, if no other instances are referencing the same file, and it’s not the default values for new instances, it’s quite likely that no references to the file remain. In those cases, the file is permanently removed from the storage system.

The uses for overriding this are clear, because the logic for when to delete the file is included directly within this method. If a

subclass needs to have different rules for this, simply

overriding this method is enough to make it happen.

The obvious example is if files should always remain, for historical reasons, even after the model instances associated with them have been deleted. Providing that behavior is a simple matter of just defining an empty implementation of this method.

74

C H A P T E R 3

฀ MODELS

Of course, there are other possible use cases for this as well, but the specifics of what those would look like will depend very much on the needs of an individual application.

attr_class

As a simple attribute, rather than a method,

might not seem like it would provide

much power or flexibility. Thankfully, looks are often deceiving, as it’s actually the gateway to some very useful features. The

attribute is set to a class that will be used to rep-

resent the field’s value when referenced in Python. That means that the value of this simple attribute is actually the primary way of specifying what features are available on the public API for data entered into a particular

instance.

The following section describes the behavior of the class specified by default for this attribute, and how its methods can be overridden to provide additional functionality.

Customizing the File Class

When a model defines a

, the value made available as the attribute on actual model instances is a special object designed specifically for managing files. Located at

, the

class provides a number of platform- independent and storage- independent methods for accessing a file’s content and properties of that content, as well as for saving new files and deleting existing ones.

Because it’s the public- facing API for accessing files, it’s often quite useful to provide additional functionality for file types that have common qualities that will need to be referenced often. This provides a nice, clean, object- oriented way to encapsulate that common code in one place, rather than requiring the rest of the application to write it over and over again.

For example, Django’s own

provides its own subclass,

, which con-

tains additional methods for accessing the width and height of an image, as well as caching it to speed up subsequent accesses. It’s an excellent example of how easy it is to provide this extra functionality.

In addition to providing new methods, though, there are a number of existing methods that could benefit from being overridden. These are a bit less likely to be of use directly, but as

shows, they can be used to perform some important tasks, such as updating or invalidating cached values.

For the most part, the methods described next map directly to file storage methods described in Chapter 8. The main difference is that these are specific to a particular file type, and can be customized for aspects that are unique to that file type, while storage systems are just designed to work with files, without regard to what type of content gets handled.

path(self)

This returns the path of the file, if it’s stored on the local filesystem. For files stored on other backends, which can’t be accessed with Python’s built- in function, this will raise an

, because the corresponding method isn’t available on the related storage system object.

C H A P T E R 3

฀ MODELS

75

This is provided mostly as a compatibility layer with older versions of Django, for those projects that were written before the introduction of this new file handling system. In the real world, projects written for newer versions of Django should avoid the use of this method, and instead use the

method listed in this section to access files in a more portable fashion.

Overriding it will also be of little use, so it’s listed here for completeness with the rest of the API.

url(self)

This method returns the URL where the file can be retrieved on the Web. It might be served up from the Django project itself, a media server operated by the site’s owners or even a storage service operated by a third party. The exact details of where this URL comes from are specified by the storage system, so this method is a portable way to access the URL for the file.

Overriding this provides little benefit for most situations, but there are a few reasons to do so, depending on the situation. One example might be a subclass that manages

HTML files with a specific structure, so that the URL might contain a name reference, to direct browsers to a specific point in the file.

size(self)

This retrieves the size of the underlying file, caching it for future reference. While this can be a very useful feature, there’s little value in overriding it in a subclass. The nature of file size is such that it doesn’t vary depending on file type, and there’s not really anything that can be done to customize how the size is obtained, so it’s just included here for completeness.

open(self, mode='rb')

This retrieves the file’s content and returns an open file or file- like object, which allows access to the file. This is the preferred method of accessing a file’s contents in a portable fashion, since it passes through to the storage system for the majority of its functionality.

The

attribute takes all the same options as Python’s own

function,8 and can

be used to open the file for read or write access. One use of overriding this method could be to change the default access mode, but only for changing whether it should be opened in binary mode by default or not. The default should always at least be to open the file for reading, rather than writing.

Another potential reason to subclass this would be to provide custom behaviors to the returned file- like object. By default, this method will return whatever object is returned by the storage system, but particular file types might have use for customizing methods on that object, such as

or

to alter how and when the file is written. Because this method is responsible for returning an open file- like object, it can wrap the true file- like object in another, passing through to the real object after doing whatever extra work needs doing.

8.

76

C H A P T E R 3

฀ MODELS

save(self, name, content, save=True)

As the name implies, this saves a new file to the storage system, replacing the file currently in place on the model instance. The arguments should be mostly self- explanatory, with being the name the new file should be saved as, and

being the actual contents of the

file to be written using that name.

Of course, invalid characters in the filename or existing files with the same name could result in the filename being changed by the storage system. Such changes will be reflected in the filename that’s stored on the model instance.

The

argument, however, merits further explanation. Because this saves a file that’s related to a model instance, the new filename will be stored on that instance for future reference. However, it’s not always beneficial to commit that change to the database immediately.

By default, it does save the instance right away, but if

is set to

, this will be

bypassed, allowing additional changes to take place before committing to the database. Take care when doing this, however. The file will already have been committed to the storage system, so failing to eventually save the instance with the new filename will result in a file with no references to it.

Overriding this can provide a way to customize or record the filename that will be used, to change the default database commitment behavior, or perhaps most commonly, to retrieve information about the file’s contents and update any cached information accordingly. The default

object does this for the file size, and

also updates its dimensions cache.

delete(self, save=True)

Also fairly self- explanatory, this deletes the file directly from the storage system, regardless of which storage system is being used. It also removes the filename from the model instance, so that it no longer references the file.

The

argument works just like the one from the

method, determining whether

the model instance is saved or not. Also like

, if

is provided, it’s important to make

sure the instance is in fact saved eventually. Otherwise, it will contain a reference to a file that has already been deleted. Perhaps worse yet, if another instance saves a file with the same name, the reference from the first instance will no longer be orphaned, but will in fact point to the wrong file entirely.

Overriding this provides most of the same benefits as overriding

, by being able to

remove any cached information so it doesn’t cause confusion if accessed later.

Signals

Chapter 9 describes the signal dispatching system bundled with Django, and how signals work in general. As will be explained, signals can be created and made available from any Python module, and can be used for any purpose. For dealing with models, several signals provided out of the box, and can be used in a number of situations.

The following signals are all available at

, and each sends the

model class as the standard

argument to the listener. In addition, many signals include a model instance as an additional argument. These and other additional arguments are detailed in the descriptions of each individual signal listed here.

C H A P T E R 3

฀ MODELS

77

class_prepared

This signal fires when Django’s

metaclass has finished processing a model class,

indicating that the class is completely configured and ready to be used. Since the metaclass operates as soon as Python encounters the class declaration, is fired before

Python even continues processing the module that contains that declaration.

One important note to consider, however, is that this fires just prior to the model being registered with

. Therefore, if a listener for

looks through

to

inspect the models that have been processed up to that point, the model that fired the signal won’t yet be present. There may be some uses for inspecting the application cache at this point in the process, but without a full application cache, its value is quite limited.

Unlike most of the other signals listed in this section,

only sends the

standard

argument. Since there isn’t any instance available at the point in time when the signal is fired and the

attribute on the new model class contains all the information about how it was declared, the model itself is enough to obtain all the information that’s available at that point in time.

Like all signals, listeners for

can be registered with or without a specific

model to listen for, though it may not seem like this would be possible. After all, if the listener must be registered prior to the signal being fired, and the signal is fired before Python even continues with the rest of the module, how can it possibly be registered with a class to listen for? Even if it could, what possible purpose could it serve?

The answer to both of these questions is

. Remember that attri-

butes on a model are given the opportunity to customize how they’re applied to the model.

When an object with a

method is encountered, that’s called instead

of the usual

, where it’s passed the model class and the attribute name, allowing the object to perform whatever functionality it wants to.

The key here is that

receives the model class as an argument. It

makes for an excellent opportunity to register a listener for specifically for the

class being processed. In fact, depending on the need at hand, this is not only possible, but could be downright essential.

Consider a situation where a field- like object needs to know everything about the model it’s attached to in order to properly configure itself. Since there’s no guarantee that all the other fields have been processed by the time

is called on the object

in question, it’s necessary to defer the rest of the configuration until the class has finished processing.

78

C H A P T E R 3

฀ MODELS

pre_init and post_init

When a model is instantiated,

fires before any other work is performed. It gets

dispatched even before any of the arguments passed into the model are assigned to their appropriate attributes. This is a good opportunity to inspect the arguments that will be assigned to the instance prior to that actually happening, especially since this allows a listener to fire before encountering any errors that might come as a result of the arguments specified.

Because this takes place prior to any of the field values being populated on the object itself, it doesn’t send the new object along when the signal is fired. Instead, it passes along two additional arguments besides

that correspond to the positional and keyword argu-

ments that were passed in to the model.

฀

฀

—A tuple containing the positional arguments that were passed to the model constructor

฀

฀

—A dictionary containing the keyword arguments that were passed to model constructor

Note that even though these are the same names as those usually given to the excess argument technique described in Chapter 2, these are passed to the listener as explicit keyword arguments, rather than using and

. Listeners must define these arguments explicitly in

order for them to work properly.

Similarly,

gets fired as part of the model instantiation process, but at the end instead of the beginning, once all the arguments have been mapped to the appropriate attributes based on the fields that were defined on the model. Therefore, as the name implies, the object is completely initialized at this point.

It would make sense, then, that when

fires, it gets passed the fully configured

model instance as well as the standard

, which is the model class. The new object is

passed in as the

argument to the listener, which can then do with it whatever is necessary, according to the application.

C H A P T E R 3

฀ MODELS

79

pre_save and post_save

When a model instance is being committed to the database, Django provides two ways to hook into that process, both at the beginning and at the end. The primary difference, therefore, between the two is that

is called before the object was committed to the database, while is called afterward. This simple distinction can be very important, depending on the needs of the application.

When triggered by

, a listener receives the model class as

, and also the

instance of the model as

. This allows the listener to get access to—and even modify—

the instance that’s about to be saved, before it hits the database. This can be a useful way to provide or override default arguments for models provided by third- party applications.

On the other hand,

is called after the save has been performed, and the instance has been committed to the database. This is a useful step in two ways, because it not only ensures that the data is in fact present in the database, which is necessary when dealing with related models, but it also occurs after Django has made the decision about whether to insert a new record into the database or update an existing record.

In addition to the

and

arguments that work the same way as in

,

listeners for

can receive another argument. The

argument is a Boolean indi-

cating whether or not the instance had to be created from scratch. A value of means it was

newly inserted into the database, while

means an existing record was updated. When

using the

signal to track database changes, this is an important distinction, and can be used to determine the behavior of other applications. To see this in action, see the history example in Chapter 11 of this book.

Because a model manager’s

method does in fact commit a new instance to the

database, it fires both of these signals. It’s also safe to assume that any time is used,

the created argument will be

, but just remember that there may well be other times when that argument is also

.

80

C H A P T E R 3

฀ MODELS

A NOTE ABOUT COMBINING PRE_SAVE AND POST_SAVE

There’s another very important difference between

and

, because they’re not always

called as a pair. Because

is triggered at the beginning of the process, you can reliably assume that it will always be called every time a

is initiated. However,

only happens at the

end, so if anything goes wrong during the save itself,

won’t get triggered.

This is an important distinction, because it may seem convenient to register a pair of listeners for the model saving signals, expecting that both will always be called every time. While that may be true for the majority of cases, and certainly when nothing goes wrong, things do go wrong sometimes. Examples include an entry with a duplicate primary key or other unique column, data being of the wrong type or a timeout connecting to the database.

In situations where this type of behavior is required, the only reasonably sane way to go about it is to override the

method on the model. This allows custom code to be run before and after the actual database interaction, but it also provides a way to identify problems that occurred in the process. In addition, it allows the code a better opportunity to pair the two pieces of functionality more fully, since if something does go wrong, it’s easier to identify, and thus any pending actions can be canceled as a result.

pre_delete and post_delete

Similar to the previous section in spirit,

and

are the pair of signals

relating to the deletion of model instances. They function almost identically to their saving counterparts, except that they both provide just the

and

arguments.

When using

, keep in mind that the instance passed in to the listener will have already been removed from the database, so many of its methods will raise exceptions if used. This is especially true if it had previously related to instances of other models. Those relationships will have been lost by the time

is triggered, so any handling of those

situations should be done in

or by overriding the

method on the model.

Also, because the instance will have been deleted, its primary key value will no longer match up with anything in the database. However, in order to more accurately keep track of which object was deleted, the primary key value is left intact on the instance, and can be read using the

shortcut described earlier in this chapter.

post_syncdb

Unrelated to a specific model,

is instead triggered as part of the

man-

agement command’s normal process. It provides a way for applications to identify when an application’s models have been installed into the database, in order to perform other tasks based on their definitions.

While there are likely other uses for this as well, the primary use for is to

either configure the application itself the first time its models are installed in the database, or to identify other applications that are being installed, taking action appropriately. Within Django itself, there are examples of both types of functionality.

C H A P T E R 3

฀ MODELS

81

฀

฀

฀

application uses it to install permissions for new models into the database, as soon as the models are installed, as well as to create a new superuser if the

application itself was just installed.

฀

฀

฀

application uses it to maintain its own record of

what models are in use, so it can provide relationships to any installed model.

฀

฀

฀

application uses it to install a default site for all new projects that use the application.

The key to making

considerably effective is that it uses a different type of value for the

argument that accompanies all signals. Instead of using a specific model, this signal sends the application’s

module, which is the object Django uses to identify

an application. This allows a listener to be configured either for all applications or just the one that registered it.

All applications listed in the

setting emit a

signal every time

the command is executed, even if nothing has changed. Therefore, in addition to

,

listeners of

receive three additional arguments to indicate with more detail the circumstances under which

was called, and help control their behavior in response.

฀

฀

—The application object (its

module) representing the application that was

just synchronized with the database. This is exactly the same as the argument,

but is named

here to make listener functions a bit more readable.

฀

฀

—A Python

containing all the models for the application that were

actually installed into the database during the execution of

. This is how a lis-

tener can identify just those models that are new, which is usually the most important thing a

handler needs to know. This will always be provided, but in the case of an application where nothing is new, it will simply be an empty

.

฀

฀

—An integer identifying the verbosity level requested by the user who executed

. Valid values are , and , with being minimal output (nothing in most cases), being normal output and being all output (including messages indicating actions being performed, even if they don’t require user input). Listeners for should always be prepared to output what activities they’re performing, and should use this argument to identify when different messages should be displayed.

82

C H A P T E R 3

฀ MODELS

Code for

listeners is generally placed in an application’s

file, which is automatically loaded whenever

is used for a project containing that

application. This ensures that it doesn’t get unnecessarily loaded for situations where it’s not needed, while also making sure that it does get loaded whenever it might be necessary.

Also, since it’s Python, code in

can do other things as well, such as inspect the

setting and decide whether the listener should even be registered at all.

Applied Techniques

Given the wide array of tools available for individual models to customize their behavior, their interaction with the database, and that of the field associated with it, the options are nearly limitless. The techniques that follow represent just a small portion of what’s possible.

Loading Attributes on Demand

When working with certain types of data, it’s sometimes quite expensive to construct a complex Python object to represent a given value. Worse yet, some parts of the application might not even use that object, even though the rest of the model might be necessary. Some examples of this in the real world are complex geographic representations or large trees of nested objects.

In these cases, we must be able to get access to the full object when necessary, but it’s very important for performance to not have that object constructed if it won’t be used. Ideally, the data would be loaded from the database when the model is instantiated, but the raw value would just sit on the instance without being loaded into the full object. When the attribute is accessed, it would be constructed at that point, then cached so that subsequent accesses don’t have to keep reconstructing the object.

Looking back again to Chapter 2, descriptors are the perfect tool for this task, since they allow code to be run at the exact moment an attribute is accessed. Some care must be taken to make sure that the fully constructed object is cached properly for future use, but by using a separate

and

, this is also fairly straightforward.

C H A P T E R 3

฀ MODELS

83

To illustrate how this would work in practice, consider a field designed to store and retrieve a pickled copy of any arbitrary Python object. There’s no way to know in advance how complicated the Python representation will be, so this is a situation where it’s ideal to delay the construction of that object until it’s actually necessary.

Storing Raw Data

The first step is to tell Django how to manage the raw data in the database, using a standard field. Since pickled objects are just strings, some form of text field would clearly be prudent, and since there’s no way to know in advance how large the pickled representation will be, the nearly limitless

seems like an obvious choice.

Of course, given that there will be some extra work going on for this new field, alone won’t suffice. Instead, we’ll create a subclass that inherits the database functionality of

, while allowing extra customizations where necessary. Since fields are just Python classes like any other, this works just like you’d expect, but with one addition. In order to interact with the database using a different value than is used to interact with other Python code, the

attribute needs to be different than the

attribute. This is controlled by a cus-

tom

method.

This much alone will suffice for getting the field set up properly for the database. At this point, it’s even possible to assign a

instance to a model and sync it with the data-

base, and the column created will be perfectly usable for the duration of this example. Of course, it only manages the raw data so far; it won’t be able to handle real Python objects at all, much less deal with pickling and unpickling as necessary.

Pickling and Unpickling Data

To make the translation between a full Python object and a string representation that can be stored in the database, Python’s pickling modules9 will be the tool of choice. There are actually two separate modules provided by Python for this purpose:

, written in C for

improved performance, and

, written in pure Python for flexibility and portability. There are some minor differences between the two,10 but they can be used interchangeably.

Having two modules available makes importing a bit trickier than usual. For obvious reasons, it’s very valuable to have the greater performance when it’s available, but a key aspect of Python and Django is the ability to be used across multiple platforms and environments.

Therefore, when looking to import a pickling module, it’s best to try the more efficient module first, falling back to the more portable module when necessary.

9.

10.

84

C H A P T E R 3

฀ MODELS

With a

module available, we can give

the ability to actually pickle and

unpickle data. By providing a couple basic methods, it’s possible to interface with the underlying module in a more object- oriented manner. In addition, it’s safe to assume that when preparing to commit to the database, the field’s value will be the full Python object, which obviously must be pickled.

On the other hand, when using a QuerySet’s

method to make comparisons

against values in the database, pickled data will be quite useless. It would technically be possible to pickle the query’s value to compare against that found in the database, but it would be comparing the pickled values, not the original Python objects, which could lead to incorrect results.

More importantly, even though a pickled value is guaranteed to be unpickled properly when necessary, it’s quite possible that the same value, pickled on different occasions or possibly on different machines, will have different strings representing the original object. This is a documented side effect of the way pickling works, and must be taken into account.

With all of this in mind, it’s unreasonable to allow any kind of comparison against pickled data, so an exception should be thrown if such a comparison is attempted. As described previously in this chapter, that behavior is controlled by

, which can be

overridden to throw such an exception. The full field thus far follows: Note that

and

only support pickled data strings as plain byte strings, not as full Unicode strings. Since everything in Django gets coerced to Unicode wherever possible, including retrieving from the database,

needs to take the extra step of forcing it

back to a byte string in order to be unpickled properly.

C H A P T E R 3

฀ MODELS

85

WHY THE EXTRA METHODS?

It may seem odd to define separate

and

methods, when the pickling module is

already available in the module’s namespace. After all, it’s not only extra lines of code for you, the developer, to write, but it’s also an extra function call that Python has to go through to get the job done, which slows things down slightly, and seemingly unnecessarily.

The biggest advantage of doing it this way is that if any other application has need to subclass and wishes to override exactly how the data gets pickled and unpickled, having explicit methods for it makes that process considerably easier. They can just be overridden like normal, and as long as the rest of

just references the methods, the subclass will work quite well.

This gets us one step closer, now that

can store values in the database prop-

erly. However, it still doesn’t solve the main issue of loading data into a Python object, and doing so only when it’s really necessary.

Unpickling on Demand

If we weren’t concerned with performance, it’d be easy to perform the unpickling step in the method and just use

to make sure it happens every time an object

is instantiated, regardless of where it came from. Unfortunately, that would incur a good deal of unnecessary overhead for those cases where this field wouldn’t be accessed, so it’s still well worth loading it up on demand, only when it’s requested.

As mentioned earlier, Python descriptors are particularly well suited for this scenario.

They get called when an attribute is accessed, and can execute custom code at that time, replacing standard Python behavior with something designed for the task at hand.

The first step is determining how to instantiate the descriptor, which also means identifying what data it will need in order to get the job done. In order to retrieve the raw data from the model instance properly, it’ll need access to the field object, from which it can gather the name of the field itself.

That will store references to all the features of the field that will be useful later on. With those in place, it’s possible to write the

and

methods that will actually do

the hard work in the long run. Actually,

is the easier of the two to implement; it just

has to assign the raw data to the instance’s namespace directly.

With that in place, the trickiest bit of this whole process is the descriptor’s method, which must be able to perform the following tasks in order to work properly.

86

C H A P T E R 3

฀ MODELS

1. Identify whether or not the full Python object needs to be created.

2. Generate a full Python object, by way of unpickling the raw data, only when necessary.

3. Cache the generated Python object for future use.

4. Return the cached copy of the object if it’s available, or the new one otherwise.

That last one’s actually a bit of a red herring, since it’s easy to make sure that a Python object is available at the end of the method, and just return that, without regard to where it came from. The rest, though, may look like quite a laundry list, but it’s really not that difficult to perform all those tasks in a small, readable method.

It should be fairly clear how this method performs each of the requirements. The first block checks for accesses from the model class, raising an appropriate exception. The second block does three more tasks, by first checking for the presences of a cached copy, and continuing otherwise. Then, it does two more in one line, unpickling the raw data and storing it in the cache if the cache wasn’t already populated. At the end, it simply returns whatever’s in the cache, regardless of whether it was in the cache when the method began.

Putting It All Together

The only thing left to make the whole thing work is to get the descriptor on the model at the right time, so it’s in place to get called when the attribute is accessed. This is precisely the intent of

, where Django already provides a way for third- party code, such as this, to tie into the model creation process. Just make sure to always call the method on the parent class as well, to make sure that all the standard Django functionality is applied as well as the application’s more specialized requirements.

With all of that now in place, we have a total of three import statements, two new classes and one new field that performs a very useful task. This is just one example of how this technique can be put to use, and there are as many more as there are applications using complicated Python data structures. The important thing to take away from this example is how to use descriptors to populate those complex objects only when necessary, which can be a big win in situations where they might not always be used.

C H A P T E R 3

฀ MODELS

87

Creating Models Dynamically at Runtime

Chapter 2 demonstrated how Python classes are really just objects like any other, and can be created at runtime by using the built- in

constructor and passing in some details about

88

C H A P T E R 3

฀ MODELS

how it should be defined. Since Django models are really just Python declared in a specific way, it’s reasonable to expect that they could also be created at runtime using this same feature. Some care must be taken, but this can be an extremely useful technique in a variety of situations.

The trick is to remember how Python processes classes, and how Django processes its models. Chapter 2 already illustrated the basic tools necessary to make this work, so it’s now just a matter of applying that to the specific details of Django models. There are a few things that set models apart from other Python classes:

฀

฀

฀

฀

฀

.

฀

฀

฀

฀

฀ ฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀ ฀

class inside the model’s declaration.

With these requirements outlined, it’s fairly easy to map a model declaration onto the arguments for

. In particular, remember that there are three arguments required to construct a class:

,

and

. The model’s name is clearly mapped to

, while the single

subclass of

can be wrapped in a tuple and passed into

. The remainder of

the class declaration would go into

, including a

class for any additional model- level

configuration options.

A First Pass

To make a first pass at what this function might look like, let’s start with just the most basic aspect of class creation and work our way out from there. To begin with, consider a function that generates a class with the correct name and base class, to illustrate the basic technique for creating a class dynamically and returning it for use elsewhere.

Unfortunately, that’s actually a little too simplistic. Trying this out in Python will result in a

, because Django expects the attribute dictionary to include a key, with

its value being the import path of the module where the model was defined. This is normally populated by Python automatically for all classes defined in source files, but since we’re generating a model at runtime, it’s not available.

This is just one of the minor details that dynamic models have to face, and there’s really no way of avoiding it entirely. Instead,

needs to be updated to provide a

attribute directly. This is also another example of why it’s a good idea to put this code in one place; imagine having to deal with this every time a dynamic model is required. Here’s what it looks like to include a module path for the class:

Now it can accept a module path and keep Django happy. Well, it can keep Django happy as long as the module path has already been imported, which means it has to actually exist.

Under normal circumstances, the model’s

attribute is set to the path of the module

C H A P T E R 3

฀ MODELS

89

where it was defined. Since the model will only be processed while executing that module, it’s always guaranteed that the module will exist and have been imported successfully. After all, if it hadn’t, the model would’ve been encountered in the first place.

For now, since the only requirement of the module path is that it be valid and already imported, Django’s own

will make a reasonable candidate. It should be

overridden where appropriate, of course, but it’s a decent default until things get rolling.

Clearly, these dynamic models shake things up quite a bit, bypassing much of how Python normally works with a process like this. The

issue is just the first issue encountered,

and one of the easiest to work around. Thankfully, even though there are a few others to be handled, it can be well worth it if used properly.

The next step in this basic example is to include a dictionary of attributes to be set as if they were declared directly on a class definition. This will allow fields to be included on the model, as well as custom managers and common methods like

. Since we’re

already passing a dictionary to be used as attributes, assigning additional items to that dictionary is a simple process.

Ordinarily, it’s not advisable to supply a mutable object, such as a dictionary, as a default argument, since modifications to it would affect all future executions of the function. In this example, however, it’s used only to populate a new dictionary, and is immediately replaced by that new dictionary. Because of this, it’s safe to use as the default argument, in an effort to keep the method reasonably succinct.

So far, we’ve set up a three-line function to create basic models with any number of attributes, which can then be used in other areas of Django. Technically, this function alone could be used to generate any model imaginable, but it already provides a shortcut for setting up

, so it would make sense to provide another shortcut for setting up the model configuration by way of a

inner class. That way, code to create a model won’t have to set up that class directly.

Adding Model Configuration Options

Django models accept configuration through an inner class called

, which contains attri-

butes for all the options that are specified. That should sound familiar, since that’s basically what models themselves do as well. Unfortunately, because of how Django processes the class, we have to take a different approach.

The attributes defined within

are passed along into a special

object, which

lives at

. As part of this process,

makes sure that no attri-

butes were supplied that it doesn’t know how to handle. Unfortunately, because the fact that is a class is just a way to separate its namespace from that of the main model.

only knows how to handle old- style Python classes—that is, classes that don’t inherit from the built- in

type.

90

C H A P T E R 3

฀ MODELS

This is an important distinction, because calling

directly creates a new- style class,

even if it doesn’t inherit from

, or any subclasses for that matter. This ends up creating two additional attributes on the class that

doesn’t know how to deal with, so it raises

a

to indicate the problem. That leaves two options for creating a class: removing the additional attributes or creating an old- style class using some other means.

While it would be possible to just remove the attributes that offend

, an even bet-

ter idea would be to provide it exactly what it expects: an old- style class. Clearly, using is out of the question, which leaves us with just declaring a class using standard syntax. Since this is possible even within functions, and its namespace dictionary can be updated with new attributes, it’s a decent way to go about solving this problem.

This will now accept two attribute dictionaries, one for the model itself, and another for the

inner class. This allows full customization of Django models that can be created at any time. While this may seem like a rather abstract concept at the moment, see Chapter 11

for a full example of how this can be used in practice to automatically record all changes to a model.

Now What?

With a solid foundation of Django’s models under your belt, the next step is to write some code that will allow users to interact with those models. The next chapter will show how views can provide your users with access to these models.

C H A P T E R 4

URLs and Views

Much of this book is split into fairly self- contained chapters, but this one covers two seemingly unrelated concepts together, because each relies very much on the other. URLs are the primary entry points to your site, while views are the code that respond to incoming events.

What goes on in a view is very open- ended. Aside from accepting a request and returning a response, there’s no particular protocol that views should adhere to, and no rules about what they are or aren’t allowed to do.

The possibilities for views are too vast to consider describing in detail, and there aren’t any utilities designed explicitly for views to use while executing. Instead, it’s possible to hook into the process Django uses to map Web addresses to the views they should execute. This makes the link between URLs and views extremely important, and a thorough understanding of it can enable further advanced techniques.

Also, in terms of how Django manages incoming requests, URL configurations exist solely to dispatch a request to a view that can handle it. Discussing URLs and URL configurations independently of views would be of little value.

URLs

Since all incoming requests to a Web server originate with the Web browser accessing a URL, a discussion of URLs is an important place to start. The process taken by the browser to transform a URL into a message to be sent to the Web server is beyond the scope of this chapter, but Chapter 7 provides more information.

One common point of confusion is whether a Web address should be called a Uniform Resource Identifier (URI) or a Uniform Resource Locator (URL). Many people use these two terms interchangeably, regardless of whether they know the difference. In a nutshell, a URI is a complete addressing mechanism that includes two pieces of information.

฀

฀

฀

฀ ฀

฀

฀ ฀

฀ ฀ ฀

฀ ฀

฀ ฀

฀

฀

฀ ฀

always followed by a single colon.

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀

฀ ฀

฀

฀

฀ ฀

-

ent for different schemes, so not all URI paths look alike.

URLs, on the other hand, are addresses from a small set of connection schemes whose path portions all conform to a single format. Included in this set are such common protocols 91

92

C H A P T E R 4

฀ URLS AND VIEWS

as HTTP, HTTPS and FTP—essentially the common protocols found on the Web today. The path format shared by these protocols is as follows:

฀

฀

฀

฀ ฀ ฀used to access the resource, such as

for standard HTTP.

This is a slight extension to the scheme portion of the URI because it is assumed that all URL protocols will include two forward slashes following the colon.

฀

฀

฀

฀

฀

฀the resource can be found, such as

.

฀

฀

฀

฀

฀

฀

฀server responds to. Each protocol has a default port that will be used if one isn’t supplied. For standard HTTP, this is

, while for encrypted

HTTP using the Secure Sockets Layer (SSL), it will be

.

฀

฀

฀

฀ ฀

฀resource on the server, such as

.

So while all URLs are certainly URIs, not all URIs are URLs. That subtle distinction can be confusing when working on the Web because either term can be used to describe the addresses found everywhere. Since Django is built for the Web—and thus the addresses covered under URL schemes—the rest of this book will refer to these addresses as URLs, as the full range of URIs might not be suitable for Django’s dispatching mechanism.

DESIGNING CLEAN URLS

In an ideal world, the URLs you choose when setting up a site the first time will never change,1 remaining intact until the documents—or the entire server—are no longer maintainable. Changing URLs simply because of a redesign or reorganization of the site is generally bad form and should be avoided.

The key to making URLs maintainable for the long haul and making it easier for your users to keep track of them, is to design them well in the first place. Django makes this easy, allowing you to design your URLs in whatever hierarchy you like, assigning variables right in the URL and splitting the URL structure into manageable chunks.

Above all, URLs are part of your application’s user interface, since users have to see them, read them and often type them in manually. Keep this in mind when designing your URLs.

Standard URL Configuration

Django doesn’t provide any features for automatically discovering or generating a URL structure for any site. Instead, each site and application is expected to explicitly declare whatever addressing scheme is most appropriate using URL configurations. This isn’t a limitation—it’s a feature that allows you to define your site’s addresses the way you’d like. After all, sites on the Web are like real estate; your Web framework shouldn’t determine your floor plan.

Defining a URL configuration may seem quite simple, but there’s a bit going on that merits some special attention, especially since Django’s own tools aren’t the only way to define this configuration. The implementation lives at

and provides two

functions that work together to manage URL configurations.

1.

C H A P T E R 4

฀ URLS AND VIEWS

93

The patterns() Function

A URL configuration consists of a list of patterns that each map a particular type of URL to a view. These patterns each have a few components but all of them are specified together as arguments to the

function.

The arguments for this function can be placed in two groups:

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀

฀ ฀

฀

Historically, all views were specified as strings, so the prefix was a great way to reduce the amount of duplication required to map URLs to views from a single application. More recently, URL patterns are allowed to specify views as callables, in which case the prefix would be ignored. It is still often useful to specify views as strings using a prefix, as it reduces the overall code by not requiring a set of imports for the views.

The URL patterns are traditionally passed in as tuples, though “The url() Function” section describes a more recent addition. Details of each portion of this tuple are as follows:

฀

฀ ฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀ ฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀ ฀

฀ ฀

฀ ฀ ฀

฀ ฀

฀

This tuple contains all the information necessary to map an incoming request to a view function. The URL’s path will be checked against the regular expression, and if a match is found, the request is passed along to the specified view. Any arguments captured by the regular expression are combined with the explicit arguments in the extra dictionary, then passed along to the view along with the request object.

MULTIPLE ARGUMENTS WITH THE SAME NAME

A single URL configuration can provide values in two separate ways: in the URL’s regular expression and in the dictionary attached to the pattern. Accepting arguments from two different sources makes it possible to provide two different values for the same key, which needs to be resolved somehow. If you try doing this with keyword arguments to a standard function, Python will raise a as described in Chapter 2.

Django allows these multiple arguments to be specified without raising an exception, but they can’t all be passed to the view together. As the second portion of this chapter shows, views are called just like any normal Python function, so these multiple arguments would cause the same described in

Chapter 2. To resolve this issue without an error, Django has to reliably choose one instead of the other. Any argument provided with a dictionary in the URL configuration will take priority over anything found in the URL.

94

C H A P T E R 4

฀ URLS AND VIEWS

It’s bad form to provide multiple arguments with the same name in this manner, since it relies heavily on Django’s handling of the situation to work properly. While that behavior isn’t likely to change on a whim, relying on it could cause problems in the future. More importantly, specifying the same argument name in multiple places greatly reduces the readability of your URL configurations. Even in closed- source applications, someone else will likely need to read your code long after you’re done with it.

The url() Function

In an effort to provide better flexibility in the long run, URL pattern tuples have been depre-cated in favor of the

utility function.

takes the same arguments that are passed

into the tuple, but can also take an extra keyword argument to specify the name of the URL

pattern being described.

This way, a site can use the same view multiple times, yet still be able to be referenced using reverse URL lookups. More information on that can be found later in this section.

The include() Function

Rather than supplying all your URL patterns in a single file, the function allows

them to be split up among multiple files. It takes a single argument: an import path where another URL configuration module can be found. This not only allows the URL configuration to be split across multiple files, but it also allows the regular expression to be used as a prefix for the included URL patterns.

One important thing to remember when using

is to not specify the end of the

string in the regular expression. The expression should never end in a dollar sign ( ). The dollar sign ( ) causes the expression to only match the full URL. This wouldn’t leave any additional URL fragments to pass along to the included configuration. This means that the extra URL patterns would only be matched if they check specifically for an empty string.

Resolving URLs to Views

Views are rarely called directly by your own code but are instead invoked by Django’s URL

dispatch mechanism. This allows views to be decoupled from the particular URLs that trigger them, and the details of how those two aspects are linked can be safely ignored for most projects. But since views don’t always have to just be simple functions, knowing how Django goes from URL to view is important in order to determine what views are truly capable of.

Mapping URLs to views is a simple, well- documented process, but it’s worth covering the basics here for reference. A typical URL pattern consists of a few distinct items:

฀

฀ ฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀ ฀

฀

฀ ฀ ฀

฀

฀ ฀

฀ ฀

฀ ฀ ฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀ ฀

฀ ฀ ฀

฀ ฀

฀

฀

฀

฀

฀

Since URL patterns are expressed in regular expressions, which can capture certain portions of a string for later use, Django uses this as a natural way to pull arguments out of a URL

C H A P T E R 4

฀ URLS AND VIEWS

95

so they can be passed to a view. There are two ways these groups can be specified, which determine how their captured values are passed into the view.

If groups are specified without names, they’re pulled into a tuple, which is expanded into excess positional arguments. This approach makes the regular expression a bit smaller, but it has some drawbacks. Not only does it make the regular expression a bit less readable, it also means that the order of arguments in your view must always match the order of the groups in the URL, because Django sends them in as positional arguments. This couples the URL to the view more than is usually preferable; in some situations, such as the object- based views described later in this chapter, it can still be quite useful.

If groups are given names, Django will create a dictionary mapping those names to the values that were extracted from the URL. This alternative helps encourage looser coupling between URLs and views by passing captured values to the view as keyword arguments. Note that Django doesn’t allow named and unnamed groups to be used together in the same pattern.

Resolving Views to URLs

As alluded to in the previous section, there’s another URL resolution process that Django provides, which can be of even more use if applied properly. Applications often need to provide links or redirects to other parts of the application or elsewhere on the site, but it’s not usually a good idea to hard- code those links directly. After all, even proprietary applications can change their URL structure, and distributed applications may not have any idea what the URL

structure looks like in the first place.

In these situations, it’s important to keep the URLs out of the code. Django offers three distinct ways to specify a location without needing to know its URL in advance. Essentially, these all work the same way, as they all use the same internal machinery, but each interface is suited for a particular purpose.

The permalink Decorator

One of the most obvious places for code to reference a URL is in the method

of most models. Providing this method is a common convention, so templates can easily provide a direct link to an object’s detail page without having to know or care what URL or view is used to display that page. It doesn’t take any arguments and returns a string containing the URL to be used.

To accommodate this situation, Django provides a decorator, living at

, which allows a function to return a set of values describing a view to be called, transforming it into a URL that calls the view. These values are provided as the return value from a function such as the

method and follow a specific structure—a tuple

containing up to three values.

฀

฀

฀

฀

฀ ฀

฀

฀ ฀

฀

฀ ฀ ฀

฀ ฀

฀

฀

฀

฀

฀

฀

should be used here. If not, the import path of the view should be used instead. This is always required.

฀

฀

฀

฀

฀ ฀ ฀

฀ ฀

฀

฀

฀

฀ ฀

฀ ฀

฀

฀

If there are no arguments to be applied to the view at all, this value doesn’t need to be provided, but if keywords are needed, this should be an empty tuple.

96

C H A P T E R 4

฀ URLS AND VIEWS

฀

฀

฀

฀

฀ ฀

฀

฀ ฀ ฀

฀

฀

฀

฀ ฀

฀

-

ues, all of which will be passed to the specified view. If no keyword arguments are necessary, this value can be left out of the tuple.

Given the following URL configuration:

a corresponding model (located in a

application) might look like this:

The url Template Tag

Another common need is to have templates provide links to views that aren’t based on models but still shouldn’t have a hard- coded URL. For instance, a link to a contact form doesn’t necessarily have any ties to the database or any models, but will still need to be linked to in a way that can accommodate future changes or distribution.

The syntax for this template looks quite similar to the

decorator because it

passes values to the same utility function. There are some slight differences, because as a template tag, it doesn’t use true Python code.

The reverse() Utility Function

Django also provides a Python function that provides the translation from a description of a view and its arguments to a URL that will trigger the specified view. Living at

, the

function does exactly that. It takes all the same arguments described for the previous two techniques, but also one other, allowing it to specify which URL configuration module should be used to resolve the URL. This function is used internally by both the

decorator and the

template tag. The

function takes up to

four arguments.

C H A P T E R 4

฀ URLS AND VIEWS

97

฀

฀

—The name of the view to be called or the import path if no name was specified. This is always required.

฀

฀

—The import path of a URL configuration module to use for lookups. This is optional and if it’s absent or

, the value is taken from the

setting.

฀

฀

—A tuple of any positional arguments that will be passed to the view.

฀

฀

—A dictionary of any keyword arguments that will be passed to the view.

Using the same example as in the previous section, here’s how would be used

to obtain a URL for a specific object.

Keep in mind that

and

are separate, distinct arguments. The

utility

function does not use any form of the argument expansion described in Chapter 2.

POSITIONAL VS. KEYWORD ARGUMENTS

To illustrate best practice, the examples in this section all use named groups in the URL’s regular expression, which allows—in fact, requires—the reverse resolution to specify arguments using keywords. This greatly improves the readability and maintainability of your code, which is a primary goal of writing Python.

It is possible, though, to specify URLs without naming the capture groups, which requires reverse resolution to use positional arguments only.

For example, if the URL pattern was defined as

, here’s how the previous

examples would have to be written in order to work properly:

฀ ฀

฀

decorator—

฀ ฀

฀

template tag—

฀ ฀

฀

function—

Since a URL configuration only allows positional arguments or keyword arguments, but not both, there’s no need to specify both types together in the same reverse resolution call.

Views

One point of confusion for programmers coming from other environments is the fact that Django uses the term “view” a bit differently than others. Traditionally, the view in a Model-View- Controller (MVC) architecture refers to the display of information to a user—essentially, the output portion of a user interface.

The Web doesn’t work like that. Viewing data is typically a direct result of a user action, and updates to that view only take place as responses to subsequent actions. This means that the output process is irrevocably linked to the user input process, which can cause some 98

C H A P T E R 4

฀ URLS AND VIEWS

confusion about how even the traditional MVC pattern should define a view. So there is no simple answer to the question of how Django’s views compare to those of other environments because there isn’t anything solid to compare against. People from different backgrounds are likely to have different expectations about what a view should be. The bad news is that Django probably doesn’t line up with any of them. The good news is that once you start working with Django, the notion of a view is clearly defined, so there’s little confusion when communicating with other Django developers.

Templates Break It Up a Bit

Django’s views do perform the basic function of the output interface, because they’re responsible for the response that is sent to the browser. In a strict sense, this response is the entire output, and it contains all the information about what the user will see. This is often too much work to do in Python while still making it readable, so most views rely on templates to generate the bulk of the content.

The most common practice is to have each view call a single template, which may make use of a number of tools to minimize the amount of template code that must be written for use by a particular view. Chapter 6 includes further details on the template language and the tools that can be used, but the important thing to know for this section is that templates are a great way to simplify the coding process as a whole. They help cut down on the amount of code that must be written, while simultaneously making that code more readable and maintainable for the future.

While Chapter 1 listed templates as a separate layer, remember that they’re really just a tool that Django makes available to other parts of an application, including views. Ultimately, whether or not templates are used to generate content, the view alone is responsible for generating the final response. Django’s template system has no concept of requests or responses; it just generates text. It’s up to views to handle the rest.

Anatomy of a View

A view is a function that takes an HTTP request and returns an HTTP response. That is a bit simplistic, given the potential power of views, but that’s really all there is to it. A view always receives, as its first argument, the

created by Django, and it should always return

an

, unless something went wrong. Full details on those objects, their purpose and their properties are covered in Chapter 7.

The first aspect of that definition is the notion that a view must be a standard function.

This definition is a bit flexible because in reality, any Python callable can be used as a view; it just happens that basic functions are easy to work with and provide everything that’s necessary for most situations. Methods—both on classes and instances—and callable objects, using the protocol described in Chapter 2, are all perfectly valid for use as views. This opens up a variety of other possibilities, some of which will be described later in this chapter.

The next point is the one immutable when it comes to views. Whenever a view is called, regardless of what other arguments are passed along, the first argument is always an

object. This also means that all views must accept at least this one object, even those views that don’t have use for any explicit arguments. Some simple views, such as those that display the server’s current time, may not even use the request object, but must always accept it anyway to fulfill the basic protocol of a view.

On the subject of arguments, another point is that a view must be able to accept whatever arguments are passed to it, including those captured from the URL and those passed into the C H A P T E R 4

฀ URLS AND VIEWS

99

site’s URL configuration. This may seem obvious, but a common point of confusion is the presumption that Django uses some kind of magic to allow a URL configuration to specify which template should be used, without requiring any supporting code in the view.

Django’s generic views all accept a separate argument to modify the template name, and many users assume that Django somehow passes this straight through to the template system to override what name the view uses by default. The truth is that the generic views have special handling for this argument, and the view itself is responsible for telling the template system which template to use. Django relies on standard Python, so there’s no magic behind the scenes that tries to interpret what your arguments are supposed to mean. If you plan to supply an argument to a function, make sure that the view knows how to deal with it.

The last notion from that original description of views is that a view must return an object, and even that isn’t entirely accurate. Returning a response is definitely the primary goal of all views, but in certain situations it’s more appropriate to raise an exception, which will be handled in other ways.

What goes on between request and response is largely unrestricted, and views can be used for as many purposes as there are needs to be met. Views can be built to serve a specific purpose or they can be made generic enough to be used in distributed applications.

Writing Views to Be Generic

A common theme in Django development is to make code as reusable and configurable as possible so that applications and snippets are useful in more than one situation, without having to rewrite code for every need. That’s the whole point of DRY: Don’t Repeat Yourself.

Views present a bit of a challenge with regards to DRY, since they’re only called by incoming requests. It may seem like it wouldn’t be possible to write a view that could be called for anything other than the request it was originally intended for. Django itself, however, is full of examples of generic views, which can be used for a variety of applications and situations with only a small amount of configuration necessary for each new use.

There are a few guidelines that can greatly aid the reuse of views, making them generic enough to be used throughout a variety of applications. Views can even be made so generic that they can be distributed to others and included in projects the original author had no concept of.

Use Lots of Arguments

Typically, a view could perform quite a few different tasks, all combining to solve a particular problem. Each of these tasks often has to make assumptions about how it should work, but these assumptions can typically be pulled out into a configurable option using arguments.

Consider the following view, designed to retrieve a blog post and pass it along to a template.

100

C H A P T E R 4

฀ URLS AND VIEWS

This view will work perfectly well for its intended purpose, but it’s quite tightly connected to a specific blog application. It’s still loosely coupled in the sense that it doesn’t need to deal with the details of how to retrieve the blog post or render the template, but still relies on details specific to the blog application, such as the model and template.

Instead, it’s possible to move these assumptions into arguments that can be swapped out for other situations. While initially this will involve some extra work, it can save a lot of time later, if this view is used in a great number of situations. More importantly, the more complex the view, the more code that can be reused using this technique. Once these options have been moved out into arguments, specific values can be passed in with a URL configuration, so a view doesn’t have to be written for each purpose.

For this particular view, a few things can be factored out in this way. The model doesn’t need to be known in advance and the view should also be able to work with a QuerySet so that a particular URL could operate on a limited set of data. Also, the field name shouldn’t be hard- coded, and the template name should be provided outside the view.

Then, when it comes time to use this view, it’s easy to customize by providing these details using a URL configuration. Simply supply the argument values as an extra dictionary in the URL

configuration, and they’ll be passed along each time the view is called from that URL pattern.

This approach can even be used with models that use other types of IDs, such as a music database using catalog numbers in the format of DJNG- 001; anything that can be guaranteed unique among all objects can be used as an object’s primary key. Since our new generic view simply passes the ID straight through to the database API, it’s easy to support these other types of IDs by simply adjusting the URL pattern appropriately.

This particular view shouldn’t have to be written in the first place, because Django provides one out of the box for this purpose,

, and it’s even more versatile than the

example shown here. It uses nearly a dozen different arguments, all of which are expected to be customized in URL configurations.

C H A P T E R 4

฀ URLS AND VIEWS

101

Once you have a view that accepts a number of arguments for customization, it can become quite easy to require far too many arguments be specified in each URL configuration.

If every use of a view requires all the configuration options to be specified, it could quickly become just as much work to use the generic view as it would be to write the view from scratch each time. Clearly, there needs to be a better way to manage all these arguments.

Provide Sensible Defaults

Since functions can define default values for any arguments that can use them, the most reasonable way to manage this complexity is to provide decent defaults wherever possible.

Exactly what defaults can be provided and what they look like will be different for each view, but it’s usually possible to come up with some sensible values for them.

Sometimes you have a number of views that each serve a different purpose but may have some code in common. This is often boilerplate, which every view needs to use, but isn’t geared toward the true functionality of any individual view.

For example, views for private pages must always verify that users are logged in and that they have the appropriate permissions. An application may have a dozen different types of views, but if they’re all private, they must all use that same code every time. Thankfully, we’re working in Python, which provides a useful alternative.

View Decorators

Most boilerplate in views is either at the very beginning or the very end. Usually it handles such tasks as initializing various objects, testing standard prerequisites, handling errors gracefully or customizing the response before it goes out to the browser. The real meat of the view is what sits in the middle, and that’s the part that’s fun to write. Described in Chapter 2, decorators are a great way to wrap several functions in some common code that can be written once and tested easily, which reduces both bugs and programmer fatigue. Since views are typically just standard Python functions, decorators can be used here as well.

Chapter 2 illustrated how decorators can be used to write a wrapper around the original function, which can then access all the arguments that were intended for that function, as well as the return value from the function itself. In terms of views, this means that decorators always have access to the incoming request object and the outgoing response object. In some cases, a decorator can be special- cased for a particular application, which would allow it to anticipate a greater number of arguments that are specific to that application.

There are a number of things decorators can offer views, and a few of them are common enough to warrant inclusion in Django itself. Living at

are a few

packages containing decorators you can use on any view in any application. The following packages are listed with just the trailing portion of their full import path provided, given that they all live at the same location.

฀

฀

—Stores the output of the view into the server’s cache so that when similar requests come in later, the page doesn’t have to be re-created each time.

฀

฀

—Prevents caching for a particular view. This is useful if you have sitewide caching set up but certain views can’t afford to go stale.

฀

฀

—Compresses the output of the view and adds the appropriate HTTP

headers so the Web browser knows how to handle it.

102

C H A P T E R 4

฀ URLS AND VIEWS

฀

฀

—Only sends the whole page to the browser if it has changed since the last time the browser got a copy of it.

฀

฀

—Accepts a list of HTTP methods (described in detail in

Chapter 7) that the view is limited to. If the view is called with any other method, it sends a response telling the browser it’s not allowed, without even calling the view.

Two included shortcut variations are

and

, which

don’t take any arguments and are hard- coded for GET and POST requests, respectively.

฀

฀

—Helps control browser- based caching of pages by indicating that the page’s content changes, depending on the values of the headers passed into the decorator. A simple variant specific to the

header is available at

.

Additional decorators are provided as part of the bundled applications living at

. These decorators all live below that path, so as in the previous list, only the relevant path is supplied:

฀

฀

—A simple decorator that checks the

current user to see if it has staff access. This is used automatically for all the views in Django’s built- in admin, but could also be used for any other staff- only views on your site. If the user doesn’t have staff permissions, the decorator redirects the browser to the admin’s login page.

฀

฀

—Accepts a single argument, which is a function to

test the current user against some arbitrary condition. The provided function should accept just the

object and return

if the test passes or

if it fails. If the test

passes, the user will be granted access to the page, but if it fails, the browser will redirect to the site’s login page, as determined by the

setting.

฀

฀

—A specialized version of

, this

decorator simply checks that the user is logged in before allowing access to the view.

฀

฀

—Another specialization of

,

this checks that the user has a given permission before the view is loaded. The decorator takes a single argument: the permission to be checked.

These are just the decorators that are bundled with Django itself. There are many other purposes for decorators, and third- party applications can provide their own as well. In order for these decorators to be of any use, however, they must be applied to views.

Applying View Decorators

Chapter 2 described how decorators can be applied to standard Python functions, both the newer syntax introduced in Python 2.4 and the older syntax, to retain compatibility with Python 2.3. Applying decorators to views works the same way, but there’s a notable difference: views aren’t always under your control.

The techniques described in Chapter 2 assume that the functions you decorate are your own. While that’s often the case, the number of distributed applications means that many Django- powered Web sites will use code from other sources, with views of their own. Applying decorators as described previously would require changes to the third- party code.

C H A P T E R 4

฀ URLS AND VIEWS

103

The goal is to apply decorators to third- party views without actually modifying third- party code. The key to doing this lies in the older- style decorator syntax from Python 2.3 and earlier.

Remember that the new syntax allows decorators to be applied above the function definition, but the older syntax relies on passing the function to the decorator directly. Since Python functions can be imported from anywhere and can be passed in as arguments at any time, this is an excellent way to create decorated views from third- party code.

Also remember that the URL configuration is defined in a Python module, which gets executed when it is read. This makes the wide array of Python available to this configuration, including the ability to pass functions into decorators to create new functions.

A NOTE ABOUT DISTRIBUTING APPLICATIONS

If you’re writing an application intended for distribution to a wider audience, it’s important to be compatible with as many installations of Django as possible. Since Django itself supports Python as far back as version 2.3, take care when using decorators in views for these distributed applications.

Although it may be convenient to use the newer, @- style syntax for applying decorators, that syntax was introduced in Python 2.4, so applications using it won’t be compatible with Python 2.3. Therefore, all applications intended for distribution should use the older syntax unless there are other reasons the application can’t be supported under Python 2.3.

This suggestion may seem obvious while reading this section, with its discussion of the difference between Python 2.3 and Python 2.4, but it’s an easy mistake to make and a simple way to break a distributed application’s compatibility with various other sites.

Writing a View Decorator

Chapter 2 covered how decorators themselves work and how they can be written to work in a variety of situations, though decorators for views have a few specific details that should be noted. These have less to do with the technical side of writing decorators and more with the nuances of how to achieve certain useful effects when working with views specifically.

The most common task decorators are used for with views is to create a wrapper function around the original view. This allows the decorator to perform extra work beyond what the view itself would ordinarily do, including

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀ ฀

฀

฀

฀

฀

฀ ฀ ฀

฀ ฀

฀

฀

฀

฀ ฀

฀

฀

฀

104

C H A P T E R 4

฀ URLS AND VIEWS

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

The first thing to consider when writing a decorator is that it receives all the arguments intended for the view itself. Previous sections covered this, but only in the usual context of using

and

to receive the arguments and pass them straight through to the wrapped function. With views, you know in advance that the first argument will always be the incoming request object, so a wrapper function can anticipate this and receive the request separately from the other arguments.

By interacting with the request object prior to executing the view, decorators can do two important things: make decisions based on the incoming request and make changes to the request to alter how the view operates. These tasks aren’t mutually exclusive and many decorators do both, such as the following example from Django.

PRESERVING A VIEW’S NAME AND DOCUMENTATION

The built- in admin interface generates documentation for your application’s views using the name and docstring of the view function itself. By using decorators to wrap the function, we’re essentially replacing the original view function with the wrapper. This causes the admin interface to see the wrapper instead of the view.

Ordinarily, this would cause the name and docstring of the view to be lost in the shuffle, so the admin’s documentation feature doesn’t work properly with these views. To get the right documentation, those attributes of the function must remain intact throughout the wrapping process.

Django provides an additional decorator, living at

, which is

designed to copy these attributes onto the wrapped function so it looks more like the original view. This process is described in more detail in Chapter 9, but all the examples in this section use it to illustrate best practices for decorating views.

Another common use of decorators is to extract some common code from the beginning or end of a set of views. This can be especially useful when looking at incoming arguments, as decorators can perform any lookups and initializations prior to calling the view. Then, decorators can simply pass fully prepared objects to the view, rather than raw strings captured from a URL.

C H A P T E R 4

฀ URLS AND VIEWS

105

The great thing about a decorator like this is that, even though the logic it contains is fairly minimal, it does cut down on the amount of code that has to be duplicated for views that all get an

object according to an ID provided in the URL. This not only makes the views themselves a bit more readable, but any time you can cut down on code that has to be written, you can help reduce bugs.

Also, by having access to the response, decorators can make some interesting decisions about how that response should behave. Middleware classes, described in Chapter 7, have much more use for accessing the response, but there are still useful things decorators can do.

Of note is the ability to set the content- type of the response, which can control how the browser deals with the content once it receives it. Chapter 7 describes this in more detail and also how it can be set when creating the response. However, it’s also possible to set it after the response has already been created and returned from a view.

This technique can be a good way to override the content- type for specific types of views.

After all, if no content- type is specified, Django pulls a value from the setting, which defaults to

. For certain types of views, especially those intended for Web services, it may be better to serve them using another content- type, such as

, while still being able to use generic views.

A lesser- used feature of view decorators is the ability to catch any exceptions that are raised by the view or any code it executes. Views typically just return a response directly, but 106

C H A P T E R 4

฀ URLS AND VIEWS

there are still many situations where a view may opt to raise an exception instead. One common example, found in many of Django’s own generic views, is raising the exception

to indicate that an object couldn’t be found.

Chapter 9 covers the exceptions Django provides in its standard distribution, many of which can be raised by views for one reason or another. In addition, many of the standard Python exceptions could be raised for various situations, and it can be useful to catch any of these. A decorator can perform a variety of additional tasks when an exception is raised, from simply logging the exception to the database to returning a different type of response in the case of certain exceptions.

Consider a custom logging application with a log entry model like this: The application providing this model could also provide a decorator for projects to apply to their own views that logs exceptions to this model automatically.

The recurring theme with all these examples is that view decorators can encapsulate some common code that would otherwise have to be duplicated in every instance of the view. In essence, view decorators are a way to extend the view’s code before or after the original code.

It’s important to generalize these examples in order to realize just how much is possible with view decorators. Any boilerplate you find yourself duplicating at the beginning or end of your views is fair game to be placed in a decorator to save some time, energy and trouble.

C H A P T E R 4

฀ URLS AND VIEWS

107

Using an Object As a View

Remember, Django views don’t always have to be standard functions; they just have to be callable. As described in Chapter 2, Python provides a way to define a class in such a way that instances of it can be called as if they were functions. If defined on a class, the method will be called when the object is passed in where a function is expected.

There are as many ways to use objects as views as there are ways to define objects themselves. Aside from using

to receive each incoming request, what happens inside the object is up for grabs. In a typical situation, the request would be dispatched to individual methods based on certain criteria. This could be some aspect of the URL, the method of the request or even some parameters provided by the user.

Because of the way classes must be declared for their instances to be considered callable, using an object as a view does require a bit more code than simple functions. Object- based views are certainly not a general- purpose solution to solve common problems, but they provide a few key advantages over traditional functions.

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀ ฀

฀

฀

฀

With typical functions, views can be configured in only two ways: adding settings or providing arguments in a URL configuration. Settings are projectwide, which makes them unsuitable for configuring views, which may need different options for different purposes.

URL configurations provide the necessary hook to specify options, but typically each URL

must be configured on its own, with little chance of reuse. The only alternative is to include a separate module, where individual views can be declared and all share the same configuration 108

C H A P T E R 4

฀ URLS AND VIEWS

options specified when the module was included. This does work, but it requires that every single view in the included module know how to accept all the arguments that are provided.

In the previous example, the object allows a model’s ID to be optional, displaying the model’s detail if the ID is provided or a list of models otherwise. The view object accepts a model class that will automatically be passed into the standard generic views to provide this functionality. It would be used in a URL configuration as follows: Note how this URL pattern captures the entire end of the URL after the prefix. This is necessary to pass that content to the view, since Django doesn’t realize there’s more than just a single function at work here.

One of the biggest advantages of this approach is how easy it is to customize this object-based view in other code. If this was provided by a third- party application, for instance, an individual project may need to alter what views are called for the individual list and detail situations. By providing the view as a class, the site can create a subclass and simply override those methods that need altering.

REVERSE() WITH OBJECT- BASED VIEWS

Because the only thing provided to the URL pattern is the object, Django doesn’t know what the object will do with it after getting passed to the

method. As far as the URL configuration is con-

cerned, there’s only one URL being mapped, using just the root URL specified in the pattern. This causes to work a bit differently than you might expect.

First, no import path is associated with the object, so the view reference passed to must use

either the object itself or the name provided to the URL pattern. The previous example represents a common case, where the object is created directly inside the configuration—which doesn’t leave you with a reference to use later, so the only option left is to name the pattern explicitly. If the object is instantiated somewhere else and simply imported into the URL configuration, that same object can also be used as the view reference to

.

Since only the URL is mapped to the object in the pattern, Django doesn’t know about any branching that takes place inside the object. The

function has no way to access any of the internal

views that the object might use, such as the list and detail methods in the previous example. The only thing can access is the root URL specified explicitly, so it’s important to make sure the root URL always maps to something useful. If any of the additional methods need to be referenced directly, you’ll have to add those to the end of the resolved URL manually.

Perhaps the best time to use an object- based view is when building Web services that don’t generate standard HTML pages. Rarely would the rest of the site need to display links to the service’s URLs or an individual Web service response need to contain a link to another one. It’s typical to write a class to manage the various actions the service provides, pass in an object those actions can be applied to and pass that object into a URL configuration and simply describe its URLs in the documentation for the service.

C H A P T E R 4

฀ URLS AND VIEWS

109

Applied Techniques

By allowing custom objects and decorators to be used with URL patterns and views, nearly any valid Python code can customize how a URL is mapped to a view and how the view itself is executed. The following is just a taste of what’s possible; the rest depends on the needs of your application.

Dual-Format Decorator

In this modern “Web 2.0” world, Web applications are often expected to make use of advanced techniques, such as

, to communicate with the server in various, interesting

ways. Django supports this, but there are no “standard” mechanisms provided. Instead, each application must decide how best to handle these special types of interactions.

These “Web 2.0” applications often need to render the same content for standard browser requests as for these special types of requests, just in different formats, in order to make sure the application works without JavaScript enabled. It also helps to maintain individual URLs that can be bookmarked and retrieved as a whole, while still allowing just the essential content to be pulled up when necessary.

Since views normally return their responses directly, complete with full contents, this type of functionality only seems possible in one of two ways, each requiring a good deal of code duplication:

฀

฀

฀ ฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

third function that each view can call.

฀

฀

฀

฀ block inside the view that decides how its content should be output.

Ideally, views for this type of situation could be written without caring which output mechanism is expected. Because decorators are a common technique for removing boilerplate and they also have the ability to create or modify outgoing responses, they’re perfectly suited for the task at hand. It’s quite easy to write a decorator that automatically handles the decision of which path to take and creates an appropriate response based on data provided by the view.

The first step is to decide how a view should provide details of the response without creating the response directly. Since most views use templates to render their content and templates can use dictionaries to provide dynamic values within that content, a dictionary seems like a logical choice. Given that Django also comes bundled with a tool for converting simple Python objects—like dictionaries—directly to JavaScript objects, dictionaries become even more attrac-tive as a way to pass data out of a view in a response- independent manner.

The next step is to determine what should be done with a dictionary returned by the view.

Each type of response will have its own way of handling the dictionary, and the decorator should be written to handle as much as possible. Consider the normal case, where the dictionary will be passed to a template, which is then returned as the response. The decorator must perform these tasks:

1. Retrieve the appropriate template.

2. Call the original view, capturing the return value for later use.

3. Create a

object with values returned by the view.

4. Render the template using this context.

5. Return an

containing the rendered content.

110

C H A P T E R 4

฀ URLS AND VIEWS

That’s a good bit of work to be done, but it’s quite simple to manage inside a decorator.

Another key to consider here is the fact that the decorator has to retrieve and render the template, which means it has to know what template to use. Rather than try to hard- code the template name in the decorator, it’s best to let it take a single argument to pass in the template name when the decorator is applied. Here’s what the code might look like before taking into account the ability to spit out two different types of responses.

This basic decorator will allow a view to have a template loaded, rendered and returned automatically. Simply provide a template name to the decorator and return a dictionary, and the decorator does the rest. Moving on to the other side of the problem, the decorator needs to be able to return a string that can be read by JavaScript, containing the same content that would otherwise be provided by the template.

Django comes built- in with a copy of

,2 a library designed for converting

Python objects into JavaScript Object Notation (JSON),3 so it can be easily consumed by code inside the Web browser. It even has a custom encoder for use with

, designed to

encode some of the particular types Django uses. This encoder can be used to transmit the dictionary’s content directly to the browser, bypassing the template entirely. Combining it with

—covered in Chapter 7—allows the decorator to decide which path to take, completing the equation.

2.

3.

C H A P T E R 4

฀ URLS AND VIEWS

111

Now What?

URLs form the foundation of your site’s architecture, defining how users access the content and services you provide. Django stays out of the way when it comes to designing your URL

scheme, so you’re free to build it however you like. Be sure to take the appropriate time and remember that URL configuration is still a form of site design.

Views are the real workhorses of any application, taking user input and turning it into useful output. While the whole of Python is available for views to use, Django does provide one very important tool to handle one of the most common user input tasks on the Web: forms.

C H A P T E R 5

Forms

One of the key ingredients to modern Web applications is interactivity—the ability to accept input from users, which helps shape their experience. That input can be just about anything, from a simple search term to entire user- submitted novels. The key is the ability to process this input and turn it into a meaningful feature that enriches the experience for all the users of the site.

The process begins by sending an HTML form to the Web browser, where a user can fill it in and submit it back to the server. When the data arrives, it must be validated to make sure the user didn’t forget any fields or enter anything inappropriate. If there was anything wrong with the submitted data, it has to be sent back to the user for corrections. Once all the data is known to be valid, the application can finally perform a meaningful task with it.

It’s possible to do all this without a framework, but doing so involves a lot of duplication if multiple forms are involved. Managing forms manually also introduces a high risk of the programmer taking shortcuts in the process. It’s very common to have a form skip essential validations, either from lack of time or a perceived lack of necessity. Many exploited security holes can be attributed directly to this type of negligence.

Django addresses this by providing a framework to manage those finer details. Once a form is defined, Django handles the details of generating HTML, receiving input and validating data. After that, the application can do whatever it likes with the data received. Like everything else in Django, you’re also still able to bypass this form handling and process things manually if necessary.

Declaring and Identifying Fields

Django’s forms, like its models, use a declarative syntax where fields are assigned as attributes to the form’s class definition. This is one of the most identifiable features of Django, and is used to great effect here as well. It allows a form to be declared as just a simple class while supplying a great deal of additional functionality behind the scenes.

The first difference between models and forms is how they recognize fields. Models don’t actually recognize fields at all; they just check to see if an attribute has a method and call it, regardless of what type of object it’s attached to. Forms do actually check the type of each attribute on the class to determine if it’s a field, looking specifically for instances of

.

Like models, forms keep a reference to all the fields that were declared, though forms do so a bit differently. There are two separate lists of fields that may be found on a form, depending on what stage it’s in, each with its own purpose.

113

114

C H A P T E R 5

฀ FORMS

The first,

, is a list of all the fields that were found when the metaclass executed. These are stored on the form class itself, and are available to all instances as well. Thus, this list should only be edited in extreme circumstances, as doing so would affect all future instances of the form. It’s always useful as a reference when looking at a form class itself or when identifying those fields that were actually declared directly on the class.

All form instances also get a

attribute, which contains those fields that will actually be used to generate the HTML for the form, as well as validate user input. Most of the time, this list will be identical to

, since it starts as just a copy of it. Sometimes, however, a form will need to customize its fields based on some other information, so that individual instances will behave differently in different situations.

For example, a contact form may accept a

object to determine whether the user is

logged in or not. If not, the form can add another field to accept the user’s name.

Binding to User Input

Since forms exist specifically to accept user input, that activity must be performed before any others. It’s so important that instantiated forms are considered to be in one of two states: bound or unbound. A bound form was given user input, which it can then use to do further work, while an unbound form has no data associated with it, and is generally used only to ask the user for the necessary data.

The difference between the two is made when the form is instantiated, based on whether a dictionary of data was passed in or not. This dictionary maps field names to their values, and is always the first positional argument to the form, if it’s passed in. Even passing an empty dictionary will cause the form to be considered bound, though its usefulness is limited, given that without data, the form is unlikely to validate. Once a form has been instantiated, it’s easy to determine whether it was bound to data by inspecting its Boolean attribute.

C H A P T E R 5

฀ FORMS

115

Also note that all values are passed as strings. Some fields may accept other types, such as integers, but strings are the standard, and all fields know how to handle them. This is to support the most common way to instantiate a form, using the

dictionary available within a view.

Sometimes a form may also accept files, which are provided a bit differently than other types of input. Files can be accessed as the

attribute of the incoming request object,

which forms use by accepting this attribute as a second positional argument.

Regardless of which way it was instantiated, any instance of a form will have a attribute, which contains a dictionary of whatever data was passed into it. In the case of an unbound form, this will be an empty dictionary. Using

on its own isn’t safe, because

there’s no guarantee that the user- submitted data is appropriate to what the form needs, and it could in fact pose a security risk. This data must always be validated before being used.

Validating Input

Once a form has been bound to a set of incoming data, it can check the validity of that data, and should always do so before continuing. This prevents your code from making invalid assumptions about the quality of the data, which can also prevent many security problems.

On the surface, the process of validating user input is quite simple, consisting of a single call to the form’s

method. This returns a Boolean indicating whether the data was indeed valid according to the rules set by the form’s fields. This alone is enough to determine whether to continue processing the form or to redisplay it for the user to correct the errors.

116

C H A P T E R 5

฀ FORMS

NEVER TRUST USER INPUT

There’s an old adage in the world of Web development, which is often phrased, “User input is evil.” That’s a bit of an extreme, but the basic idea is that Web applications don’t run in a vacuum, but are instead exposed to the outside world, for a wide variety of users to interact with. Most of these users are upstanding citizens of the Web, looking only to use a site the way it was intended to be used. Others, however, would like nothing more than to bring your precious application to its knees.

Any application that takes action based on user input potentially opens itself up to some risks. Since decisions are being made based on what a user supplies, that user has a great deal of control over how the application behaves. In some cases, user input is passed directly through to database or filesystem operations, with an assumption that the input will be within some established range of known values.

Once someone comes along with malicious intent, he can use this fact to his advantage, pushing other data into the application, in hopes of convincing it to do something it shouldn’t, such as read content the user shouldn’t have access to, write to areas that should be read- only or even bring the application down so no one can use it at all. These types of attacks are generally placed into categories, such as SQL injection, cross-site scripting, cross-site request forgery and form manipulation, but one theme ties them together: they all rely on an application being too trusting of incoming data.

The solution to these types of attacks is to vigorously guard your application from malicious input, by meticulously validating everything that comes in. Django’s forms have a variety of ways to control this validation, but the

method makes sure they all run, so that the application can know if the input should be used. This step should never be skipped, as doing so will make your application vulnerable to many of these attacks.

It’s also important to realize that validation must always take place on the server, by way of

, regardless of what happens inside the user’s Web browser. In this age of Web 2.0 and rich Web applications, much work is done in JavaScript inside the browser, and it’s easy to think that this is a sufficient way to ensure the quality of incoming data, before it even arrives at the server.

However, a lot can happen between the browser and the server, and there are a great many tools freely available to help users manipulate the submitted data after it’s been processed by JavaScript. No amount of client- side validation is sufficient to keep an application safe from attack; everything must be checked on the server.

Behind the scenes,

does even more work, by indirectly calling the form’s

method, which populates two more attributes. The first,

, is a dic-

tionary analogous to the

attribute previously mentioned, except that its values have already been processed by the form’s fields and converted to appropriate Python data types.

The second is

, a dictionary containing information about all the problems that were encountered with the incoming data.

These two attributes are somewhat tied to each other, in that no field should be identified in both attributes at the same time. That is, if a field’s name is in

, it’s not in

, and vice versa. Therefore, in an ideal situation,

would contain data for

every field, while

would be empty.

The exact details regarding what data is considered valid and what errors would be returned otherwise are typically specified by each field, using its method. For most

forms, this is sufficient, but some may need additional validation that goes beyond a single field. To support this, Django provides a way to inject additional validation rules into a form.

C H A P T E R 5

฀ FORMS

117

Special methods may be defined on the form to assist in this process, and are named according to the fields they’re associated with. For example, a method designed to validate and clean the

field would be called

. Each method defined this way is

responsible for looking up its value in

, validating it against whatever rules are

appropriate for the form. If the value needs additional cleaning, the method must also replace the value in

with an appropriately cleaned value.

Custom Fields

While the fields included with Django are suitable for most tasks, not every application fits neatly into a list of situations somebody else expected to be common. For those applications where the existing fields aren’t enough, it’s easy to define custom fields for forms, much like how fields for models can be created. It’s even easier to create form fields than model fields, since they don’t have to interact with the database.

The main difference between model fields and form fields is that forms only have to deal with string input, which greatly simplifies the process. There’s no need to worry about supporting multiple backends, each with its own complexities, much less all the different lookup types and relationships that add to the bulk of model fields.

As mentioned, all form fields inherit from

, living at

. Because

forms use this fact to distinguish fields from methods or other attributes, all custom fields must be part of this inheritance chain in order to work properly. Thankfully, provides

a number of useful features that can make it much easier to implement a specific type of field.

Like many other classes, fields define a few attributes and methods to control specific behaviors, such as what widget to use and what error messages to display, as well as how to validate and clean incoming values. Any or all of them can be overridden to customize the functionality of a specific field.

Validation

Perhaps the most important behavior of a field is how it validates and cleans user input. After all, fields exist as a bridge between dangerous incoming data and a safe Python environment, so it’s essential that this translation be done properly. The field’s method is primarily

responsible for this, both for raising exceptions for improper data and for returning a cleaned value if the input is valid.

The method’s signature is simply

, accepting the field object itself and

also the incoming value. Then, if the value is deemed inappropriate according to the field’s requirements, it should raise an instance of

with a mes-

sage indicating what went wrong. Otherwise, it should convert the value to whatever native Python data type is appropriate for the field and return it.

In addition to making sure error messages are as descriptive as possible, it’s important to keep maintenance of error messages simple, while still allowing individual instances to override them. Django facilitates this by way of a pair of attributes called and

, as well as an argument called

. This may seem like

a tangled nest of values, but the way it works is rather simple.

A field class defines its standard error messages in a class- level attribute called

. This is a dictionary mapping an easily-identifiable key to the actual error message string. Since fields will often inherit from other fields, which may define their own 118

C H A P T E R 5

฀ FORMS

attributes, Django automatically combines them all into one dictionary when the field is instantiated.

In addition to using

, Django allows individual field instances to

override some of these messages by way of the

argument. Any values in that

dictionary will replace the default values for the keys specified, but only for that particular field instance. All other instances of the field will remain unaffected.

That means that error messages can come from three separate places: the field class itself, the field’s parent classes and the arguments used to instantiate the field. When looking to raise an exception as part of the

method, there needs to be a simple way to retrieve a spe-

cific error message, regardless of where it was actually defined. For this, Django populates an attribute of every field instance, which contains all the messages that were defined in all three ways. This way,

can simply look up a key in

and use its value as the argument to

.

Note the use of

here to call the

method of the parent

class,

which makes sure that the value is first a valid decimal before bothering to check if it’s a valid latitude or longitude. Since invalid values result in an exception being raised, if the call to allows code to continue executing, then it is assured that the value is a valid decimal.

Controlling Widgets

Two other attributes defined on field classes specify which widgets are used to generate HTML

for the field in certain situations. The first,

, defines the default widget to be used when

the field instance doesn’t specify one explicitly. This is specified as a widget class, rather than an instance, as the widget is instantiated at the same time as the field itself.

C H A P T E R 5

฀ FORMS

119

A second attribute, called

, controls which widget is used when the field should

be output into the HTML, but not shown to the user. This shouldn’t have to be overridden, as the default

widget is sufficient for most fields. Some fields, like the

,

need to specify more than one value, so a special

is used on those cases.

In addition to specifying individual widget classes for these situations, fields can also define a

method to specify a set of attributes that should be added to whatever widget is used to render the field in HTML. It receives two arguments, the usual as well as

, a

fully-instantiated widget object that any new attributes will be attached to. Rather than attaching the attributes directly,

should return a dictionary of all the attributes that should be assigned to the widget. This is the technique the built- in uses to assign a

attribute to the HTML input field.

Defining HTML Behavior

Widgets, as mentioned in the previous section, are how fields represent themselves in a Web page as HTML. While fields themselves deal more with data validation and conversion, widgets are concerned with presenting the form and accepting user input. Each field has a widget associated with it, which handles the actual interaction with the user.

There are a variety of widgets provided by Django, from basic text inputs to checkboxes and radio buttons, even multiple- choice list boxes. Each field provided by Django has, as its attribute, the widget that is most appropriate for the most common use cases for that field, but some cases may find need for a different widget. These widgets can be overridden on an individual field basis by simply supplying a different class to the field’s constructor as the argument.

Custom Widgets

Like fields, the widgets provided with Django are useful for the most common cases, but will not fit every need. Some applications may need to provide additional information, such as a unit of measurement, to help users enter data accurately. Others may need to integrate with client- side JavaScript libraries to provide extra options, such as calendars for selecting dates.

These types of added features are provided with custom widgets, which satisfy the requirements of the field they are associated with, while allowing great flexibility in HTML.

While not strictly enforced like fields, all widgets should inherit from to receive the most common functionality from the start. Then, each custom widget can override whatever attributes and methods are most appropriate for the task it needs to perform.

Rendering HTML

The most common need for a custom widget is to present a customized field display for the user, by way of HTML. For example, if an application needs a field to handle percentages, it would make it easier for users to work with that field if its widget could output a percent sign ( ) after the input field. This is possible by overriding the method of the widget.

In addition to the normal

, the

method receives three additional arguments:

the

of the HTML element, the

currently associated with it and

, a dictionary of

attributes that should be applied to the element. Of these, only is optional, and should

default to an empty dictionary if not provided.

120

C H A P T E R 5

฀ FORMS

Obtaining Values from Posted Data

Since widgets are all about dealing with HTML, and values are posted to the server using a format specified by HTML, in a structure dictated by HTML elements, widgets serve the extra purpose of translating between incoming data and the fields that data maps to. This not only insulates fields from the details of how HTML inputs work, it’s also the only way to manage widgets that use multiple HTML inputs, and allows widgets to fill in defaults, like

, in situ-

ations where nothing at all was submitted by the HTML input.

The widget method responsible for this task is

, which takes three

arguments in addition to the standard

.

฀

฀

—The dictionary provided to the form’s constructor, usually

฀

฀

—The files passed to the form’s constructor, using the same format as

฀

฀

—The name of the widget, which is essentially just the name of the field plus any prefix that was added to the form

The method uses all of this information to retrieve the value submitted from the browser, make any necessary changes and return a value suitable for fields to use. This should always return a value, defaulting to

if no suitable value could be found. All Python functions return

by default, if they don’t return anything else, so this rule is easily followed simply by ensuring that

doesn’t raise any exceptions, but for the sake of read-

ability, it’s always best to explicitly return

.

C H A P T E R 5

฀ FORMS

121

Splitting Data Across Multiple Widgets

Since widgets are a bridge between fields and HTML, they have a great deal of control over what HTML gets used, and how it reports back to the field. So much so, in fact, that it’s possible to split up a single field across multiple HTML field controls. Because of where the and

hooks are placed in the flow, this can even be done without the field having to know it’s happening.

Exactly how this works depends largely on what HTML inputs the widget would use, but the general idea is simple. A field passes its value to the widget’s method, which breaks

it up into multiple HTML inputs, each containing a piece of the original value. An example of this is having a separate text box for each of the date and time components of a

.

Then, when the widget receives the data back through its

method,

it assembles these pieces back together into a single value, which is then handed back to the field. At no point does the field have to deal with more than one value, regardless of what the widget does.

Unfortunately, that all requires each widget to be responsible for all the HTML markup, as well as reassembling the value when it’s received. Sometimes it’s just as useful to simply combine two or more existing fields, relying on their widgets to do the job instead. Since it’s quite handy to have a utility to help with this, Django provides one.

To be accurate, Django provides two utilities: a field,

, and a widget,

, which are designed to work together. By themselves, they’re not terribly useful in the real world. Instead, they provide a significant share of the necessary features, while allowing subclasses to fill in the details that are specific to a particular use case.

On the field side of things,

takes care of the details when cleaning data,

by validating it against each of the individual fields that make up the composite. The only two things it leaves to the subclass are the definition of which fields should be combined and how their values should be compressed into a single value suitable for use by other Python code. In Django itself, for example, the

combines a

with a

and

compresses their values to a single

object.

The process of defining which fields should be used is simple, and is handled in the method of the new field class. All it takes is to populate a tuple with the field instances that should be combined. Then, simply pass this tuple as the first argument to the

method of the parent class, which handles the rest from there. This keeps the method definition on the specific field quite simple, typically only a few lines long.

Compressing the values generated by those multiple fields takes place in the method. This takes a single value in addition to the usual

, a sequence of values that should

be combined into a single native Python value. What happens within can be a bit more complicated, though, as there are a few situations to take into account.

First, there’s the possibility that no value was submitted at all, for any part of the field, which would mean that the incoming data would be an empty list. By default, fields are required, in which case an exception would be thrown prior to calling

. If a field was declared

with

, this is a very likely scenario, and the method should return in this

case.

In addition, it’s quite possible for just part of the value to be submitted, since it’s split across multiple HTML inputs. Again, if the field is required, this is handled automatically, but if the field is optional,

must still do some extra work to ensure that if any of the value is submitted, all of it is submitted. This is typically handled by just checking each item in the value sequence against the standard

tuple, also located at

.

122

C H A P T E R 5

฀ FORMS

Any portion of the field containing an empty value should then raise an exception informing the user of which portion of the field was missing a value.

Then, if all the values were submitted and were valid,

does its real work,

returning a value suitable for use in Python when the form is being processed. The exact nature of this return value will depend entirely on the type of field being created, and how it’s expected to be used. Consider the following example of a field to accept latitude and longitude coordinates as separate decimals, combining them into a simple tuple.

With the field side of things out of the way, the next step is to create a widget that captures both of these elements separately. Since the intended display is simply two text boxes, it makes sense to make the custom widget a simple composite of two widgets, which solves

the first challenge of identifying the widgets to be used. The base does a good job

of rendering output and retrieving values from the incoming data, so the only challenge left is to convert the single compressed value into a list of values to be rendered by the individual widgets.

The counterpart to the field’s

method is, as you might expect, the widget’s

method. Its signature is quite similar, taking just a single value, but its task is to split that value into as many pieces as there are widgets to render them. Ordinarily, this would be a matter of taking bits and pieces from a single value and putting them into a sequence, such as a tuple or a list. Since the

shown previously outputs its value as a tuple

directly, the only thing that’s left is to supply a tuple of empty values if none was provided.

C H A P T E R 5

฀ FORMS

123

Customizing Form Markup

In addition to defining custom widgets, it’s also possible to customize how forms themselves are rendered as HTML. Unlike the previous examples, the following techniques are used inside Django’s template language, where it’s a bit easier to make changes that are specific to an individual form.

The most obvious thing that can be customized is the actual element, because

Django forms don’t even output that at all. This is primarily because there’s no way to assume whether the form should use GET or POST, and what URL it should be sent to. Any form that needs to be submitted back to the server needs to have this specified by hand, so it’s a perfect opportunity for some specialization. When using a form that includes a

, for exam-

ple, the

element needs to include an attribute such as

a .

In addition to the form’s submission behavior, one common thing to configure is the presentation of the form, using Cascading Style Sheets (CSS). There are a number of ways to reference an element with CSS, but two of the most useful are by assigning an ID or a class, both of which are often placed on the

element itself. Since that element has to be

defined, it’s easy to add these extra attributes as well.

In addition, there are often uses for configuring how the form’s fields are displayed, depending on how the overall look of a site is achieved. Different sites may use tables, lists or even simple paragraphs to present forms, so Django tries to make it as easy as possible to accommodate these different scenarios.

When outputting a form in a template, there are a few methods available to choose which of these output formats to use. The default,

, wraps each field in a row, suitable for

use in a standard table, while

wraps the fields in list items, and

wraps them in

paragraphs. None of these output any kind of element around all the fields, however; that’s left to the template, so that additional attributes can be added, such as IDs and classes for CSS

referencing, just like the

element.

While these three provided methods are useful for their own purposes, they’re not necessarily enough for every situation. In keeping with DRY, each of them is in fact a customized wrapper around a common method, which wraps any kind of markup around all the fields in the form. This common method,

, shouldn’t be called directly from outside

the form, but is perfectly suitable for use by another custom method designed for a more specific purpose. It takes a number of arguments, each specifying a different aspect of the HTML

output.

฀

฀

—HTML to be used for a standard row. It’s specified as a Python format string that will receive a dictionary, so there are a few values that can be placed here:

,

,

and

. Those should be fairly self- explanatory, except that

actually contains the HTML generated by the field’s widget.

฀

฀

—HTML used for a row consisting solely of an error message, primarily used for form- level errors that aren’t tied to a specific field. It’s also used for forms that are configured to show field errors on a separate row from the field itself, according to the option described at the end of this list. It’s also a Python format string, taking a single unnamed argument, the errors to be displayed.

124

C H A P T E R 5

฀ FORMS

฀

฀

—Markup used to identify the end of a row. Rather than appending this to the rows, since the rows specified above must have their endings specified directly, this is used to insert any hidden fields into a last row, just before its ending. Therefore, always make sure that the following is true:

.

฀

฀

—HTML to be used when writing out help text. This markup will be placed immediately after the widget and takes the help text as a single unnamed argument to this format string.

฀

฀

—A Boolean indicating whether field errors should be ren-

dered using the

prior to rendering the field itself. This doesn’t impact what values are passed to

, so if the form expects errors to be on separate rows, be sure to leave errors out of that format string. Otherwise, errors will be printed twice.

Accessing Individual Fields

In addition to being able to customize a form’s overall markup in Python, on the form itself, it’s also quite simple to specify a form’s markup directly in a template. This way, forms are as reusable as possible, while still allowing templates to have final control over the rendered markup.

Form objects are iterable, using techniques described in Chapter 2. This means that templates can simply loop over them using the

block tag, with each iteration being a field on

the form, which has been bound to a value. This bound field object can then be used to display the various aspects of a field, inside whatever markup makes the most sense to a template. It has a nice selection of attributes and methods to help the process along.

฀

฀

—The original field object, with all of its associated attributes

฀

฀

—The current value bound to the field

฀

฀

—An

(as described in the next section) containing all the errors for the field

฀

฀

—A Boolean indicating whether the default widget is a hidden input

฀

฀

—The HTML

element and its contents, for use with the field

฀

฀

—The default rendering of the field, using the widget defined for it

฀

฀

—The field rendered using a basic

instead of its own widget

฀

฀

—The field rendered using a

instead of the widget defined for it

฀

฀

—The field rendered using a hidden input instead of any visible widget Customizing the Display of Errors

By default, the markup used to display errors is specified by a special Python class called

, which lives at

. This behaves just like a standard Python list,

except that it has some extra methods for outputting its values as HTML. In particular, it has two methods by default,

and

, which output errors as an unordered list or

as unadorned text, respectively.

C H A P T E R 5

฀ FORMS

125

By creating a custom error class, as a subclass of

, it’s easy to override these

methods to provide custom markup when errors are displayed. This markup includes any containing elements, such as

, as the entire markup will be dropped in place wherever the field’s errors are displayed, whether as part of the default markup, or by accessing the field’s attribute directly.

By default, the

method is used to render errors, though templates that wish to do further customizations can call whichever method makes the most sense for the template.

In fact, it’s possible to add entirely new methods and even override which method is used by default by also overriding the

method. It’s also possible for templates to simply

loop through the errors in this list and wrap each one in whatever markup makes sense for the situation.

Writing a custom

subclass isn’t quite enough; it also has to be passed into the form somehow to make sure it gets used. This is also quite simple: just pass the custom class into the form’s constructor as the

argument.

In addition to displaying errors on individual fields, a form’s method allows errors

to be shown for form- wide validation failures. Displaying this in the template requires accessing the form’s

method.

Applied Techniques

While Django’s forms are primarily designed to handle a fairly common user input requirement, they can be made to do some complicated legwork. They can be used either individually or in groups to extend the user interface even further. Nearly any form of user input can be represented using a Django form; the following is just a sample of what’s available.

Pending and Resuming Forms

Forms are generally intended to receive input all at once, process that input and behave accordingly. This is something of a one- off cycle, where the only reason a form would have to be redisplayed would be to show validation errors, allowing the user to fix them and resubmit.

If a user needs to stop working on a form for a time and come back later, that means starting over from scratch.

While this is generally the accepted approach, it can also be a burden for complex forms or those where the user might need to provide information that takes time to gather, such as tax information. In these situations, it would be much more useful to be able to save the form in a partially-filled state and return to it at a later point in time. That’s not how forms typically work, so there’s clearly some work to be done, but it’s really not that hard.

Since forms are declared as classes, and there’s no reason to violate that presumption, the class developed hereafter will be usable as a parent class, just like

. In fact, for all

intents and purposes, it should be a drop- in replacement for the standard class, simply imbu-ing its subclasses with extra functionality. Consider the following form for making an offer on a house in a properties application, something which usually won’t be taken lightly. By allowing the form to be pended and resumed at a later time, users can take the necessary time to review an offer before committing to such an investment.

126

C H A P T E R 5

฀ FORMS

Note that, aside from the switch to

, this is defined like any other standard

Django form. The advantages of this simple change are described in the following sections, which outline a new

application.

Storing Values for Later

In order to save a form in a partially completed state, its current values must be stored in the database somehow. They’d also have to be tied to field names, so they can be used later to re-create the form. This sounds like a job for dynamic models, which can be created automatically, based on the form’s definition, to store values efficiently. However, they aren’t appropriate for this use case, for a few reasons.

For one thing, form fields don’t have directly equivalent model fields. Since the dynamic model would have to be filled with fields that can contain the same data as the form fields, there would have to be some way to determine a model field based on a form field. Model fields do define form fields that can be used with them, but not the other way around.

Technically, it would be possible to manually provide a mapping of form fields to model fields, so that such models could be created anyway. This would have its fair share of problems as well, since it wouldn’t be able to support custom form fields. Essentially, any form field that isn’t present in that mapping wouldn’t have a matching model field, and the technique would fail.

Also, storing field values in model fields that are based on the form’s field types would require converting those values into Python objects first, which would mean that they’d all have to be valid values. It should be possible to pend a form, even with invalid values, so that they can be corrected later. This wouldn’t be at all possible if the values had to be stuffed into model fields with specific data types, which included either data validation or type- checking.

Instead, we can rely on the fact that all form data, when submitted back to the server, arrive as strings. These strings must be converted to native Python objects as part of the form validation process, so the strings themselves are the last chance to get the actual raw data from the submitted form. Better yet, since they’re all strings, Django provides an easy way to store them for later use:

. A

is necessary, because different form values provide different lengths of data, some of which will likely extend beyond the 255- character limit of

.

With a reliable way to store values, the next step is to identify what other information must be stored in the database, in order to reconstitute the form. Obviously the names of the fields would be included, so the values could get put back in the right place. Also, since different forms could have different structures, with different numbers of fields, it would be best to give each field’s value its own row in the database. That means there would need to be a way of keeping fields together as part of a form.

The trick here is that forms don’t have a unique identifier. After all, they’re not normally expected to exist outside of a specific request/response cycle, except for validation corrections, C H A P T E R 5

฀ FORMS

127

where the entire form is resubmitted as part of the new request. There’s simply no built- in way to identify an instance of a form, so something different will have to be used.

One very common way of identifying complex structures like this is to create a hash based on the data. While hashes aren’t guaranteed to be unique, they’re close enough for most purposes, and there are some things that can be included along with a hash to get better odds of uniqueness.

In the case of a form, this hash can be taken from the complete collection of field data, so that a change in any name or value would result in a change in the hash that data would produce. Another piece of information that can be stored alongside the hash is the import path to the form, which allows for differentiation among multiple sets of data, if there are multiple forms with the same collection of fields.

Now that there are a few pieces of information to store, consider how they should relate to each other. There are essentially two levels here: the form and its values. These could be taken as two separate models, relating multiple values to a single form by way of a standard foreign key relationship. The form side would contain the form’s path as well as the hash of all its values, while the value side would contain the names and values of each field, as well as a reference back to the form it belongs with.

The

module of the

application looks like this:

This simple structure is now capable of storing any amount of data for any form. It wouldn’t be very efficient if the application needed to make complex queries on the form’s data, but since it’s just being used to save and restore the contents of a form all at once, it’ll work quite well.

Now that there are models in place to contain the form’s data, there needs to be a way to actually store that data for later retrieval. Thankfully, forms are just standard Python classes, so it’s easy enough to just write an extra method that handles this task directly. Then, when the time comes to write a specific form that needs this capability, it can simply subclass the following form, rather than the usual

. This is placed in a new

module in

our

application.

128

C H A P T E R 5

฀ FORMS

Note the liberal use of

here. If an instance of a form already exists with

exactly the same values, there’s no sense saving the whole thing twice. Instead, it simply relies on the fact that the previous copy will be functionally identical, so it’ll work for both.

Reconstituting a Form

Now that forms can be placed in the database without being fully processed, or even validated, their usefulness is still limited if they can’t be retrieved later, for the user to continue working on them. The data is stored in such a way that it can be reassembled into a form, all that’s left is to actually do so.

Since the code to do this must, by definition, be called prior to having a form instance to work with, it may seem like it must be in a module- level function. Remember that methods can be declared to be used on the class, rather than the instance, if the need arises. Since the goal here is to have all of this functionality encapsulated on a subclass, without having to worry about where all the machinery itself is written, a class method will do the trick here quite well.

What actually goes on in this new class method is a bit more interesting. In order to instantiate a form, it takes a dictionary as its first argument, which is usually just

,

available to all views. When loading the form later, the new request has absolutely nothing to do with the form, much less does it contain the appropriate data, so that dictionary must be constructed manually, from the data previously stored in the database.

This data may be referenced by the form hash described earlier, along with the import path of the form being used. Those two pieces of information are all that’s needed to properly locate and retrieve all the field’s values from the database. Since the form already knows how to get its import path, thanks to one of the methods described previously, all that’s left is to provide the form’s hash manually. This would most likely be captured in a URL pattern, though different applications may have different ways to go about that.

Once the hash is known, the method for resuming a form should be able to accept that, combine it with its own import path, retrieve the values from the database, populate a dictionary based on those values, instantiate a new copy of the form with those values and return that new form for other code to use. That sounds like an awful lot of work, but it’s a lot easier than it may seem.

One thing that comes to the rescue here is how Python’s own dictionaries can be instantiated. The built- in

can accept a variety of different argument combinations, but one of the most useful is a sequence of 2- tuples, each of which contains the name and value of an entry in the intended dictionary. Since QuerySets return sequences already, and tools like list C H A P T E R 5

฀ FORMS

129

comprehensions and generator expressions can easily create new sequences based on them, it’s quite easy to create something suitable.

Getting the import path and looking up the saved form is easy, and that object’s attribute provides easy access to all of its values. Using a generator expression, the data’s name/value pairs can be easily passed into the built- in

, creating a dictionary that can

be passed into the form object’s constructor. All is made clear by the code.

This simple method, when called with a form’s generated hash value, will return a fully-formed form object, ready to be validated and presented to the user for further review. In fact, validation and presentation will be the typical workflow in this case, giving the user a chance to see if there was anything to add or correct, before deciding to commit the form or pend it again for later.

A Full Workflow

As mentioned earlier, the normal workflow is fairly standard, with little variation across all the various forms that are in use in the wild. By allowing forms to be pended or resumed, there’s an optional extra step added to the workflow, which requires some added handling in the view.

Adding this new piece to the puzzle, the overall workflow looks a bit like this: 1. Display an empty form.

2. User fills in some data.

3. User clicks Submit.

4. Validate data submitted by the user.

5. Display the form with errors.

6. User clicks Pend.

7. Save form values in the database.

8. Validate data retrieved from the database.

9. Display the form with errors.

10. Process the completed form.

In order to maintain this entire workflow, the view gets a bit more complicated. There are now four separate paths that could be taken, depending on which part of the workflow is being processed at any given time. And remember, this is all just to take the necessary steps to handle the form. It doesn’t take into account any of the business logic required for a specific application.

130

C H A P T E R 5

฀ FORMS

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

From there, the typical workflow steps still apply, such as checking the validity of the input data and taking the appropriate steps that are specific to the application’s functionality.

Once this is all rolled up together in a view, it looks something like this: There’s a lot going on here, but very little of it has anything to do with making an offer on a house. The vast majority of that code exists solely to manage all the different states the form could be in at any given time, and would have to be repeated every time a view uses a subclass, and that’s not efficient.

Making It Generic

While it’s easy to see which aspects of the view are repetitive, and should thus be factored out into something reusable, it’s a bit trickier to decide how to do so. The main issue is that the portion of the code that’s specific to this particular view isn’t just a string or a number, like has been shown in most of the previous examples, but rather a block of code.

This is something of a problem, because previous examples had shown how generic views can be used to factor out commonalities, while allowing specific differences to be specified in a URL

pattern. That works well for basic data types, such as strings, numbers, sequences and dictionaries, but code is handled differently. Instead of being able to just specify the value inline in the URL pattern, this code must be defined in a separate function, which is then passed in to the pattern.

C H A P T E R 5

฀ FORMS

131

While that’s certainly possible, it makes the URL configuration module a bit more cumber-some, given that there might be a number of top- level functions declared above each block of URL patterns. Lambda- style functions could be a way around this, but since they’re restricted to executing simple expressions, with no loops or conditions, they’d severely limit the type of code that could be used.

One alternative is a decorator, which could be applied to a standard function, providing all of the necessary functionality in a wrapper. This way, any function can be used to contain the code that will actually process the form, with the full capabilities of Python at its disposal.

That code also wouldn’t have to deal with any of the boilerplate necessary to pend or resume the form, because the decorator could do all that before the view code itself even executes, simply passing in a form as an argument. Here’s how the previous view could look, if a decorator was used to remove the boilerplate.

Now all that’s left is to write the decorator itself, encapsulating the functionality that was removed from the previous example, wrapping it around a view that would be passed in. This would be placed in a new

module.

132

C H A P T E R 5

฀ FORMS

Now, all that’s necessary is to set up a URL configuration that provides both a form class and a template name. This decorator will handle the rest, only calling the view when the form has been completed and submitted for processing.

Now What?

In order to be truly useful in the real world, forms must be presented to users as part of an HTML page. Rather than trying to generate that HTML content directly inside Python code, Django provides templates as a more designer- friendly alternative.

C H A P T E R 6

Templates

While Chapter 2 made it clear that Django is built entirely on Python, and standard Python rules apply, templates are the exception to the rule. Templates are Django’s provided way of generating text- based output, such as HTML or emails, where the people editing those documents may not have any experience with Python. Therefore, templates are designed to avoid using Python directly, instead favoring an extensible, easy-to- use custom language built just for Django.

By disallowing arbitrary Python expressions, templates are certainly restricted in some ways, but there are two things to keep in mind. First, the template system is backed by Python, just like everything else in Django, so it’s always possible to add Python- level code for specific features. It’s just bad form to include the actual Python code in the template itself, so Django provides other means for plugging in that extra code.

More importantly, drawing a clear line between templates and the Python code that powers them allows for two separate groups of people, with different backgrounds and skill sets, to work together. For many hobbyist projects, this probably sounds like a waste, since the only people working on the site are developers. In many commercial environments, however, developers are often a separate group of people from those tasked with maintaining the site’s content and visual structure.

By clearly separating the tasks of development and template editing, it’s easy to set up an environment where developers work on the things that they’re really needed for, while content editors and designers can work on things that don’t really require development experience.

Django’s templates are fairly simple in nature, and easy to pick up by most anyone, even those without any programming experience.

The basic details of what the template syntax looks like and the included tags and filters are well described elsewhere. Instead of focusing on those higher- level details, this chapter will cover how templates are loaded, parsed and rendered, how variables are managed within a template and how new tags and filters can be created. Essentially, it’s all about what developers can do to make life as easy as possible for their content editing counterparts.

What Makes a Template

Even though templates aren’t written in Python directly, they’re backed by Python, making all the good stuff possible. When a template’s code is read in from a file or other source, it’s compiled into a collection of Python objects, which are then responsible for rendering it later.

Exactly how these objects work can be ignored for basic template usage, but as with anything else, a proper understanding unlocks a world of possibilities.

133

134

C H A P T E R 6

฀ TEMPLATES

Taking a peek inside the

package, the

class stands out as the

starting point for template operations, and rightly so. When a template is loaded, its content is passed to a new instance of

, along with some optional information about where the

template itself came from. There are three arguments passed in to new objects, on

which everything is based.

฀

฀

—The only required argument, this contains the actual content of the template, as read in from a file. The great thing here is that accepts a string,

not a filename or an open file object. By accepting just a string—either a Unicode string or a regular string encoded in UTF- 8—it’s possible to set up templates from any source.

Some interesting uses of this can be found in this chapter’s “Applied Techniques”

section.

฀

฀

—An object representing where the template came from, such as a template loader or just a raw string. It’s only used when the

setting is

, and

can often be left out without penalty, but it’s always best to include it for development situations, where it can help debug problems involving multiple template loaders.

฀

฀

—The name of the template, as passed to whatever loader requested it, if any. This is often just a relative path to the template, but could theoretically be anything that makes sense to a particular situation. After all, what Django really cares about is the

; the rest is just useful when debugging problems.

The actual code for

is fairly minimal, deferring most of its work to a utility function called

, which parses the raw text, compiling it into a sequence of nodes.

These nodes are just Python objects, each configured for a specific part of the template. Taken together, they represent the entire template, from start to finish, in a way that can be more easily and efficiently rendered.

These nodes are attached to the template as an attribute called

. When rendering

the template with data, it simply iterates over this list, rendering each node individually. This keeps the

code very minimal, while allowing maximum flexibility. After all, if each individual theme is responsible for rendering itself, it has the full power of Python at its disposal. Therefore, creating or customizing template nodes is a simple matter of writing some real Python code.

Exceptions

All of this assumes that the template works correctly all the way through. When working with templates, there are a number of things that can go wrong, and thus a few different exceptions that could be raised. While the following exceptions are handled automatically in a way that works for most cases, it’s possible to catch these instead and handle them separately.

฀

฀

—The template code doesn’t validate as proper

syntax, usually due to the use of an invalid tag name. This is raised immediately when trying to instantiate a

object.

฀

฀

—The requested template couldn’t be loaded

by any of the known template loaders. This is issued by the template loading functions described in the “Retrieving Templates” section of this chapter.

C H A P T E R 6

฀ TEMPLATES

135

฀

฀

—The template string provided couldn’t be

forced to a Unicode string by the

object. Template strings must either be

a Unicode string already, or be encoded in UTF- 8; any other encoding must be converted to one of those two types prior to being passed to a new

. This will be

raised immediately when trying to construct a new

object.

฀

฀

—A specified variable name couldn’t be

resolved in the current context. See the “Context” section later in this chapter for details on this process and what situations will cause this exception to be raised.

฀

฀

—A template tag specified some invalid

parameters to one of the tag library registration functions. A single tag issuing such an error will cause the entire tag library to stop loading, and none of the tags will be available to the template. This is raised when using the

template tag.

The Process at Large

Once a string is obtained from a loader, it must be converted from a single string to a set of Python objects that can be rendered. This happens automatically, and no intervention is necessary for most cases, but as with most of Django, an understanding of these internals can be quite useful. The following steps explain how a template is processed. All the classes involved live at

.

1. A new

object accepts the raw string of the template’s contents, forming the object that will be used later.

2. A

object also receives the raw template string, to begin processing the template contents.

3.

uses a regular expression to split the template into individual components, called tokens.

4. These tokens populate a new

object.

5.

goes through the available tokens, creating nodes along the way.

6. For each block tag,

calls an external function that understands the tag’s

syntax and returns a compiled

for that tag.

7. The list of compiled nodes is stored on the

object as its

attribute.

Upon completion, you’re left with a

object that contains references to Python

code, rather than the raw string that started the process. That original string is discarded after the node list is created, because those nodes contain all the necessary functionality to render the template. The

,

and all the

objects are also discarded once the process

completes, but they can be very useful along the way.

Content Tokens

The

object is responsible for making a first pass through the template’s contents, identifying different components that are present. In addition to the template string itself, also

accepts an

, which indicates where the template came from. This processing is done by the

method, which returns a list of

objects. This could be seen as

136

C H A P T E R 6

฀ TEMPLATES

processing the template’s syntax, but not its semantics: individual components are identified, but they don’t yet carry much meaning.

Tokens contain all the information necessary to create nodes, but tokens themselves are relatively simple. They have just two attributes:

and

. The value for

will be one of four constants defined in

, while its

will be

defined by the type of token it is.

฀

฀

—Variable tags, using the

syntax, are placeholders for data that

won’t be provided until the template is rendered. The

attribute contains the

full variable reference string, unparsed.

฀

฀

—Block tags—commonly called “template tags”—use the

syntax and are populated by a Python object that can execute custom code during template rendering. The

attribute contains the full contents of the tag, including the tag’s name and all its arguments.

฀

฀

—Comment tags use a

syntax and are essentially ignored

by the template engine. Tokens are generated for them as part of the lexing process, but their

are empty and they don’t become nodes later in the process.

฀

฀

—Text tokens are generated for all other content in the template, storing the text in

.

A

always gets created and utilized automatically during standard template processing, but can also be used directly. This is a useful way to inspect and analyze templates without the overhead of compiling them completely. To illustrate, consider the following example, which parses a simple one- line template into a series of tokens. Note that the is printed

only by value; it’s far more useful to compare this value to the constants named previously.

Parsing Tokens into Nodes

Once a

has split the template string into a list of tokens, those tokens are passed to a

,

which examines them in more detail. This is the semantic side of template processing, where each token is given meaning by attaching a corresponding

object to the template. These nodes

vary greatly in complexity; comment tokens don’t produce nodes at all, text nodes have very simple nodes and block tags could have nodes that encompass the whole remainder of the template.

The

object itself is a bit more complicated than

, because it’s responsible for

more of the process. Its

method has to work its way through the list of tokens, identifying which tokens require nodes and which type of nodes to create along the way. Each token C H A P T E R 6

฀ TEMPLATES

137

is retrieved and removed from the list using

. That token is then used to

determine what type of node to create.

For text and variable tokens, Django supplies standard nodes that are used for all instances.

These are

and

, respectively, and they are also available at

.

Comment tokens are simply ignored, with no node generated at all. Block tokens go through the template tag library, matching the name of the tag with a node compilation function.

These compilation functions, described in the “Template Tags” portion of the “Adding Features for Templates” section later in this chapter, are each responsible for parsing a token’s and returning a

object. Each function receives two arguments: the

object

and the current token. Having access to the

object, node compilation functions can access

a few additional methods to help control how much of the template that node has access to.

฀

฀

—This is the same method that gets called when the tem-

plate is first processed, and it can also be called from within a node. By supplying a tag name for the

argument, this method will return just those nodes up to that tag name. This is how tags such as

,

and

wrap around additional content

between the opening and closing tags. Note that this returns fully compiled nodes.

฀

฀

—This retrieves and returns one token from the list. It also removes that token, so that future nodes don’t receive any tokens that have already been processed.

Note that this returns a token that has not yet been compiled into a node.

฀

฀

—This method is similar to

, accepting a tag that marks

the end of where the template should be processed. The main difference is that doesn’t parse any of the tokens into nodes along the way, nor does it return any of the tokens that were found. It simply advances the template to beyond the end tag, ignoring anything in between.

Template Nodes

While it may seem like a complicated concept, template nodes are fairly simple. All template nodes extend the basic

class, located at

. In addition to an

method to customize the node’s behavior, nodes have just a few methods that need to be included.

First, to maintain a common structure across all objects in a template, every template node is iterable, yielding all nodes that are contained within the node in question, rather than rendering their contents. This allows an easy way to get at all the nodes in a template.

By default,

simply yields itself, which works well for simple template tags that just render a small snippet of text. For more complicated tags that encapsulate other content, this should return all the nodes that were contained within it.

In addition, nodes must also provide a method called

, though the default

usually works well enough for most nodes. This method takes a single argument,

, the

class of node to retrieve. The node where the method was called will be checked to see if it’s an instance of that class, as well as any other nodes within it. All nodes found that are indeed instances of the specified type will be returned in a list, or an empty list will be returned if none were found.

The most important method on a node is

, which is used to output the final text.

Since rendering to text requires the data that was passed to the template, this method accepts a single argument, a context object as described in the upcoming “Context” section.

138

C H A P T E R 6

฀ TEMPLATES

Rendering Templates

Since a template is really just a collection of compiled instructions, getting those instructions to produce output text requires a separate step. Templates can be rendered using the simple method, which takes a context object as its only argument.

The

method returns a string, containing the fully-rendered output, based on the compiled nodes and the context variables. This output will often be HTML, but can be anything, since Django templates are designed to work with any text- based format. The bulk of the work of rendering gets put off to the individual nodes themselves, with the template just iterating over all the nodes, calling

on each in turn.

By offloading this work onto each node itself, the overall template code can be less complex, while also maximizing the flexibility of the template system in general. Since each node is fully responsible for its behavior, the possibilities are nearly limitless.

Context

A template itself is mostly just a bunch of static content, logic and placeholders for data to be filled in later. Without having data to fill in the blanks, it’s relatively useless to a Web application. On the surface, it seems like a standard Python dictionary would suffice for this, since template variables are just names, which can be mapped to values. In fact, Django will even allow a dictionary to be used in certain cases.

One drawback of this approach is that there are some situations where a template tag might need to alter some data, and have that alteration only persist for a specific portion of the template. For example, when looping through a list, each item in the list should be available for use by other tags, but once the loop completes, that variable should no longer be accessible to the rest of the template. Beyond that, if a loop defines a variable that already had a value, that existing value should be restored once the loop finishes executing.

CONTEXTS VS. NAMESPACES

In Python, variables are assigned to namespaces, where they can later be retrieved by name, making template contexts very similar. There are also some notable differences that may cause some confusion.

Python allows namespaces to be nested, but only inside a defined class or function. In these nested namespaces, new variables aren’t accessible to the other namespaces that enclose them. Other types of code blocks, such as conditionals and loops, share the namespace with whatever code surrounds them, so new variable assignments persist after the block finishes executing. This works well because namespaces are based on where the code is written, rather than where it executes, so the programmer can easily make sure that there aren’t any conflicts with related names.

When writing a template tag, there is no way of knowing what variables will be defined in the template where the tag gets used. If it adds any new variables to the context, those could very well overwrite something else that was already set in the template. To overcome this, templates offer and

methods to allow tags to manually create a new nesting level and remove it when finished with it.

This makes templates work a bit differently from Python code in this respect, since blocks like loops essentially create a new namespace for the duration of their execution, removing it when finished. These differences may be a bit confusing at first to programmers, but designers working just with templates will only have one behavior to get used to.

C H A P T E R 6

฀ TEMPLATES

139

To accomplish all this, Django implements its data mapping as a special object,

which behaves much like a standard dictionary, but with some extra features. Most notably, it encapsulates a list of dictionaries internally, each of which represents a certain layer in the data map. This way, it can function like a stack as well, with the ability to new values

onto it and

a layer off when no longer needed.

Neither

nor

take any arguments. Instead, they simply add or remove a dictionary at the front of the list, adjusting which dictionary will be used first when looking up variables, as described next. This functionality prevents a standard dictionary from being used in most cases; it’ll work fine as long as the template is simple, but as soon as one of these tags is encountered, it’ll raise an

because it’s missing these extra methods.

Simple Variable Resolution

Looking up data in the context is one of the most basic operations, though there’s a lot that happens when a variable is referenced in a template. First, when using the standard syntax, Django automatically checks the context dictionaries in order from the one added most recently to the one added first. This lookup can also be performed manually on the context itself, using standard dictionary lookup syntax, which works just as well for retrieving values as setting them.

If the given name doesn’t exist in the topmost dictionary, the context falls back to the next dictionary in line, checks for the name again and the process continues. Often, the phrase

“current context” is used to describe the values that are available to a template tag at any specific point in time. Even though a template will use the same context object throughout the rendering process, the current context at any given point will change depending on what tags are in use and what values are retrieved by those tags.

If it gets through all available dictionaries without finding anything, it raises a as

a standard dictionary would. That

is normally handled by Django directly, replacing

the variable reference with a constant value defined in the site’s settings. By default, the setting is set to an empty string, but this may be overridden by any site that wishes to display something different for this case.

140

C H A P T E R 6

฀ TEMPLATES

Complex Variable Lookup

In addition to simple name lookups, variables can also contain references to certain portions of an object, using a period to separate one layer from the next. This allows a variable node to reference not just an object, but perhaps an attribute of that object, a method call or an entry in a dictionary or a list. This is also nested, so each time a dot resolves a new variable, another dot can resolve the next layer deep.

This is handled using a separate class, appropriately named

. It’s instantiated

with a single argument, the string to be used as the variable’s path, including any periods separating portions of the path. Once instantiated, it provides a single method,

, which

is used to perform all the necessary steps of retrieving the requested value. This method takes a single argument, the context where the variable should be found.

If the variable was declared with a literal value, such as a number or a quoted string, rather than a named variable, that value will always be returned directly, without even referencing the provided context. Otherwise, this resolves the first portion of the variable using the simple lookup described previously. If that part is found, it continues on to the next portion, and so on.

Each step in the chain after the first is based on the object that was retrieved in the step before it. When determining what to get at each stage,

goes through a few different

stages, with an error at each stage causing the lookup to continue on to the next stage.

฀

฀ Dictionary lookup—The name provided is used as a dictionary key.

฀

฀ Attribute lookup—The name is used in the standard method.

฀

฀ Method call—If the attribute lookup retrieved a callable, such as a function, that callable is executed without any arguments. If this succeeds, the return value is used, but if the function requires any arguments, it will be skipped. Also, if the function has an attribute set to

, the function will be skipped, as a security precaution.

฀

฀ List-index lookup—The variable name is coerced to an integer, if possible, and used as an index lookup to see if the value is present in a list.

Since this provides a much more robust and feature- rich way to access variables, it’s always best to use

when a node needs to be able to access data from a template. This C H A P T E R 6

฀ TEMPLATES

141

will ensure that template authors have as much flexibility as possible when referencing variables, even to custom tags.

Including Aspects of the Request

It’s often necessary to include certain attributes from the incoming HTTP request, or at least to look up some other useful information based on those attributes, and include them in the template context. There’s no way for Django to magically get the request from the view into the template system, so it has to be passed in manually.

Since

on its own only accepts a dictionary as an argument, a different object is necessary to make this happen.

, also located at

,

accepts a request object as its first argument, while the normal dictionary is pushed back to the second argument instead. Aspects of the request can then be retrieved when preparing the context for use by the template.

It’s always best to use

whenever rendering a template as part of an HTTP

cycle. Django’s own generic views use it consistently, and most third- party applications also use it reliably. Failing to use

may result in templates not having access to nec-

essary data, which can cause the template to render incorrectly.

For many sites, templates might get rendered as part of an automated process, such as a nightly job to send out billing notification emails. In these situations, there is no HTTP request coming in, so

is inappropriate. Simply using a standard

will be sufficient

in these cases.

Once a

is instantiated with a request, it has to populate context variables based on attributes of that request. It doesn’t do this arbitrarily, but rather runs through code specified by another hook in Django.

Retrieving Templates

So far, all that’s been illustrated is how to work with templates once they already exist. In the real world, templates will have to be loaded on demand, according to the needs of a particular view, so there’s clearly more work to be done.

One particular requirement of retrieving templates is that they be referenced by name only, so that they can be loaded from different locations between development and production environments, without changing the code for any of the views. Chapter 8 shows how to write your own template loader, further increasing the available options. To handle this abstraction, Django provides two utility functions that should be used when retrieving templates.

django.template.loader.get_template(template_name)

Most of the time, a view knows about exactly one template, so only one name is given. The function takes the name of the requested template and returns a fully instantiated

object. Then, that template can be rendered according to the needs of the view.

Behind the scenes,

checks each template loader for the presence of

a template with the given name, then returns the first one it finds. If no template was found matching the specified name, it raises a

exception.

142

C H A P T E R 6

฀ TEMPLATES

django.template.loader.select_template(template_name_list) Sometimes, it’s necessary to retrieve a template using one of a few different names. This is often the case when an application would like to provide some kind of default template every time a view is accessed, while allowing a different template to be loaded in certain cases.

Consider a real estate site, where every property listing is expected to look the same. Naturally, the view for the property listing would simply use the same standard template for every listing in the database. If, however, a property comes along that has special requirements for its listing, such as additional buyer incentives or a special notice about an urgent need to close quickly, the standard template might not have a place for that. That information might also need to be rearranged on the page for a particular listing.

To handle these cases,

takes a list of template names, rather than just

a single value. For each name in the list, it calls

to try to retrieve it, and if that

fails, it simply moves on to the next name in the list. That way, a more specific name can be supplied first—often based on an object’s ID or slug—followed by a more generic fallback.

In a real application, the number included in the most specific template name would be supplied by something dynamic, such as the URL being requested. That way, new property listings would use the generic template by default, but customizing an individual listing is as simple as dropping in a new template using the more specific name.

Shortcuts to Load and Render Templates

While it’s definitely nice to have full control over how templates get loaded and rendered, the common flow is to just load the template, render it with a given context, and access the resulting string. This involves a few steps, which can easily get repetitive, so Django provides a couple ways to make the process simpler.

render_to_string(template_name, dictionary=None, context_instance=None) Living at

, this simple function takes a few arguments and returns

a string resulting from the template rendering. A template name is retrieved according to the name provided, and is then immediately rendered by passing the given dictionary into the provided context.

C H A P T E R 6

฀ TEMPLATES

143

If the dictionary isn’t provided, an empty dictionary is used instead, while if no context is provided, Django will simply use a

. Most of the time, it’s most appropriate to use

, so that all context processors get applied as well. Since Django can’t magically find the request being used, a

must always be first instantiated with the

request, then passed in as the

.

render_to_response(template_name, dictionary=None, context_instance=None, mimetype=None)

Living at

, this function works almost identically to

,

except that it uses the resulting string to populate an

object, which is covered in

detail in the next chapter. The only other difference is that this accepts an optional

,

which will be used when populating the

.

Adding Features for Templates

Perhaps the most powerful feature of Django’s templates is the ease with which new features can be added to them, without having to modify the framework itself. Each application can provide its own set of new features, rather than expecting site developers to provide their own.

Django’s own template features can be split into two types, variables and tags, and custom add- ons fit right into those two areas. Variables can’t really be added in code, since they’re controlled by the template’s context, but variable filters are a way for applications to allow variables to be modified easily. Tags, on the other hand, can do just about anything, from adding or modifying variables in the context to branching based on variables to injecting other templates.

Setting Up the Package

In order to make things easier for template authors, Django requires your template features to live at a specific package structure within an application. The tag uses this structure

to locate a specific module among all the installed applications, without the need for complex configurations that would make life more difficult for template designers.

Any application can supply new template features by creating a package

within the application’s main package. This new package can contain any number of modules, each containing a group of features that relate to each other. For example, a mail application could provide features that format text, perform basic math and show relationships between messages. The package structure would look something like this: 144

C H A P T E R 6

฀ TEMPLATES

When writing templates for this application—or any other application you use in your site—the

tag makes those features available, accepting the names of the modules to load. These modules can come from any application in your setting. Django

first looks for a

package in each application, then looks for the module named in the

tag.

Variable Filters

When variables are used in templates, they’re normally just shown exactly as they were passed into the context by the current view. Sometimes, it’s necessary to format or otherwise modify some of those values to suit the needs of a particular page. These types of presentational details are best placed in the template, so the view can just pass the raw values through, without regard to what the templates might do with them.

Django provides a number of these filters in its core distribution, intending to handle many of the most common situations you’re likely to encounter. Full documentation is available online,1 but here are a few of the most common filters:

฀

฀

—Returns a string with the first letter capitalized

฀

฀

—Returns the number of items in a given sequence

฀

฀

—Formats a date using a string as an argument

Filters are just Python functions that take the variable’s value as input, and return the modified value as a return value. This is really as simple as it sounds, though there is still a good bit of flexibility. Here’s what a simple filter function might look like, for displaying the first few characters of a variable, used as

.

Note This uses the Python 2.4 decorator syntax, but remember from Chapter 2 that distributed applications should retain compatibility with Python 2.3.

1.

C H A P T E R 6

฀ TEMPLATES

145

Accepting a Value

The first argument is the variable’s value, and is always passed in, so it should always be required by the filter function. This value is typically the variable that was passed into the template’s context, but filters can be chained together, so this value may actually be the result of another filter having already executed. Filters should therefore be made as generic as possible, accepting a wide range of input and handling it as gracefully as possible.

Tip This notion of “be liberal in what you accept” has been long considered a best practice for interoper-able systems. It has been documented as far back as 1980, during the formation of technologies that today’s Web is built on. The counterpart is to be “conservative in what you send,” which recommends in this case that filters should always return the same data type.

Since the value could contain data from anything the view provides, or the result from any previous filters, care should be taken when making assumptions about its type. It will often be a string, but could be a number, model instance or any number of other native Python types.

Most filters are designed to work with strings, so Django also provides a useful shortcut for dealing with those. It’s not guaranteed that the input will be a string, so string- based filters would always need to start by coercing the value to a string before continuing. There’s already a decorator for this process, called

, which is located at

. This automatically coerces the incoming value to a string, so the filter itself doesn’t have to.

It’s also important to make sure that no changes are made to this object directly. In the event that the input is a mutable object, such as a list or a dictionary, any changes made within the filter will also be reflected in any future uses of that variable in the template. If there are any changes to be made, such as prefixing or reorganizing items, it’s essential to make a copy first, so those changes are reflected only in the filter itself.

Accepting an Argument

In addition to receiving the variable itself, it’s also possible for a filter to accept an argument to customize its use. The only change necessary to accept an argument is to define an additional argument on the function. It’s also easy to make an argument optional, simply by providing a default value in the function’s definition.

Like the variable itself, this can also be of any type, since it can either be specified as a literal or supplied through another variable. There isn’t any provided decorator for coercing this value to a string, as numbers are very common as filter arguments. Whatever argument your filter expects, just make sure to explicitly coerce it to the type it needs, and always catch any exceptions that might occur during that coercion.

Returning a Value

The vast majority of the time, a filter should return a string, since it’s expected to be sent into the output of the rendered template. There is definite use for filters that return other types of data, such as a filter that averages the numbers in a list, returning the result as a number, but 146

C H A P T E R 6

฀ TEMPLATES

those are far less common, and should be well documented in case other filters are chained with them, to avoid unexpected results.

More importantly is the fact that filters should always return a value. If anything goes wrong during the filter’s processing, no exceptions should be raised. This also means that if the filter calls some other function that might raise an exception, the exception should be handled by the filter so it doesn’t raise up beyond that. In the event of any of these problems, the filter should either return the original input or an empty string; which one to use will depend on the purposes of the filter in question.

Registering As a Filter

Once the function is all written up, it’s registered with Django by using the class

provided at

. Once instantiated,

has a

method that can be

used as a decorator, which, when applied to the filter function, automatically registers it with Django. That’s all that’s necessary on the code side.

This doesn’t make it globally available to all templates by default, but rather just tells Django that the application provides it. Any template that would like to use it must still load the application’s template features using the

tag.

Template Tags

Filters serve a very useful and practical purpose, but given they can only receive up to two values—the variable and an argument—and can only return one value, it’s easy to see how quickly an application can outgrow them. Getting more functionality requires the use of template tags, which allow just about anything.

Like filters, Django provides a number of tags in its core distribution, which are documented online. Some of the more common tags are listed here, along with a brief description of their functionality.

฀

฀

—Allows a template to loop over the items in a sequence

฀

฀

—Applies a template filter, such as those described previously, to all the content contained within the tag

฀

฀

—Prints out the current time, using some optional arguments to format it Template tags are implemented as a pairing of a function and a class, with the former configuring the latter. The node is just like the nodes described earlier, representing the compiled structure of the template tag. The function, on the other hand, is used to accept the various allowable syntax options for the tag and to instantiate the node accordingly.

A Simple Tag

The simplest form of tag only exists in and of itself, typically to inject additional content into the page, based on some arguments. The node for this case is extremely simple, just taking and storing those arguments and formatting them into a string during rendering. Here’s how it would look if the filter from the previous section were instead implemented as a tag. In the template, this would look like

.

C H A P T E R 6

฀ TEMPLATES

147

The function to compile it, on the other hand, is a bit more complicated. Tag functions, unlike filter functions, always take two arguments: the template parser object, and a token representing the text contained within the tag. It’s up to the compilation function to extract the necessary bits of information from these two objects. For a simple tag like this, it’s not necessary to worry about the parser object, but the token is still necessary in order to get the argument, if one was specified.

The most important thing to know about the token is the

method,

which intelligently breaks apart a tag’s declaration into individual components, including the tag’s name and its arguments. It correctly handles variable references, quoted strings and numbers, though it doesn’t do any variable resolution, and leaves the quotes around quoted strings.

In order to get the two necessary bits of information out of our template tag, is used to extract them from the declared string. Then, these can be coerced to the correct types and used to instantiate the node described previously.

A Shortcut for Simple Tags

Thankfully, there’s a shortcut that makes this process a whole lot easier. The object

contains another decorator method,

, which handles simpler cases like this.

Behind the scenes, it handles the parsing and resolution of arguments, and even the creation of the node class, so all that’s left for the template tag is a single function that looks quite similar to a variable filter.

148

C H A P T E R 6

฀ TEMPLATES

This is still of limited use, but there are many such situations where a simpler tag is necessary, and the shortcut can become quite a time- saver. For more advanced needs, manually creating the node offers much more power and flexibility.

Adding Features to All Templates

Django doesn’t automatically load all applications’ template filters and tags by default; instead, it uses just a default set for all templates. For those templates that need to access a specific application’s template features and using the

tag is too much overhead, there’s also

an option where an application can be added to the default set of tags for all templates.

Also living at

, the

function takes the name of the

application to be included by default. Specifically, this is the for that application, as

described in Chapter 3. Once an application is supplied to this function, all of its tags and filters will be made available to all templates. This can be called anywhere that will be executed when the application loads, such as its

module.

This should be used sparingly, as it does incur some added overhead for even those templates that don’t use any of that application’s features. In some cases, however, it’s necessary to override the behavior of default filters or tags, and

provides that option.

Keep in mind that more than one application can do this, so there’s still no guarantee which application’s version of a particular feature will be used. Django will simply overwrite them as they’re encountered, so the last application to load is what will be used. Use it with care.

Applied Techniques

Django templates are designed to make things as easy as possible for the people who have to write templates on a regular basis. Advanced techniques are used to reinforce this idea, simplifying tasks that might otherwise be too complex to perform in a template. An application will often have its own unique template needs and should provide tags and filters to satisfy them.

Even better is to provide features that can be reused by other applications if necessary.

Embedding Another Template Engine

While Django’s template engine is suitable for most common cases, its limitations may cause frustration in situations where more power or flexibility is needed. Template tags extend Django’s functionality, but only by involving programmers to write them for each individual need.

Another template engine with a different design philosophy might be more suitable to some of these needs. By allowing template designers to switch to an alternative engine for portions of templates, additional functionality is exposed without requiring additional programming. Tags can still simplify common tasks, but switching template engines can be an easy way to support corner cases.

One such alternative template engine is Jinja,2 which has a syntax fairly similar to Django.

There are fundamental differences in the design philosophies, making Jinja a better choice for situations where the output requires complex conditions and logic. These aspects make it a perfect candidate for embedding within Django templates.

2.

C H A P T E R 6

฀ TEMPLATES

149

To illustrate this, consider a template that needs to calculate a composite value to display in a template. This feature isn’t available in Django, so it would ordinarily require a custom template tag or a view that calculated the value before sending it to the template.

Django will automatically process everything up to the

tag, passing all remaining

tokens to the Jinja compilation function along with the

object. The parser and tokens

can be used to extract the content written between the

and

tags. This then

needs to be converted back into string content before being passed to Jinja for rendering.

Converting a Token to a String

Before diving into the full compilation function, first notice that tokens must be converted back into strings for Jinja to process them. Jinja uses a fairly similar syntax for its templates, so Django’s

accurately identifies variable, block and comment tags. Though Jinja also creates token for those tags, tokens from the two template engines aren’t compatible with each other, so they must be converted back to strings. Jinja can then process them as it would any template from any source.

To accomplish this conversion, the node compilation function will rely on a separate function, which takes a token and returns a string. It works on the fact that also contains constants for the beginning and ending portions of these tags. With this information and the structure of tokens, a suitable string can be created from a given token.

150

C H A P T E R 6

฀ TEMPLATES

This won’t produce an exact replica of the original template string. Some whitespace gets removed during Django’s

processing and comments lose their contents entirely. All functional aspects of the tags are retained, so the template will still work as advertised, but know that some minor formatting issues may arise as the result of this technique.

Compiling to a Node

With a function in place to reproduce strings for the tokens within the blog, the next step

is to generate a

that will be used to render the content along with the rest of the template.

When gathering up the content between an opening tag and its closing tag, compilation functions often make use of the

method, passing in the name of the end tag, which

will return a list of

objects representing the inner content.

Since Jinja tags can’t be processed using Django’s node functions, will

cause problems due to incorrect syntax. Instead, the Jinja compilation function must access the tokens directly, which can then be converted back to strings. There are no provided functions to do this entirely, but combining

with some extra logic will work

quite well.

The compilation function can loop over the available tokens, calling each time. This loop will execute until either an

block token is found or there are no

more tokens in the template. Once a token is obtained from the parser, it can be converted to a string and added to an internal template string that can be used to populate a

.

C H A P T E R 6

฀ TEMPLATES

151

Caution By not using the parser’s

method, you won’t be able to use any other Django tags

inside of the

tag. That’s not a problem here, since the contents are processed by Jinja instead, but using this technique without a good reason can cause problems with other types of tags.

Preparing the Jinja Template

Once the compilation function retrieves the Jinja template contents from the Django template tokens, a

is created to access that template. Jinja provides its own object that

compiles content into tangible objects, so it makes sense to make use of it when a is

created.

Then, when it comes time to render the

, all it takes is to render the compiled

Jinja template and return that output back to Django’s template. This task is trickier than it may seem on the surface, because Django’s

object, which contains variables that

should be passed to Jinja, don’t behave entirely like Python dictionaries. They support the common dictionary- style syntax for accessing keys, but internally, their structure is quite different from what Jinja expects.

To pass a nested

object to the Jinja template properly, it must first be flattened to a single, standard Python dictionary. This can be done fairly easily, simply by looping through the individual dictionaries stored in the context and assigning them to a new dictionary, maintaining the precedence that Django itself uses: the first appearance of a particular key takes priority over any other instances of that same key. Only if a key doesn’t exist in the new Jinja context dictionary should it be added, so that no existing values get overwritten in the process.

Once the dictionary is available, that data can be passed to Jinja’s own method. The result from that method is the properly rendered content that can be returned from the

, placing that content in the page.

152

C H A P T E R 6

฀ TEMPLATES

Enabling User- Submitted Themes

Earlier in this chapter, we discovered that templates can be loaded from any source, as long as there’s an appropriate loader that knows how to retrieve them. One shortcoming of that approach is that it’s only valid for loading templates for everybody; there’s no way of associat-ing templates with a specific user.

That’s not really a failure in any way, since most applications would need it to work exactly as it does. Also, user information is only available once a request comes in, so there wouldn’t be any way to access it in a generic fashion. Every tool has its time, and there are certainly times where it’s useful to have templates tied to users.

Consider a site where users are encouraged to customize their own experience, by supplying custom themes that will be used while they’re logged in. This gives users a great deal of control over how they engage in the site, and can pull them further into the experience.

This can be enhanced still further if they’re given the opportunity for their own custom themes to be made available for others to use. This idea isn’t good for all sites, but for heavily community- oriented sites, especially those in artistic circles, it can be a great boost to the user experience.

A WORD ABOUT ADVERTISING

Many sites on the Web today are funded at least in part by advertisements placed on their various pages.

This advertising only works if it’s actually shown to users, so they have a chance to click on ads and buy products or services. By introducing user- editable themes to a site, users have a perfect opportunity to remove any ads a site may rely on, so it’s important to carefully consider whether this is right for your site.

Any themes that a site’s staff approves for the use of the site’s general audience can be checked first to ensure that they don’t cause any harm to the advertising on the site, or to the site’s own branding. This is a great way to enforce at least some quality control on the process. The problem is that users can create themes to behave however they like, prior to submitting them for approval, and may use them on their own through the site, removing ads from their own experiences.

One way to minimize the impact of this problem is to offer paid site memberships, with one of the benefits being the ability to create custom themes. This way, unpaid users will always see advertising as a way of funding their use of the site, while paid users are offsetting their lack of advertising with an annual fee.

In fact, if your site adopts this model, it’s best to remove ads for paid users altogether, regardless of what theme they’re using. Nobody likes paying for the use of a site, only to still be presented with advertising designed to bring further revenue to that same site.

C H A P T E R 6

฀ TEMPLATES

153

On the surface, it may seem like this is a perfect job for Cascading Style Sheets (CSS). CSS

is all about the presentation of Web sites, but it’s always limited by the ordering of content on a page. For example, markup placed higher in the document is difficult to place at the bottom of a page, and vice versa. By allowing users to edit the template that determines those posi-tions, it’s easy to unlock many more possibilities.

Using Django templates poses some technical and security challenges that must be overcome, and solving these challenges exposes a number of interesting ways to use templates.

First, consider the problems that need to be solved.

฀

฀ ฀

฀

฀ ฀ ฀

฀

฀

฀ ฀

฀ ฀

฀

฀

฀

฀

฀ ฀ ฀

฀ ฀ ฀

฀

฀ ฀

฀

฀

฀

฀

฀

and also for assigning credit to the proper authors when themes get promoted.

฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

a security risk that would expose way too much information to just anyone.

฀

฀

฀

฀ ฀

฀ ฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀ ฀

everyone.

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

shouldn’t be able to make any changes.

฀

฀ ฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀

฀ ฀

others—should be used on all portions of the site.

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀

฀ ฀

to better style other aspects of the site.

That’s quite a list of things that need to be covered, and individual sites may have even more requirements. It’s not quite as bad as it may seem on the surface, as Django already has many things in place to make those problems easy to solve.

Setting Up the Models

The first order of business is to make a place for templates to be stored in the database. In standard Django fashion, this is done with a model, with fields for the various properties of the template. For this application, a theme consists of a few various pieces of information:

฀

฀ ฀

฀ ฀

฀ ฀ ฀

฀ ฀

฀

฀ ฀

฀

฀

฀ ฀

฀ ฀ ฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀ ฀

฀ ฀

฀

฀

฀

฀

฀ ฀

฀ ฀ ฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

selected a theme still have one to use

Most of this information will only be used by the theme object itself, as only the main block of text will be passed in to the template. It’s easy to think of a theme as a template in its own right, where it’s simultaneously a set of data that gets stored in the database and a set of instructions that are used to render HTML. Python provides a way to make that notion explicit and offers a simple way to deal with themes.

154

C H A P T E R 6

฀ TEMPLATES

By using multiple inheritance, it’s possible for a theme to be both a model and a template, behaving in whichever way is necessary for the task at hand. The class inherits from and

, and

is overridden to initialize

both sides separately:

That’s enough to get the themes themselves stored in the database, but it still doesn’t cover how a user can select a theme to use while browsing the site. Ordinarily, that would be set up as a

on the model that references

, but since the

model is outside

our control, something else will need to be done.

C H A P T E R 6

฀ TEMPLATES

155

Whenever an application needs to store user- centric information, such as preferences, the proper way to go about it is to add a user profile model. Django’s official documentation3

covers this in detail, but the basic idea is that a model can be declared to hold user profiles, allowing applications to use it easily.

A site can only have one user profile model, and it makes little sense to hijack that model solely for the purpose of supporting themes. An ideal situation would be to simply add a

to a general- purpose profile and reference that field on the profile model in the

application.

Note This could be done automatically, by using the setting to identify the profile model and add a new field using the model’s

method. But since it only ever needs to

be applied to one model and won’t change after being applied, it makes more sense to just add the field by hand.

Doing this right requires some extra effort beyond just making it work. Like any good Django application, this should be as generic and reusable as possible, so it’s important to keep assumptions to an absolute minimum. Even hard- coding things like the name of the field used to store a user’s theme can seriously impair a site’s ability to use themes.

The matter of the theme’s field name is one of the biggest issues facing the reusability of this application. The key to solving it is to specify the field name on a per- site basis, a job best suited for site- wide settings. Adding settings shouldn’t be a first choice for implementing new features, but this is an example where it just isn’t feasible any other way.

In order to avoid potential clashes with other applications that may also use their own settings, it’s important to always choose a name with a prefix that’s specific to the application. Since this is a theme application, and the setting is for specifying the field to be used for a user’s selected theme, we’ll call it

. It’s also important to provide

defaults wherever possible. In this example, the name of the field is fairly arbitrary, but is as sensible as any, so we’ll use that.

Note This isn’t the only available approach. We could also go with a that relates

directly from

to

, avoiding the issues with the profiles entirely. That would raise another issue to deal with, since the

application would have to make sure only one theme gets stored per user.

That’s not difficult on its own, but going with the profile adds one additional benefit: users can manage their preferred theme right alongside their other profile settings, using whatever application already manages their profiles.

Since this is a way to retrieve themes from the database, it’s best to place the code for it on a custom manager, rather than directly on the

model itself. There are actually two

3.

156

C H A P T E R 6

฀ TEMPLATES

different methods that would be useful here, both for getting themes based on the user. One is for retrieving a user’s selected theme, using the

setting, while the other

is for retrieving the themes a user has created.

With this manager in place, it’s easy to retrieve themes for a specific user, both those that user can edit, and the one that user should use when browsing the site. Having these shortcuts in place helps make views simpler, allowing them to focus on the business they really have to do. The whole point of a site- wide theme is that it’s used for every view, so clearly something else needs to be done to accommodate that.

Supporting Site- Wide Themes

Individual views have enough to worry about, and shouldn’t be responsible for managing themes. Instead, there needs to be a way to retrieve a user’s selected theme—or the default—

and have that automatically applied to whatever template a view uses. Ideally, all this should happen without any changes to the views, so there’s little extra work that needs to be done.

This is a job best suited for a context processor, a concept described earlier in this chapter.

By using a context processor, every view that uses

will automatically have access

to the proper theme. This makes the ordinarily good advice of always using now

an absolute requirement. As will be seen in the next section, templates will explicitly rely on the theme being available, and failing to use

will violate that assumption.

The context processor required for this process is fairly straightforward, but it has to provide a few specific features. It must determine whether the current user is logged in or not, identify the user’s selected theme, fall back to a default theme if no theme is selected or if the user isn’t logged in, and it must return the proper theme so that it may be added to the template’s context. This code would be placed in a module called

, in

keeping with the conventions used within Django itself.

C H A P T E R 6

฀ TEMPLATES

157

Note the use of

here in the test to see whether a user is logged in. That may seem unnecessary, but by adding that simple condition to the test, it allows this context processor to be used with no middleware requirements. Otherwise, it would always require

, which places the

attribute

on the request. If that middleware isn’t in use, every user will simply receive the default theme.

Also, note that the name of the context variable is driven by another new setting, this time called

. Like

, this defaults to the

, so that it’s not

necessary to supply a name explicitly unless that causes a clash with some other feature. This is a bit of a recurring theme (pun intended), because with an application that has to interact with a good deal outside of itself, such as user profiles and template contexts, it’s important to make sure conflicts are kept to a minimum.

With this file in place, the only thing left is to add

to

the

setting to make sure it gets applied to all the templates. Once the theme is made available to the template, it’s still necessary to make sure the template can access it and make use of it.

Setting Up Templates to Use Themes

The end goal of themes is to reorder the components of a page, so it’s important to identify what a “component” is. In terms of Django templates, this would mean a block of markup, identified by the

template tag. Each component of a page could be defined in a separate block, separating each bit into its own space.

With Django’s template inheritance, it’s possible to define blocks in one template that will be filled in with content from another template. This way, a page- specific template can define what goes in each block, while a base template can specify where those blocks are rendered, and what other markup gets placed around them. This would be an excellent way to reorder significant portions of a page, as long as there’s a way to dynamically specify where the base template places all the blocks.

Django supports template inheritance through the

tag, which takes a single

argument to identify the base template to extend. Typically, this is a hard- coded name of the template to use as a base. It can also take a context variable, containing a string to use as this base template. If that context variable points to a template instance, Django will use that instead of bothering to look up a template anywhere else.

Taking advantage of this in a template is easy; just put

at the top of

the template. If you’ve specified a

explicitly for your site, make sure to

change

to whatever you’ve entered for that setting. That still only covers part of it. It’s still necessary to get the templates to make use of the blocks defined in the theme.

158

C H A P T E R 6

฀ TEMPLATES

There’s no universal way to do this, since each site will have its own template inheritance setup, and its own set of blocks that every page will need to fill in. Typically, these blocks would be used for things like page title, navigation, page content and footers, but different sites may have different needs.

In addition, a site may have more blocks that can’t be rearranged, but are instead defined inside of other blocks. These wouldn’t be taken into consideration for our purposes at the moment, since themes are only concerned with blocks that can be moved around. Consider an application with the following blocks that can be customized:

฀

฀

—The site’s logo, as an image

฀

฀

—The title of the current page

฀

฀

—A search box, possibly with advanced options

฀

฀

—A collection of links or other interface used for getting around the site

฀

฀

—A bit of content related to the current page

฀

฀

—The flesh of the current page, whether that be a product listing, press release, search results or contact form

฀

฀

—A copyright disclaimer, along with a few links for job openings, investor relations and contact information

Every theme must define all of these blocks in order to make sure the whole site gets displayed, so it’s important to outline them explicitly. Every template on the site needs to define content to be placed into these blocks, so that there’s always something to put in the right places. Many of those blocks aren’t specified to any particular page, so template inheritance comes to the rescue here as well.

By placing another template layer between the theme and the individual page, some blocks can be populated automatically for all pages, while others are left for individual pages to fill in. The individual page template still has final authority, with the ability to override any block with new content, if necessary. That just leaves the issue of making sure that templates do in fact define all the blocks required by the site’s inheritance scheme.

Validating and Securing Themes

Any time a site accepts input from users, it must be scrutinized to make sure that it fulfills a certain set of requirements and stays within acceptable limits. Themes are no exception there, but user- editable templates also represent a very real security risk. Django takes steps to ensure that templates can’t execute any common functions that make changes to the database, but there are a number of other things a template can do.

By default, only Django’s own data- altering methods are secured from templates by using the

attribute. Any application’s models may define other methods that make changes to the database, and if those aren’t marked with

, they’re fair game for

use in templates. Even read- only access, if not kept in check, can be a problem. A theme is used on every page, and many pages will have access to a wide array of objects through model relationships.

There are so many ways to access things that should be kept private that no blacklist approach can ever hope to be complete. Instead, a whitelist approach is necessary, where C H A P T E R 6

฀ TEMPLATES

159

themes are only allowed to use a small subset of features provided by Django’s template system. The trick is determining the right way to approach a problem like this.

On the surface, it may seem like regular expressions are the way to go. After all, Django itself uses a regular expression to parse templates and break them up into nodes, so surely it would be trivial to write a more limited expression to secure templates. That may be true for now, but remember that Django is constantly improving, and the future may bring new syntax to templates.

However unlikely that may be, if it does happen, no amount of careful crafting of our regular expression can predict what new syntax might be included in the future. Anything that slips past this protection has the potential to harm the site or divulge confidential information.

That’s a lot to pin on the hope that the template syntax will remain constant.

Instead, we’ll rely on Django’s own regular expression to compile the template into a list of nodes, just like normal. Then, once it’s been compiled to a

, it’s easy to peek at

those nodes to make sure they’re all doing the right thing. Using this, forms can easily verify that the template defines all the right blocks and nothing else. Theme templates must

฀

฀

฀

฀

฀

฀referenced by the

setting.

฀

฀

฀

฀

฀

฀the name referenced by a

setting.

฀

฀

฀

฀

฀

฀all the blocks referenced in the

setting.

฀

฀

฀

฀

฀ ฀

฀ ฀

฀

blocks.

฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

setting.

฀

฀

฀

฀

฀

฀

฀

฀

160

C H A P T E R 6

฀ TEMPLATES

An Example Theme

Even with an application in place, it may be difficult to understand how a theme would be written to work with the site. Consider a site using this application with the following

settings:

C H A P T E R 6

฀ TEMPLATES

161

The

template at the root of the inheritance chain might look like this: A theme can then be written to fill in the application’s requirements: extend from

, provide a

block and fill it with empty

,

and

blocks.

Now, individual templates for the rest of the site can be written to extend from the variable and fill in the

,

and

blocks. Consider the template for the root of

a real estate site:

With all of these templates in place, loading up the root of the site will yield a full HTML

document like the following:

162

C H A P T E R 6

฀ TEMPLATES

Now What?

Views and templates combine to determine what content should be sent to users, but it still has to make its way to the browser. Django speaks HTTP fluently, so there are a number of ways to customize that journey.

C H A P T E R 7

Handling HTTP

The Hypertext Transfer Protocol (HTTP) is the fundamental language for communication over the Web. It’s spoken by both Web servers and Web browsers, along with a variety of specialty tools for dealing with the Web.

The Python community has done a tremendous amount of work to standardize the behavior of applications that interact with HTTP, culminating in PEP- 333,1 the Web Server Gateway Interface (WSGI). Since Django follows the WSGI specification, many of the details listed in this chapter are a direct result of compliance with PEP- 333.

Requests and Responses

Because HTTP is a stateless protocol, at its heart is the notion of a request and a response.

Clients issue a request to the server, which returns a response containing the information requested by the client or an error indicating why the request couldn’t be fulfilled.

While requests and responses follow a detailed specification, Django provides a pair of Python objects that are designed to make the protocol much easier to deal with in your own code. A basic working knowledge of the protocol is useful, but most of the details are handled behind the scenes. These objects are described in this section, along with notes indicating the relevant portions of the specification that should be referenced.

HttpRequest

As described in Chapter 4, every Django view receives, as its first argument, an object representing the incoming HTTP request. This object is an instance of the class, which

encapsulates a variety of details concerning the request, as well as some utility methods for performing useful functions.

The base

class lives at

, but individual server connectors will

define a subclass with additional attributes or overridden methods that are specific to the Web server being utilized. Any overridden methods or attributes should behave as documented here, and any additional information will be best documented in the code for the server interface itself.

1.

163

164

C H A P T E R 7

฀ HANDLING HTTP

HttpRequest.method

The HTTP specification outlines a variety of verbs that can be used to describe the type of request being performed. This is typically referred to as its method, with different request methods having specific expectations of how they should be handled. In Django, the method being used for the request is represented as the

attribute of the

object. It

will be included as a standard string, with the method name in all uppercase letters.

Each method describes what the server should do with the resource identified by the URL. Most Web applications will only implement GET and POST, but a few others are worth explaining here as well. Further details on these—and others not listed here—can be found in the HTTP specification,2 as well as many other resources on the Web.

฀

฀

฀

฀

฀

฀be deleted. Web browsers don’t implement

this method, so its use is limited to Web service applications. In typical Web browser applications, such operations are done with a POST request, since GET requests aren’t allowed to have side effects, such as removal of the resource.

฀

฀

฀

฀

฀

฀by the URL. This is, by far, the most com-

mon type of request made on the Web, as every standard retrieval of a Web page is done with a GET request. As noted in the “‘Safe’ Methods” section, GET requests are assumed to have no side effects on the server; they should retrieve the specified resource and nothing else.

฀

฀

฀

฀

฀

฀the resource without getting the entire

contents. Specifically, the response to a HEAD request should return exactly the same headers as a GET request, only without anything in the body of the response. Web browsers don’t implement this method, but since the server- side operation is essentially just a GET request without a response body, it is rarely missed. In Web service applications, a HEAD request can be a low- bandwidth way to retrieve information about a resource, such as whether it exists, when it was last updated or the size of its content.

฀

฀

฀

฀

฀

฀data be stored in some way related to the resource

specified by the URL. This could mean comments on a blog post or news article, answers to a question, replies to a Web- based email or any number of other related situations.

This definition is only valid in Web service environments, where a differentiation can be made between PUT and POST. In standard Web browsers, only GET and POST are reliably available, so POST is used for any situation that modifies information on the server. Using POST to submit data from a form is more of a footnote in the official HTTP specification, but is the most popular use of the method.

฀

฀

฀

฀

฀

฀data be stored at the resource specified by the URL.

This could be seen as a “create” or “replace” operation, depending on whether the resource already exists. This method isn’t traditionally available in Web browsers, though, so its use is limited to Web service applications. In a standard Web browser, the operation specified by PUT is done with a POST request instead.

2.

C H A P T E R 7

฀ HANDLING HTTP

165

“Safe” Methods

As alluded to in the previous section, there is an important distinction to be made among various types of HTTP requests. The specification refers to GET and HEAD as “safe” methods, which only retrieve the resource specified by the URL, without making any changes on the server at all. To be explicit, a view that processes a GET or HEAD request shouldn’t make any changes except those that are incidental to retrieving the page.

The goal of safe methods is to allow the same request to be made more than once and at various times, without any adverse effects. This assumption allows GET requests to be used by bookmarks and browser histories without a warning to the user when the request is made more than once. An example of an allowed change is updating a count that indicates how many times the page was viewed.

“Idempotent” Methods

In addition to safe methods, the HTTP specification describes PUT and DELETE as “idempotent,” meaning that, even though they are intended to make changes on the server, those changes are reliable enough that calling the same request with the same body multiple times will always make the same changes.

In the case of PUT, the resource would be created the first time the request is performed, and each subsequent request would simply replace the resource with the same data that was originally submitted, thus leaving it the same. For DELETE, each subsequent request after the resource was originally deleted would result in an error, indicating that the resource isn’t present, thus leaving the state of the resource the same each time.

This behavior is contrary to POST, which is expected to always make changes or additions on each request. To represent this situation, Web browsers present a message when a POST

request is performed more than once, warning the user that subsequent requests could cause problems.

HttpRequest.path

This attribute contains the complete path that was requested, without any query- string parameters attached. This can be used to identify the resource being requested, without relying on which view will be called or how it will behave.

Accessing Submitted Data

Any time a request comes in, it can potentially be accompanied by a variety of data provided by the Web browser. Processing this information is key to making a Web site dynamic and interactive, so Django makes it easy and flexible. Just as there are many ways to submit data to a Web server, there are as many ways to access that data once it arrives.

Data that comes in using the standard query- string format3 sent by most browsers is automatically parsed into a special type of dictionary class called

. This is an immutable

subclass of

, which means that it functions mostly like a dictionary, but with a few added options for handling multiple values for each key in the dictionary.

3.

166

C H A P T E R 7

฀ HANDLING HTTP

The most significant detail of

is that it’s instantiated with a query- string from an

incoming request. For more information on the details of how to access values in a

,

see the details for

in Chapter 9.

HttpRequest.GET

If the request came in with the GET method, its

attribute will be a

containing all

the values that were included in the query- string portion of the URL. Of course, while there’s no technical restriction on when

can be used to get parameters out of a URL, the goal of

clean URLs limits the situations where it’s most advantageous.

In particular, it’s important to separate parameters that identify a resource from those that customize how the resource is retrieved. This is a subtle, but important, distinction. Consider the following examples:

฀

฀

฀

฀

฀

฀

As you can see, most of the data sent to the view for GET requests should be placed in the URL itself, rather than the query- string. This will help search engines index them more efficiently, while also making it easier for users to remember them and communicate them with others. As with many other principles, this isn’t an absolute rule, so keep query- strings and the attribute in your toolbox, but use them with care.

HttpRequest.POST

If the request comes in with a PUT or POST method using a standard HTML form, this will be a

containing all the values submitted with the form. The

attribute will be popu-

lated for all standard forms, regardless of the encoding type, with or without files.

However, the HTTP specification allows these requests to supply data in any format, so if the incoming data doesn’t fit the format of a query- string, will be empty,

and the data will have to be read in directly through

.

HttpRequest.FILES

If an incoming PUT or POST request includes any uploaded files, those files will be stored away in the

attribute, which is also a

, with each value being an

object.

Living at

, this is a subclass of the

object described in

Chapter 9, providing a few extra attributes specific to uploaded files.

฀

฀

—The

associated with the file, if any was provided. Web

browsers typically assign this based on the last part of the filename, though a Web service call could specify this more accurately based on the actual type of content.

฀

฀

—The character set that was specified for the uploaded file’s content.

HttpRequest.raw_post_data

Any time a request comes in with data in the body of the request, as is done for PUT and POST, the

attribute provides access to this content, without any parsing. This isn’t C H A P T E R 7

฀ HANDLING HTTP

167

typically necessary for most Web sites, as the

and

attributes are more appropriate for

the most common types of requests. Web services may accept data in any format, and many use XML as a primary means of data transfer.

Dictionary Access

In addition to accessing

and

separately, Django provides a shortcut for accessing val-

ues provided with the request, regardless of which method is used. To use this shortcut, simply access the request as a dictionary. This will check values in before

. Like any diction-

ary, if a value can’t be found in either location, a

is raised.

It’s important to note that accessing values from the request as a dictionary doesn’t provide the full functionality of

. Instead, accessing a value by key will behave just

like doing the same with a

, returning the last instance of that value that was

specified in the request.

HttpRequest.META

When a request comes in, there is a significant amount of information related to the request that doesn’t come through in a query- string and isn’t available in the or

attributes on

the request. Instead, data regarding where the request came from and how it got to the server is stored in the request’s

attribute. Details of which values are available in

can be

found in PEP- 333.

In addition, each request is accompanied by a number of headers, which describe various options the client would like to make known. Exactly what these types of headers can contain is specified in the HTTP specification,4 but they typically control things like a preferred language, allowable content- types and information about the Web browser.

These headers are also stored in

, but in a form slightly altered from how they came in

originally. All HTTP header names become uppercase, are prefixed with and have all of

their dashes replaced with underscores.

฀

฀

becomes

.

฀

฀

becomes

.

฀

฀

becomes

.

4.

168

C H A P T E R 7

฀ HANDLING HTTP

HttpRequest.COOKIES

Since each HTTP request is a fresh connection between the client and the server, cookies are used as a way to identify clients that make multiple requests. In a nutshell, cookies are little more than a way to send a name and associated value to a Web browser, which that browser will then send back each time it makes a new request to the Web site.

While cookies are set during the response phase of the process, as documented under

, the task of reading cookies from an incoming request is quite simple. The attribute of the request is a standard Python dictionary mapping names of cookies to the values that were previously sent.

Keep in mind that this dictionary will contain entries for all cookies sent by the browser, even if they were set by another application on the same server. The section later

in this chapter covers the specific rules of how a browser decides which cookies to send with a particular request and how to control that behavior.

HttpRequest.get_host()

Many server configurations allow a single Web application to respond to requests sent to multiple different domain names. To help with these situations, the method of the incoming

request allows a view to identify the name that the Web browser used to reach the Web site.

In addition to the host name used to make the request, the value returned from this method will include a port number if the server was configured to respond on a nonstandard port.

HttpRequest.get_full_path()

In addition to the host information, the

method returns the entire path portion

of the URL; everything after the protocol and domain information. This includes the full path that was used to determine which view to use as well as any query- string that was provided.

HttpRequest.build_absolute_uri(location=None)

This method generates an absolute URL for the provided location, if any. If no location is supplied explicitly, the request’s current URL is returned, including the query- string. The exact behavior of the method if the location is provided depends on what value is passed in.

฀

฀ ฀

฀

฀

฀ ฀฀

฀

฀

฀

฀

฀ ฀

฀

absolute and is returned as provided.

฀

฀ ฀

฀

฀

฀with a forward slash ( ), it is appended to the protocol and domain information of the current URL, then returned. This will generate an absolute URL for the provided path, without having to hard- code the server information.

฀

฀

฀

฀

฀ ฀

฀ ฀ ฀ ฀

฀

฀ ฀

฀

฀

฀

฀

฀

the two will be joined together using Python’s

utility function.

HttpRequest.is_secure()

This simple method returns

if the request came in using the Secure Sockets Layer (SSL) protocol or

if the request was unsecured.

C H A P T E R 7

฀ HANDLING HTTP

169

HttpRequest.is_ajax()

Useful for “Web 2.0” sites, this method returns

if the request has an

header with a value of

. Most JavaScript libraries designed to make calls to the server will provide this header, providing a convenient way to identify them.

HttpRequest.encoding

This is a simple attribute representing the encoding to be used when accessing the and

attributes described previously. Values in those dictionaries are forced to objects

using this encoding, if one is set. By default, its value is

, which will use the default encod-

ing of

when accessing values.

In most cases, this attribute can be left as is, with most input being converted properly using the default encoding. Specific applications may have different needs, so if the application expects input with a different encoding, simply set this attribute to a value that will decode those values properly.

HttpResponse

After a request is received and processed, every view is responsible for returning a response—

an instance of

. This object maps cleanly to an actual HTTP response, including headers, and is the only way of controlling what is sent back to the Web browser. Like its cousin for requests,

lives at

, but several shortcuts are available to create

responses more easily.

Creating a Response

Unlike the request, the author of a view has full control over how its response is created, allowing a variety of options. The standard

class is instantiated rather simply,

but accepts three arguments to customize its behavior. None of these are required; options described later in this section can set these values in other ways.

฀

฀

—This accepts text—or other content—to be used as the body of the request.

฀

฀

—This sets the HTTP status code5 to be sent with the request.

฀

฀

—This controls the Content- Type header to be sent with the request. If this is supplied, make sure it also contains the

value when appropriate.

5.

170

C H A P T E R 7

฀ HANDLING HTTP

There is also a

argument, provided for backwards- compatibility with older Django applications, but

should be used instead. It’s still important to keep

in mind, though, as it means that

and

should be specified as

keyword arguments if supplied at all.

Dictionary Access to Headers

Once a response has been created, it’s simple to customize the headers that will be sent out along with its content, using standard dictionary syntax. This is quite straightforward and works just as you’d expect. The only notable variation from a standard dictionary is that all key comparisons are case- insensitive.

File-Like Access to Content

In addition to the ability to specify body content as a string when creating the response object, content can be created by many third- party libraries that know how to write to open files. Django’s implements a few file protocol methods—most notably

—that enable it to

be treated as a write- only file for many of these libraries. This technique can be especially useful when using Django to generate binary content, such as PDF files, dynamically within views.

One important thing to note regarding file- like access to the response body is that not all file protocol methods are implemented. This means that certain libraries, such as Python’s own

class, which require those extra methods, will fail with

an

, indicating which method was missing. This is by design, as HTTP

responses aren’t true files, so there is no predictable way to implement those methods.

HttpResponse.status_code

This attribute contains the numerical status code representing the type of response being sent to the client. As described earlier, this can be set immediately when instantiating the response C H A P T E R 7

฀ HANDLING HTTP

171

object, but as a standard object attribute, it can also be set any time after the response has been created.

This should only be set to known HTTP response status codes. See the HTTP specification for details on valid status codes. This status can be set while instantiating the response, but it can also be set as a class attribute on a subclass, which is how Django configures many of its specialized responses.

HttpResponse.set_cookie(key, value="[, ...])

When looking to store values across multiple requests, cookies are the tool of choice, passing values to the Web browser through special headers, which are then sent back to the server on subsequent requests. By calling

with a key and a value, the HTTP response sent

to the client will contain a separate header, telling the browser what to store and when to send it back to the server.

In addition to just the key and value,

can take a few extra arguments that

configure when the browser should send the cookie back to the server. While a quest for readability suggests that these arguments be specified using keywords, this list uses their positional order. More details on what values are allowed for each of these options can be found in the official specification for HTTP state management.6

฀

฀

—Corresponding to the

option from the specification, this speci-

fies the number of seconds the cookie should remain active.

฀

฀

—Not all browsers accept and respect

as required by the offi-

cial specification but instead follow an early pattern set out by Netscape. The attribute takes an exact date when the cookie should expire, rather than an offset in seconds. The specified date is in the following format:

.

฀

฀

—This specifies a base path under which the browser should send this cookie back to the server. That is, if the path of the URL being requested begins with the value specified here, the browser will send the cookie’s value along with the request.

฀

฀

—Similar to

, this specifies the domain under which the cookie will be sent. If left as

, the cookie will be restricted to the same domain that issued it, while providing a value will allow greater flexibility.

฀

฀

—If set to

, this will indicate that the cookie contains sensitive information and should only be sent to the server through a secure connection, such as SSL.

6.

172

C H A P T E R 7

฀ HANDLING HTTP

Keep in mind that this will set the cookie in the browser only after the response has made its way across the wire. That means that the cookie’s value won’t be available on the request object until the browser’s next request.

COOKIES AND SECURITY

Although cookies can be a tremendously useful way to maintain state across multiple HTTP requests, they’re stored on a user’s computer, where knowledgeable users will have access to view them and alter their contents. Cookies on their own are not secure, and should not be used to store sensitive data or data that controls how the user can access the site.

The typical way around this problem is to only store a reference in the cookie, which can be used to retrieve the “real” data from somewhere on the server, such as a database or a file, where users don’t have access. The “Applied Techniques” section near the end of this chapter provides an alternative method of storing data securely in cookies so that their data can in fact be trusted.

HttpResponse.delete_cookie(key, path='/', domain=None)

If a cookie has already been delivered to the Web browser and is no longer needed or has become invalid, the

method can be used to instruct the browser to remove it.

As mentioned, the path and domain provided here must match an existing cookie in order to have it deleted properly.

It does this by setting a new cookie with

set to and

set to

. This causes the browser to overwrite any existing cookie matching the same

,

and

, then expire it immediately.

HttpResponse.cookies

In addition to being able to explicitly set and delete cookies during the response phase, you can view the cookies that will be sent to the Web browser. The attribute uses Python’s

standard

module,7 with the attribute itself being a

object, which behaves

much like a dictionary, with each value being a

object.

Using a cookie’s name as the key, you can retrieve a

representing a specific cookie

value, along with its associated options. This object may be used as a dictionary to reference these additional options, while its

attribute contains the value that was set for the cookie.

Even deleted cookies are accessible using this dictionary, since the process involves setting a new cookie that will simply expire immediately.

7.

C H A P T E R 7

฀ HANDLING HTTP

173

HttpResponse.content

This attribute provides access to the string content of the response body. This can be read or written, and is particularly useful during the response phase of middleware processing.

Specialty Response Objects

Since there are several common HTTP status codes, Django provides a set of customized

subclasses with their

attribute already set accordingly. Like

itself, these all live at

. Some of them take a different set of argu-

ments than the standard

, and those differences are also listed here.

฀

฀

—Takes a single argument, a URL that the browser will redirect to. It also sets the

to 302, indicating a “Found” status, where the resource

is located.

฀

฀

—Takes a single argument, a URL that the browser

will redirect to. It sets the

to 301, indicating the resource was permanently

moved to the URL specified.

฀

฀

—Sets the

to 304, indicating a “Not Modified”

status, to be used in response to a conditional GET, when the response hasn’t changed from the conditions associated with the request.

฀

฀

—Sets the

to 400, indicating a “Bad Request”

where the syntax used in the request couldn’t be understood by the view.

฀

฀

—Sets the

to 403, “Forbidden,” where the

requested resource does exist, but the requesting user doesn’t have permission to access it.

฀

฀

—Perhaps most common of all custom classes, this sets the to 404, “Not Found,” where the URL in the request didn’t map to a known resource.

฀

฀

—Sets the

to 405, “Not Allowed,” indicating that

the method used in the request isn’t valid for the resource specified by the URL.

฀

฀

—Sets the

to 410, “Gone,” to indicate that the resource

specified by the URL is no longer available and can’t be located at any other URL.

฀

฀

—Sets the

to 500, “Server Error,” used whenever

the view encountered an unrecoverable error.

Some of these specialized responses aren’t supported by Web browsers, but they’re all quite useful for Web service applications, where a wider range of options are available. It often makes more sense to set these statuses on a site- wide basis, so individual views don’t have to worry about managing them directly. For this, Django provides HTTP middleware.

174

C H A P T E R 7

฀ HANDLING HTTP

Writing HTTP Middleware

While Django itself creates an

and each view is responsible for creating an

, applications commonly need certain tasks to be performed on every incoming request or outgoing response. This portion of the process, called middleware, can be a useful way to inject advanced processing into the flow.

Common examples of middleware processing are compressing response content, denying access to certain types of requests or those from certain hosts and logging requests and their associated responses. Although these tasks could be done in individual views, that would not only require a great deal of boilerplate but would also require each view to know about every piece of middleware that would be applied.

This would also mean that adding or removing HTTP processing would require touching every single view in an entire project. That’s not only a maintenance issue in its own right, but it also causes additional maintenance problems if your project uses any third- party applications. After all, changing third- party code restricts your ability to upgrade it in the future without unnecessary hassle. Django solves these problems by performing middleware operations in a separate part of the request/response cycle.

Each piece of middleware is simply a Python class that defines at least one of the following methods. There are no other requirements for this class; that is, it doesn’t have to subclass any provided base class, contain any particular attributes or be instantiated in any specific way.

Just provide the class at an importable location and a site will be able to activate it.

There are four distinct points where middleware can hook into Django’s HTTP handling, performing whatever tasks it needs along the way. Each part of the process is controlled simply by specifying a method on the middleware class. Remember, it’s all just Python, so anything that’s valid Python is valid in middleware as well.

MiddlewareClass.process_request(self, request)

As soon as the incoming HTTP request is made into an

object, middleware has its

first chance to change how things get handled. This hook occurs even before Django analyzes the URL to determine which view to use.

Being standard Python, the

method can perform any task, but com-

mon tasks include prohibiting access to certain clients or request types, adding attributes to the request for use by context processors or returning a previously-cached response based on details of the request.

This method can change any attribute on the request, but keep in mind that any changes will affect how Django handles the request throughout the rest of the process. For example, because this method is called prior to the URL resolution, it can modify to

redirect the request to an entirely different view than would’ve otherwise been used. While something like this is often the desired behavior, it can possibly be an unintended side effect, so take care when modifying the request.

MiddlewareClass.process_view(self, request, view, args, kwargs) This method is called after the URL has been mapped to a view and arguments have been extracted from it, but before the view is actually called. In addition to the request, the arguments passed to this method are as follows:

C H A P T E R 7

฀ HANDLING HTTP

175

฀

฀

—The view function that will be called. This is the actual function object, not the name, regardless of whether the view was configured using a string or a callable.

฀

฀

—A tuple containing the positional arguments that will be passed to the view.

฀

฀

—A dictionary containing the keyword arguments that will be passed to the view.

Now that the view’s arguments have been extracted from the URL, it is possible to verify these against what the configuration was supposed to obtain. This can be quite useful during development as a way to verify that everything is configured properly. Simply set up a middleware to print out the

and

variables along with

. Then, if anything goes

wrong with a view, the development server’s console will have a handy way to identify or rule out a potential problem.

This may seem like a perfect opportunity to do some detailed logging of the view that’s about to be executed as well, since the view function object is available too. While this is true, the common use of decorators on views complicates matters. Specifically, the view function passed to this method will often be a wrapper function created by the decorator, rather than the view itself.

This means that the introspection features detailed in Chapter 2 can’t reliably be used to line up positional arguments with the names they were given in the function definition. There is still some good, though, as you should still be able to access the module and name of the view, as long as the decorators use the special

decorator described in Chapter 9.

MiddlewareClass.process_response(self, request, response) After the view has been executed, the new response object is made available for middleware to view it and make any necessary changes. This is where middleware could cache the response for future use, compress the response body for faster transmission over the wire or modify the headers and content that will be sent with the response.

It receives the original request object as well as the response object returned by the view.

At this point, the request has already exhausted its usefulness to the HTTP cycle, but it can be useful if some of its attributes are used to determine what to do with the response. The response object can be—and often is—modified at this stage, before being returned by the method.

The

method should always return an

object, regardless

of what’s done with it beforehand. Most often, this will be the response it was given in the first place, just with some minor modifications. Sometimes, it may make more sense to return an entirely different response, such as when redirecting to a different URL.

MiddlewareClass.process_exception(self, request, exception) If something goes wrong during any part of the request- handling process, including the middleware methods, an exception will usually be thrown. Most of these exceptions will be sent to the

to be logged or handled in a special way. The exception argument 176

C H A P T E R 7

฀ HANDLING HTTP

passed to this method is the exception object that was thrown, and it can be used to retrieve specific details about what went wrong.

A common task for this stage of the process is to log exceptions in a way that’s specific to the site currently in use. The exception’s string representation is usually sufficient for this, along with its type, though the exact usefulness of this will depend on the exception that was raised. By combining details of the original request with details of the exception, you can generate useful and readable logs.

Deciding Between Middleware and View Decorators

Chapter 4 showed how views can use decorators to perform extra work before or after the view is executed, and keen readers will notice that middleware can perform a similar function. View decorators have access to the incoming request as well as the response generated by the view. They can even access the view function and the arguments that will be passed to it, and they can wrap the view in a

block to handle any exceptions that are raised.

So what makes them different, and when should you use one over the other? That’s a rather subjective topic, and there’s no one answer to satisfy all cases. Each approach has advantages and disadvantages, which should help you decide which route to take for a particular application.

Differences in Scope

One of the most notable differences between the two is how much of the site is covered. Middleware is activated in a site’s

, so it covers all requests that come in on any URL.

This simple fact provides a few advantages:

฀

฀

฀

฀ ฀

฀ ฀

฀

฀

฀

฀

฀

request on the site; middleware makes these tasks easy to implement.

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀

having to make any special allowances for the behavior they provide.

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀ ฀

฀

฀ ฀

middleware behavior.

Decorators, on the other hand, are applied to individual functions, which means that every view must have decorators added manually. This makes decorators a bit more time- consuming to manage, but some operations—such as access restriction or specialized cache requirements—

are more appropriate for limited parts of the site, where decorators can be used to great effect.

Configuration Options

Middleware are specified as strings containing the import path to the class, which doesn’t allow any direct way to configure any of their features. Most middleware that accept options do so by way of custom settings that are specific to that middleware. This does provide a way to customize how the middleware works, but like middleware itself, these settings are sitewide, by definition. There isn’t any room for customizing them for individual views.

As shown in Chapter 2, decorators can be written to accept configuration options when they’re applied to a function, and view decorators are no different. Each view could have a C H A P T E R 7

฀ HANDLING HTTP

177

separate set of options or

could be used to create a brand- new decorator with a set of preconfigured arguments.

Using Middleware As Decorators

Given the similarities between middleware and decorators, Django provides a utility to transform an existing middleware class into a decorator. This allows code to be reused across an entire site, using the best tool for the job in any situation.

Living at

, the special

function

takes, as its only argument, a middleware class that should be applied to a single view. The return value is a perfectly functional decorator, which can be applied to any number of views.

Allowing Configuration Options

Since decorators can accept options to configure their behavior, provides a way for middleware classes to utilize this same flexibility. Doing so is a simple task of providing an

method on the middleware class that accepts additional arguments besides

. This allows a class to be written from the beginning to be used either as middleware or as a view decorator.

One thing to keep in mind is that middleware will be most commonly called without any arguments, so any additional arguments you define must use defaults. Failing to do so will result in a

whenever it is used as a standard middleware, regardless of how it is expected to be used as a decorator.

When used as middleware, this class will pad all responses to be at least 1,024 characters in length. If it’s used as a decorator, individual views can be given specific values to be used for this minimum length.

Also, be aware that if a middleware class is already defined as middleware and as a decorator, any views that use the decorator will actually be calling the middleware twice for every request. For some, such as those that set attributes on the request object, this won’t be an issue. For others—especially those that modify the outgoing response—this can cause a world of trouble.

178

C H A P T E R 7

฀ HANDLING HTTP

HTTP-Related Signals

Since requests are spawned outside the control of any application code, signals are used to inform application code of the beginning and completion of all request/response cycles. Like all signals, these are simply

objects, and they live at

. For more

information on signals, how they work and how to use them, refer to Chapter 9.

django.core.signals.request_started

Whenever a request is received from the outside, this signal is fired without any additional parameters. It fires early in the process, even before the object has been created.

Without any arguments, its uses are limited, but it does provide a way to notify applications when a request is received, before any middleware has a chance to get access to the request object.

One potential use for this would be as a way to register new listeners for other signals, which should only operate during requests coming in over HTTP. This is in contrast to situations where those other signals might get fired due to some non- HTTP event, such as a scheduled job or a command- line application.

django.core.signals.request_finished

Once the response has been generated by the view and middleware has been processed, this signal fires just prior to sending the response back to the client that sent the original request.

Like

, it doesn’t provide any parameters to the listener, so its use is fairly limited, but it could be used as a way to disconnect any listeners that were attached when fired.

django.core.signals.got_request_exception

If an exception occurs any time while processing a request but it isn’t handled explicitly somewhere else, Django fires the

signal with just one parameter: the request

object that was being processed.

This is in contrast with the

method of middleware, which is only

fired for errors that occur during execution of the view. Many other exceptions will fire this signal, such as problems during URL resolution or any of the other middleware methods.

Applied Techniques

By providing so many hooks into the protocol handling, Django makes possible a great variety of options for modifying HTTP traffic for an application. This is an area where each application will have its own needs, based on what type of traffic it receives and what type of interface it expects to provide. Therefore, take the following examples as more of an explanation of how to hook into Django’s HTTP handling, rather than an exhaustive list of what can be done to customize this behavior.

C H A P T E R 7

฀ HANDLING HTTP

179

Signing and Validating Cookies

As mentioned previously, cookies are inherently insecure, and their content can’t be trusted reliably without some extra care. Most often, data that shouldn’t be seen or edited by the user should be stored on the server, with just a reference to that data, such as an ID, stored in a cookie.

Sometimes, it makes sense to store some simple data in a cookie, saving time and resources by avoiding database calls, but that data shouldn’t be altered by the user, because it’s used for sensitive operations, such as identifying the user or what that user has done on the site. This should most often be limited to simple data, such as an OpenID8 URL or a date the user last logged in, as opposed to more complex data like a shopping cart.

Security Through Digital Signatures

The key to writing trustworthy cookies is to attach a signature along with the value, which can be easily created by the server when writing it out the first time and validated by the server on subsequent requests, but which can’t be edited by the user without the server knowing that it was tampered with.

This may sound like a complicated task best left to security experts, but it’s actually quite a simple process. All it takes is combining some aspects of the cookie with aspects of the server that Django knows but to which users don’t have access. A hash is created, based on this combination of data, and attached to the cookie’s value before it gets written to the client.

The signature can be generated from scratch again on the server and compared to the signature that was supplied with the cookie. If they don’t match, the cookie is assumed to have been tampered with and should be discarded, so it never reaches the rest of the system. This creates a combination of values that must all remain intact; if any one of them is changed, the cookie is known to be compromised.

In this particular example, four values are included in this data combination:

฀

฀ ฀

฀

฀

฀

฀ ฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀ ฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀these values

Flexibility Considerations

Only three of the four values in this combination are easy to determine, as they are aspects of the cookie itself or are generated based on the remaining values. The last—the secret known only to the server—is a bit different, but Django provides an excellent facility for this already.

Every project created with Django contains a special setting, called

, which is

already intended to be a secret known only to the server. The default even contains a warning not to share its value with anyone. This is an excellent candidate for use in cookie signatures, but there may be other values that make more sense for a given application.

Since this middleware is planned to also be used as a decorator, it could be written to accept an argument that would specify what to use as the secret when generating signatures. This would default to

but could be overridden for individual instances of the decorator.

8.

180

C H A P T E R 7

฀ HANDLING HTTP

The Code

The following code supplies a middleware to transparently handle signed cookies. It provides all the necessary functions to ensure that cookie values can be trusted, and does so without views having to change a single thing about their behavior. Everything is handled in the middleware, including all the following features:

฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

having to do anything special.

฀

฀

฀

฀

฀

฀ ฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

receive just the raw value, with no knowledge that the cookie was signed.

฀

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀ ฀

are removed from the request, so views never see them at all.

฀

฀

฀

฀

฀ ฀

but can be changed on a per- decorator basis.

C H A P T E R 7

฀ HANDLING HTTP

181

Now What?

The request and response cycle is the primary interface Django applications use to communicate with the outside world. Just as important is the collection of utilities available behind the scenes that allow applications to perform their most fundamental tasks.

C H A P T E R 8

Backend Protocols

As a framework, Django’s purpose is to provide a cohesive set of interfaces to make the most common tasks easier. Some of these tools are contained entirely within Django itself, where it’s easy to maintain consistency. Many other features are—or at least, could be—provided by external software packages.

Although Django itself supports some of the most common software packages for these various features, there are many more out there, especially in corporate environments. In addition to a developer’s preferences for one type of database over another, many other servers are already in use by existing applications that can’t be easily converted to use something different.

Since these types of problems do come up in real life, Django provides easy ways to reference these features without worrying about what implementation actually makes it happen in the background. This same mechanism also allows you to swap out many of these lower- level features with third- party code, to support connecting to other systems or just to customize some facet of behavior.

The sections listed throughout this chapter serve something of a dual purpose. In addition to documenting Django’s generic API for each of these features, each section will also describe how a new backend should be written to implement these features. This includes not only what classes and methods to declare, but also what the package structure might look like, as well as how each piece of the puzzle is expected to behave.

Database Access

Connecting to databases is one of the most fundamental requirements of a modern Web application, and there are a great deal of options available. Currently, Django ships with support for some of the more popular open source database engines, including MySQL, PostgreSQL and SQLite, and even some commercial offerings such as Oracle.

Given the unique features and SQL inconsistencies of different database systems, Django requires an extra layer between its models and the database itself, which must be written specifically for each database engine used. The supported options each ship within Django as a separate Python package containing this intermediary layer, but other databases can also be supported by providing this layer externally.

183

184

C H A P T E R 8

฀ BACKEND PROTOCOLS

While Python provides a standardized API for accessing databases, PEP- 249,1 each database system interprets the base SQL syntax in a slightly different way and supports a different set of features on top of it, so this section will focus on the areas provided by Django for hook-ing into the way models access the database. This leaves to the reader the nitty- gritty details of formulating the right queries in each situation.

django.db.backends

This is a reference to the backend package’s

module, from which the entirety of the

database can be accessed. Accessing the database backend in this manner ensures a unified, consistent interface, regardless of which database package is being used behind the scenes.

Django does a lot of work to make this level of access unnecessary, but there’s only so far it can go without overcomplicating things. When the ORM fails to offer some necessary bit of functionality—for instance, updating one column based on the value of another column in pure SQL—it’s always possible to go straight to the source and peek at what’s really going on, adjust the standard behavior or replace it altogether.

Because this is really just an alias for a backend- specific module, the full import paths listed throughout this chapter are only valid when trying to access the database in this manner. When implementing a new backend, the package path will be specific to that backend.

For instance, if a backend for connecting with IBM’s DB22 were placed in a package named

, this module would actually be located at

.

DatabaseWrapper

One of the main features of a database backend is the

, the class that acts as

a bridge between Django and the features of the database library itself. All database features and operations go through this class, in particular an instance of it that’s made available at

.

An instance of

is created automatically, using the

set-

ting as a dictionary of keyword arguments. There isn’t any mandated set of arguments for this class, so it’s essential to document what arguments the backend accepts, so developers can customize it accordingly.

There are a few attributes and methods on the

class that define some of

the more general aspects of the backend’s behavior. Most of these are suitably defined in a base class provided to make this easier. By subclassing

,

some sensible default behaviors can be inherited.

Though individual backends are free to override them with whatever custom behavior is appropriate, some must always be explicitly defined by a backend’s

. Where

that’s the case, the following sections will state this requirement directly.

DatabaseWrapper.features

This object, typically an instance of a class specified as

,

contains attributes to indicate whether the backend supports each of a variety of database- related 1.

2.

C H A P T E R 8

฀ BACKEND PROTOCOLS

185

features Django can take advantage of. While the class could technically be named anything, since it’s only ever accessed as an attribute of

, it’s always best to remain consistent

with Django’s own naming conventions to avoid confusion.

Like

itself, Django provides a base class specifying defaults for all of the available attributes on this object. Located at

, this

can be used to greatly simplify the definition of features in a particular backend. Simply override whatever feature definitions are different for the backend in question.

This is a list of supported features and their default support status:

฀

฀

—Indicates whether the database can iterate over portions of the result set without reading it all into memory at once. Defaults to

; if

,

Django will load all results into memory before passing them back to an application.

฀

฀

—Specifies what value the database library returns to indicate that no more data is available, when fetching multiple rows; defaults to an empty list.

฀

฀

—Indicates whether the database treats an empty

string as the same value as

; defaults to

.

฀

฀

—Indicates whether dates need to be converted from

a string to a

object after being retrieved from the database; defaults to

.

฀

฀

—Indicates whether the database requires relationship

fields to be of the same type as the fields they relate to. This is used specifically for the and

types; if

, the actual type of

the related field will be used to describe the relationship; if

—the default—Django

will use an

instead.

฀

฀

—Indicates whether the database is capable of performing

a

subquery on a table that’s currently being modified with an query;

defaults to

.

฀

฀

—Indicates whether the backend supplies its own

class,

which would be used to customize how queries are performed; defaults to

.

฀

฀

—Indicates whether the database supports savepoints in addition to full transactions. Savepoints allow database queries to be rolled back on a more granular basis, without requiring the entire transaction to be undone if something goes wrong.

This attribute defaults to

; setting it to

will also require implementations for

the

,

and

methods described in the next section.

DatabaseWrapper.ops

This is the gateway to most of the database- specific features, primarily to handle the various differences in how each database handles certain types of SQL clauses. Each database vendor has its own set of special syntaxes that need to be supported, and defining those in the backend allows Django to operate without needing to worry about those details.

Like the situations described previously, backends only need to write those operations that deviate from the standard.

, also living at

,

provides default behaviors for many of these operations, while others must be implemented by the backend itself. The following list explains their purposes and default behaviors.

186

C H A P T E R 8

฀ BACKEND PROTOCOLS

฀

฀

—Returns the SQL necessary to create an automatically-

incrementing primary key. If the database has a field to support this natively, that field will be chosen using the

module described in the “Creation of New Structures”

section, and this method should return

instead of any SQL statements, which is also

the default behavior.

฀

฀

—Returns an SQL statement that pulls

out just a portion of a date, so it can be compared to a filter argument. The will be one of

,

or

, while

is the name of the table column

that contains the date to be checked. This has no default behavior, and must be defined by the backend to avoid a

.

฀

฀

—Returns an SQL statement that drops

off that portion of the date that’s beyond the specificity provided by

. The

possible values are the same as those for

, but this differs in that

if

is

, for instance, this will return a value that specifies both the month and the year, while

will return the month without the year.

Also like

, there is no default behavior, and must be implemented.

฀

฀

—Returns the SQL required to force a

value into what-

ever format the database library uses to return a true

object in Python. The

return value will be used as a Python format string, which will receive just the field name, to be referenced as

in the string. By default, it simply returns

, which will

work just fine for databases that don’t require any special type casting.

฀

฀

—Returns the SQL necessary to append to a constraint definition in order to make that constraint initially deferred, so that it won’t get checked until the end of the transaction. This will be appended immediately after the constraint definition, so if a space is required, the return value must include the space at the beginning.

By default, this returns an empty string.

฀

฀

—Returns the SQL fragment that will drop a foreign key reference as part of an

statement. The name of the reference will be appended

automatically afterward, so this only needs to specify the command itself. For example, the default return value is simply

.

฀

฀

—Returns an SQL statement to drop the auto- incrementing sequence from the specified table. This forms something of a pair with

,

since the sequence only needs to be dropped explicitly if it was created explicitly. By default, this returns

to indicate no action is taken.

฀

฀

—Returns an SQL fragment for casting the specified database column type to some value that can be more accurately compared to filter arguments in a

clause. The return value must be a Python format string, with the only argument being the name of the field to be cast. The default return value is

.

฀

฀

—Returns an SQL fragment for issuing a fulltext

search against the specified field, if supported. The string returned should also include a

placeholder for the user- specified value to be searched against, which will be quoted automatically outside this method. If fulltext search isn’t supported by the database, the default behavior will suffice by raising a

with an

appropriate message to indicate this.

C H A P T E R 8

฀ BACKEND PROTOCOLS

187

฀

฀

—Returns the last query that was issued

to the database, exactly as it was sent. By default, this method has to reconstruct the query by replacing the placeholders in the

argument with the parameters supplied

by

, which will work correctly for all backends without any extra work. Some backends may have a faster or more convenient shortcut to retrieve the last query, so the database cursor is provided as well, as a means to use that shortcut.

฀

฀

—Returns the ID of the row inserted by

the last

into the database. By default, this simply returns

, as

specified by PEP- 249, but other backends may have other ways of retrieving this value.

To help access it accordingly, the method also receives the name of the table where the row was inserted and the name of the primary key column.

฀

฀

—Returns the SQL necessary to cast a value to a format that can be used with the specified

. The return value must also include a

placeholder for the actual value to be cast, and by default it simply returns

.

฀

฀

—Returns the maximum number of characters the database engine allows to be used for table and column names. This returns by default, which indicates there’s no limit.

฀

฀

—Returns the value that should be used to indicate a limit of infin-ity, used when specifying an offset without a limit. Some databases allow an offset to be used without a limit, and in these cases, this method should return

. By default,

this raises a

, and must be implemented by a backend in order to

allow offsets to be used without limits.

฀

฀

—Returns the value to be used when issuing an

statement

to indicate that the primary key field should use its default value—that is, increment a sequence—rather than some specified ID; defaults to

.

฀

฀

—Returns a modified form of , suitable for use with a

comparison in the query’s

clause. By default, this escapes any percent signs

( ), underscores ( ) or double backslashes (

) found in with extra backslashes as

appropriate.

฀

฀

—If the backend provides a custom

class, as

indicated by

, this method must

return a custom

class based on the supplied

. If

is

, this method is never called, so the default behavior is to simply return

.

฀

฀

—Returns a rendition of the given

with quotes appropriate for

the database engine. The name supplied may have already been quoted once, so this method should also take care to check for that and not add extra quotes in that case.

Since there’s no established standard for quoting names in queries, this must be implemented by the backend, and will raise a

otherwise.

฀

฀

—Returns the necessary SQL for generating a random value; defaults to

.

188

C H A P T E R 8

฀ BACKEND PROTOCOLS

฀

฀

—Returns the SQL for performing a regular expression

match against a column. The return value should contain two placeholders, the first

for the name of the column and the other for the value to be matched. The lookup type would be either

or

, the difference being case sensitivity. By default, this raises a

, which would indicate that regular expressions aren’t

supported by the database backend, but for simple cases,

and

can be sup-

ported using the

dictionary described in the next section.

฀

฀

—Returns an SQL statement for creating a new savepoint.

The

argument is the name to give the savepoint, so it can be referenced later.

฀

฀

—Explicitly commits the savepoint referenced by the

argument.

฀

฀

—Rolls back a portion of the transaction according to the savepoint referenced by the

argument.

฀

฀

—Returns the SQL necessary to remove all the

data from the specified structures, while leaving the structures themselves intact. Since this is so different from one database engine to another, the default behavior raises a

, and must be implemented by the backend.

฀

฀

—Returns a list of SQL statements neces-

sary to reset the automatically incrementing sequences for the specified models. Like and

, this is only useful for databases that maintain

independent sequences for automatic IDs, and can return an empty list of those that aren’t needed, which is the default behavior.

฀

฀

—Returns the SQL used to enter a new transaction; defaults to

.

฀

฀

—Returns the SQL to declare a tablespace,

or

if the database doesn’t support them, which is the default.

฀

฀

—Converts a

object to an object suitable for use with the

database for

columns.

฀

฀

—Converts a

object to a value suitable for use

with

columns.

฀

฀

—Converts a

object to a value that can be used with the

database for

columns.

฀

฀

—Converts a

object to a value that the database

can place in a

column.

฀

฀

—Returns a two- item list representing the lower and upper bounds of a given year. The

argument is an

year, and each of the return val-

ues is a string representing a full date and time. The first return value is the lowest date and time that is considered part of the supplied year, while the second is the highest date and time that is considered part of that same year.

฀

฀

—Also returns a two- item list represent-

ing the upper and lower date and time boundaries for the year supplied as

. By

default, this defers to

but can be overridden in case the database

can’t compare a full date/time value against a

.

C H A P T E R 8

฀ BACKEND PROTOCOLS

189

Comparison Operators

Many of the comparisons that can be done in a database follow a simple format, with one value being followed by some kind of operator, then followed by another value to compare it to. Since this is such a common case, and is quite simple to work with, Django uses a much simpler method for defining the operators for these types of comparisons.

Another attribute on the

object,

, contains a dictionary map-

ping various lookup types to the database operators that implement them. This relies very heavily on the basic structure, because while the key for this dictionary is the lookup type, the value is the SQL fragment that should be placed after the name of the field being compared.

For example, consider the common case where the

lookup is handled by the stan-

dard operator, which would be handled by a dictionary like the following: This dictionary would then be filled out with the other operators supported by Django.

Obtaining a Cursor

Combining all of these database- specific features with Django’s object- oriented database API makes available a world of possibilities, but they’re all designed to cover the most common cases. Databases support a wide variety of additional functionality that’s either less commonly used or extremely disparate across different implementations. Rather than try to support all of these features in all databases, Django instead provides easy access straight to the database itself.

The

method of

returns a database cursor straight from the

third- party library used to connect with the database itself. In keeping with standard Python policy, this cursor object is compatible with PEP- 249, so it may even be possible to use other database abstraction libraries with it. Since the behavior of the attributes and methods on this object are outside Django’s control—often varying wildly across implementations—it’s best to consult the full PEP and your database library’s documentation for details on what can be done with it.

Creation of New Structures

One of the more convenient features Django’s database connection provides is the ability to automatically create tables, columns and indexes based solely on model definitions declared in Python. Along with a powerful database querying API, this is a key feature in avoiding the use of SQL code throughout an application, keeping it clean and portable.

While the SQL syntax itself is reasonably well standardized with regards to creation of data structures, the names and options available for individual field types are quite varied across different implementations. This is where Django’s database backends come in, providing a mapping of Django’s basic field types to the appropriate column types for that particular database.

This mapping is stored in the backend package’s

module, which must contain

a single dictionary, named

, at the module level. The keys in this dictionary match

190

C H A P T E R 8

฀ BACKEND PROTOCOLS

up with the available return values from the various

subclasses, while the values are

a string that will be passed to the database as the column’s definition.

The value can also be a Python format string, which will be given a dictionary of field attributes, so that customized field settings can be used to determine how the column is created.

For example, this is how

passes along the

attribute. While many field

types have common attributes, the ones that are of most use to the column type are likely specific to each individual field. Consult the field’s source code to determine what attributes are available for use in this mapping.

There are a number of basic field types available as internal column types:

฀

฀

—An automatically incrementing numeric field, used for primary keys when one isn’t defined explicitly in the model.

฀

฀

—A field representing just two possible values: on and off. If the database doesn’t have a separate column that represents this case, it’s also possible to use a single- character

to store

and

to simulate this behavior.

฀

฀

—A field containing a limited amount of free- form text. Typically, this uses a variable- length string type in the database, using the extra attribute to

define the maximum length of a stored value.

฀

฀

—A field containing a list of integers, typically repre-

senting IDs, which are stored in a single string, separated by commas. Since the list is stored as a string, this also uses a variable- length string type on the database side.

While some databases may have a more intelligent and efficient means of storing this type of data, the field’s code still expects a string of numbers, so the backend should always return one.

฀

฀

—A standard date, without any time information associated with it. Most databases should have a date column type, so this should be easy to support. Just make sure that the column type used returns a Python

upon retrieval.

฀

฀

—A date, but with associated time information attached, excluding time zones. Again, most reasonable databases will support this easily, but make sure that the Python library for it returns a

when retrieving from the database.

฀

฀

—A fixed- precision decimal number. This is another example of using field attributes to define the database column, since the and

field attributes should control the database column equivalents.

฀

฀

—The name and location of a file stored elsewhere. Django doesn’t support storing files as binary data in the database, so its files are referenced by a relative path and name, which is stored in the associated column. Since that’s text, this again uses a standard variable- length text field, which also utilizes the field attribute.

฀

฀

—The name and path of a file in a storage system. This field is similar to in many respects, but this is intended to allow users to choose from existing files, while

exists to allow saving new files. Since the data actually being stored is essentially the same format, it works the same way, using a variable- length string specified using the

attribute.

C H A P T E R 8

฀ BACKEND PROTOCOLS

191

฀

฀

—A field containing a floating point number. It doesn’t matter if the database stores the number with fixed precision internally, as long as the Python library returns a

for values stored in the column.

฀

฀

—A field containing a signed 32- bit integer.

฀

฀

—An Internet Protocol (IP) address, using the current IPv43 standard, represented in Python as a string.

฀

฀

—A Boolean field that also allows

values to be stored in the

database.

฀

฀

—A string with a maximum length of 20 characters, for storing phone numbers.

฀

฀

—A field containing an unsigned 32- bit integer.

฀

฀

—A field containing an unsigned 8- bit integer.

฀

฀

—A field containing a signed 8- bit integer.

฀

฀

—An unlimited- length text field, or at least the largest text field the database makes available. The

attribute has no effect on the length of this field.

฀

฀

—A field representing the time of day, without any associated date information. The database library should return a

object for values in this

column.

฀

฀

—A field for storing the abbreviation for a US state. Since all state abbreviations are just two letters, this should be a string fixed at two characters in length.

While Django typically handles all of the necessary accesses to this module automatically as part of the

4 command, it’s also possible to access this module directly, in a backend- agnostic manner. It’s accessible by calling the method

from

, which returns the entire module. This field mapping is then available as the attribute of that module.

Introspection of Existing Structures

In addition to being able to create new table structures based on model information, it’s also possible to use an existing table structure to generate new models. This isn’t a perfect process, since some model information doesn’t get stored in the table’s own definition, but it’s a great starting point for new projects that have to work with existing databases, usually to run alongside a legacy application that’s being phased out.

The backend should provide a module called

for this purpose, which

provides a number of functions for retrieving various details about the table structures. Each 3.

4.

192

C H A P T E R 8

฀ BACKEND PROTOCOLS

function receives an active database cursor; all arguments and return values of each of these functions are documented in the following list, as well as another mapping for picking the right field types based on the underlying column types.

฀

฀

—Returns a list of table names that are present in the

database.

฀

฀

—Given the name of a specific table,

found using

, this returns a list of tuples, each describing a column in the table. Each tuple follows PEP- 249’s standard for the cursor’s attribute:

. The

here is an internal type used by the database to identify the column type, which will be used by the reverse mapping described at the end of this section.

฀

฀

—Given a table’s name, this returns a dictionary

detailing the relationships the table has with other tables. Each key is the column’s index in the list of all columns, while the associated value is a 2- tuple, with the first item being the index of the related field according to its table’s columns, and the second item is the name of the associated table. If the database doesn’t provide an easy way to access this information, this function can instead raise

,

and relationships will just be excluded from the generated models.

฀

฀

—Given the name of a table, this returns a dictionary

of all the fields that are indexed in any way. The dictionary’s keys are column names, while the values are additional dictionaries. Each value’s dictionary contains two keys: and

, each of which being either

or

. If both are

, the column is still indicated as indexed, by virtue of being in the outer dictionary at all; it’s just an ordinary index, without primary key or unique constraints. Like

, this can also raise

if there’s no easy way to obtain

this information.

In addition to the preceding methods, the introspection module also provides a dictionary called

, which maps the

values in the dictionary returned from

. The keys are whatever values are returned as

, regardless

of whether that’s a string, an integer or something else entirely. The values are strings containing the names of the Django fields that will support the associated column type.

DatabaseClient

Living in the database backend’s

module, this class is responsible for calling the

command- line interface (shell) for the current database specified by

. This is

called using the

command, allowing users to manage the underlying tables’

structure and data manually if necessary.

The class consists of just a single method,

, which takes no arguments. This

method is then responsible for reading the appropriate database settings for the given backend and configuring a call to the database’s shell program.

C H A P T E R 8

฀ BACKEND PROTOCOLS

193

DatabaseError and IntegrityError

Pulled in from

, these classes allow exceptions to be handled easily, while still being able to swap out databases.

should be a subclass of

,

so that applications can just check for

if the exact type of error isn’t important.

Third-party libraries that conform to PEP- 249 will already have these classes available, so they can often just be assigned to the

module’s namespace and work just fine. The only

time they would need to be subclassed or defined directly is if the library being used doesn’t behave in a way that’s similar to other databases supported by Django. Remember, it’s all about consistency across the entire framework.

Authentication

While the combination of a username and password is a very common authentication method, it’s far from the only one available. Other methods, such as OpenID, use completely different techniques, which don’t even include a username or password. Also, some systems that do use usernames and passwords may already be storing that information in a different database or structure than Django looks at by default, so some extra handling still needs to be done to verify credentials against the right data.

To address these situations, Django’s authentication mechanism can be replaced with custom code, supporting whatever system needs to be used. In fact, multiple authentication schemes can be used together, with each falling back to the next if it doesn’t produce a valid user account. This is all controlled by a tuple of import paths assigned to the setting. They will be tried in order from first to last, and only if all backends return will it be considered a failure to authenticate. Each authentication backend is just a standard Python class that provides two specific methods.

get_user(user_id)

Any time a user’s ID is known in advance, whether from a session variable, a database record or somewhere else entirely, the authentication backend is responsible for converting that ID

into a usable

instance. What it means to be an ID could be

different for different backends, so the exact type of this argument may also change depending on the backend being used. For

, the default that

ships with Django, this is the database ID where the user’s information is stored, but for others, it might be a username, a domain name or something else entirely.

authenticate(**credentials)

When the user’s ID isn’t known, it’s necessary to ask for some credentials, with which the appropriate

account can be identified and retrieved. In the default case, these credentials are a username and password, but others may use a URL or a single- use token, for example.

In the real world, the backend won’t accept arguments using the syntax, but rather it would

accept just those arguments that make sense for it. But, since different backends will take different sets of credentials, there’s no single method definition that will suit all cases.

194

C H A P T E R 8

฀ BACKEND PROTOCOLS

PASSING INFORMATION TO CUSTOM BACKENDS

You may have noticed from the previous sections that the data passed in to an authentication backend depends very much on the backend being used. Django, by default, passes in a username and password from its login form, but other forms can supply whatever other credentials as appropriate for the form.

Storing User Information

One aspect of authentication that might not seem obvious is that all users must, for all intents and purposes, still be represented in Django as

objects in the

appli-

cation. This isn’t strictly required by Django as a framework, but most applications—including the provided admin interface—expect users to exist in the database, and will make relationships with that model.

For backends that call out to external services for authentication, this means duplicating every user in Django’s database to make sure applications work correctly. On the surface, this sounds like a maintenance nightmare; not only does every existing user need to be copied, but new users need to be added, and changes to user information should also be reflected in Django. If all this had to be managed by hand for all users, it would certainly be a considerable problem.

Remember, though, that the only real requirement for an authentication backend is that it receives the user’s credentials and returns a

object. In between, it’s all just standard

Python, and the whole of Django’s model API is up for grabs. Once a user has been authenticated behind the scenes, the backend can simply create a new if one doesn’t already

exist. If one does exist, it can even update the existing record with any new information that’s updated in the “real” user database. This way, everything can stay in sync without having to do anything special for Django. Just administer your users using whatever system you’re already using, and let your authentication backend handle the rest.

Files

Web applications typically spend most of their time dealing with information in databases, but there are a number of reasons an application may need to work directly with files as well.

Whether it be users uploading avatars or presentations, generating images or other static content on the fly, or even backing up log files on a regular basis, files can become a very important part of an application. As with many other things, Django provides both a single interface for working with files and an API for additional backends to provide additional functionality.

The Base File Class

Regardless of source, destination or purpose, all files in Django are represented as instances of

. This works very much like Python’s own file object, but with a few additions and modifications for use on the Web and with large files. Subclasses of can

alter what goes on behind the scenes, but the following API is standard for all file types. The following attributes are available on all

objects:

C H A P T E R 8

฀ BACKEND PROTOCOLS

195

฀

฀

—A Boolean indicating whether the file has been closed. When instantiated, all

objects are open, and its contents can be accessed immediately. The method sets this to

, and the file must be reopened using

before its

contents can be accessed again.

฀

฀

—Typically an attribute of the file’s class rather than an instance of it, this determines what size chunks should be used with the method.

฀

฀

—The access mode the file was opened with; defaults to

.

฀

฀

—The name of the file, including any given path relative to where it was opened.

฀

฀

—The size of the file’s contents, in bytes.

The following methods are also available on

objects:

฀

฀

—Iterates over the file’s contents, yielding it in one or more smaller chunks, to avoid filling up the server’s available memory with large files.

If no

is provided, the

, which defaults to 64KB, will be

used.

฀

฀

—Closes the file, so its contents become inaccessible.

฀

฀

—Writes any new pending contents to the actual filesystem.

฀

฀

—Returns

if the file is big enough to

require multiple calls to

to retrieve the full contents, or

if it can all be read

in one pass. The

argument works the same as in

. Note that this will

not actually read the file at this point; it determines the value based on the file’s

.

฀

฀

—Reopens the file if it had been previously closed. The

argument is optional and will default to whatever mode the file had used when it was last open.

฀

฀

—Retrieves a certain number of bytes from the file. If called without a

argument, this will read the remainder of the file.

฀

฀

—Retrieves the content of the file as a list of lines, as indicated by the presence of newline characters (

and

) in the file. These newline characters are left

at the end of each line in this list.

฀

฀

—Moves the internal position of the file to the specified location.

All read and write operations are relative to this position, so this allows different parts of the file to be accessed by the same code.

฀

฀

—Returns the position of the internal pointer, as the number of bytes from the beginning of the file.

฀

฀

—Writes the specified contents to the file. This is only available if the file was opened in write mode (a mode beginning with

).

฀

฀

—A generator version of

, yielding one line, includ-

ing newline characters, at a time. In keeping with Python’s own transition away from

, this functionality is also provided by iterating over the object itself.

196

C H A P T E R 8

฀ BACKEND PROTOCOLS

Handling Uploads

When accepting files from users, things get a little bit trickier, because these files shouldn’t necessarily be saved alongside the rest of your files until your code has had a chance to review them. To facilitate this, Django treats uploaded files a bit differently, using upload handlers to decide what subclass of

should be used to represent them. Each upload handler has a chance to step in during the upload and alter how Django proceeds.

Upload handlers are specified with the

setting, which takes a sequence

of import paths. As uploaded files are being processed, Django calls various methods on each of these handlers in turn, so they can inspect the data as it comes in. There’s no need to call these directly, as it’s automatically handled by Django’s request processing code, but the API for new upload handlers provides ample opportunity to customize how incoming files are managed.

฀

฀

—The handler is initialized every time a request

comes in with files attached, and the incoming request is passed in so the handler can decide if it needs to handle the files for the request. For example, if it’s designed to write details of the upload to the console of the development server, it might check if the

setting is

and if

is in the

set-

ting. If a handler should always process every request, this doesn’t need to be defined manually; the inherited default will suffice for most cases.

฀

฀

—This is called for each file submitted in the request, with various details about the file, but none of its actual content. The

is the form field name that

was used to upload the file, while the

is the name of the file itself as reported

by the browser. The

,

and

are all properties of the

file’s contents, but they should be taken with a grain of salt, since they can’t be verified without accessing the file’s contents.

While not strictly required, the primary function of this method is to set aside a place for the file’s content to be stored when

is called. There’s no

requirement on what type of storage is used, or what attribute is used for it, so nearly anything’s fair game. Common examples are temporary files or objects. Also,

this method provides a way to decide whether certain features should be enabled, such as automatically generated thumbnails of images, determined by the

.

฀

฀

—This is one of only two

required methods, and is called repeatedly throughout the processing of the file, each time receiving a portion of the file’s contents as

, with

being the offset

within the file where that content was found. The amount of data called each time is based on the handler’s

attribute, which defaults to 64KB.

Once this method has completed processing the data chunk, it can also control how other handlers deal with that data. This is determined by whether the method returns any data or not, with any data returned being passed along to the next handler in line.

If it returns

, Django will simply repeat the process with the next chunk of data.

฀

฀

—As a complement to

, this

method is called when Django finds the end of the file in the request. Since this is also the only time the file’s total size can be known with certainty, Django gives each handler a chance to determine what to do with that information.

C H A P T E R 8

฀ BACKEND PROTOCOLS

197

This is the only other required method on an upload handler, and should return an

object if the file was processed by this handler. The

returned will be used by the associated form as the content for the field used to upload the file. If the handler didn’t do anything with the file, for whatever reason, this can return

, but be careful with this, because at least one upload handler must return an

to be used with forms.

฀

฀

—While

is called when each

file is finished loading,

is called once per request, after all uploaded

files have been processed completely. If the handler needs to set up any temporary resources while dealing with all the files, this method is the place to clean up after itself, freeing up resources for the rest of the application.

Notice that many of the features made possible by these methods rely on one method knowing what decisions a previous method has already made, but there’s no obvious way to persist this information. Since handlers are instantiated on every incoming request and process files one at a time, it’s possible to simply set custom attributes on the handler object itself, which future method calls can read back to determine how to proceed.

For example, if

sets

to

,

can read

that attribute to determine whether it should process the chunks it receives or just pass them through to the next handler in line. It’s also possible for to set the same or similar

attribute, so those types of decisions can be made on a per- file basis as well as per- request.

Since each handler works in isolation from the others, there isn’t any standard imposed on which attributes are used or what they’re used for. Instead, interaction among the various installed upload handlers is handled by raising a number of exceptions in various situations.

Proper operation of an upload handler doesn’t require the use of any of these, but they can greatly customize how a number of them can work together. Like

, these are

all available at

.

฀

฀

—Tells Django to stop processing all files in the upload, preventing all handlers from handling any more data than they’ve already processed. It also accepts a single optional argument,

, a Boolean indicating whether Django

should stop without reading in the remainder of the input stream. The default value of

for this argument means that Django will read the entire request before passing control back to a form, while

will stop without reading it all in, resulting in

a “Connection Reset” message shown in the user’s browser.

฀

฀

—Tells the upload process to stop processing the current file, but continue on with the next one in the list. This is a much more appropriate behavior if there was a problem with a single file in the request, which wouldn’t affect any other files that might be uploaded at the same time.

฀

฀

—Only valid if thrown from the

method, this indicates

the current upload handler will handle the current file directly, and no other handlers should receive any data after it. Any handlers that process data before the handler that raises this exception will continue to execute in their original order, as determined by their placement within the

setting.

198

C H A P T E R 8

฀ BACKEND PROTOCOLS

Storing Files

All file storage operations are handled by instances of

, which lives at

, with the default storage system specified by an import path in the setting. A storage system encompasses all the necessary functions for dealing with how and where files are stored and retrieved. By using this extra layer, it’s possible to swap out which storage system is used, without having to make any changes to existing code.

This is especially important when moving from development to production, since production servers often have specialized needs for storing and serving static files.

In order to facilitate this level of flexibility, Django provides an API for dealing with files that goes beyond the standard

function and associated

object provided by Python.

Earlier in this chapter, Django’s

object was described, explaining what features are avail-

able for dealing with individual files. When looking to store, retrieve or list files, however, storage systems have a different set of tools available.

฀

฀

—Deletes a file from the storage system.

฀

฀

—Returns a Boolean indicating whether the specified name references a file that already exists in the storage system.

฀

฀

—Returns a version of the given name that’s suitable for

use with the current storage system. If it’s already valid, it will be returned unchanged.

One of only two methods with default implementations, this will return filenames suitable for a local filesystem, regardless of operating system.

฀

฀

—Given a valid name, this returns a version of it

that’s actually available for new files to be written, without overwriting any existing files. Being the other method with a default behavior, this will add underscores to the end of the requested name until an available name is found.

฀

฀

—Returns an open

object, through

which the file’s contents can be accessed. The

accepts all the same arguments as

Python’s

function, allowing for both read and write access. The optional argument accepts a class to be used alongside the

subclass provided by the stor-

age system, to enable additional features on the file returned.

฀

฀

—Returns the absolute path to the file on the local filesystem, which can be used with Python’s built- in

function to access the file directly.

This is provided as a convenience for the common case where files are stored on the local filesystem. For other storage systems, this will raise a if

there is no valid filesystem path at which the file can be accessed. Unless you’re using a library that only accepts file paths instead of open file objects, you should always open files using

, which works across all storage systems.

฀

฀

—Saves the given content to the storage system, prefer-

ably under the given name. This name will be passed through and

before being saved, and the return value of this method will be the name that was actually used to store the content. The argument provided

to this method should be a

object, typically as a result of a file upload.

฀

฀

—Returns the size, in bytes, of the file referenced by

.

C H A P T E R 8

฀ BACKEND PROTOCOLS

199

฀

฀

—Returns an absolute URL where the file’s contents can be accessed directly by a Web browser.

฀

฀

—Returns the contents of the directory specified by the

argument.

The return value is a tuple containing two lists: the first for directories located at the path and the second for files located at that same path.

By default, Django ships with

, which, as the name implies, stores files

on the local filesystem. Typically this means the server’s hard drive, but there are many ways to map other types of filesystems to local paths, so there are already a number of possibilities. There are even more storage options available, though, and there are plenty of ways to customize how even the existing options behave. By subclassing

, it’s possible to

make available a number of other options.

There are a number of things a storage system must provide, starting with most of these methods. One of those methods,

, doesn’t strictly need to be supplied by

the new storage class, since its default implementation is suitable for many situations; overriding it is a matter of preference, not requirement. On the other hand, the method has a default behavior that’s suitable for most backends, but some may have different file naming requirements, and would require a new method to override it.

Two other methods,

and

, have still further requirements. By definition,

both of these require special handling for each different storage system, but they shouldn’t be overridden directly in most situations. They provide additional logic beyond what’s necessary to store and retrieve files, and that logic should be maintained. Instead, they defer the interaction with the actual storage mechanism to

and

, respectively, which have

a simpler set of expectations.

฀

฀

—The

and

arguments are the same as

, but it no longer has the

logic to deal with, so

can focus solely on

returning a

object suitable for accessing the requested file.

฀

฀

—The arguments here are the same as

, but

the name provided here will have already gone through

and

, and the content is guaranteed to be a

instance. This allows

the

method to focus solely on committing the file’s content to the storage system with the given name.

In addition to providing these methods, most custom storage systems will also need to provide a

subclass with

and

methods that are designed to access the

underlying data in the most efficient manner. The

method defers to

internally,

so there shouldn’t need to be anything done there to make large files more memory- friendly for applications to work with. Keep in mind that not all filesystems allow reading or writing just part of a file, so the

subclass may also need to take additional steps to minimize both memory usage and network traffic in these situations.

Session Management

When users are casually browsing a Web site, it’s often useful to track some information for them temporarily, even if there are no

accounts associated with them yet. This can range

from the time they first visit the site to a shopping cart. The typical solution in these cases is 200

C H A P T E R 8

฀ BACKEND PROTOCOLS

a session—a server- side data store referenced by a key stored in a browser- side cookie. Django comes with built- in support for sessions, with a bit of room for configuration.

Most of the session process is constant: identifying a user without a session, assigning a new key, storing that key in a cookie, retrieving that key later on and acting like a dictionary the whole time. There are some basic settings for the name of the key and how long to use it, but in order to actually persist any information across multiple page views, the key is used to reference some data stored somewhere on the server, and that’s where the bulk of the customization comes in.

Django uses the

setting to identify which data store class should handle

the actual data itself. Three data stores ship with Django itself, covering common tactics like files, database records and in- memory cache, but there are other options available in different environments, and even the stock classes might require additional customization. To accommodate this,

accepts full import paths, allowing a session data store to be placed in any Django application. This import path points to a module containing a class named

, which provides the full data store implementation.

Like most of Django’s swappable backends, there’s a base implementation that provides most of the features, leaving fewer details for the subclass to cover. For sessions, that base class is

, located at

. That’s what handles

the session key generation, cookie management, dictionary access and access to the data store only when necessary. This leaves the custom

class to implement just five meth-

ods, which combine to complete the entire process.

฀

฀

—Returns

if the provided session key is

already present in the data store, or

if it’s available for use in a new session.

฀

฀

—Loads session data from whatever storage mechanism the data store uses, returning a dictionary representing this data. If no session data exists, this should return an empty dictionary, and some backends may require the new dictionary to be saved as well, prior to returning.

฀

฀

—Commits the current session data to the data store, using the current session key as an identifier. This should also use the session’s expiration date or age to identify when the session would become invalid.

฀

฀

—Removes the session data associated with the

given key from the data store.

฀

฀

—Creates a new session and returns it so external code can add new values to it. This method is responsible for creating a new data container, generating a unique session key, storing that key in the session object and committing that empty container to the backend before returning.

Also, to help session data stores access the necessary information to do their work, Django also provides a few additional attributes that are managed by

.

฀

฀

—The randomly-generated session key stored in the client- side cookie.

฀

฀

—A dictionary containing the session data associated with the current session key.

C H A P T E R 8

฀ BACKEND PROTOCOLS

201

฀

฀

—Returns a

object representing when the ses-

sion should expire.

฀

฀

—Returns the number of seconds after which the session should expire.

By implementing just five methods on a subclass of

, it’s possible to store ses-

sion data nearly anywhere. Even though this data isn’t tied to a object, it’s still specific to

individual people browsing the site. In order to store temporary information that’s useful for everyone, a little something else is in order.

Caching

When an application has a lot of seldom- changing information to deal with, it’s often useful to cache this information on the server so it doesn’t have to be generated each and every time it’s accessed. This can save on memory usage on the server, processing time per request, and ultimately helps the application serve more requests in the same amount of time.

There are a number of ways to access Django’s caching mechanism, depending on just how much information needs to be cached. The online documentation5 covers the many general cases on how to set up site- wide caching and per- view caching, but the lower- level details merit a bit more explanation.

Specifying a Backend

Specifying a cache backend in Django works quite a bit differently than other backends discussed in this chapter. Even though there are multiple configuration options to consider, there’s just one setting to control them all. This setting,

, uses the URI syntax6 to

accept all of the necessary information in a way that can be parsed reliably. It can be split up into three separate parts, each with its own requirements.

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀ ฀

฀

฀

฀

cache. Django ships with four backends that cover most cases—

,

,

and

7—which are well documented online, and cover the majority of cases.

For custom backends, this portion of the setting can also accept a full import path to a module that implements the protocol described in the next section.

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀ ฀

฀

฀

฀

depending on the backend used. For example,

expects a single database name,

expects a full directory path, while

expects a list of server addresses and

doesn’t require anything at all. The host can also include by a trailing slash, which can help readability, since it makes the whole setting look more like a URI.

5.

6.

7.

202

C H A P T E R 8

฀ BACKEND PROTOCOLS

฀

฀

฀

฀

฀

฀

฀be provided to customize how caching takes place

within the backend. They’re provided using the query- string format, with one argument required for all backends:

, the number of seconds before an item should

be removed from the cache. Two more arguments are also available for most backends (including all those supplied by Django except for

):

, the total

number of items that should be stored in the cache before culling old items; and

, which controls how many items to purge from the cache when it reaches

.

One important thing to realize about

is that its value isn’t actually how

often items should be removed. Instead, the value is used in a simple formula,

, which determines how many items are affected. So, if you’d like to purge 25% of the items at a time, that’s equivalent to ¼, so you’d pass as an argument

to the cache backend, while half (½) of the entries would require passing

.

Essentially,

is the number of times the cache must be culled to guarantee that all items are purged.

Using the Cache Manually

In addition to the standard site- wide and per- view caching options, it’s also quite simple to use the cache directly, storing specific values so they can be retrieved later without having to perform expensive operations for data that doesn’t change often. This low- level API is available in a generic form through the

object, living at

. Most of the usefulness of

this object comes from three methods—

,

and

—which work mostly how

you’d expect.

There are a few details about these methods that bear a little more explanation, and also some additional methods that prove useful. Here is a full list of the available methods, along with their functional details.

฀

฀

—This sets the specified

in the cache,

using the provided

. By default, the timeout for values to expire from the cache is determined by the timeout passed into the

setting, but that can be over-

ridden by specifying a different timeout as an argument to this method.

฀

฀

—This method returns the value contained in

the cache for the specified

. Normally,

returns

if the key doesn’t

exist in the cache, but sometimes

is a valid value to have in the cache. In these

cases, just set

to some value that shouldn’t exist in the cache, and that will be returned instead of

.

฀

฀

—This deletes the value associated with the given key.

C H A P T E R 8

฀ BACKEND PROTOCOLS

203

฀

฀

—Given a list of keys, it returns a corresponding list of their values. For some backends, like

, this can provide a speed increase over call-

ing

for each individual key.

฀

฀

—This method returns

if the specified key has a value

already in the cache or

if the key wasn’t set or has already expired.

฀

฀

—This method only attempts to add a new

key to the cache, using the specified value and timeout. If the given key already exists in the cache, this method will not update the cache to the new value.

A common idiom when working with cache is to first check to see if a value is already present in the cache, and if not, calculate it and store it in the cache. Then, the value can be retrieved from the cache regardless of whether it was there to begin with, making the code nice and simple.

To make this a bit more Pythonic, the

object also functions a bit like a dictionary, support-

ing the operator as an alias for the

method.

Template Loading

While Chapter 6 showed that when a view or other code requests a template to render, it just passes in a name and a relative path, the actual retrieval of templates is done by special loaders, each of which accesses templates in a different way. By supplying the import paths to one or more of these to the

setting, Django doesn’t need to know in advance

how or where you’ll store your templates.

Django ships with three template loaders, representing the most common ways templates are expected to be used, loading files from the filesystem in certain configurations. When these options aren’t enough, it’s fairly straightforward to add your own template loader to locate and retrieve templates in whatever way is best for your environment.

This is actually one of the easiest pluggable interfaces to write, since it’s really just a single function. There isn’t even any assumption of what that function should called, much less what module it should be in, or any class it needs to be a part of. The entry in points directly at the function itself, so no other structure is necessary.

load_template_source(template_name, template_dirs=None) While the loader can be called anything, the name Django uses for all of its template loaders is

, so it’s generally best to stick to that convention for ease of understanding. This is also typically placed in its own module, but again, the import path has to be supplied explicitly, so just make sure its location is well- documented.

The first argument is obviously the name of the template to be loaded, which is usually just a standard filename. This doesn’t have to map to an actual file, but views will typically request templates using a filename, so it’s up to the template loader to convert this name to 204

C H A P T E R 8

฀ BACKEND PROTOCOLS

whatever reference is used for templates. That may be database records, URLs pointing to external storage systems or anything else your site may use to store and load templates.

The second argument to

is a list of directories to use when

searching for the template. Within Django itself, this is typically not provided, so the default of

is used, indicating that the

setting should be used instead. A loader that

uses the filesystem should always follow this behavior, to maintain consistency with the way other template loaders work. If the loader retrieves templates from somewhere else, this argument can simply be ignored.

What goes on inside the template loader will be quite different from one template loader to the next, varying based on how each loader locates templates. Once a template is found, the loader must return a tuple containing two values: the template’s contents as a string, and a string indicating where the template was found. That second value is used to generate the argument to the new

object, so that it’s easy to find a template if anything

goes wrong.

If the given name doesn’t match any templates the loader knows about, it should raise the exception, described in Chapter 6. This will instruct Django to move on to the next template loader in the list, or to display an error if there are no more loaders to use.

load_template_source.is_usable

If the Python environment doesn’t have the requirements for a template loader to operate, Django also provides a way for the loader to indicate that it shouldn’t be used. This is useful if a template loader relies on a third- party library that hasn’t been installed. Adding an attribute to the function, set to

or

, will tell Django whether the template loader can

be used.

Context Processors

When a template gets rendered, it’s passed a context of variables, which it uses to display information and make basic presentation decisions. If a special type of context is used,

,

available from

right alongside the standard

, Django runs through a list

of context processors, each of which gets the opportunity to add new variables to the context of the template. This is not only a great way to add common variables to every template used on the site, but it’s a really easy way to supply information based on information from the incoming object.

The interface for a context processor is quite simple; it’s nothing more than a standard Python function that takes a request as its only argument, and returns a dictionary of data to be added to the template’s context. It should never raise an exception, and if no new variables need to be added, based on the specified request, it should just return an empty dictionary.

Here’s an example context processor to add an

variable that contains the request-

ing user’s IP address.

Installing a context processor is as easy as adding a string to the setting list, with each entry being a full Python import path, including the name of the function on the end of it. Also, remember that context processors are only called when templates are C H A P T E R 8

฀ BACKEND PROTOCOLS

205

rendered using

. Since context processors accept the incoming request as an argument, there’s no way to call them without this information.

Applied Techniques

The available uses of the tools described in this chapter are many and varied, but there are a few simple examples of how they can be put to good use for some common needs. Take these with a pinch of salt and a sprig of parsley, and make them your own. Without prior knowledge of an application’s working environment, any examples that can be given will, by definition, be fairly abstract, but they should serve as a good outline of how these techniques can be put to good use.

Loading Templates Using a Different Engine

Chapter 6 explained how a template tag can provide a wrapper around content from a different template engine, using Jinja8 as an example. That technique works well for including small snippets of template code inside a Django template, but flexible template loaders raise the question of whether a Jinja template can be loaded in its entirety.

Normally, using something like Jinja would require importing the Jinja package directly in a view and using its API rather than Django’s, but that isn’t an option for third- party applications, where the code is out of your control. Having a loader for Jinja templates allows any application—whether written for Jinja or not—to make use of Jinja templates.

On the surface, this may seem easy. The example in Chapter 6 includes a template tag that executes Jinja code inside of a Django template, and template loaders are designed to return Django template code. The problem with that tag is that Jinja, like Django, supports template inheritance, so one template can include content from another. If all the loader provides is the content of one template with a simple tag around it, there’s no way for Jinja to locate other templates it might reference.

Just like Django needs a set of template loaders to be configured in your site’s settings, Jinja requires an environment be set up, which includes a loader that can locate related templates.

More specifically, Jinja requires that the template be loaded directly from its environment in order to access the associated template loader. And that’s where things get trickier. We have Django’s template loading mechanism that returns just a string, and we have Jinja’s template system that needs a full Python object. The solution involves two new template tags and a new template loader that bridges the gap.

The Template Loader

Since template nodes are Python objects, it’s easy for a node to configure a Jinja environment and pull up the appropriate template. The trouble is getting the necessary information from a template loader to a compiled node. The answer seems as simple as you might expect: put the information in the string sent back by the template loader.

It’s a bit more complicated than that, because we need some new template tags to interpret that information, but the basic idea is indeed to just pass information from the loader to the template tags by way of the template string. Think of it as a kind of indirect function call.

8.

206

C H A P T E R 8

฀ BACKEND PROTOCOLS

There’s a template tag that represents the function, with a set of arguments to pass to it: a template name and a list of directories. Here’s what the loader’s template string might look like on a system configured to load templates from two different directories: Getting this string out of the template loader is fairly easy, but there’s an important aspect of template loaders that must be addressed along the way. In the event that a loader can’t locate a particular template, it must raise

to tell Django to continue on

to the next template loader. To do so, the template loader first needs to try to locate the template itself before generating a template.

Notice also that in the event that

—the most common scenario—

this template loader looks at a new

setting, rather than the standard

settings. Using a new setting allows both template engines to be used side-by- side, which would otherwise be very difficult, because their syntaxes aren’t completely compatible.

There are enough similarities that certain types of base templates can be shared across the two template engines, but only limited templates can support both.

Jinja supports multiple template loaders as well, and this example uses its most common,

. It works much the same way as Django’s filesystem template loader, accepting a list of directories to search for templates. Its

method attempts to retrieve just

the template string, without doing any parsing. This is important because there’s no way to render the template at this point or save it for later; parsing would be wasted effort that could be better used on other aspects of the site. Since that template won’t be used yet, the return value from

is simply discarded by not assigning it to anything.

Now that the rest of the template loader function can be certain that the template does exist, the only thing left to do is generate the template string that will pass the necessary information to the appropriate template tags.

C H A P T E R 8

฀ BACKEND PROTOCOLS

207

This addition loads up the template name and all template directories into template tag definitions to be returned from the function. With that in place, the template loader function is complete, but it still relies on additional template tags to function properly. The only thing remaining before we can move on is to set the

attribute on the function so Django

knows whether it should even use this or not. Since this relies on Jinja being installed to work properly, checking for the existence of the

package is an adequate test.

208

C H A P T E R 8

฀ BACKEND PROTOCOLS

Now, if this template loader is added to the

settings without Jinja actu-

ally being installed, Django will issue a warning and ignore the loader entirely.

WHY INCLUDE DIRECTORIES?

Although Jinja’s template inheritance requires a loader to be populated with a list of directories where it can find additional templates, that doesn’t explain why those directories have to be passed in from the Django template loader. The code in this section relies on a setting,

, which could just as

easily be accessed from within the template tags rather than passing them in the template string like this.

The reason for this particular approach lies in the optional argument for

.

In addition to the template name, it’s possible for the template loader to receive a list of template directories to search for templates. Functions like

can

accept a list of template directories other than the defaults and pass those along to the template loader.

Those custom directories aren’t made available directly to the template tags, so the only way to make sure that everything works properly is to pass these directories along with the template name. Then, the associated template tag can simply pick out the directories that were provided and configure Jinja appropriately. This has the added benefit of making the template loader solely responsible for which directories are used to locate templates, rather than offloading that responsibility to a template tag.

The loadjinja Template Tag

The workhorse of this approach is the

template tag, which must collect all the

information that is passed from the template loader and compile a Jinja template from it. This tag—and its partner,

—should be added to the same template tag module that was created in Chapter 6. These new tags rely on the

class defined there, reducing the

new code that must be written.

First, the compilation function needs to pull the template name out of the template to use. It will always be the only argument to the template tag, so anything else will be considered an error. Since

includes the quotation marks around quoted strings,

the tag also needs to explicitly remove those before using the template name.

C H A P T E R 8

฀ BACKEND PROTOCOLS

209

With a template name in place, the only information left to obtain from the template tags is the list of directory names that should be used to initialize Jinja. Since those directories are supplied as template tags of their own, the parser can be used to retrieve them all in one shot. By checking specifically for instances of

, other template tags can be

included inside the

tag without causing problems; any such nodes will simply be ignored.

Note also that, since

doesn’t remove the token for the template tag that

was supplied—in this case,

—the compilation function needs to explicitly remove

that token. If it’s left in the parser, Django will try to locate a tag compilation function for it, which won’t exist.

The last step is to set up Jinja with all of this information and generate a template. The API used here is specific to Jinja, which is well- documented on its own site.9 This code just initializes a Jinja template loader and environment, then retrieves a fully- parsed template.

9.

210

C H A P T E R 8

฀ BACKEND PROTOCOLS

Note that the template retrieved from

goes straight into the

same

that was written in Chapter 6. This way, the same context handling logic in the method can be reused for these templates.

The jinjadir Template Tag

The Django template produced by this new template loader includes a number of tags inside the main

tag. Since the

tag is basically just a way to pass a value

around, its implementation is quite simple. It simply has to take the directory provided and store it in the

attribute of a new node.

That’s all it takes to preserve the directory information long enough to get it into the tag. Once the

node is parsed and the directory is retrieved, it’s not used for anything else, so it doesn’t need any special functionality.

Scanning Incoming Files for Viruses

For sites that allow users to upload files to be distributed to other users, a large amount of trust is placed on the quality of those incoming files. As with any form of user input, there must be a certain level of distrust in this information, since there’s always someone out there who wants to do harm to your site and its users.

When looking to let users share specific types of files, it’s often easy to validate using third- party libraries designed to understand those files. Sharing arbitrary files, on the other hand, opens up a world of other possibilities, many of which put your site and its users at risk.

Protecting against viruses is an important part of the safety of such an application, and Django’s upload handlers make this an extremely simple task.

C H A P T E R 8

฀ BACKEND PROTOCOLS

211

For this example, we’ll use an excellent open source virus scanning application, ClamAV,10

which is designed for use in servers, along with pyclamd,11 a Python library for interacting with ClamAV. Together, these provide an easy-to- use interface for scanning any incoming file before it’s even passed to the rest of the application. If a virus is found, the offending file can simply be removed from the input stream immediately, before it can do any harm to anyone.

Your application may have more specific requirements, like explaining to users which virus was found and that they should consider cleaning their own system before attempting to share files with others. The key to this example is how easy it is to implement this type of behavior, which might seem very difficult on the surface.

10.

11.

212

C H A P T E R 8

฀ BACKEND PROTOCOLS

Now What?

As much as there is to learn about accessing the protocols for these various types of backends, putting them to good use requires a good deal of imagination. There’s only so much a book like this can say about how and why to access or replace these lower- level interfaces, so it’s up to you to determine what’s best for your environment and your applications.

While this chapter discussed how to use and overhaul major portions of Django’s infra-structure, sometimes all that’s needed is a simple utility to replace or avoid a lot of redundant code. It’s important to know the difference, and the next chapter will outline the many basic utilities provided in Django’s core distribution.

C H A P T E R 9

Common Tools

While Django aims to provide a foundation for you to build your own Web application, the framework has its own underpinnings that tie it all together. These common tools and features help everything remain consistent and easier to maintain, and those same benefits can be used by your own applications. After all, what’s available in Django is available for anything that uses it.

Core Exceptions

While Python comes with its own set of exceptions that can be raised in various situations, Django introduces enough complexity on top of that to merit some more. Since Django serves a specialty audience, these exceptions are considerably more specialized, but they’re still usable by more than just core code. Some of these exceptions have been mentioned previously, because they deal more specifically with a particular Django feature, but they’re also useful in other situations, as the following sections will explain.

django.core.exceptions.ImproperlyConfigured

This is one of the first exceptions most new users run into, because it’s the one raised when an application’s models aren’t set up correctly, a view can’t be found or a number of other common configuration mistakes occur. It’s typically raised during execution of and helps users identify and correct whatever mistakes were discovered.

Not all applications require any particular configuration, but those that do can make good use of this exception, since most users have seen it before. Common situations where this can be useful include missing or incorrect settings, a URL configuration used without an accom-panying

entry, invalid arguments given to custom model fields and missing a required third- party library.

The most important thing to remember is to indicate not only that something went wrong, but also how the user should go about fixing it. Typically, exceptions indicate that some bit of code ran awry, and there’s little to no way of informing a user how to fix it. With an application’s configuration, however, there are a finite number of acceptable ways to set it up, and this error should be used as a way to steer users in the right direction.

For example, if an application is designed to work with audio files, it might require the presence of Mutagen,1 a well- established Python library for extracting information from such 1.

213

214

C H A P T E R 9

฀ COMMON TOOLS

files. A simple import of this library at the top of the

, where it’s likely to be used,

could identify if the library is installed correctly, and instruct the user how to proceed if not.

django.core.exceptions.MiddlewareNotUsed

Chapter 7 described how middleware can be used to adjust how HTTP is handled, but an interesting side effect is that not all middleware is always useful. While each project has the option of setting just those middleware that are necessary by way of the setting, there are still differences between development and production or among the various developers’ computers.

Each middleware has the ability to decide whether its environment is suitable to be used and indicate if there’s a problem. Middleware classes are instantiated automatically when first needed, at the beginning of the first request, which is where this check would take place. By overriding the class’s

method, middleware can check right away whether everything’s set up to work properly and react accordingly.

Specifically, this reaction is to either return without doing anything if everything looks fine or raise

. If raised, Django will always catch this exception and take it to mean that class should be removed from the list of middleware that get applied on every request.

This is an important distinction to make because without being able to tell Django to not use the middleware at all, it would be up to each individual method to decide whether it should execute. While that would work, it would take up valuable time and memory on every request, checking for something that could be determined just once. By taking the middleware out of the list entirely, it never consumes any additional cycles or memory at all.

django.core.exceptions.MultipleObjectsReturned

When retrieving objects from the database, it’s often expected that exactly one row will be returned. This is always the case whenever the query is a primary key but slugs—and perhaps even dates—can be made unique in certain applications. Django supports this situation with a QuerySet’s

method, and if it matches more than one result, it can throw off the whole execution of the application.

Note Django’s

is almost always set as

because it’s used to identify objects in

a URL.

Since

is expected to return exactly one record from the database, a query matching multiple records is marked by an exception,

. It’s not raised for other

types of queries, since multiple records are to be expected in most situations. Catching this C H A P T E R 9

฀ COMMON TOOLS

215

exception can be useful in a number of ways, from displaying more useful error messages to removing unexpected duplicates.

django.core.exceptions.ObjectDoesNotExist

The other side of the

expectation is that one row will always be returned; that is, there must always be a row in order to succeed. If a query that expects a row to exist finds instead that no such rows are present, Django responds accordingly with

. It works in

much the same way as

, differing only in the situation where it’s raised.

Simply called

, this subclass avoids an extra import because the class it’s used on is typically already imported when the

method is called. In addition, by being called

and being an attribute of a model class, it looks like perfectly readable English:

.

django.core.exceptions.PermissionDenied

Most applications will have some form of permissions in place to prevent access to restricted resources; this follows the pattern of a rule with exceptions. The rule is that the user attempting to access the resource will indeed have the correct permissions, so any user that doesn’t will result in an exception—this time

. This serves as a convenient way to

indicate the problem and stop processing the rest of the view, since the view itself could make changes that aren’t valid if the user doesn’t have the correct permissions.

Django also catches this exception automatically inside its request handler, using it as an instruction to return an HTTP

response instead of the usual

. This will

indicate to the client that the credentials provided didn’t have sufficient permission to request the resource and that the user shouldn’t try again without rectifying the situation. This behavior is provided by default in Django’s own admin application but can also be used in any other.

Like other exceptions,

can be either raised or caught, though the default

behavior of returning a special HTTP response code is appropriate most of the time. If some other behavior is desired, it’s easy enough to create a middleware that catches this exception in the

phase, possibly redirecting users to a form where they can contact the site administrators to request permission to access the page.

216

C H A P T E R 9

฀ COMMON TOOLS

Adding a reference to this in

or creating a decorator out of it using

as described in Chapter 7 is all that’s necessary to redirect users to another page when their permissions weren’t valid for the original request. Even without a custom handler for this exception, though, it’s quite useful to raise it in any of your own views where a user doesn’t satisfy the appropriate permissions. That response will then result in whatever handling is used for all other similar situations, helping make your site as cohesive and consistent as possible.

django.core.exceptions.SuspiciousOperation

While users typically obey the rules and use your site the way it’s expected to be used, any reasonable developer prepares for those who don’t. Django takes a number of precautions to protect against unauthorized access to things like the administration interface and provides decorators to restrict access to application views, but there are still more subtle things to take into account.

For instance, the sessions framework needs to worry about users altering the session ID

in an attempt to hijack another user’s session. These types of things don’t fall under authentication or permissions themselves, but rather a user is attempting to circumvent these usual protections. It’s important to identify when this occurs, so it can be dealt with appropriately.

To identify these across the board, Django provides a

exception

that can be used any time something like this happens. In many situations, this is thrown and caught in the same application but is provided so that it’s possible to reach into the application and use just the portion that raises the exception. In other cases, it’s left exposed to other applications to handle in whatever way makes the most sense.

The signed cookies application from Chapter 7 is a good example of where suspicious activity can be easily identified and handled. If a cookie comes in without a valid signature, it’s clear that something fishy is going on and the signature validation code raises a to signify it. Since it’s designed to work as a hands- free middleware, it also provides code to catch this exception and perform a more useful function by removing the offending cookie from the request before it reaches the view. But since it’s possible for other applications to sign and validate values outside the middleware, it’s useful to raise an exception that accurately identifies what’s going on.

django.core.exceptions.ViewDoesNotExist

When resolving URLs, it’s quite possible for an incoming URL to match a pattern in the URL

configuration, but not match any known views. This could be for a variety of reasons, including a truly missing view, but it’s also often due to an error that causes the view not to be loaded properly. After all, Django can only identify a proper view if Python can parse it and load it as a function. When any of these situations occur, Django raises to indicate,

as best it can, what went wrong.

There’s typically no need to manually catch this error or do anything special with it, since Django handles it as best as can be reasonably expected. In development, with

, it

displays a useful error page with details on which view was attempted and a Python error message indicating why it couldn’t be loaded. In production, that level of detail is unsafe, so it falls back to a standard HTTP 500 error, notifying the administrators behind the scenes.

C H A P T E R 9

฀ COMMON TOOLS

217

Text Modification

At its core, the Web is a written medium, using text to convey the vast majority of ideas. Typically, this text is supplied as a combination of templates and database content, but it often needs a bit of massaging before it can be sent to users. It might have to be capitalized for use in a title, line- wrapped for use in an email or otherwise altered.

get_text_list(items, last_word='or')

There are a number of ways to present a list of items to users, each appropriate for different situations. Rather than listing each item on its own line, it’s often useful to display the list in plain English as a comma- separated list, such as “red, blue and green.” This may seem like a daunting task, but

simplifies it considerably. Simply pass in a list of items as the first argument and an optional conjunction to be used as the second argument, and it returns a string containing the items separated by a comma and the conjunction at the end.

javascript_quote(s, quote_double_quotes=False)

When writing strings out to JavaScript, whether in source code or in a response code in JavaScript Object Notation (JSON),2 there are certain considerations that have to be taken into account for special characters. This function properly escapes these special characters, including Unicode characters, in a way that JavaScript can understand.

normalize_newlines(text)

When an application needs to work with text content coming from unknown sources, it’s quite possible that input will be generated on a combination of Windows, Apple and Unix- style systems. These different platforms have different standards for what characters they use to encode line- endings, which can cause problems when the application needs to do any text processing on them. Given input like this,

looks for the common line- ending alter-

natives and converts them all to the Unix- style that Python expects.

2.

218

C H A P T E R 9

฀ COMMON TOOLS

phone2numeric(phone)

Businesses often offer phone numbers as words to make them easier to remember. If phone numbers like that are offered as input to an application, they’re typically only useful as- is if they’re only ever displayed directly to users. If the application ever has to use those numbers as part of an automated system or show them to employees who make calls on a regular basis, it’s more useful to work with them as raw numbers instead of marketing text. By passing phone numbers through

, you can be sure that you’ll always get a real phone number to work with.

recapitalize(text)

Given a string that may have already been converted to lowercase, perhaps for search or other comparison, it’s usually necessary to convert it back to regular mixed case before displaying it to users. The

function does this, capitalizing letters that fol-

low sentence- ending punctuation, such as periods and question marks.

Caution Although Django provides many features for international audiences, the function only works for basic English text. Punctuation used in other languages may not be properly identified, causing the capitalized output to be incorrect.

smart_split(text)

Originally developed as a way to parse template tag arguments, takes a string

and breaks it apart at spaces, while still leaving quoted passages intact. This is a good way to parse arguments for any other application, as it allows a great deal of flexibility. It recognizes both single and double quotes, safely handles escaped quotes and also leaves the quotes intact at the beginning and end of any quoted passages it comes across.

C H A P T E R 9

฀ COMMON TOOLS

219

truncate_words(s, num)

Given a long string as its first argument, this function drops all words after the number specified in the second argument, returning the modified string. It preserves punctuation adjacent to words, and automatically adds ellipses after the shortened string before returning it. This is mostly useful for showing brief summaries of content prior to displaying the full text in a separate view.

truncate_html_words(s, num)

This is equivalent to

, except that it knows about HTML, taking care to pre-

serve HTML tags along the way. In fact, it even closes any tags that are left open as a result of truncating the text. This should be used not only any time the original content is raw HTML, but also when the content is a markup language meant to be converted to HTML. Converting the markup prior to passing it through

will ensure that the specialized

markup language doesn’t get adversely impacted.

wrap(text, width)

This takes the specified text and inserts newline characters as necessary to make sure that no line exceeds the width provided. It makes sure not to break up words, and also leaves existing newline characters intact. It expects all newline characters to be Unix- style, though, so it’s best to run the text through

first if you are not controlling the source of

the text to be sure it works properly.

220

C H A P T E R 9

฀ COMMON TOOLS

Data Structures

When working with any complex system, it’s often necessary to work with data in a very specific structure. This might be a sequential list of items, a mapping of keys to values, a hierarchical tree of categories, any combination of those or something else entirely. While Django doesn’t pretend to provide objects for every arrangement of data an application might need, there are a few specific things that the framework itself requires, and these are made available to all applications based on it as well.

django.utils.datastructures.MergeDict

When multiple dictionaries need to be accessed together, the typical approach is to create a new dictionary that contains all the keys and values of both dictionaries together. This works well for simple applications, but it may well be necessary to maintain the mutability of the underlying dictionaries so that changes to them are reflected in the combined dictionary. The following shows how that breaks down with standard dictionaries.

This illustrates a simple approach at combining dictionaries, using the fact that can accept both a dictionary and keyword arguments, combining them into a new dictionary. Thanks to the

syntax described in detail in Chapter 2, this make it a convenient way to achieve the desired result, but the example also shows where it starts to fail.

First, it only accepts two dictionaries; adding more would require calling more than

once, adding a new dictionary each time. Perhaps more importantly, updates to the source dictionaries don’t get reflected in the combined structure. To be clear, this is ordinarily a good thing, but in cases like

, which combines

and

, changes made

to the underlying dictionaries should also be revealed in the combined output.

To facilitate all of this, Django uses its own class that acts like a dictionary in many respects, but transparently accesses multiple dictionaries behind the scenes. There’s no limit to the number of dictionaries that can be accessed this way. Simply supply as many dictionaries as needed when instantiating the object, and they’ll be accessed in the order they’re provided. Since it stores references to the real dictionaries and accesses them instead of creating a new one, modifications to the underlying dictionaries are reflected in the composite.

C H A P T E R 9

฀ COMMON TOOLS

221

Since keys are checked in the internal dictionaries in the same order they were passed in to

,

is in the second example, while it was in the first one.

django.utils.datastructures.MultiValueDict

On another extreme, it’s sometimes useful to have each key in a dictionary potentially reference more than one value. Since Web browsers send data to the server as a series of name/

value pairs, without any more formal structure, it’s possible for a single name to be sent multiple times, probably with a different value each time. Dictionaries are designed to map one name to only one value, so this presents a challenge.

On the surface, it seems like the solution is simple: just store a list of values under each key. Digging a bit deeper, one problem is that the vast majority of applications only use one value for each key, so always using a list would make more work for everybody. Instead, the majority case should be able to use a single key to access a single value, while still allowing all the values to be accessed for those applications that need them.

Django uses

to handle this case, basing its default behavior on what most other frameworks do in this situation. By default, accessing a key in a returns

the last value that was submitted with that name. If all the values are required, a separate method is available to return the full list, even if it only contains one item.

Caution This doesn’t automatically coerce each value to a list. If you pass in a single item for any of the values, that value will be returned as expected, but

will return the original value as it was passed

in. That means

will return the single item only, not a list containing a single item.

222

C H A P T E R 9

฀ COMMON TOOLS

django.utils.datastructures.SortedDict

One of the more obscure features of Python dictionaries is that they’re technically unsorted.

Inspecting a variety of dictionaries may seem to yield some patterns, but they can’t be relied on, as they will differ between Python implementations. This can be quite a stumbling block at times because it’s easy to accidentally rely on the implicit ordering of dictionaries, only to find it change out from under you when you least expect.

It’s quite common to need a reliably ordered dictionary, so that both Python code and templates can know what to expect when they encounter a dictionary. In Django, this feature is provided by the

, which keeps track of the order its keys were added to the dictionary. The first step in utilizing this functionality is to pass in an ordered sequence of key/

value pairs. This order is then preserved, as well as the order that any subsequent keys are given new values.

Functional Utilities

Python treats functions as first- class objects. They have certain attributes and methods associated with them that are obviously different from other objects, but the core language treats them just like any other object. This handling allows for some very interesting uses of functions, such as setting attributes at run time and assembling functions in a list, to be executed in order.

django.utils.functional.curry

It’s often necessary to take a function with a complex set of arguments and simplify it so that code that calls it doesn’t always need to supply all the arguments. The most obvious way to do this is by providing default values wherever possible, as described in Chapter 2. In many situations, though, there isn’t a sensible default at the time the function is written or the default value might not be suitable to the needs of the situation. Normally, you can just call the function with whatever argument values you need, which works just fine for most needs.

Sometimes, though, the function’s arguments are determined at a different time than when it actually needs to be called. For instance, it’s quite common to pass a function around so it can be used later, whether as an instance method or a callback, or even a module- level function.

When using a function that accepts more arguments than will be provided later, the remaining arguments must be specified in advance.

Since Python 2.5, this functionality is provided in the standard library, by way of the function. While being bundled with Python is convenient, it’s only useful for subsequent installations, while Django supports versions of Python that have been C H A P T E R 9

฀ COMMON TOOLS

223

around far longer. Instead, Django provides its own implementation at

.

The first argument to curry is always a callable, which won’t be called right away, but will be tucked away to be used later. Beyond that, all positional and keyword arguments are saved as well, and will be applied to the supplied callable when the time comes. The return value is then a new function that, when called, will execute the original callable with both the original arguments and any arguments that were provided in the call that came later.

django.utils.functional.memoize

When working with a lot of information, it’s often necessary for functions to make certain basic calculations where the only true variables—that is, values that change from one call to the next—are the arguments that are passed in. To reuse a term mentioned in Chapter 7, this behavior makes the function idempotent; given the same arguments, the result will be the same, regardless of how many times the function is called. This is, in fact, the original math-ematical meaning of the term, which was borrowed for use with HTTP methods.

Idempotence provides an interesting disconnect between humans and computers. While humans can easily identify when a function is idempotent and learn to memorize the result rather than continue carrying out the function each time (remember learning your multiplica-tion tables?), computers aren’t so lucky. They’ll happily churn away at the function time and time again, never realizing how much unnecessary time it takes. This can be a big problem in data- intensive applications, where a function might take a very long time to execute or be executed with the same arguments hundreds or thousands of times.

It’s possible for a program to take the same shortcut that we humans learn as children, but not without a little help. Django provides this assistance by way of the function, also

located at

. It simply takes any standard function and returns a wrapper around it that records the arguments being used and maps them to the value the function returns 224

C H A P T E R 9

฀ COMMON TOOLS

for those arguments. Then, when those same arguments are passed in again, it simply finds and returns the value that was previously calculated, without running the original function again.

In addition to the function to be called,

takes two other arguments, used to

determine how its cache of return values should be managed.

฀

฀

—A dictionary where the values will be stored, with the key being the arguments passed in to the function. Any dictionary- like object will work here, so it’s possible, for instance, to write a dictionary wrapper around Django’s low- level cache—described in Chapter 8—and have multiple threads, processes or even entire machines all share the same memoization cache.

฀

฀

—The number of arguments that are combined to form the key in the dictionary cache. This is typically the total number of arguments the function accepts, but can be lower if there are optional arguments that don’t affect the return value.

C H A P T E R 9

฀ COMMON TOOLS

225

NOTE ABOUT MEMOIZING ARGUMENTS

Because the function’s arguments will be used in a dictionary to map to return values, they must be hashable values. Typically, this means anything immutable, but certain other types of objects may be hashable as well. For example, the

function described in this section would throw an error if passed a list instead of a tuple. Because the contents of a list can change, it can’t be used as a dictionary key.

django.utils.functional.wraps

Chapter 2 described decorators in detail, but there’s one aspect of them that can cause problems in some situations, because decorators often return a wrapper around the original function. This wrapper is, in fact, an entirely different function than what was written in the source file, so it has different attributes as well. When introspecting functions, this can cause confusion if several functions are passed through the same decorator, because they would all share similar properties, including their names.

To help ease this situation, Django includes a copy of Python’s own function,

which was first introduced in Python 2.5.

is actually another decorator, which copies

details of the original function onto the wrapper function, so it looks more like the original when everything’s done. Just pass in the original function to and use it as you would

any other decorator on your wrapper, and it’ll do the rest.

226

C H A P T E R 9

฀ COMMON TOOLS

Caution Unfortunately,

can’t make the wrapper completely identical to the original function. In particular, its function signature will always reflect that of the wrapper function, so attempting to introspect the arguments of decorated functions will likely result in some confusion. Still, for automated documentation and debugging purposes, having

update the name and other information is quite useful.

Signals

An important aspect of a large application is knowing when certain things happen in other parts of the application. Even better is the ability to do something the instant that event happens. For this purpose, Django includes a signal dispatcher that allows code to broadcast the occurrence of an event, while providing a method for other code to listen for those broadcasts and react accordingly, the instant the event occurs. It identifies the type of event being broadcast by allowing code to define unique signals to dispatch.

This concept of dispatching and the code that enables it isn't unique to Django, but its implementation is customized for the needs of a Web application. This implementation is located at

, though it’s designed to be used through the simple

object, available at

. Django uses signals in a variety of places, many

of which have been documented elsewhere in this book, in the areas where they’re used. The following sections discuss in more generality how signals and dispatching work, and how to register listeners for particular events.

How It Works

The basic process is fairly simple. Each step will be explained in more detail in individual sections, but the following should serve as a good overview.

First, some Python code defines a signal. As described in the next section, this is a object that is placed in a reliable location. This object represents an event that is expected to occur at some point in time—possibly multiple times. The dispatcher doesn’t use any central registration of signals; it’s up to your own code to know which signal to use at any given time.

When your code triggers an event that you’d like other code to know about, your code sends some information to the signal, including a “sender” object representing where the event is coming from and any arguments that describe other details of the event. The signal itself identifies just the type of event; these additional arguments describe what’s happening at a particular time.

The signal then looks at its list of registered listeners to see if any of them match the provided signal and sender, and calls each function it finds in turn, passing along whatever arguments the signal was given when the event was triggered. Registration of listeners can happen at any time, and the signal will update its registry when a new listener is added, so that future events will include the new listener.

Defining a Signal

A signal doesn’t need to implement any kind of protocol, or even supply any attributes. They’re really just vehicles to use for advertising when an event occurs; they’re simply instances of

.

The real key to defining a successful signal is just in making sure that it doesn’t get replaced.

A signal object must always be available from the same import location, and it must always be C H A P T E R 9

฀ COMMON TOOLS

227

the same object. The dispatcher requires this because it uses the object as an identifier, to match the event being dispatched with the appropriate listeners that have been registered.

Sending a Signal

Whenever you’d like to notify other code of an event occurrence, signals provide a method to send that signal to any registered listeners. This method requires a

, which

represents the object that was responsible for dispatching the signal, which allows listeners to respond to events coming from a particular object. Typically, Django uses a class—such as a model—as the

, so that listeners can be registered prior to any instances being created, while also allowing for listeners to respond to events on all instances of that class.

In addition to a sender,

also accepts any number of additional keyword arguments,

which will be passed through directly to listeners. As shown in the next section, listeners must always accept all keyword arguments, regardless of what they actually use. This allows the sending code to add new information to a signal later on, without causing any problems with listeners that haven’t yet been updated to use that new information. It’s quite likely that the code that sends a signal will have features added to it later on, and this keyword argument support makes it easy to incorporate those features into existing signals.

Once all the listeners have been called,

returns a list of the responses returned by

the registered listeners. This list contains a sequence of 2- tuples, of the format

. Django’s own signals don’t typically use any return values, but they can be quite useful to support plugins that send information back to the application itself.

Capturing Return Values

Functions are often expected to return a value, and signals can take full advantage of that.

When each listener is called with the signal’s arguments, Django captures its return value and collects all of them together in a list. Once all the listeners have been called, the full list of return values is then returned from

, allowing the calling code to access any

information provided by the listeners. This allows signals to be used for more than just extra actions; they can also be used for data processing and related tasks.

Defining a Listener

When sent, the signal passes the sender and all appropriate arguments to each listener function that is registered with that signal. A listener is simply a Python function like any other; the only difference is the fact of having been registered as a listener for a particular signal. Since the signal simply calls the listener as a function, it can actually be any valid Python callable, 228

C H A P T E R 9

฀ COMMON TOOLS

many of which are described in Chapter 2. In practice, standard functions are the most common.

While listeners are allowed a great deal of flexibility, signals do make one important assumption about how they’re defined: all listeners must accept any keyword arguments that are passed in. Which arguments are actually used depends entirely on how a particular listener intends to use the signal, but it must accept unused arguments without error. As shown previously, signals may be sent with any number of keyword arguments, and these will all be passed along to all listeners.

The value in this approach is that listeners don’t need to know about everything the signal is responsible for. A listener can be attached for one purpose, expecting a specific set of arguments.

Then, additional arguments can be added to the signal dispatch, and all previously defined listeners will continue to function properly. As with any other function call, if a listener expects an argument that isn’t provided with the signal, Python will raise a

.

Registering Listeners

Once you have a signal to work with and a listener intended to work with it, connecting them is a simple call to the signal’s

method. In addition to one required argument, there

are a few options that can be specified when registering a signal, customizing how that listener should be handled when the signal is dispatched later on.

฀

฀

—The callable that will receive the signal and its associated arguments. This is obviously required for all registrations.

฀

฀

—A specific object to watch for signals. Since every signal must include a sender, this allows a listener to respond to just that one sender. If omitted, the listener will be called for all senders that issue the given signal.

฀

฀

—A Boolean indicating whether weak references should be used, a topic described in more detail in the next section. This defaults to

, using weak references by

default.

฀

฀

—A unique string used to identify the listener on the given signal. Since modules can sometimes get imported more than once, it’s possible for listeners to get registered twice, which will often cause problems. Supplying a unique string here will ensure that the listener only gets registered once, no matter how many times a module gets imported. If omitted, an ID will be generated based on the listener itself.

Forcing Strong References

While weak references are a fairly complex topic, well beyond the scope of this book,3 signals’

use of them can cause confusion in certain situations, so it’s worth giving a basic overview of the problem and its solution. When an object is referenced using a weak reference, as done by Django’s dispatcher, this reference alone will not keep the object from being garbage 3.

C H A P T E R 9

฀ COMMON TOOLS

229

collected. It must still have a strong reference somewhere else, or Python will automatically destroy it and free the memory it occupies.

While standard references in Python are strong, the dispatcher, by default, uses weak references to maintain its list of registered listeners. This is generally preferable with signals, because it means that listener functions which belong to code no longer in use won’t use up valuable time and energy by being called.

However, some situations in Python would ordinarily cause an object to be destroyed, and these situations require special attention when using signals. In particular, if a listener function is defined inside another function—perhaps to customize a function for a particular object—the listener will be destroyed when its container function finishes executing and its scope is removed.

As you can see, the default form of registering the listener allows the function to be destroyed once its customization function finishes executing. By specifying explicitly, it survives

to be called when the signal is sent at a later point in time.

Now What?

The tools laid out in this chapter won’t provide major new features for your applications, but they can help with many of the simpler tasks many applications need. These little things can really help tie it all together. How the application actually gets used is another issue, with some of the more interesting options described in the next chapter.

C H A P T E R 1 0

Coordinating Applications

Writing software for a business is hard work. There is no single rule book that outlines which applications to write, how they should be written, how they should interact with each other or how customizable they should be. The answers to all of these concerns are best left to developers on each project, but the examples shown throughout this chapter and Chapter 11 may help you decide the best approach for your project.

Much of a site’s functionality is outward- facing, providing features to users outside the organization. Many times, more functionality is focused inward, intended to help employees perform their daily tasks more effectively. Consider a basic real estate Web site that needs to keep track of its clients and available properties. In addition to just displaying properties to the outside world, agents also need to manage those properties and the people who help the process move along.

Rather than building one large application geared toward a specific need, it’s more valuable to try to pull those needs apart, having multiple applications that work together to achieve the final goal. Doing so will require a bit more work in the beginning, but as new features keep getting added, clean separation of applications will help determine what should go where and how everything should work together.

Contacts

While it may seem like everything in the real estate world revolves around property, people are still the most fundamental piece of the puzzle. For example, a given property could have an owner, a real estate agent and several prospective buyers. These people each fill a different role in the real estate process, but the data necessary to represent them is the same, regardless of the roles they play. They can all be generalized into a “contact” that simply contains the data necessary to identify and communicate with them.

This abstraction provides us a simple model that can be used for people related to a specific property, others who haven’t yet expressed interest in a property, employees within our fictional real estate office itself and even third- party contacts like quality inspectors and value assessors. What roles each person plays can be defined later, by relating them to another model, such as a property.

231

232

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

contacts.models.Contact

Contact information typically consists of things like a person’s name, address, phone number and email address, some of which can already be captured by Django. The model from

contains fields for a person’s first and last names as well as an email address, so all that’s left is to include some of the more real- world contact information. Relating it to

allows a single contact to contain both types of data, while also opening up the possibility of contacts who can log in later on.

WHY NOT MODEL INHERITANCE?

One Django model can directly inherit from another, automatically creating a reference similar to the one used here. Since that also adds some extra ease-of- use options, you may be wondering why doesn’t just inherit from

directly.

Model inheritance is best suited for situations where you won’t be using the base model directly, because Django doesn’t provide a way to add an inherited instance to existing models. In our case, that means that if a

already exists in the database, we wouldn’t be able to create a new based

on it. Since there are many other applications, including Django’s admin application, that might create users directly, we need to be able to create contacts for either new or existing users without any trouble.

By using a

explicitly, we’re defining the exact same relationship that model inheritance would use, but without the different syntax that restricts us in this case. We lose a few of the syntax benefits that true inheritance provides, but those can be accommodated another way.

Because a contact is essentially just a user with some added attributes, it’s useful to have all the attributes available on a single object. Otherwise, template authors would have to know not only which model a given attribute comes from, but also how to refer to the other model to retrieve those attributes. For example, given a

object named

, the following list

shows many of its attributes and methods:

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

233

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

This introduces an unnecessary burden on template authors who shouldn’t need to know what type of relationship exists between contacts and users. Model inheritance alleviates this directly, by placing all attributes on the contact directly. This same behavior can be achieved here by simply using a set of properties that map various attributes to the related user object behind the scenes.

Not all methods of

make sense on

. For instance, the

and

methods are best left on

. Views and templates won’t be using a contact

to determine authentication or permissions, so a contact will instead serve as a central location for all aspects of a person’s identity information.

contacts.forms.UserEditorForm

Rather than requiring users to manage their contacts through the admin interface, it’s more useful to have a separate form that can be devoted solely to contact management. This is even more important for contacts than most other models because contacts are actually composed of two separate models. Django’s provided

helper1 maps one form to one model,

requiring the

application to use two separate forms to manage a single person.

A single form could contain all the fields necessary for both models, but that wouldn’t work with

, because the form would have to contain all the necessary logic for populating and saving the models manually. Instead, two independent forms can be used, with a view tying them together—see the description of

for details.

Since Django provides the

model on its own, it would seem logical to reuse whatever Django uses for managing users. Unfortunately, the forms provided for user management are designed for very different use cases than contact management. There are two forms available, both living at

, each with a different purpose:

1.

234

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

฀

฀

—Intended for the most basic user creation possible, this form only accepts a username and two copies of the password (for verification). The fields needed for contacts—name and email—are unavailable.

฀

฀

—Used for the admin interface, this form contains every field available on the

model. Although this does include name and email, it also includes a host of fields intended for authentication and authorization.

Since neither of these forms really fits the use case for contact management, it makes more sense to simply create a new one for this application.

makes this easy, allow-

ing a form to just specify those things that differ from the defaults. For contact management, that means only including fields like username, first name, last name and email address.

With that information,

can manage the rest of the form’s behavior, based on

details provided on the underlying model. All that’s left is to supply a complementary form for managing the new contact- level details.

contacts.forms.ContactEditorForm

The form for managing contacts works very similarly to the one for users, using to

handle most of the details. The only difference is that the fields used for contacts have much more specific validation requirements than were set out in the model already defined.

Phone numbers, for example, are stored in the model as plain text, but they follow a specific format that can be validated by the form.

These validations are already provided by Django as part of the package, which

lives at

. Within

, Django provides individual modules for several dif-

ferent localities, including the United States, where our real estate company will operate. Inside each module is a selection of form fields and widgets that are specific to that region’s common data types. For the United States, that includes such as things as

฀

฀

for validating a two- letter code against current states

฀

฀

to display a list box containing all valid states

฀

฀

to validate ten- digit phone numbers, including dashes

฀

฀

that validates five- or nine- digit ZIP codes

Note The

Django provides also includes the US territories: American Samoa, the District of Columbia, Guam, the Northern Marianas Islands, Puerto Rico and the US Virgin Islands.

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

235

There are others as well, but those four classes will suffice for customizing the validation of a contact. The only remaining editable fields, address and city, don’t have established formats that can be verified programmatically. Applying these overrides, looks

like this:

Note the use of

here instead of fields, as was used in

. This tells

to use all fields in the model except those explicitly listed. Since the user will be provided by

, there’s no need to include that as a separate selection here. Address and city don’t need to be provided as explicit field declarations, since will use standard text fields for those automatically.

contacts.views.edit_contact

Contacts are made up of two models—and thus, two forms—but the users who manage those contacts should only need to deal with one form that contains all the appropriate fields. No generic views are available to ease this process, but a custom view isn’t hard to write.

The first choice to make is what arguments to accept, in addition to the incoming request that is required for all views. In order to pull up a single contact for editing, the view must have a way of identifying which contact should be used. This identifier must be unique and should be reasonably understandable for users to look at. Since every contact relates to a user and every user has a unique username, that username will serve the purpose quite well.

In addition to a username to specify which user to edit, there is another argument that can be provided to keep it somewhat generic: a template name. Even though this view is being written for one specific purpose, it could still be used across many different sites, each having its own template structure. We can provide a default template name, but allowing a URL pattern to override it is a big win for reusability.

Notice that the username is optional. Having an optional identifier allows this same view to be used for adding new contacts as well as editing existing ones. Both situations require essentially the same behavior: accept contact details from a user, check them for valid data and save them in the database. The only difference between adding and editing is whether a

object already exists.

236

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

With this goal in mind, the view must be prepared to create a new object and possibly even a new

, should either of them not exist. There are four distinct situations that must be handled:

฀

฀ ฀

฀ ฀

฀

฀

฀ ฀

and a

exist for that username. The

view should proceed to edit both existing records.

฀

฀ ฀

฀ ฀

฀

฀ ฀

exists, but no

is associated with it. A new

should be created and associated with the

, so that both can be edited.

฀

฀ ฀

฀ ฀

฀

฀

฀

exists for it, which also means no

exists.

Requesting a username implies an existing user, so requesting one that doesn’t exist should be considered an error. In this case, this is an appropriate use of the HTTP 404

(Not Found) error code.

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

and

objects should be created, ignoring any that might already exist. The form will ask for a new username to be provided.

Tip Using a 404 error code doesn’t always mean you have to serve a generic “Page Not Found” page.

You can supply whatever content you like to the

class instead of the default

class. These examples simply rely on the standard 404 error page for simplicity, but it may make more sense for your site to show a 404 page that says something like, “The contact you requested doesn’t exist yet.” This allows you to take advantage of a known HTTP status code, while still displaying more useful messages to your users.

These situations can be handled rather easily in the first code block of the view: Once both objects are known to exist, the view can then proceed to process the form and populate those objects with the appropriate information. It must instantiate, validate and save each form independently of the other. This way, each form only needs to know about the data it’s designed to manage, while the view can tie the two together.

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

237

If both forms were saved correctly, the view should redirect to a new URL where the edited contact information can be viewed. This is especially useful for new contacts, which wouldn’t have a URL assigned to them prior to processing the form. In any other case, including when the form is first viewed—that is, no data has been submitted yet—and when the submitted data fails to validate, the view should return a rendered template that can display the appropriate form.

238

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

Admin Configuration

Since this application has its own views for adding and editing contacts, there isn’t much need to work with the admin interface. But since the

model described later will both relate

to

and make heavy use of the admin, it’s a good idea to configure a basic interface for managing contacts.

It doesn’t offer the convenience of editing the

and

models at the same time,

but does offer value for related models that are managed through the admin.

URL Configuration

In addition to adding and editing contacts, this application must also supply a way to view all existing contacts and details about any specific contact. These features in turn require four distinct URL patterns to be accounted for in the

application’s URL configuration. Two

of these will map to the

view described in the previous section, while two more

will be mapped to Django’s own generic views.

฀

฀

—The list of all existing contacts, with links to individual contact details

฀

฀

—An empty form where a new contact can be added

฀

฀

—A simple view of all the contact information for a given user

฀

฀

—A form populated with any existing data, where that

data can be changed and new data can be added

The

portion at the beginning of these URLs isn’t integral to any of the contact views themselves; it’s a site- level distinction, pointing to the application as a whole.

Therefore, it won’t be included in the URL configuration for the application, but in the configuration for the site. What remains is a set of URL patterns that can be made portable across whatever URL structure the site requires.

The first pattern, the list of all existing contacts, is quite simple on the surface. Once is removed from the URL, there’s nothing left—rather, all that’s left is an empty string. An empty string is indeed easy to match with a regular expression, but the view that we’ll use for it,

, requires some additional

customization to behave properly.

To start, it requires both a

and a

, controlling where it can find the

objects and how they should be displayed. For the purposes of this application, all contacts are available, without any filtering. The template name could be whatever works best according to your style; I’ll call it

.

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

239

By showing all contacts, the list could get quite long, so it would be more useful to be able to split up the results across multiple pages, if necessary. The view provides this

as well, by way of its

argument. If provided, it supplies the maximum number of

results that should be shown on a single page before spilling over to the next. The template can then control how page information and links to related pages are displayed.

Next is the URL for adding new contacts, using the custom view. Like the

contact list, the regular expression for this URL pattern is quite simple, as it doesn’t contain any variables to capture. In addition to matching the

portion of the URL, this pattern just

needs to point to the correct view and pass along a template name.

The remaining URL patterns both require a username to be captured from the URL itself, which is then passed to the associated view. Usernames follow a fairly simple format, allowing letters, numbers, dashes and underscores. This can be represented by the regular expression

]+, a pattern often used for recognizing textual identifiers known commonly as “slugs.”

Note Slugs have their roots in the news industry, just like Django itself. A slug is the name an article is given for internal communications within a news organization, prior to going to print. Just before being printed, a proper title is given to the article, but the slug remains a way to uniquely reference a specific article, whether it’s available for public viewing or not.

240

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

The first of these views to write, the basic contact detail page, will use another of Django’s provided generic views,

, so some care has

to be taken with the name of the variable the username is assigned to. The custom view calls it

, but

doesn’t know to look for something with that name.

Instead, it allows a URL pattern to capture a

variable, which functions the same way.

Another requirement is to supply a

argument that contains the name of the field to

match the slug against.

Ordinarily, this

argument would be the name of the field on the model where the

value can be found. Like most generic views, though,

requires a

argument to be given a valid QuerySet, from which an object can be retrieved. The view then adds a

call to the QuerySet, using the

/

combination to locate a specific

object.

This implementation detail is important, because it allows a URL pattern additional flexibility that wouldn’t be available if the view matched the to an actual field on the

model. More specifically,

can contain a lookup that spans related models, which is

important given the fact that contacts are made up of two different models. The URL pattern should retrieve a

object by querying the username of its related

object. To do this,

we can set

to

.

The last URL pattern, editing an individual contact, closely follows the pattern used to add a new contact. The only difference between the two is the regular expression used to match the URL. The previous pattern didn’t capture any variables from the URL, but this one will need to capture a username in order to populate the form’s fields. The expression used to capture the username will use the same format as the one from the detail view, but will use the name of instead of

.

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

241

The only things missing from this application now are the four templates mentioned in the URL patterns. Since this book is targeted at development, rather than design, those are left as an exercise for the reader.

Real Estate Properties

The meat and potatoes of a real estate company is, of course, real estate. Individual buildings or pieces of land are typically called properties, but that term shouldn’t be confused with Python’s notion of properties, described in Chapter 2. This name clash is unfortunate, but not unexpected; it’s quite common for entirely different groups of people to use the same terms with different meanings.

When this situation arises in general, it’s often best to use whatever term is most widely understood by your audience. When meeting with real estate agents, you should be able to use

“property” to refer to a piece of real estate, without any confusion or explanation. When talking to programmers, “property” might refer to a model, an object or a built- in function.

Python’s

decorator is useful for many situations, but the majority of this chapter will be focusing on other Python techniques. In light of this, the term “property” will refer to a real estate property unless otherwise specified.

properties.models.Property

The most basic item in a property management application is a

. In real estate terms,

a property is simply a piece of land, often with one or more buildings attached. This includes things like houses, retail stores, industrial complexes and undeveloped land. Although that covers a wide range of options, there are a number of things that are shared across the spec-trum. The most basic of these shared features is that all properties have an address, which is made up of a few components:

242

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

This model also includes a slug, which will be used to identify the property in a URL.

Note This model just uses one field for the address, while many address forms use two. Two address lines are always appropriate for mailing addresses, because they allow for divisions within a building, such as apartments or office suites. Real estate is often focused on the building itself and the land it sits on, rather than how the building is divided, so one field will suffice. Condominiums are subdivisions of a building that are sold individually, so in markets that deal in condos, an extra address field would be necessary to uniquely identify properties within a building.

In addition to being able to locate the property, more fields can be added to describe the size of the property and the building that occupies it. There are a number of ways to contain this information, and

will make use of more than one, all of which are optional.

Typically, all of these would be filled before a listing is made public, but the database should support managing properties with incomplete information, so agents can populate it as information becomes available.

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

243

The

field refers to the available area inside the building. When designing or remodeling a building, it’s necessary to break this down into individual room dimensions, but for the task of buying and selling property, the total amount works just fine on its own. The field represents the total land area occupied by the property, as measured in acres—a unit equal to 43,560 square feet.

Tip If an agent does obtain the sizes of individual rooms within the property, those can be included as individual property features using the

model described in the “properties.models.Feature” section later in this chapter.

So far, most aspects of the

model have been focused on describing the property

itself, but there are also aspects of the sales process that can be included. Price is perhaps the most important aspect of a property listing, and even though it’s not a physical attribute, each property can only have one price at a time, so it still makes sense to have it as a field here. The next chapter will explain how we’ll keep track of past prices, but this model will just store the current price.

Another such attribute is the property’s

—where in the sales process it currently

is. For new entries in the database, there may not be any status at all. Perhaps some property information is being recorded for a home owner who is considering selling but hasn’t yet decided to list it on the market. Once the owner decides to sell, it can be listed for public consideration and the rest of the process begins.

244

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

In addition to the attributes that can be stored once on a model, there are other features of properties that may occur more than once or in many varying combinations. These amenities, such as fireplaces, basements, garages, attics and appliances, aren’t part of a checklist of features that every property either does or doesn’t have. This makes it difficult—if not impossible—to create a field for each feature, without having to modify the model’s structure every time a new property comes along that doesn’t fit with prior assumptions.

Instead, features should be stored in another model, indicating which features are present on the property and describing them in detail. Another model can step in to generalize these features into common types, so they can be browsed and searched. For instance, a user might be interested in finding all properties with a fireplace. Having one model defining a fireplace, with a related model describing individual fireplaces, helps enable this type of behavior. See the “properties.models.Feature” and “properties.models.PropertyFeature” sections later for more details on how this works.

Properties also have a number of people associated with them, such as an owner, real estate agent, architect, builder and possibly several prospective buyers. These all qualify as contacts and are stored using the

model already defined. For the purposes of making

it as generic as possible, they will be called “interested parties,” since each person has some stake in the dealings regarding the property.

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

245

Not all properties should be listed publicly. Until a property is listed, and after it is sold, it should be hidden from the general public, available only to staff members to manage. Rather than typing a query for this every time a property is needed for public display, it’s easy to create a custom manager with a method to narrow down the list.

This can be attached to a model through a simple assignment; any name can be used, but the convention is to call the standard manager

, so this will do so.

246

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

properties.models.Feature

A feature is simply something noteworthy that the property offers. It could be a common necessity, such as a basement or a laundry room, but it could also be very distinct, like a fireplace or a sun room. These features are often listed in an attempt to distinguish one property from another, since buyers often have a list of features they’d like to have.

The

model contains just the information necessary to define a particular type of feature. Rather than describing a specific fireplace, a

simply defines what a fireplace

is, offering an anchor point for individual fireplaces to relate to. That way, properties can be searched by feature, using this model as a starting point.

properties.models.PropertyFeature

Instead of defining a feature at a high level, specific details are far more useful when viewing a specific property. The

model forms a bridge between

and

,

providing a way to describe the individual features of a specific property.

properties.models.InterestedParty

Contacts that have an interest in a particular property come in many varieties, from owners and buyers to real estate agents and safety inspectors. Each of these people can be connected to a specific property by way of a relationship that includes some detail about the nature of the relationship.

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

247

Note These roles can overlap, such as an owner who is also the builder and the real estate agent. Some databases allow for a field to be used as a bitmask, where you can toggle individual bits to indicate which roles a contact fulfills. Since Django doesn’t support creating or searching on those types of fields, we instead store just one role per row; a contact with multiple roles would simply use multiple rows to describe the situation.

Admin Configuration

Property listings are meant to be viewed by the general public, but only edited by employees of the real estate agency who have extensive training and experience in the field and can be trusted with this task. That description is the same as the intended audience for Django’s built- in admin application.

With the features available from the admin, it’s easy to put together interfaces for users to be able to edit and maintain all the various models in the properties application. No separate editor views are required, and only minor changes are necessary to customize the admin to work with these models in a user- friendly way.

248

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

“THE ADMIN IS NOT YOUR APP”

If you spend much time in the Django community, you’ll likely run across the phrase, “The admin is not your app.” The general sentiment being conveyed here is that the admin has a fairly limited focus, far more limited than most sites. It’s expected to be used by trusted staff members who can work with a more rudi-mentary data- entry interface. When you find yourself struggling to find ways to get the admin to do what you want, chances are you need to start writing your own views and stop relying on the admin.

That doesn’t mean that the admin is only ever useful during development. If a basic editing interface is suitable for staff members to work with, it can save both time and energy. With a few simple customizations, the admin can perform most of the common tasks that such editing interfaces require. The contacts application described earlier in this chapter couldn’t rely on the admin because it required two forms to be combined, which is outside the scope the admin was intended for.

For properties, the admin is quite capable of generating an adequate interface. Since only staff members will need to edit property data, there’s no need to create custom views that integrate with the rest of the site. More of your time can be focused on building out the public- facing aspects of the application.

The first model to set up is

, but due to the workings of related models, some

configurations for

and

need to be in place first. These are

each configured using a simple class that tells the admin to add them to the property editor as tables at the end of the page. In addition to any existing relationships, the admin should show one empty record that can be used to add a new relationship.

In order to customize some of the more specialized fields on the model’s admin

page, a custom

subclass is required. This allows the form to specify what widgets should be used for its

and

fields, since they adhere to a more specific format than

just a free- form text field. All the other fields can remain as they were, so they don’t need to be specified on this form.

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

249

Now we can finally configure the admin interface for

itself. The first customiza-

tion is to use the

instead of the plain

that would be used normally.

Within that form, not all of the fields should display in a simple list from top to bottom.

The full address can be displayed in a more familiar format by putting the

,

and

fields in a tuple so they all end up on the same line. The slug is placed next to the address, since it will be populated based on that information. The sales fields can be placed in a separate grouping, as can the fields related to size, with a heading to set each group apart.

The related models are added to a tuple called

, which controls how other models

are attached to an existing admin interface. Since they were already configured in their own classes, all we need to do here is add them to the

.

250

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

Lastly, the declaration for generating the slug requires a dictionary assigned to the attribute. The key in this dictionary is the name of the

to gener-

ate automatically. The associated value is a tuple of field names where the slug’s value should be pulled from. All properties should be unique according to their address and ZIP, so those two fields can be combined to form a slug for the property.

Note Slug fields are prepopulated using JavaScript while editing the model instance in the admin application. This is a useful convenience, saving the time and trouble of having to visit a separate field as long as the default slug is suitable. When creating objects in Python, the only way a field gets populated without an explicit value is through a function passed in as its

argument or through the field’s

method.

With that in place, the only model left to set up is

. Since it’s a simpler model

than

, the admin declaration is considerably simpler as well. There are three fields to arrange and a

to configure.

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

251

URL Configuration

Since the actual management of properties is handled by the admin interface, the only URLs to configure are for users to view property listings. These types of read- only views are best handled by Django’s own generic views, configured to work with the models in question.

Specifically, these URLs will use the views from

.

A view for the property listings can be set up using the

view. This view requires

a QuerySet to locate items, which is where the

proves useful. Its

method narrows down the query to the items that should be displayed for the general public.

Although the detail view requires fewer configuration options—since it doesn’t need the

argument—the regular expression gets a bit more complicated. Looking up a property in a URL is best handled by a slug, but slugs can be typically made up of any combination of letters, numbers and basic punctuation. The slug for a property is a more specific format, starting with the street number from the address and ending with a ZIP code. The street name in the middle could still be anything, but it’s always surrounded by numbers.

This simple fact helps shape the regular expression used to capture the slug from the URL.

The idea here is to be as specific as reasonably possible, so that one URL pattern doesn’t inter-fere with others that might look for similar patterns. The URL configuration for the detail view of a single

object looks like this:

252

C H A P T E R 1 0

฀ COORDINATING APPLICATIONS

This regular expression adds explicit rules for digits at the beginning and end of the slug, separate from the middle portion by dashes. This will match property slugs just as well as the usual

]+, but with an important added bonus: these URLs can now be placed at the site’s root. Having a more specific regular expression allows for smaller URLs like 5/. This is a great way to keep URLs small and tidy, while not impeding other URL configurations that might also use slugs.

Now What?

With a few applications in place and ready to work together, a basic site takes shape. The next chapter will show how to bring together all the tools you’ve learned so far to add significant new features to applications like these.

C H A P T E R 1 1

Enhancing Applications

Once a site has a set of basic applications in working order, the next step is to add more advanced functionality to complement the existing behavior. This can sometimes be a matter of simply adding more applications, each providing new features for users and employees alike. Other times, there are ways of enhancing your existing applications so they grow new features directly, without a separate application that can stand on its own.

These “meta- applications” or “sub- frameworks” are built with the goal of easily integrating into an existing application, using hooks that are already provided. This book has illustrated many such hooks, and they can be used in combination to great effect. It’s often possible to write a tool that performs a lot of tasks but only requires adding a single line of code to an existing application.

Recording the Current User

One common need among larger companies is surveillance of the company’s data. It’s important to know who is making changes; if anything goes wrong, there’s someone who can be held accountable. This is also useful for smaller businesses, but the need is less urgent with fewer employees who have access to the data in question.

Nearly any organization of any size can benefit from knowing who last changed a model instance, but it’s often left to “those other guys” who really need it. It doesn’t have to be that way. Combining a few of the tools available in Django, recording the user who last made changes to a model instance doesn’t have to be a difficult task.

On the surface, it seems ridiculously simple. A standard way to override what values get saved for a field is to override the model’s

method and set the value there. Chapter 10

showed how this can be used to calculate one field’s value based on the values of other fields.

It seems reasonable that this same technique could be used to insert the current user into a

field.

253

254

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

There’s just one problem: where does

come from? Django’s models are separated

from views, so the request isn’t just magically available inside a model method. One possible solution is to pass the user in manually. After all, the request isn’t what’s really important; it’s the user we’re after.

Then, when the time comes for a view to save an

instance, it goes some-

thing like this:

That would certainly do the trick, but only in views that are specifically written to use this custom

method. All existing applications, including Django’s own admin application, expect

to work without arguments. Writing a model with that kind of a method

will mean writing—or at least, modifying—all your own applications to work with it.

The core problem now shows itself: how do we get the current user into a model method without changing how the function gets called?

The Thread- Local Approach—Useful but Dangerous

The most common approach for a long time was to make use of the fact that an incoming request is processed all the way through to outgoing response in a single thread. Regardless of what server environment Django was running in, these simple facts remained: one thread processes one request at a time and one request goes through just one thread. Coupled with Python’s own

module,1 a solution was born.

The

module, which provides tools for working with threaded applications, provides a function called

. This function returns a dictionary that can be used as

a namespace for a thread. Functions can read to it and write from it, without worrying about interfering with other threads. Each thread gets its own private dictionary, a task managed by Python itself.

This feature was once useful enough that Django itself contains a version of it for compatibility with Python 2.3 (

was introduced in Python 2.4). This copy included

with Django formed enough justification for many programmers to begin using a thread- local dictionary to store the request, which could then be retrieved by a model’s method.

Since the request is always available to middleware, we ended up with a middleware module that looked something like this:

1.

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

255

After that, a model method such as

just has to import the

vari-

able and read the user out of it.

Every time a model instance is saved,

updates the

attribute to be the user

who is currently logged in at the time. However, there are a few potential pitfalls with this approach.

First,

becomes a bit of a dumping ground for data. Nothing actively manages this dictionary, so there’s no central place to look to find out what might be in it. Any code can put things in there, without any clear way of indicating that it did so.

Taking that problem a step further, applications don’t know what other code is already using

, so there’s no way to prevent name clashes. If two applications running in the same thread—perhaps a view that calls out to a third- party library as part of its processing—

assign data to the same variable name, the later assignment wins. Even if an application actively checks for the presence of a name before assigning to it, there’s no reasonable way to know if that application itself had previously assigned the variable or if another application is using the same name.

The worst part about these issues is that they’re intermittent. Whether they cause any real problems depends entirely on what applications use

and which ones

256

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

of those are executed within the same thread. Everything could be working fine one day and break the next, because a new third- party library makes use of in a way you

didn’t anticipate.

Python’s module- level global variables are at least safe enough that one module can’t accidentally overwrite a global variable in another module—it’s possible to do, but not accidentally. Without offering that level of protection,

becomes a dangerous

tool to rely on without fully understanding how all the code in your site is using it.

The Admin Approach

Instead of relying on thread locals, another approach is to consider the actual use case. Typically, people turn to thread locals as a way of storing the current user in a table while using the built- in admin application. Custom views make it easy enough to insert the user manually, so it’s usually the admin interface that prompts developers to investigate these techniques.

Shortly before Django 1.0 was released, the admin application was overhauled in order to provide more flexibility and features. One of the additions was a result of the desire to access the current user while working in the admin. Now there is a way to override how the admin saves a given model, also providing access to the request object along the way. Remember the definition of

from the previous section:

By adding an

module to the application, it’s possible to supply a new method to use when the admin attempts to save an instance of this model.

Now, when a user adds a new instance of

or changes an existing instance,

the admin will use this method, and the current user will be added to the instance accordingly.

There’s no need to worry about anonymous users, because the admin is only available to authenticated users.

This works quite well, following the “ideal” approach where the request is simply passed to those methods that need it. The only problem is that it only works for the admin application. Other applications that could benefit from storing the current user on a related model, without having to rewrite existing views, are still left needing another solution.

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

257

Introducing the CurrentUserField

Another alternative approach is to keep track of the fields that need to contain the current user and update those fields whenever instances of their associated models are changed. This goes back to being a model- based approach, rather than being view- based, but it no longer requires an override of the

method.

The first step is to mark a field as needing the current user to be inserted when the model is saved. This task is traditionally handled by

or a view, but this approach will

use a new type of field to manage it. This new

will live in the

module of a new

application.

It’s currently little more than a specialized

that’s been hard- coded to relate

with Django’s built- in

model. It specifies

to account for applications where the

model may be edited by an anonymous user and other non- Web applications that might add or update records. As it stands, it would be usable in that regard alone, by simply replacing an existing

with a new

. Here is how it looks on the

from the preceding sections:

Very little has changed in this incarnation: one field has a new type, and one import was updated accordingly. The notable effect of this simple change is that no additional changes will be necessary on the model after this point. You can go through and add to

whatever models you like now, knowing that they’ll continue to work while we work through the remainder of the code to support them properly.

Keeping Track of CurrentUserField Instances

The next thing to take care of is to keep a record of all the models that have instances attached to them. This is important for performance; without it, the user- updating code described later would have to look at every model that gets saved and cycle over its fields, looking for instances of

. Instead, we can supply a registry of known

instances that can speed things up considerably.

A new module,

, will contain the code necessary to maintain a record of every

in use and supply information about that registry to other code that 258

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

asks for it. It uses a slightly modified notion of the Borg pattern,2 looking fairly similar to the plugin architecture registry from Chapter 2.

The internal

dictionary exists only on the

class and is never cop-

ied out to any instances. All methods operate on that class- level dictionary, so it doesn’t matter how many instances of

get created. All instances will use the same dictionary all the time. Take a look at it in action:

Note also that this allows for more than one field to be registered on a given model. Since Django allows the same field to be used multiple times on a single model, supports that as well. If just one instance of the field was stored in the registry, the last one assigned to the model would overwrite the first, which could cause confusion about what’s going on behind the scenes. By explicitly supporting multiple fields per model, we can avoid that problem entirely.

The last step in the registration process is actually adding instances of to the registry when they’re added to models. Remember from Chapter 3 that fields provide a

method that executes while Django processes a model’s contents.

Overriding that method on

gives access to the model class as well as the

name it was given, but the registry is only interested in the model and field objects.

2.

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

259

Now

can register itself on any model it’s attached to, without any additional intervention from you, the developer; simply assigning it to a model will suffice. This registration is the sole purpose of

, so its job is now done. All the rest of the

work happens when a request is processed.

The CurrentUserMiddleware

Like the thread- local approach,

relies on a middleware class to get access to

each incoming request and retrieve the current user. Middleware updates the fields without having to write your views specifically to do so, which opens it up for use in all applications, including the admin and other third- party applications where modifying code is problematic.

The real trick here is how to update

records without resorting to thread

locals, and the answer is signals. Since Django provides a signal that fires just before

an instance gets committed to the database, this new middleware can register a handler to execute at just the right time. These pieces come together in a new module in

the

application, starting with the workhorse: the

method.

As a signal handler, it gets two arguments from the

handler:

and

;

the

argument will be supplied by the

method. Since

is the

model whose instance is currently being saved, it can be used to check whether the model is registered as having any

attributes. If so, it simply loops over them, setting

the instance attribute for each one to the current user.

That won’t do anything unless registered for the

signal, which is a job for the

method.

260

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

This method starts out by checking whether the user is authenticated or not. Remember, uses

to support anonymous users, so this step is necessary to

make that distinction. By also checking to see if the request even has a attribute at all,

this handles cases where the default

is disabled or is placed after

in the

setting.

Continuing on,

is curried into a new function, with the current user

preloaded as its first argument. The resulting function is now configured for use as a signal handler. This signal will be registered for every incoming request, since the only way to know the current user is to get it when the request comes in. It must be removed when the request is finished; otherwise, multiple signal handlers would be competing to update the same fields.

Since

is curried, there won’t be a reference for it once

finishes executing. In order to keep it from being destroyed before it can be useful, it gets registered with

. Since the middleware doesn’t get to keep a reference to the curried function, the

argument provides an alternative reference for the handler. There will only be one signal handler for each incoming request, so the request object is a suitable unique identifier.

Once the curried

is then registered on the

signal, Django con-

tinues on with other middleware and executes the view. Any models updated during that time will be checked by

and updated as necessary. Once the view finishes, Django

enters the response phase of middleware processing, where needs to

remove the listener, using the request to identify it.

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

261

Performance Considerations

As mentioned,

will register a signal handler for every request that Django processes, checking for instances of

every time a model is saved within

a request. On most small sites, this additional overhead is hardly noticeable, but high- volume sites may notice a reduction in the quality of the user experience. The benefits of data surveillance aren’t worth degrading the experience provided to your users.

One way to keep overhead to a minimum is by restricting it to situations where updates are likely to take place. Chapter 7 explained how the HTTP standard expects certain methods to be “safe”—simply viewing a document shouldn’t make any changes. These safe methods are GET, HEAD, OPTIONS and TRACE;

can be written to special- case these

methods, bypassing any further handling.

262

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

Even among requests that do modify data, not all views modify the models that are being managed by

. There’s no way for a third- party application like this to programmatically know which models are modified by which views, so the middleware simply looks at all of them. This can be avoided by only applying this middleware on those views that you know modify the affected models.

To achieve this, we turn to

, which contains the useful

function that was shown in Chapter 7. This utility function takes a middleware, like our

, and converts it into a decorator that can be

applied to just those views that need its features. This new decorator can be provided in the middleware module, right alongside the middleware it accesses.

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

263

Now it’s possible to import this decorator and apply it to the views that modify any of the models with a

attached. One obvious example is the admin interface, which shouldn’t be open to the general public and should therefore have a limited number of users.

This does raise one last problem: the decorator produced by only

works on functions, not on callable objects.

The admin site uses an object—typically

—as the view in

URL configurations, so the

decorator won’t work with it directly. Instead,

a small wrapper function needs to be placed between the two, which can satisfy the decorator while passing everything through to the admin site object.

If the vast majority of the site’s views do update models that have attributes—to be expected if you’re tracking all the models in your applications—the programmer overhead of having to import and apply the decorator to every view may not be worth it. Since nearly all views would need the decorator, applying the middleware makes more sense in that situation.

Keeping Historical Records

Capturing the last user to make a change is useful to a point, but finding out more information requires talking to that user in person and asking what was changed. Worse yet, there’s no record of who else changed anything previously, so there’s no way to know what path a record took from beginning to end.

By bringing together even more of the techniques listed in this book—dynamic models, custom field- like objects, descriptors and curried functions for a start—we can supply a framework for tracking the changes of any model in any application under your control. This includes who changed the model, when it was changed and what it looked like at the time. In keeping with DRY, it’s even possible to add this functionality to a model with a single line.

264

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

Intended Usage

Managing the history of objects requires a fairly detailed application, and it can be hard to understand the end goal when looking at everything individually. This section provides an overview of the features that will be available when the application is completed, so you can start seeing these features fall into place as the code progresses.

First is the act of assigning a history manager to the model that will be archived. This should be as simple as possible, preferably just a single attribute assignment. Simply pick a name and assign an object, just like Django’s own model fields.

That’s enough to get everything configured. From there, the framework is able to set up a model behind the scenes to store old records and the

attribute can be used to access

those records using Django’s standard database API methods. Consider a object for

the author of this book:

This object will function just as it normally would, but with the addition of the attribute, additional information about the author’s history is available. To start, one historical record is available from when the

object was first created.

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

265

Note Historical records won’t magically exist for data that was already in the database. Only new records and updates will get tracked. If you’d like to make a historical record for each row in your existing database, simply save them all one at a time, using a loop such as

.

This may be problematic for large databases, where some custom SQL may be more appropriate.

If the contact changes his phone number, it’s necessary to update the Contact record accordingly. That change also shows up as a new historical record.

Notice that they’re sorted with the most recent record first. This allows for some simple methods to be added, making it easier to get older copies. For instance, the history manager also has a

method, which returns a

object with its attributes set to those

found in the most recent historical record.

Even though each historical record is a different model than the original, a true object is available from any

by using the

attribute.

In the event that a specific date is known, the historical manager also includes a shortcut function to return a

object containing the values that were true for the given object as of the date specified.

266

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

Even after a contact has been deleted, a record of it still remains. In fact, a new record is added to indicate when the contact was deleted. Given the original ID, an empty object can be used to retrieve historical records, including the method for

a kind of “undo” functionality.

Each historical record is identified as one of three types: created, changed or deleted.

Overview of the Process

The whole registration of a history manager begins by assigning a object

to a model, so that’s a good place to start defining code. This will live in the module

of a new

application. There are a number of things that have to happen in sequence to get the history system initialized for a particular model; at a high level, manages all of the following tasks:

1. Create a copy of the model it’s attached to, with additional fields for auditing purposes.

2. Register signal handlers to execute when the original model is saved or deleted. These in turn add new historical records each time the model is modified.

3. Assign a manager to the original model, using the attribute name where the was assigned. This manager will then access historical information.

That’s a short list, but each step requires a fair amount of code, combining several of the techniques described throughout this book. Before any of those steps can really begin, there’s a small amount of housekeeping that must be done. Since the object gets

assigned as an attribute of a model, the first chance it gets to execute is in the method.

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

267

So far it’s not much, but this is the only point in the process where Django tells what name it was given when assigned to the model. This is stored away for future reference during Step 3.

Step 1: Copy the Model

In order to store the data from a model instance in a historical record that can be easily added, searched and retrieved, we need a new model behind the scenes. In theory, we could use any structure that can contain data; perhaps a single

that contains pickled objects.

But to search and browse the historical data more easily, it makes sense to use the same data structure as the original model itself.

Chapter 3 showed that a model’s

attribute contains all the information about how

that model was defined, including all of its fields in the order they were declared. This information is crucial, because it allows us to create a new model that matches that same data structure.

The only trouble is that

gets called on each field in turn, in the order

they appear in the namespace dictionary Python created for the model’s definition. Since standard dictionaries don’t have a guaranteed order, there’s no way to predict how many fields will already have been processed by the time

gets a chance to peek at the model.

To solve this, we turn to a signal:

. Django fires this signal once all the

fields and managers have been added to the model and everything is in place to be used by external code. That’s when

will have guaranteed access to all the fields,

including the order in which they were defined, so

continues by set-

ting up a listener for

.

Django will now call

with the fully- prepared model once

everything is in place to continue processing it. That method is then responsible for performing all of the remaining tasks, all the way through Step 3. Most of the details are delegated to other methods, but

coordinates them.

The first thing

needs to do is copy the original model to create a new model with extra fields attached. It defers this task to the

method, which in

turn relies on a few other methods.

268

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

There are a few different sub- steps required in creating a model like this. Adding all the logic in one method would hamper readability and maintainability, so it’s been broken up into three additional methods.

Copying the Model’s Fields

The

method is tasked with copying the existing fields on the model, returning a dictionary with new fields that can be applied to the history model. This is a more complicated task than it may sound, because there are a few special cases that need to be accounted for, but Python provides a tool to help with the common case.

Python’s

module3 is designed to copy an object and all of its attributes into a new object. This operation is necessary in the event that any of these field attributes get changed; changing one field shouldn’t affect another. If we simply assign the existing field to the new model, they would be the same object, sharing a namespace. A change to one would affect the other, which isn’t a good thing. The

function takes care of our needs.

After copying each field,

has to take care of two special cases. The first is

that a model can only ever contain one

attribute, and it must be the primary key.

Django provides an

as the primary key for any model that doesn’t explicitly declare a different primary key, so this is a common case. Since there will likely be multiple historical records for a given ID, the history model has a separate for its primary key. Any

existing

instances that are found on the original model must be changed to a standard

on the history model.

The next special case to take care of is that uniqueness can no longer be guaranteed on any field. Both the

and

arguments imply that a field’s value must be

unique across all rows in the model, which won’t be true in a historical context. Any field found with either of these attributes set to

is changed to

, with

set to

3.

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

269

instead. Having a unique field on the original model implies some importance, so adding an index to it will help speed up queries that rely on that field’s content.

In addition to the fields themselves, every model needs an attribute named that Django can use to determine what application it belongs to. Since this history model will be tied to the original model,

can be copied straight over to the new model along

with its fields. This way,

provides everything necessary to make the history

model function like the original model as much as possible.

Adding Record- Keeping Fields

So far, the history model is only set up to store the same values as the original model. It does keep a historical record of each stage an instance went through, but without anything else, it’s of little real- world value. It needs some extra information along with that data to make it useful. There are a few basic components that are useful in nearly all situations.

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀the change, if any. This is controlled using the

explained earlier in this chapter.

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀ ฀

฀ ฀

฀

฀

for a new

instance,

for an update to an existing instance and

for a deleted instance.

270

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

Most models also include a

method that controls how a model instance

will be displayed when printed to a console or written to a string, such as a template. To preserve this while still indicating its historical status, provides a new

method that simply uses the original method and adds a date to the end of the string. This is done with the help of a special

attribute, which will be explained

in the next section.

One advantage of providing this extra information in a separate method is that offers a chance for customization. Many projects have some additional information, such as a

, that could be logged alongside this information to give greater insight into the data. Overriding

provides an opportunity to easily add those extra

fields.

In addition, overriding

allows other types of customizations. If the

provided field names or the

implementation don’t suit your taste, feel free to

replace them. This makes

the method for the more flexible aspects of the

history model.

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

271

Accessing a True Model Instance

Since the history model is designed for storing and retrieving information about what a model instance looked like at points in the past, it makes sense to have access to an instance of the original model with the historical values. This provides access to any custom methods or other attributes that didn’t get copied over to the history model. This is especially necessary for implementing a proper

representation.

Since the historical record of an instance contains all of the field values of the instance itself, no additional database calls are necessary to populate an instance of the original model.

This is accessible on a historical instance through the

attribute, which is imple-

mented as a descriptor.

It needs to take the original model as an argument and use that instead of the argument to the

method because

is the history model, not the original model.

Using the original model’s collection of fields as a guide, the descriptor pulls the appropriate values from the instance and creates a new instance. This new instance has all the original methods, including

, which can be used to restore an older copy of an instance.

Adding Meta Options

Another necessary item for creating the history model is a dictionary of options to be included as the model’s

inner class. The only option that is actually required is

, which

makes sure that the records are sorted in descending order by date.

Other implementations can override this method to add more options as well, if necessary. With nearly everything in place, all that’s left is for to create

a new name for the history model and pass everything to

to create it. Then,

can use that new model to perform additional tasks.

272

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

Step 2: Register Signal Handlers

There are two ways of modifying a model instance, and Django provides signals to hook into both of them; the

and

signals are fired when saving and deleting an

instance, respectively.

The

object isn’t used for anything after its initial setup, so it’s discarded by Python’s garbage collection fairly quickly. Because the signal handlers are methods of that object and signals use weak references by default, the handlers get removed from the signal as soon as

goes away. Passing

forces the signals to use strong ref-

erences for these methods, keeping them alive long enough to do their jobs.

Like most of this system, the actual implementations of these two signal handlers each delegate to a separate method to reuse code. They both perform the same task, adding an entry to the history model, so it makes sense to share code as well. The only difference between the two is the value each provides for the

field of the historical record.

The manager used to create this entry in the history model is determined according to the attribute that was set aside when

was called at the begin-

ning of the process. The manager assigned there is the last step of the process.

Step 3: Assign a Manager

In order to access the historical records for a given model, a manager is attached to the original model using the name where the

object was assigned. The object that gets

assigned is actually a descriptor, which creates a customized manager when accessed. All the manager code is located in a new module,

, which is referenced from

as

follows:

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

273

The addition of those lines completes the code necessary in

; the remainder of

the functionality is implemented in

instead. The descriptor that gets assigned to

the original model is fairly simple, storing the history model and using that to create customized managers. The

then accepts the history model and instance and stores

them for later. Note that

is an optional argument, allowing the manager to be used

on the original model itself. This, in turn, will retrieve all historical records for that model, regardless of what instance they are attached to.

Notice also that the

attribute is received and stored, but not used by any of this code. Django’s own

class uses

to determine what model it should refer-

ence in database queries. Assigning the right model to

is all that’s necessary to

tell Django how to get data from the correct table and formulate results using the appropriate instances.

274

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

This overridden

method is what allows a

to retrieve

objects matching the ID of a given instance of the original model. No special ordering is required because the history model’s

inner class already has an

option set. In

addition to simply retrieving a list of related historical records, can contain

extra methods to perform more specific searches.

Retrieving the Most Recent Copy of an Instance

In the event that a model instance has changed since the last time it was saved or was even deleted previously, it becomes necessary to quickly and easily retrieve the last known state of the instance. Since that information is stored in a history model, which in turn is accessible by the

, a new manager method can do this job without requiring any arguments at all. The first requirement is that this method should not be available on the model itself, only on instances.

Now that we can be sure there is a valid model instance to work with, the next step is to gather up the field names that exist on the model, so that only those fields are retrieved.

This method returns an instance of the original model, not the history model. Retrieving any additional fields would not only be wasteful, it would also require more code to remove them before populating the model instance, since those extra fields aren’t supported by that model.

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

275

Note that this needs to use the

attribute of

, rather than

because

is the history model, not the original model that we’re keeping track of.

With a list of fields in place, a simple call to

retrieves the values for all recorded

states for the given instance. Because those states are sorted descending by date, the first row is always the most recent, so using an index of will issue the appropriate query.

Catching

allows for a more useful error message in the event that there is no history data available for the given instance. In this case, a different exception is raised, using the model’s own

class to keep in line with the way Django’s own instance lookups work. If no error is raised,

will have all the values necessary to populate an instance of the original model. It then does exactly this and returns it for other code to use.

276

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

Retrieving an Instance As It Existed at a Specific Point in Time Similar to

, it’s also sometimes useful to see what a model instance looked like on some specific date or at a particular time. This is useful, for instance, when customers ask about products or resources that they heard about some time ago. Being able to retrieve the item as it existed on the date in question can be a valuable tool in serving those customers’

needs. Like

, the new

method starts by making sure it only gets used on

a model instance, rather than the model itself.

The list of fields is retrieved the same way as in

, but it’s not passed directly

into a

query. The

query needs to limit its results to the data that was

accurate on the date supplied, so we must first apply a

to satisfy that condition.

This new QuerySet is what we need to retrieve the instance values for the particular date.

Again, a

query is used and limited to the first result to obtain the record nearest to the date provided. If no records are found, the same

exception is raised, but

with a slightly different message to indicate that there may be records for the object, but none before the date specified.

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

277

Note also that the

query used here includes an extra field not present in the query. One last check must be performed on the data before it’s used to populate an instance of the model, and the

is necessary for that. If the row returned

from the query has a

of

", that means the instance was deleted prior to the date specified, so it technically didn’t exist as of that date. Rather than return an object that didn’t exist,

raises

, explaining what happened.

With all the sanity checks completed, we can be certain that the values retrieved are valid for an object that existed on the date passed to the method. Since the first value retrieved in the QuerySet was the

, which isn’t part of the original model, a slice is taken to retrieve the rest of the values, which are then passed to the model instead.

278

C H A P T E R 1 1

฀ ENHANCING APPLICATIONS

Now What?

The tools and techniques discussed in this book go well beyond the official Django documentation, but there’s still a lot left unexplored. There are plenty of other innovative ways to use Django and Python; the rest is up to you.

As you work your way through your own applications, be sure to consider giving back to the Django community. The framework is available because others decided to distribute it freely; by doing the same, you can help even more people uncover more possibilities. The Appendix explains how you can give back to the community and to the framework itself.

A P P E N D I X

Contributing to Django

As with other open source projects, Django relies heavily on community contributions. The community is responsive to those willing to give back to the framework, whether in terms of code, documentation, answers to common questions or even just reporting problems so that others can solve them. And while everyone is encouraged to contribute, there are a few simple expectations that should be followed, which help ensure the contribution will be well received.

As this appendix explains, there are several ways to help out your colleagues, each with its own set of intended situations and expectations to follow.

Reporting a Ticket

For many people, the first step in community contributions is to report a problem with the code as it stands. It doesn’t have to be a significant bug; every problem that gets reported—

large or small—helps move the framework along.

Issues and feature requests for Django’s internal code are kept in an instance of Trac,1

an open source project management utility. It allows you to browse the source, register new issues, view and comment on existing issues, submit patches, review changes that were made to the codebase and even keep track of any or all of this using automated news feeds.

Perhaps the most important of these features is the ability to report a new ticket for the community to consider. This is often a bug with the code as it stands, but it could be anything from an error in the documentation to a request for a new feature or a suggestion for performance improvements.

If you’re entering a bug, just clearly explain the situation, along with any information required to duplicate the problem, what you expect to occur and what actually occurs when you encounter the situation.

Supplying a Patch

If you enter a new issue, or perhaps find an existing one that piques your interest, and you feel comfortable with the subject, you might consider writing a patch to address the issue.

The ability of community members to submit patches is an incredibly valuable feature of the open source model, and every patch helps solidify the framework.

1.

279

280

A P P E N D I X

฀ CONTRIBUTING TO DJANGO

Patches in Trac will be applied to trunk, so make sure you’ve checked out the SVN version of Django and updated it prior to creating the patch. If the patch will be significant in size or scope, you may even want to check out a fresh copy into a new directory, so that any problems with your changes don’t adversely impact any of your other Django projects while you get it just right.

When you’re confident that you’ve successfully addressed the problem, generate a file—

sometimes called a “diff file” or often just a “diff”—containing your changes. All SVN clients provide a way to create a diff, so check out your client’s documentation for details, but if you’re using the command- line client, it’s quite a simple process.

Then, simply attach the patch to the ticket in Trac, describing what changes you made and what you did to verify that the ticket’s primary issue was indeed addressed. It will be reviewed in time by the appropriate developers, and if it seems to do the job well enough and the ticket merits fixing, your patch will help guide the core developers when they update trunk with a fix for the issue.

Please be aware that patches rarely make it from Trac to trunk without any modifications.

The core developers are very experienced with Django’s internal code, so they may make stylistic changes, add code to address more situations or reduce code to make it more readable or maintainable.

Writing Tests

Whenever a ticket in Trac has a patch for a significant problem, it’s expected to also include automated tests that will not only prove that the issue is solved, but help prevent it from resurfacing in the future. Tests are covered in detail in Django’s documentation2, but they’re particularly important for tickets in Trac.

Tests will often be supplied in subsequent patches to a ticket, but may be supplied with the initial patch, if they’re easy enough to include. If you’re looking at an existing ticket without tests, feel free to supply tests to help the process along. Patches intended solely to add tests for existing features are also appreciated, even if those features are currently working fine; tests will make sure they stay that way.

Writing Documentation

When new features are added to Django, they should be documented so other developers can easily find and understand them. This documentation is usually provided in tickets alongside the patches that implement the new functionality. This way, when the patches are applied to trunk, documentation is included at the same time, so they’re always in sync with each other.

Documentation for Django is included in the SVN distribution, in reStructuredText3

format, located in the

directory. In addition to being in the distribution, these files are used to generate the pages found on the project’s Web site. Simply edit the appropriate file 2.

3.

A P P E N D I X

฀ CONTRIBUTING TO DJANGO

281

to include whatever new or modified features the ticket provides, and attach it to the ticket. If it’s being included as part of a new code patch as well, the two can be combined into a single patch, along with tests, if necessary.

As you’re writing documentation, be sure to read through what already exists to get a feel for the general style of writing that’s expected. Of course, the core developers are always the final authority on what makes it into trunk, though, so any necessary changes in style or structure can be made at that point. The closer you get to the appropriate style, the easier it will be to get the patch ready for check- in to the repository.

Development Sprints

Periodically, the Python community—or a subset of it, like Django—gets a large number of programmers together for sprints. These gatherings are a time for developers to set aside a chunk of time to devote exclusively for giving back to the community. Given the volunteer nature of open source development, time is often precious, so sprints are a way to schedule thousands of man- hours to occur simultaneously, crammed into a period ranging from a few hours to a few days.

Like many Python projects, the Django community schedules sprints periodically as a way to fix issues, commit new features, make decisions and allow developers to get together and have fun working with the code we love. These are often scheduled a few times a year, with several weeks—or even months—of advance notice, allowing people to get time off work or even arrange travel plans to meet in person. For information regarding community- wide sprints, keep an eye on the Django weblog.

For full community- wide sprints, there are often a few locations where local developers can get together in person and communicate directly, with the rest of the development community taking part over the Internet, communicating over IRC. These are usually scheduled by the core developers, and will sometimes coincide with other Python events, such as PyCon,4

an annual conference for Python enthusiasts, which dedicates time and space for sprints following its own activities.

Regional sprints, however, may be scheduled by any group of local developers at any time, and typically only involve local developers who can get together in person, and are often called to tackle very specific problems. Regional sprints in the past have focused on such tasks as supporting Oracle databases or geographic information systems. These may be scheduled by any member of the local community, and may even be coordinated with the local Python User Group5 to get additional help.

Publishing Code

For individual contributions, everyone is encouraged to publish any code or research they’d like to share with the rest of the community. Any time you’ve developed something of interest, simply write it up with comments and examples and share.

The key to sharing code or techniques with the community is making sure people know about it. After all, if nobody knows it’s out there, nobody can learn from it. There are a few ways 4.

5.

282

A P P E N D I X

฀ CONTRIBUTING TO DJANGO

to go about this, and they can be used individually or in combination, depending on how heavily you’d like it publicized.

฀

฀

฀

฀

฀

฀

6 by adding your blog to the list of those it com-

bines. Instructions for doing so are provided on the page.

฀

฀

฀

฀

฀ ฀

฀฀

฀

฀

฀

฀

฀

฀ ฀

฀ ฀

฀

accompanied by a blog post or software release, with the mailing list email providing a brief overview and a link to more information.

฀

฀

฀ ฀

฀ ฀

฀ ฀ ฀

฀

฀

7 along with a brief description of

what it does and how to use it.

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀

8

where other people can pick up where you leave off and make your code even better.

฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀

9 where it might be dis-

cussed on the air and promoted in that way.

These aren’t the only ways to share code, so if you have other ideas, feel free to do so however you like, but these are the most commonly used.

Releasing an Application

If you’ve written an application, beyond just a small snippet of code, the most useful way to share it is to bundle up the code, write some tests and documentation, and release it to the community as a complete application. One option for hosting the application is Google Code, which is favored by many in the community, but any hosting option will suffice.

Details of what it takes to make an application distributable are outside the scope of this book, but an excellent resource is James Bennett’s Practical Django Applications (Apress, 2008), which is written with exactly that situation in mind. Once you decide to distribute it, it’s a good idea to do so as a single compressed file as well as a freely viewable repository. That way, people who choose to keep up with your latest code can update regularly without having to wait for official releases.

An open repository is also a great way to allow others to observe your development process. A primary benefit of open source is the ability to learn from other developers, but that only works as long as developers are willing to share this type of activity. Many developers also document the decisions they made along the way, often in a blog where they can encourage fruitful discussion.

Another important thing to consider when releasing an application is that developers will likely use it and come to depend on its functionality. As such, they’re also likely to submit tickets, supply patches and request new features. Therefore, by releasing it to the public, you’re agreeing to continue to support and develop the application for the future. It’s generally considered bad form to simply release the application and forget about it.

6.

7.

8.

9.

A P P E N D I X

฀ CONTRIBUTING TO DJANGO

283

Also, since Django itself makes sure to keep its trunk version as stable as possible, most Django developers will expect third- party applications to do so as well. This isn’t an explicit requirement of releasing an application, but it would be beneficial to make sure that people can update your code regularly without fear of breakage. When backwards- incompatible changes are necessary, just be sure to document them thoroughly.

Index

Symbols and Numbers

excess, 26–28

@-style syntax, 29, 103

keyword, 26

* (asterisk), with argument name, 26

memoizing, 223–225

$ (dollar sign), 94

mixing types, 27

/ (forward slash), 168

multiple, with same name, 93

403 Forbidden response, 215

order of declaration, 39

404 error code, 236

passing collections of, 27–28

positional, 26

A

positional vs. keyword, 97

abstract attribute, 50

variable filters and, 145

address forms, 241

views and, 98–101

add() method, 203

See also specific arguments

add_to_builtins() function, 148

as_hidden() method

add_to_class() method, 46–47, 155

as_of() method, 276–278

admin application, 215, 232, 247–248, 256

as_p() method, 123

admin interface

associative arrays, 21

for contact application, 238

as_table() method, 123

for properties application, 247–251

asterisk, with argument name, 26

purpose of, 248

as_text() method, 124

admin.views.decorators.staff_member_

as_textarea() method, 124

required decorator, 102

as_ul() method, 123–125

admin.py module, 256

as_widget() method, 124

advertisements, 152–158

attname attribute, 57, 83

ALTER TABLE statement, 186

attr_class attribute, 74

app argument, 81

AttributeError, 22, 74, 139

AppCache class, 52–56, 77

attribute lookup, 140

app_label attribute, 46, 50, 54, 56, 158

attributes

application-based development, 10–11

classes, 18

application labels, 54

common, 37

applications

descriptors and, 34–36

coordinating, 231–252

field, 57–60

compatibility, 103

loading on demand, 82–88

dealing with individual models in, 55–56

ordering, 18

distributing, 103

setting, on models, 46–47

enhancing, 253–278

See also specific attributes

releasing, 282–283

attrs dictionary, 119

retrieving all, 51–54

augmenting functions, 25–33

retrieving single, 54–55

auth application, 81, 194, 232

reusable, 10–11

authenticate() method, 193–194

as seen by Django, 52–53

AUTHENTICATION_BACKENDS setting, 193

separation of, 231

AuthenticationMiddleware, 260

archives, model, 264–266

auth.decorators.login_required decorator,

args argument, 26, 27, 33, 39, 78, 97, 175

102

arguments

auth.decorators.permission_required

decorators and, 29–33

decorator, 102

default values for, 101

auth.decorators.user_passes_test decorator,

102

285

286

I N D E X

AUTH_PROFILE_MODULE setting, 155

can_use_chunked_reads feature, 185

AutoField attribute, 49, 190, 268

capfirst filter, 144

autoinc_sql(), 186

capitalization, recapitalize() function, 218

Cascading Style Sheets (CSS), 123, 153

B

CharField, 190

backend protocols, 183–323

choices attribute, 57

applied techniques, 205–211

chunks() method, 199

authentication, 193–194

ClamAV, 211

caching, 201–203

__class__ attribute, 37

context processors, 204–205

class declarations, 18–19

creating new structures, 189–191

class definitions, 13–14

database access, 183–193

classes

files, 194–199

attributes, 17–18

scanning for viruses, 210–211

building, 13–19

session management, 199–201

callable, 107

template loading, 203–204, 208–210

common attributes, 37

backwards compatibility, 170

declaring at runtime, 14

base class, 16–17

instances of, 13

BaseDatabaseFeatures, 185

metaclasses, 15–16

BaseDatabaseOperations, 185

vs. models, 88

BaseDatabaseWrapper, 184

namespaces, 13

base_fields, 113

new style vs. old style, 14

base module, 184

old style, 14, 37

Benevolent Dictator for Life (BDFL), 9–10

using base, with metaclasses, 16

blank attribute, 57

class information, on models, 47–48

{% block %} tag, 157

class objects, 13

block tokens, 136, 137

class_prepared signal, 77, 267

BooleanField, 190

cleaned_data dictionary, 116

Boolean values, 66

clean() method, 116–118

Borg pattern, 257–258

client.py module, 192

bound field objects, 124

closed attribute, 195

bound forms, 114

close() method, 22, 195

bracket syntax, 7

cls argument, 38, 63

bugs, reporting, 279

code

build_absolute_uri() method, 168

comments, 8

don’t repeat yourself (DRY), 5–6

C

loose coupling, 5

cache, model, accessing, 52–56

publishing, 281–282

cache.app_cache_ready() method, 52

readability, 6

cache argument, 224

resusable, 5–6

CACHE_BACKEND setting, 201

code divisions, 3–4

CacheClass.add() method, 203

column attribute, 58

CacheClass.delete() method, 202

CommaSeparatedIntegerField, 190

CacheClass.get_many() method, 203

comments, 8

CacheClass.get() method, 202

comment tokens, 136–137

CacheClass.has_key() method, 203

community, 8–12

CacheClass.set() method, 202

community contributions, 279–283

cache.get_apps() method, 53–56

comparison operators, 189

cache object, 202, 203

compilation functions, 137, 147, 150–151

cache.cache_page decorator, 101

compile_string() function, 134

cache.never_cache decorator, 101

compress() method, 121–122

cache.get_models() method, 55–56

computer viruses, scanning for, 210–211

caching, 201–203

configuration mistakes, 213

manual, 202–203

configuration options, on models, 50–51, 89

specifying a backend, 201–202

connect() method, 228

callables, 20–21, 27

ContactEditorForm, 234–235

__call__ method, 20, 107, 108

contact information, editing, 237–238

I N D E X

287

contact management, 233, 235–238

cross-site scripting, 116

Contact model, 232–233

cull_frequency argument, 202

contacts application, 231–241

current context, 139

admin interface, 238

current user, recording, 253–263

ContactEditorForm, 234–235

admin approach, 256

UserEditorForm, 233

CurrentUserField, 257–263

Contact model, 232–233

with save() method, 253–254

edit_contact view, 235–238

thread-local approach, 254–256

URL configuration, 238–241

CurrentUserField, 257–263, 269

contacts.forms.ContactEditorForm, 234–235

CurrentUserMiddleware, 259–263

contacts.forms.UserEditorForm, 233

curry() function, 30, 222–223

contacts.models.Contact, 232–233

cursor() method, 189

contacts.views.edit_contact, 235–240

custom backends, passing information to,

__contains__() method, 21

194

content, file-like access to, 170

custom fields, 62–63, 117–119

content argument, 169

custom widgets, 119–122

content attribute, 173

contents attribute, 136

D

content tokens, 135–136

data

content-type, setting of responses, 105

accessing submitted, 165

content_type argument, 169

altering behavior, in fields, 64–68

Context object, 138–141, 151

historical records of, 264–266

context processors, 141, 156–157, 204–205

obtaining values from posted, 120

CONTEXT_PROCESSORS setting, 204–205

pickling and unpickling, 83–86

contexts

splitting across multiple widgets, 121–122

current, 139

storing raw, 83

vs. namespaces, 138

tracking users for modify, 253–263

template, 138–141

validation of, 115–118

contribute_to_class() method, 46, 59–60, 63–

data attribute, 116, 124

64, 73, 77, 86–87, 113, 258, 266–267

database backends . See backend protocols

contribute_to_related_class() method, 64

database behavior, controlling, 68–71

Cookie header, 102

DatabaseClient class, 192

cookies

DATABASE_ENGINE, 192

deleting, 172

DatabaseError, 193

digital signatures, 179

DATABASE_OPTIONS setting, 184

security, 172

databases

signing and validating, 179–181

accessing, 183–193

viewing, 172

interaction of fields with, 68–71

cookies attribute, 168, 172

DatabaseWrapper class, 184–189

copy_fields() method, 268–269

DatabaseWrapper.features, 184–185

copy module, 268

DatabaseWrapper.ops, 185–188

core developers, 9

data mapping, with Context object, 138–141

core exceptions, 213–216

data structures, 220–222

ImproperlyConfigured, 213

creating new, 189–191

MiddlewareNotUsed, 214

introspection of existing, 191–192

MultipleObjectsReturned, 214–215

MergeDict, 220–221

ObjectDoesNotExist, 215

MultiValueDict, 221

PermissionDenied, 215–216

SortedDict, 222

SuspiciousOperation, 216

data types

ViewDoesNotExist, 216

handling of, by fields, 64–68

cPickle module, 83–86

support for complex, with SubfieldBase,

created_models argument, 81

66–68

create_history_model() method, 267–268, 271

DATA_TYPES dictionary, 189–190

create() method, 79, 200

DATA_TYPES_REVERSE dictionary, 192

creation_counter technique, 48

date_extract_sql() method, 186

creation module, 186, 189–190

DateField, 190

cross-site request forgery, 116

date filter, 144

288

I N D E X

datetime_cast_sql() method, 186

Django

DateTimeField, 190

as Python, 13

date_trunc_sql() method, 186

community, 8–12

db_column attribute, 58

contributing to, 279–283

db_index attribute, 58

declarative syntax, 17–19, 46

db_table attribute, 51

framework management, 9–10

db_tablespace attribute, 51, 58

introduction to, 12

db_type() method, 60, 68

loose coupling, 5

DecimalField, 190

MVC pattern and, 2–4

declarative syntax, 17–19, 46

philosophy, 1–8

decompress() method, 122

versions, 11

decorator_from_middleware() function, 177,

weblog, 10

262–263

django.contrib.auth application, 81, 194, 232

decorators

django.contrib.contenttypes application, 81

applying, 28–33, 102–103

django.contrib.sites application, 81

arguments and, 29–33

django.core.signals.got_request_exception, 178

configuration options, 176–177

django.core.signals.request_finished, 178

dual-format, 109–111

django.core.signals.request_started, 178

extra arguments, 29

django.core.exceptions.

for making forms generic, 131–132

ImproperlyConfigured, 213

permalink, 95–96

django.core.exceptions.MiddlewareNotUsed,

scope, 176

214

tasks of, 103

django.core.exceptions.

uses of, 104–106

MultipleObjectsReturned, 214–215

using middleware as, 177

django.core.exceptions.ObjectDoesNotExist,

view, 101–106, 176–177

215

wrappers and, 225

django.core.exceptions.PermissionDenied,

writing, 103–110

215–216

default argument, 58, 250

django.core.exceptions.SuspiciousOperation,

default_error_messages dictionary, 117–118

216

DEFAULT_FILE_STORAGE setting, 198

django.core.exceptions.ViewDoesNotExist,

default values, handling, 39

216

deferrable_sql() method, 186

django.db.backends, 184–189

delete_cookie() method, 172

DatabaseWrapper class, 184–189

delete_file() method, 73–74

DatabaseWrapper.features, 184–185

DELETE method, 165

DatabaseWrapper.ops, 185–188

delete() method, 76, 80, 200, 202

django.db.models.Model, 154

DELETE request, 164

django.dispatch.dispatcher, 226

descriptors, 34–36, 82

django.template.loader.get_template(),

for loading attributes on demand, 85–88

141–142

for unpickling data, 85–86

django.utils.datastructures.MergeDict,

development sprints, 281

220–221

dict() function, 128–129, 220

django.utils.datastructures.MultiValueDict,

__dict__ attribute, 36

165, 167, 221

dictionaries, 21

django.utils.datastructures.SortedDict, 222

lookups, 139–140

django.utils.functional.curry, 222–223

merging multiple, 220–221

django.utils.functional.memoize, 223–225

ordering, 222

django.utils.functional.wraps, 225–226

referencing multiple values, 221

__doc__ attribute, 37, 40

with templates, 138–139

docstrings, 8, 40

dictionary access, 167, 170

documentation

diff files, 280

of views, 104

digital signatures, 179

writing, 280–281

directory name, 71–72, 209

documentation, 8, 11

discard() method, 7

DoesNotExist class, 46, 215, 275–276

dispatcher, 226–229

dollar signs, 94

dispatch_uid argument, 228, 260

domain argument, 171

I N D E X

289

Don’t Repeat Yourself (DRY) philosophy, 5–6,

FieldDoesNotExist exception, 49

99

field methods, 60–62

drop_foreignkey_sql() method, 186

FieldRegistry, 258

drop_sequence_sql(table) method, 186

fields

dual-format decorator, 109–111

altering data behavior, 64–68

duck-typing, 19–25, 37, 62

attributes, 57–60

checking contents of, for accuracy, 65–66

E

complex datatype support, 66–68

editable attribute, 58

controlling widgets for, 118–119

edit_contact view, 235– 240

copying, 268–269

empty_fetchmany_value attribute, 185

custom, 117–119

empty_strings_allowed attribute, 58

declaring and identifying, 113–114

EMPTY_VALUES tuple, 121

duck-typing principles with, 62

encoding attribute, 169

interaction of, with database, 68–71

endjinja tag, 149

inventing or extending, 62–63

environment.get_template() function, 210

mapping form to model, 126

ErrorList class, 124

methods, 60–62

error messages, 236

populating, in Python, 250

custom, for forms, 124–125

primary key, 49–50

with form validation, 117

processing of, by ModelBase class, 63–64

error_row argument, 123

record-keeping, adding, 269–270

errors, 6, 71

splitting data across multiple widgets,

errors attribute, 124

121–122

errors dictionary, 116

storing values of, 66

errors_messages attribute, 118

string values for, 115

errors_on_separate_row argument, 124

subclassing, 62–71

exceptions

tracking, for current user data, 257–263

catching, with view decorators, 105–106

using, 57–62

core, 213–216

validating, 117–118

ImproperlyConfigured exception, 213

values in, storing, 126–128

MiddlewareNotUsed, 214

widgets for, 119–125

MultipleObjectsReturned, 46, 214–215

fields attribute, 48–49, 114

ObjectDoesNotExist, 215

field types

PermissionDenied, 215–216

attributes, 190

process_exception() method, 175–176

basic, 190–191

SkipFile, 197

mapping of, 189

StopFutureHandlers, 197

File class, 74–76, 194–195

StopUpload, 197

FileField class, 71–76, 190

SuspiciousOperation, 216

file-like objects, 22

TemplateDoesNotExist, 134, 141, 204

filename, 72

TemplateEncodingError, 135

FilePathField, 190

with templates, 134–135

File.chunks() method, 195

TemplateSyntaxError, 134

File.closed attribute, 195

VariableDoesNotExist, 135

File.close() method, 195

ViewDoesNotExist, 216

file_complete() method, 196

excess arguments, 26–28

File.DEFAULT_CHUNK_SIZE attribute, 195

exists() method, 200

File.flush() method, 195

expires argument, 171

File.mode attribute, 195

{% extends %} tag, 157, 161

File.multiple_chunks() method, 195

File.name attribute, 195

F

File.open() method, 195

Feature model, 243, 244, 246, 250

File.read() method, 195

features, adding to templates, 143–148

File.readlines() method, 195

field attributes, 57–60, 124

File.seek() method, 195

field_cast_sql(db_type), 186

File.size attribute, 195

Field class, 62

File.tell() method, 195

field definitions, of models, 48–49

File.write() method, 195

290

I N D E X

File.xreadlines() method, 195

unbound, 114

files, 22, 194–199

for user management, 233–234

deleting, 73–74, 76

validation, 113–128

File class, 194–195

for tag, 146

as input for forms, 115

forward slash (/), 168

management of, 71–76

frequently asked questions (FAQs), 11–12

open, 75

full_clean() method, 116

path of, 74–75

fulltext_search_sql(field_name), 186

saving, 73, 76

functional utilities

scanning incoming for viruses, 210–211

curry, 222–223

size of, 75

default values for, 222

storing, 198–199

memoize, 223–225

uploads, 196–197

wraps, 225–226

FILES attribute, 115, 166

functions

FileSystemStorage class, 199

augmenting, 25–33

FileUploadHandler.file_complete() method,

common attributes, 37

196

decorating (wrapping), 28–33

FileUploadHandler.__init__() method, 196

excess arguments, 26–28

FileUploadHandler.new_file() method, 196

idempotent, 165, 223

FileUploadHandler.receive_data_chunk()

introspecting, 225

method, 196

partial application of, 29–30

FileUploadHandler.upload_complete()

return values, 227

method, 197

specifying arguments for, to be used later,

FILE_UPLOAD_HANDLERS setting, 196

222–223

filter() method, 69, 146, 276

views and, 107

filters, variable, 144–146

wrappers around, 225–226

filter tag, 146

See also specific functions

finalize() method, 267–268

function signatures, 39

flatten_date() method, 60

functools module, 29

FloatField, 191

functools.partial function, 222–223

flush() method, 195

func value, 33

for block tag, 124

ForeignKey, 154, 155

G

form element, 123

generate_filename() method, 72–73

formfield() method, 60

generators, 24–25

form fields. See fields

generic views, 99–101

form objects, iteration of, 124

get_absolute_url() method, 95

forms, 113–132

get_apps() method, 53–56

accessing individual fields, 124

get_attname_column() method, 61

binding to user input, 114–117

get_attname() method, 60, 83

bound, 114

GET attribute, 166

custom error messages, 124–125

GET method, 166, 167

custom fields, 117–119

getattr() method, 46

customizing markup, 123–124

get_available_name() method, 199

custom widgets for, 119–122

get_cache_name() method, 61

declaring and identifying fields, 113–114

get_choices() method, 61

decorators with, 131–132

get_db_prep_lookup() method, 61, 69–71, 84

defining HTML behavior, 119–125

get_db_prep_save() method, 61, 69

hashes, 127–128

get_db_prep_value() method, 69

instantiation of, 114–115

get_default() method, 61

manipulation, 116

get_directory_name() method, 71–72

vs. models, 113

getdoc() function, 40

outputting in template, 123

get_expiry_age() method, 201

pending and resuming, 125–132

get_expiry_date() method, 200

presentation of, 123

get_extra_fields() method, 270

reconstituting, 128–129

get_filename() method, 72

specifying markup in template, 124

get_host() method, 168

I N D E X

291

get_indexes() function, 192

_html_output() method, 123

get_internal_type() method, 61, 64–65, 68

HTTP middleware

__getitem__() method, 21, 25

configuration options, 176–177, 177

get_latest_by attribute, 51

deciding between view decorators and,

getlist() method, 221

176–177

get_many() method, 203

for handling signed cookies, 180–181

__get__() method, 35, 85–86, 271

process_exception(self, request,

get() method, 7, 202, 214, 215

exception), 175–176

get_models() method, 55–56

process_request(self, request), 174

get_nodes_by_type() method, 137

process_response(self, request, response),

get_or_create() method, 128

175

get_query_set() method, 274

process_view(self, request, view, args,

get_relations() method, 192

kwargs), 174–175

GET request, 164, 166

scope, 176

get_source() method, 206–192

using, as decorators, 177

get_table_description() function, 192

writing, 174–177

get_table() function, 192

http.require_http_methods decorator, 102

get_template() function, 141–142

HTTP (Hypertext Transfer Protocol), 92,

get_template_source() function, 208

163–181

get_text_list() function, 217

applied techniques, 178–181

get_user() method, 193

HttpRequest class, 163–169

get_valid_name() method, 199

HttpResponse class, 169–173

global variables, module-level, 256

requests and responses, 163–173

Google Code, 10–11

HTTP-related signals, 178

got_request_exception, 178

HttpRequest class, 163–169

gzip.gzip_page decorator, 101–102

accessing submitted data, 165

dictionary access, 167

H

HttpRequest.build_absolute_uri(), 168

hasattr() method, 157

HttpRequest.COOKIES, 168

has_default() method, 61

HttpRequest.encoding, 169

hashes, 21, 127, 128

HttpRequest.FILES, 166

has_key() method, 203

HttpRequest.GET, 166–168

headers, dictionary access to, 170

HttpRequest.is_ajax() method, 169

HEAD request, 164

HttpRequest.is_secure() method, 168

help resources, 11

HttpRequest.META, 167

help_text attribute, 58

HttpRequest.method, 165

help_text_html argument, 124

HttpRequest.path, 165

hidden_widget attribute, 119

HttpRequest.POST, 166

historical_object attribute, 271

HttpRequest.raw_post_data, 166–167

historical records

idempotent methods, 165

adding meta options, 271

safe methods, 165

intended usage, 264–266

HttpRequest object, 98

keeping, 263–278

HTTP requests

step 1, copy the model, 267–271

including aspects of, 141

step 2, register signal handlers, 272

maintaining state, with cookies, 171–172

step 3, assign a manager, 272–278

views and, 98

HistoricalRecords.finalize() method, 267–268

HttpResponseBadRequest, 173

HistoricalRecords object, 267, 272

HttpResponse class, 169–173

history attribute, 264

creating a response, 169–170

history_object attribute, 265–270

dictionary access to headers, 170

history_type field, 272

file-like access to content, 170

Holovaty, Adrian, 10

HttpResponse.content, 173

host domain, 92

HttpResponse.cookies, 172

HTML (Hypertext Markup Language),

HttpResponse.delete_cookie() method, 172

rendering with custom widget, 119–120

HttpResponse.set_cookie() method, 171–172

HTML behavior, defining with widgets,

HttpResponse.status_code, 170–171

119–125

subclasses, 173

292

I N D E X

HttpResponseForbidden, 173

is_bound attribute, 114

HttpResponseGone, 173

is_hidden attribute, 124

HttpResponseNotAllowed, 173

isinstance() function, 38

HttpResponseNotFound, 173, 236

is_secure() method, 168

HttpResponseNotModified, 173

issubclass() function, 38

HttpResponse object, 19, 22, 99, 143

is_usable attribute, 207

HttpResponsePermanentRedirect, 173

is_valid() method, 115, 116–117

HttpResponseRedirect, 173

iterables, 23–25

HTTP responses

iterators, 23

codes, 215

iter() function, 23

process_reponse() method, 175

__iter__() method, 23, 25

setting content-type of, 105

specialty response objects, 173

J

views and, 98

JavaScript Object Notation (JSON), 110, 217

HttpResponseServerError, 173

javascript_quote() function, 217

HttpResponse.set_cookie() method, 171–172

jinjadir template tag, 210

HttpResponse.status_code, 170–171

JinjaNode class, 150–151, 208

Hypertext Transfer Protocol. See HTTP

jinja template tag, 149

JINJA_TEMPLATE_DIRS setting, 206, 208

I

Jinja template engine, 148–152

ImageField class, 71, 74

compiling to a node, 150–151

ImageFile class, 74

converting tokens to strings, 149–150

ImproperlyConfigured exception, 213

preparing template, 151–152

include() function, 94

template loading, 205–210

idempotent methods, 165, 223

JSON (JavaScript Object Notation), 110, 217

IndexError, 275

index lookup, 140

K

inheritance

Kaplan-Moss, Jacob, 10

model, 232

KeyError, 88, 139, 167

multiple, 154

keyword arguments, 26, 97

template, 157

kwargs argument, 26–27, 78, 97, 175

__init__() method, 16, 121, 137,177, 196, 214

__init__.py module, 52

L

inlines tuple, 249

label_tag() method, 124

in operator, 21

Lambda-style functions, 131

inspect.getargspec() function, 39

last_executed_query(), 187

inspect module, 36–40

last_insert_id(), 187

INSTALLED_APPS setting, 47–48, 52–53, 56,

length filter, 144

81, 144

__len__() method, 25

installed attribute, 48, 52, 56

Lexer objects, 135–136

instance argument, 35, 73

Lexer.tokenize() method, 135–136

instance data, keeping track of, 36

Library class, 146

IntegerField, 191, 268

line-endings, converting, to Unix-style, 217

IntegrityError, 193

list_detail view, 251

interactivity, 113

listdir() method, 199

InterestedParty model, 246–247

listed() method, 251

Internet Relay Chat (IRC) channel, 12

listeners

interprets_empty_strings_as_nulls feature,

defining, 227–228

185

registration of, 226, 228

interval column type, 66–67

list-index lookup, 140

introspection, 6, 36–40, 175, 191–192

lists, 217

introspection.py module, 191–192

loadjinja template tag, 208–210

InvalidTemplateLibrary exception, 135

load() method, 200

IPAddressField, 191

{% load %} tag, 143, 144, 146, 148

is_ajax() method, 169

load_template_source() function, 203–204,

is_anonymous() method, 233

208

is_authenticated() method, 233

load_template_source.is_usable, 204

I N D E X

293

localflavor package, 234

process_response(), 175

local() function, 254–256

process_view(), 174–175

login_required decorator, 102

scope, 176

lookup_cast(lookup_type) function, 187

using, as decorators, 177

lookup_type argument, 69–70, 186

writing, 174–177

loose coupling, 5

MiddlewareNotUsed exception, 214

middleware.py module, 259

M

mimetype argument, 143, 170

mailing addresses, 242

mixin argument, 198

mailing lists, 12

mode attribute, 75, 195

manage.py dbshell command, 192

ModelBase class, 46, 52, 63–64

manage.py validation, 213

model cache, accessing, 52–56

manager, assigning for historical records,

model classes, processing of, 46–47

272–278

model fields

Manager class, 273

altering data behavior, 64–68

manager module, 272–278

attributes, 57–60

ManyToManyField, 155

complex datatype support, 66–68

maps, 21

copying, 268–269

max_age argument, 171

definitions, 48–49

max_entries argument, 202

duck-typing principles with, 62

max_length attribute, 58, 190

interaction of, with database, 68–71

max_name_length() function, 187

inventing or extending, 62–63

median() function, 225

mapping form fields to, 126

memoize() function, 223–225

methods, 60–62

MergeDict, 220–221

model class registration, 63–64

META attribute, 167

primary key, 49–50

_meta attribute, 47–48

subclassing, 62–71

configuration options, 50–51

using, 57–62

fields attribute of, 48–49

ModelForm subclass, 233, 234, 248

installed attribute, 48, 52, 56

model instances

pk, 50

accessing true, 271

_meta.auto_field, 50

recording users making changes to,

Meta class, 89, 271

253–263

metaclass arguments, 15

retrieving most recent copy of, 274–276

__metaclass__ attribute, 15, 16, 41

retrieving, from specific point in time,

metaclasses, 15–16, 41–42

276–278

base class with, 16

models, 3, 45–90

for processing model definitions, 46

accessing model cache, 52–56

metadata, 36

applied techniques, 82

_meta.get_field() method, 49

vs. classes, 88

_meta.has_auto_field attribute, 50

class information, 47–48

metaprogramming, 15

configuration options, 50–51, 89

method attribute, 164

copying, 267–271

methods

creating dynamically, at runtime, 87

as views, 98

dealing with individual, 55–56

field, 60–62, 165

definitions, 46

idempotent, 165

determining installed, 47–53

safe, 165

file management, 71–76

middleware classes, decorators and, 105

vs. forms, 113

MIDDLEWARE_CLASSES setting, 214, 216

getting information about, 47–56

middleware, HTTP

inheritance, 232

configuration options, 176–177

INSTALLED_APPS setting, 47–48, 52–53

deciding between view decorators and,

introduction to, 45

176–177

loading attributes on demand, 82–88

for handling signed cookies, 181

processing of model classes, 46–47

process_exception(), 175–176

setting attributes on, 46–47

process_request(), 174

signals, 76–82

294

I N D E X

subclassing fields, 62–71

O

tracking changes to

object_detail view, 240, 251

historical records, 263–278

ObjectDoesNotExist exception, 215

recording current user, 253–263

object_list view, 239

user profile, 155–156

object-relational mapper (ORM), 17, 45

Model-Template-View (MTV) pattern, 3

objects

Model-View-Controller (MVC) pattern, 2–4, 97

class, 13

modules, centralized access to, 17

file-like, 22

__module__ attribute, 16, 27, 48, 88–89, 269

pickled, 83–88

Morsel object, 172

request, view decorators and, 104

most_recent() method, 265–266, 276

response, 173

mount point, 42–43

using as views, 107–108

multiple arguments, with same name, 93

object types, 19–20

multiple_chucks() method, 195

callables, 20–21

multiple inheritance, 154

checking for specific, 38–39

MultipleObjectsReturned exception, 46,

descriptors, 34–36

214–215

files, 22

MultiValueDict, 165, 167, 221

getting arbitrary, 37–38

MultiValueField, 121

identifying, 37–39

MultiWidget, 121

iterables, 23–25

Mutagen, 213–214

sequences, 25

MySQL, 183

old-style classes, 14, 37

OneToOneField, 232

N

OpenID, 193

name attribute, 59, 63, 83, 134, 195

OpenID URL, 179

__name__ attribute, 37, 38, 48, 54

open() method, 75, 195, 199

name lookups, 139

operators attribute, 189

namespace dictionaries, 18

Options object, 89–90

namespaces

Oracle, 183

vs. contexts, 138

ordering tuple, 51

nested, 138

order_with_respect_to attribute, 51

needs_datetime_string_cast feature, 185

origin, 135–136

nested namespaces, 138

origin argument, 134

__new__() method, 16

ORM (object-relational mapper), 17, 45

new_file() method, 196

os.path.join() method, 72

news aggregator, 10

owner argument, 35

new-style classes, 14

next() method, 23–24

P

next_token() method, 137

paginate_by argument, 239, 251

Node class, 137

parse() method, 136–137, 151

node compilation functions, 137, 147,

Parser object, 135, 136–137

150–151

Parser.next_token() method, 136–137, 150

nodelist attribute, 134

Parser.parse() method, 135, 150

nodes

parse_until argument, 137

compiling to, 150–151

partial objects, 29

parsing tokens into, 136–137

patches, supplying, 279–280

template nodes, 137

path argument, 171, 199

template tags and, 146

path() method, 74–75

nodetype argument, 137

patterns() function, 93

no_limit_value() function, 187

PEP. See Python Enhancement Proposal

normalize_newlines() function, 217

PEP-8, 6

normal_row argument, 123

PEP-20, 2

NotImplementedError, 22, 192, 198

PEP-249, 184, 193

now tag, 146

PEP-333, 163

null attribute, 59

permalink decorator, 95–96

NullBooleanField, 191

PermissionDenied exception, 215–216

num_args argument, 224

permission_required decorator, 102

I N D E X

295

permissions tuples, 51

Python

Peters, Tim, 2

API for accessing databases provided by,

phone2numeric() function, 218

184

PhoneNumberField, 191

applied techniques, 41–44

phone numbers, converting text to numbers,

augmenting functions, 25–33

218

class building in, 13–19

pickled objects, storing and retrieving, 83–88

declarative syntax, 17–19

pickle() method, 85

descriptors, 34–36

pickling modules, 83–86

Django as, 13

pk attribute, 50

duck-typing protocols, 19–25, 37

pk_default_value() method, 187

exceptions, 6

plugin architecture, 42–44

introspection, 36–40

pop() method, 138, 139

loose coupling, 5

port numbers, 92

philosophy, 2

positional arguments, 26, 97

pickling modules, 83–86

PositiveIntegerField, 191

populating fields in, 250

PositiveSmallIntegerField, 191

readability, 6

post_delete signal, 73, 80, 272

templates and, 133–135

PostgreSQL, 66–67, 183

Python dictionaries, 18

post_init signal, 78

Python Enhancement Proposal (PEP), 2, 6,

POST request, 164, 166, 167

163, 184, 193

post_save signal, 79, 272

Python protocols, 19–25

post_syncdb signal, 80–82

callables, 20–21

Practical Django Applications (Bennet),

dictionaries, 21–22

282

iterables, 23–25

pre_delete signal, 80

pre_init signal, 78

Q

prep_for_like_query(x), 187

query_class() method, 187

prepopulated_fields attribute, 250

QueryDict, 165

pre_save() method, 61, 250

QuerySet, 238–240

pre_save signal, 79, 259

queryset argument, 240

primary_key attribute, 49, 59

query strings, parading, 165

primary key fields, of models, 49–50

quote_name() method, 187

problems, reporting, 279

process_exception() method, 175–176

R

process_response() method, 175

random_function_sql() method, 187

process_request() method, 174, 259, 261

rapid development, 3

process_view() method, 174–175, 215

raw data, storage of, 83

properties application, 241–252

raw_post_data attribute, 166–167

admin interface, 247–250

readability, 6

Feature model, 243, 244, 246, 250

read() method, 22, 195, 199

InterestedParty model, 246–247

readlines() method, 195

PropertyFeature model, 246

read-only views, 251

Property model, 241–250

real estate Web site (example), 231–252

URL configuration, 251–252

contacts application, 231–241

properties.models.Feature, 243, 244, 246, 250

admin interface, 238

properties.models.InterestedParty, 246–247

contacts, 231–238

properties.models.Property, 241–246

URL configuration, 238–241

properties.models.PropertyFeature, 246

properties application, 241–252

PropertyAdmin model, 247–250

admin interface, 247–250

property decorator, 241

Feature model, 243, 244, 246, 250

PropertyFeature model, 246

InterestedParty model, 246–247

PropertyForm, 249–250

PropertyFeature model, 246

PropertyManager class, 245, 251

Property model, 241–250

Property model, 238, 241–250

URL configuration, 251–252

push() method, 138, 139

recapitalize() function, 218

PUT request, 164, 165

receive_data_chunk() method, 196

296

I N D E X

receiver argument, 228

sender argument, 73

record-keeping fields, 269–270

senders, 227–228

references, 229

send() method, 227

regex_lookup() method, 188

sequence_reset_sql() method, 188

registration.py module, 257–258

sequences, 25

_registry dictionary, 258

serialize attribute, 59

regular expressions

_session attribute, 200

dollar signs in, 94

SessionBase class, 200–201

URL patterns and, 93–95

SESSION_ENGINE, 200

related_fields_match_type feature, 185

session_key attribute, 200

rel attribute, 59

session management, 199–201

render() method, 119, 121, 137–138

SessionStore class, 200

render_to_response() function, 143

SessionStore.create() method, 200

render_to_string() function, 142–143

SessionStore.delete() method, 200

RequestContext object, 141, 143, 156, 204

SessionStore.exists() method, 200

request_finished, 178

SessionStore.load() method, 200

request objects, view decorators and, 104

SessionStore.save() method, 200

request.POST dictionary, 115

set_attributes_from_name() method, 61

requests. See HTTP requests

setattr() method, 36, 46–47, 63–64

request_started, 178

set_cookie() method, 172

reserved names, 18–19

__setitem__() method, 21, 25

resolve() method, 141

set() method, 202

resource path, 92

__set__() method, 35, 85–86

resources, 10

signal handlers, registering, 272

response objects, specialty, 173

signals, 76–82

responses. See HTTP responses

capturing return values, 227

response status codes, 170–171

class_prepared, 77, 267

reStructuredText format, 280–281

defining, 226–227

return values, capturing, 227

HTTP-related, 178

reusable applications, 10–11

listeners and, 227–228

reverse() utility function, 96–97, 108

post_delete, 80, 272

Rossum, Guido van, 10

post_init, 78

row_ender argument, 124

post_save, 79, 272

rules

post_syncdb, 80–82

defining, 7–8

pre_delete, 80

documenting, 8

pre_init, 78

runshell() method, 192

pre_save, 79

runtime, creating models dynamically at, 87

sending, 227–229

workings of, 226–229

S

simplejson, 110

save argument, 76

simple_tag() method, 147–148

save_form_data() method, 61, 73

site memberships, 152

save() method, 76, 80, 199, 200, 253–254

site-wide themes, 156–157

savepoint_commit_sql() method, 188

size attribute, 195

savepoint_create_sql() method, 188

size() method, 75

savepoint_rollback_sql() method, 188

SkipFile exception, 197

SECRET_KEY setting, 179

skip_past() method, 137

secure argument, 171

slug fields, 214, 240, 250

security

slugs, 239, 250, 252

cookies and, 172

slug variables, 240

digital signatures, 179

SmallIntegerField, 191

with user input, 116

smart_split() function, 218

with user-submitted themes, 158–160

SortedDict, 222

select_template() function, 142

special characters, escaping, 217

seek() method, 195

specialty response objects, 173

self.instance attribute, 275

split_contents() method, 147

self.model attribute, 273

sprints, 281

I N D E X

297

sql_flush() method, 188

TemplateDoesNotExist exception, 134, 141,

sql_for_tablespace() method, 188

204, 206

SQL injection, 45, 116

TemplateEncodingError exception, 135

SQLite, 183

template engine

staff_member_required decorator, 102

embedding another, 148–151

start_transaction_sql() method, 188

Jinja, 148–151

status argument, 169

template features, 143–148

status_code attribute, 170–171, 173

adding to all templates, 148

status field, 243

package structure, 143–144

StopFutureHandlers exception, 197

template tags, 146–148

StopUpload exception, 197

variable filters, 144–146

StorageBase class, 198, 199

template loaders, 203–204, 210

Storage.delete() method, 198

TEMPLATE_LOADERS setting, 203, 208

Storage.exists() method, 198

template_name, 238

Storage.get_available_name() method, 198,

template nodes, 137

199

Template objects

Storage.get_valid_name() method, 198,

arguments, 134

199

string acceptance by, 135

Storage.open() method, 198, 199

Template.render() method, 151

Storage.path() method, 198

templates, 4, 133–162

Storage.save() method, 198, 199

adding features for, 143–148

Storage.size() method, 198

alternate engine for, 148–151

Storage.url() method, 199

applied techniques, 148–162

storage systems, 198–199

content tokens, 135–136

stringfilter decorator, 145

contexts, 138–141

strings

dictionaries with, 138–139

as input for forms, 115

enabling user-submitted themes, 152–162

breaking apart, 218

exceptions, 134–135

converting tokens to, 149–150

introduction to, 133

converting to mixed case, 218

parsing tokens into nodes, 136–137

in templates, 134

processing, 135–137

truncating, 219, 219

providing links to views, 96

Unicode, 135

Python code and, 133–135

strong references, 229

Python objects and, 133–134

Structured Query Language (SQL), 45

rendering, 138

structures . See data structures

RequestContext object, 141

subclasses, tracking, 41–42

retrieving, 141–143

subclassing, of fields, 62–71

setting up, to use themes, 157–158

SubfieldBase class, 66–68

shortcuts for loading and rendering,

Subversion, 10–11

142–143

SuspiciousOperation exception, 216

variable resolution, 139–141

syncdb command, 80

view, 98, 109

syntax

template_string argument, 134

@-style, 29, 103

TEMPLATE_STRING_IF_INVALID setting, 139

bracket, 7

TemplateSyntaxError exception, 134

declarative, 17–19, 46

template tags, 134, 143–148

common, 146

T

package, 143–144

table structures

shortcut for, 147–148

creating new, 189–191

simple, 146–147

introspection of existing, 191–192

tests, 8, 280

tag functions, 147

text, wrapping, 219

tell() method, 195

TextField, 126, 191

Template class, 134, 154

text modification tools, 217–219

TEMPLATE_CONTEXT_PROCESSORS

get_text_list(), 217

setting, 157

javascript_quote(), 217

TEMPLATE_DEBUG setting, 134

normalize_newlines(), 217

298

I N D E X

phone2numeric(), 218

U

recapitalize(), 218

unbound forms, 114

smart_split(), 218

Unicode characters, escaping, 217

truncate_html_words(), 219

__unicode__() method, 89, 125, 271

truncate_words(), 219

Unicode strings, 135

wrap(), 219

Uniform Resource Locators (URLs), 91–97,

TextNode, 137

111

text tokens, 136, 137

designing clean, 92

THEME_BLOCKS setting, 159

include() function, 94

THEME_CONTAINER_BLOCK setting, 159

patterns() function, 93

THEME_CONTEXT_NAME setting, 157

resolving, to views, 94–95

THEME_PROFILE_FIELD setting, 155–156

resolving views to, 95–97

themes

standard configuration, 92–94

enabling user-submitted, 152–162

vs. URIs, 91–92

example, 160–162

url() function, 94

field name, 155

views and, 91

multiple inheritance with, 154

unique attribute, 59

setting up models for, 153–156

unique_for_date attribute, 59

setting up templates to use, 157–158

unique_for_month attribute, 59

site-wide, supporting, 156–157

unique_for_year attribute, 59

storing in database, 154

unique_together attribute, 51

validating and securing, 158–160

unpickle() method, 84–85

THEMES_EXTENDS setting, 159

update_can_self_select feature, 185

third-party applications, releasing, 282–283

update_users() method, 259–260

threading module, 254–256

upload_complete() method, 197

thread-local dictionary, 254–256

UploadedFile object, 73, 197

tickets, reporting, 279

uploaded files, 196–197, 210–211

tight coupling, 5

upload handlers, 196–197

timedelta object, 66–67

upload_to attribute, 71–72

TimeField, 191

urlconf argument, 97

timeout argument, 202

URL configurations, 4

TOKEN_BLOCK, 136

for contacts application, 238–241

TOKEN_COMMENT, 136

for properties application, 251–252

token.split_contents() method, 208

view configuration and, 107

tokens

url() method, 75, 94

content, 135–136

URL patterns, 93, 94, 108

converting to strings, 149–150

for contacts application, 238–241

parsing into nodes, 136–137

for properties application, 251–252

text, 136, 137

url template tag, 96

TOKEN_TEXT, 136

UserChangeForm, 234

token_type attribute, 136

UserCreationForm, 234

TOKEN_VAR, 136

user credentials, 193

tools, 213–229

UserEditorForm, 233

core exceptions, 213–216

user ID, 193

data structures, 220–222

user information, storage of, 155, 194

functional utilities, 222–226

user input

signals, 226–229

binding forms to, 114–117

text modification, 217–219

risks associated with, 116

to_python() method, 61, 66, 67

validating, 115–118

TrackedClass, 41–42

user management, forms for, 233–234

tracking trunk, 11

User model, 232–233

truncate_html_words() function, 219

usernames, for contacts, 235–236, 239

truncate_words() function, 219

user_passes_test decorator, 102

tuples, 93, 95

user profile model, 155–156

TypeError, 71, 93, 177

user-submitted themes

type() function, 15, 87–88

advertisements and, 152

type object, 14, 15, 37

enabling, 152–162

I N D E X

299

example, 160–162

uses of, 104–106

setting up models, 153–156

writing, 103–106

site-wide, 156–157

ViewDoesNotExist exception, 216

validating and securing, 158–160

viewname argument, 97

uses_custom_query_class feature, 185

views, 3–4, 91, 97–111

uses_savepoints feature, 185

anatomy of, 98–99

USPhoneNumberField, 234

applied techniques, 109–111

USStateField, 191

arguments, 98–101

USStateSelect, 234

default values, 101

generic, 99–101

V

in Django, 97

validate() method, 62, 65–66

object-based, 107–108

validation

object_detail, 240, 251

of forms, 113–128

object_list, 239

of user input, 115–118

preserving name and documentation of, 104

ValidationError, 66

process_view() method, 174–175

value argument, 35

read-only, 251

ValueError, 71

resolving, to URLs, 95–97

value_from_datadict() method, 120, 121

resolving URLs to, 94–95

value_from_object() method, 62

reusing, 99–101

values_list() method, 275, 276–277

tasks of, 103

value_to_db_datetime() method, 188

templates for, 98, 109

value_to_db_date() method, 188

URLs and, 91

value_to_db_decimal() method, 188

using objects as, 107–108

value_to_db_time() method, 188

viruses, scanning for, 210–211

varargs, 39

Variable class, 141

W

VariableDoesNotExist exception, 135

weak option, 228–229

variable filters, 143–146

Web 2.0 applications, 109

accepting an argument, 145

Web advertisements, 152

accepting a value, 145

weblog, 10

registering, 146

Web Server Gateway Interface (WSGI), 163

returning a value, 145–146

widget attribute, 118, 119

VariableNode, 137

widget_attrs() method, 119

variable resolution

widgets

complex, 140–141

controlling, in forms, 118–119

simple, 139

custom, 119–122

variables

defining HTML behavior with, 119–125

global, module-level, 256

for obtaining values form posted data, 120

namespaces and, 138

splitting data across multiple, 121–122

in templates, 143, 144

wrap() function, 219

variable tokens, 136, 137

wraps() function, 225–226

varkwargs, 39

write() method, 22, 170, 195, 199

vary.vary_on_header decorator, 102

WSGI (Web Server Gateway Interface), 163

verbose_name attribute, 51, 60

verbose_name_plural attribute, 51

X

verbose_name_raw attribute, 51

XmlHttpRequest, 109

verbosity argument, 81

xreadlines() method, 195

versions, 11

X-Requested-With header, 169

view argument, 175

view decorators, 101–106

Y

applying, 102–103

year_lookup_bounds_for_date_field()

configuration options, 176–177

method, 188

deciding between middleware and,

year_lookup_bounds() method, 188

176–177

exception catching with, 105–106

Z

scope, 176

Zen of Python, 2, 6

Índice

CHAPTER 2

CHAPTER 4

CHAPTER 2

CHAPTER 4

CHAPTER 7 Handling

CHAPTER 8 Backend

CHAPTER 9 Common

CHAPTER 10 Coordinating

CHAPTER 11 Enhancing