Chicken Scratches

Facebook made a splash a year or two ago when they opened up their API. Now developers could write applications that integrate with the site. Instantly, users -- many of whom had fled to Facebook from a spam-filled MySpace -- were inundated with Requests to battle ninjas and News Items bearing obscene pictures. To Facebook's credit, they did clamp down and put some restrictions on what apps can do. A few entertaining or useful applications have risen to the top, and the potential of the API is ready to be exploited. It's not an easy task for a developer, however. In the next few Chicken Scratches posts, I'll talk about my experience developing two Facebook apps from scratch: The Limerick Book, a site that works both in and outside of Facebook and allows users to share and rate Limericks, and Play Scopa, a traditional Italian card game that users can play against each other in realtime (this one is not yet launched to the public). This first post discusses some of the difficulties I have run into. In the next couple posts, I'll discuss how I dealt with them. First, to set the scene:

My setup

I am using the Django web framework for my backend development, the latest SVN version running with Python version 2.4 on a shared host at Dreamhost. In fact, it's the same server I use for this site and weblog. To connect Django to Facebook, I am using the nice PyFacebook library, which is pretty mature, though I had to modify the code to support some of the latest features of the Facebook API. And now, on to the perils. Read the rest of this entry »

I have written two previous articles about how I edit Django template files in Emacs and XEmacs. Here is How I edit Django templates. And here is More on editing Django templates in XEmacs. Here today is another little tip that can be used in conjunction with those two other posts or independently.

Django templates involve a lot of punctuation. Between the angle brackets and slashes of HTML and the curly braces and percent signs of the Django template language, it's enough to make your pinky fingers hurt just thinking about it. Therefore any little trick to reduce some of this typing burden can be helpful. Presented here is some Emacs Lisp code to provide auto-closing of Django template tags. So even if you still have to type things like curly-brace percent-sign space ifequal blah blah2 percent-sign close-curly-brace, you won't have to type the {% endifequal %}. (Of course, if you're using the abbrev tips I gave previously, you won't even need to type the opening tag very often, but sometimes you still do.)

Read the rest of this entry »

Air travel is in a downward spiral. Not literally of course, but clearly and consistently, and everybody knows it. The airlines know it, the airline employees know it, travel agents found out a while ago, and consumers sure as Hell know it. There are a lot of reasons for it, but this post talks about just one: Expedia.com.

Airlines claim the degradation in quality comes from increased competition. They say they are forced to cut corners and reduce features in order to stay competitive. Does this make sense?

Read the rest of this entry »

When writing web applications, sooner or later (usually sooner), everybody is going to need a template language. String-interpolation just doesn't cut it. We need a way to write something that is almost all text (or HTML, or XML, or whatever), but with some dynamic pieces thrown in.

Since this need is so universal, and the basic requirements are so easy to describe, many different groups of people have taken it upon themselves to create Yet Another Template Language. As developers, we can join the fray and roll our own, or we can wade through the myriad options available to us to find the one that meets our needs or philosophy. Those who use PHP or ASP pretty much have the choice made for them, since the languages themselves are glorified template processors. Python programmers have a lot more options.

Here I'm just going to focus on the two Python templating languages I have used in real applications: Cheetah and the Django templating engine. (Django, of course, is more than just templates, but the template subsystem can be used independently.) I use and enjoy both of these, but there are significant differences that are worth comparing and contrasting, when deciding which to use for your particular needs. There are other comparisons out there, including one by the Benevolent Dictator for Life himself (though that's a bit out of date and inaccurate). When choosing which to use, you should read as many opinions as you can, then make the decision yourself. Presented here are just my personal thoughts.

Read the rest of this entry »

Previously, I wrote about how I set up syntax-highlighting for Django template files using MMM (multiple major modes) in XEmacs. This entry builds on the previous one, so if you haven't read that one, I suggest doing so now.

This post addresses the age old question, "If (X)Emacs is so good, why can't it do the typing for me?" Well, the answer, of course, is it can. You just have to write a few lines of Lisp first. Or you can copy and paste the Lisp code off of sites like this one. ;)

Introducing dynamic abbreviations

Pressing M-/, (that is, holding down the Alt key and pressing the forward slash key), runs the command dabbrev-expand, which tries to finish whatever word you are in the middle of typing. For example, if as I'm typing this entry, I type the letter d, then press M-/, "dabbrev-expand" shows up, because that was the last word I typed that started with d. If I then type M-/ again, "dabbrev-expand" changes to "down". If I keep pressing it, the word cycles through different guesses for what I'm going for. If you've ever used VisualStudio, you may see some similarities to the Intellisense feature. I find I almost never type a whole word anymore, I've become so dependent on dynamic completion.

Templates and more abbreviations

dabbrev-expand is useful to avoid having to type long words over and over, but there are also longer patterns that seem to need typing frequently. Thats where the tempo package comes in handy, especially when combined with the abbrev library. Here is some documentation for tempo. Here is some documentation for abbrev.

Essentially, tempo allows you to specify "templates," or blocks of standard text that can be parameterized as they are filled in: the same concept as Django templates, but meant for interactive use. From now on, I will say "tempo template" or "Django template" to avoid confusing the two types.

abbrev lets you define your own pre-set abbreviations, which can be filled in automatically as you type or upon request. This can be combined with tempo to do some pretty powerful stuff very easily.

I have a few Django-specific tempo templates and abbrevs set up in my XEmacs initialization file. For example, as soon as I type "{% block" followed by a space, the entire framework of a Django template block is filled in for me. I've also added a special menu to the menu bar for tempo templates I use frequently.

HTML and templates can be very repetitive, so I've found this saves me a lot of typing.

My initialization file

Here is the subset of my ~/.xemacs/init.el file. This includes the mmm-mode stuff I discussed last time, as well as the tempo templates and abbrevs. I haven't put a tempo template for every possible Django template tag, just the ones I use most frequently. I'll leave further extension as an exercise for the reader.

Also, some people don't like the expansion to happen automatically as they are typing. To turn this off, comment out the line that says "(setq abbrev-mode t)" by putting a semicolon (;) in front of it. Then you can manually expand tempo templates by pressing C-\ (control-backslash).

Once again, this is known to work in XEmacs version 21.4 on Windows XP. It will most likely work in other versions of XEmacs, and possibly in GNU Emacs.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; CSS-Mode
(autoload 'css-mode "css-mode" "Mode for editing CSS files" t)
(add-to-list 'auto-mode-alist '("\\.css\\'" . css-mode))
(setq cssm-indent-function #'cssm-c-style-indenter)
(setq cssm-indent-level '2)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Use hm--html-mode for files that end in .tmpl (Django templates)
(add-to-list 'auto-mode-alist '("\\.tmpl\\'" . hm--html-mode))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Multiple Major Modes.
(require 'mmm-vars)
(require 'mmm-mode)
(require 'mmm-sample)
(setq mmm-global-mode 'maybe)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Custom MMM classes for Django templates
(mmm-add-classes
 '((my-django-expr
    :submode python-mode
    :face mmm-declaration-submode-face
    :front "{%"
    :back "%}"
    :include-front t
    :include-back t)))

(mmm-add-classes
 '((my-django-var
    :submode python
    :face mmm-output-submode-face
    :front "{{"
    :back "}}"
    :include-front t
    :include-back t)))

(mmm-add-mode-ext-class nil "\\.tmpl\\'" 'embedded-css)
(mmm-add-mode-ext-class nil "\\.tmpl\\'" 'my-django-var)
(mmm-add-mode-ext-class nil "\\.tmpl\\'" 'my-django-expr)
(mmm-add-mode-ext-class nil "\\.tmpl\\'" 'html-js)

;; Use different colors for different sub-modes.
(setq mmm-submode-decoration-level 2)
;; Make the code submode a little more readable.
(set-face-background 'mmm-code-submode-face "#EEEEFF")


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Tempo templates and abbrevs

;; Control-backslash to expand a tempo template manually.
(global-set-key "\C-\\" 'tempo-complete-tag)

(require 'adapt) ; Must be done before hm--html for some reason.
(require 'hm--html-mode) ; Needed for hm--html-mode-abbrev-table
(require 'tempo) ; Tempo templates.


;; Expands <span into an HTML span with a class defined.
(tempo-define-template "span"
                       '("<span class=\"" (p "Class? ") "\">" r "</span>" >)
                       "<span "
                       "Insert a template for a span with a class")
(define-abbrev hm--html-mode-abbrev-table
  "<span" "" 'tempo-template-span)

;; Expands <script into an HTML javascript block.
(tempo-define-template "script"
                       '(> "<script type=\"text/javascript\">" r "</script>" > %)
                       "<script "
                       "")
(define-abbrev hm--html-mode-abbrev-table
  "<script" "" 'tempo-template-script)

;; Expands a Django template block.
(tempo-define-template "block"
                       '("{% block " (p "Block name? ") " %}" p "{% endblock %}")
                       "{% block "
                       "")
(define-abbrev hm--html-mode-abbrev-table
  "{% block" "" 'tempo-template-block)

;; Expands a Django template if tag.
(tempo-define-template "django-if"
                       '("{% if " (p "Conditional? ") " %}" p "{% endif %}")
                       "{% if "
                       "")
(define-abbrev hm--html-mode-abbrev-table
  "{% if" "" 'tempo-template-django-if)

;; Expands a Django template for tag.
(tempo-define-template "django-for"
                       '("{% for "
                         (p "Variable? ") " in "
                         (p "List? ") " %}" p
                         "{% endfor %}")
                       "{% for "
                       "")
(define-abbrev hm--html-mode-abbrev-table
  "{% for" "" 'tempo-template-django-for)

;; Create a menu for inserting these templates.
(defun add-templates-menu ()
  (interactive)
  (add-submenu nil '("Tem%_plates"
                     ["{%% %_block %%}" tempo-template-block]
                     ["{%% %_if %%}" tempo-template-django-if]
                     ["{%% %_for %%}" tempo-template-django-for]
                     ["<%_span>" tempo-template-span]
                     ["<s%_cript>" tempo-template-script])))

;; When entering hm--html-mode, turn on abbrevs and add the template menu.
(add-hook 'hm--html-mode-hook
          (lambda ()
            (setq abbrev-mode t)
            (add-templates-menu)))
            

NOTE: This post is pretty old, and I no longer use mmm-mode for Django template editing. This post and this post are still accurate, though.

Every programmer has their favorite editor and mode of working. Some people have more than one. For example, I use Microsoft Visual Studio when editing .NET code, DrScheme for editing Scheme code, and XEmacs for pretty much everything else.

This post is about how I use XEmacs for editing Django template files, in the hopes that others may find this useful.

The "Other" One True Editor

It's rare that people will get passionate about something as pedestrian as a way of editing plain text, but the brief history of the internet is awash with flamewars and heated discussions with titles like "Emacs vs. XEmacs," "Emacs vs vi," and so on. I'm not about to go into the relative merits, but the fact that certain editors pop up time and time again in these debates must mean there is something special about them.

Early on in my college education I started using Emacs because it was all that was available on the school's servers. (Well, that or vi, but vi was and still is black-magic to me.) I got over the learning curve, and now I'm hooked.

At some point, I switched from Emacs to XEmacs, for reasons I can't remember. At the time, it had some feature which was to me essential. That reason no longer applies, but neither have I had a reason to switch back. These tips may apply even if you use GNU Emacs, but they've only been tested in XEmacs.

My current setup is Xemacs 21.4 (patch 20), running on Windows XP. *gasp!* Yes, I use Windows for Django development. Shocking, I know. I have my reasons. In any case, these tips should work equally well with XEmacs on other platforms.

Multiple Major Modes

I won't include a full XEmacs tutorial here, since there is already plenty of info on the web about it. The key point is that there is a "major mode" for each programming environment. There is a Python mode, a Java mode, and so on. Django templates tend to combine more than one language in a file, so that's when the mmm library comes in handy. It stands for "multiple major modes," and it turns out to be just exactly what we need. We can have html-mode for the HTML parts, JavaScript-mode for the JavaScript parts, css-mode for embedded CSS, and python-mode for the Django template filters and tags.

Which HTML mode?

As often happens in the world of Free Software, there are several options to choose from when setting up HTML editing in XEmacs.

• html-mode. This has the fullest support for HTML parsing and validation. The problem is, when dealing with templates, the HTMl will often not validate, so all kinds of error messages show up.
• sgml-mode. This is a more general mode for SGML (of which XML and HTML 4 are subsets).
• xml-mode. SGML mode specialized for XML.
• hm--html-mode. I have no idea what the HM stands for, but this is a lightweight HTML mode, with basic syntax highlighting and indentation.

I use html-mode for full-fledged HTML documents, and hm--html-mode for templates. So that XEMacs can tell the difference, I use the suffix ".tmpl" for template files.

One problem

I did run into some problems getting mmm-mode to work with XEmacs. It turns out the version of mmm-mode that is distributed with the XEmacs package system is ancient - from 2001. I had to download the newer version of mmm-mode and unzip it into my site-packages directory.

How it looks

Here's a screenshot of me editing an example Django template (borrowed from my beta-registration Django app). I've chosen fairly bright colors to make the syntax highlighting more obvious, but that's all customizable. Notice how the Django tags and variables are easy to find in the file. (Click on the image for a larger size.)

My initialization file

Here is the subset of my ~/.xemacs/init.el file dealing with setting up mmm-mode for XEmacs. I hope someone finds this useful. Let me know if you do, or if you encounter problems.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; CSS-Mode
(autoload 'css-mode "css-mode" "Mode for editing CSS files" t)
(add-to-list 'auto-mode-alist '("\\.css\\'" . css-mode))
(setq cssm-indent-function #'cssm-c-style-indenter)
(setq cssm-indent-level '2)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Use hm--html-mode for files that end in .tmpl (Django templates)
(add-to-list 'auto-mode-alist '("\\.tmpl\\'" . hm--html-mode))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Multiple Major Modes.
(require 'mmm-vars)
(require 'mmm-mode)
(require 'mmm-sample)
(setq mmm-global-mode 'maybe)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Custom MMM classes for Django templates
(mmm-add-classes
 '((my-django-expr
    :submode python-mode
    :face mmm-declaration-submode-face
    :front "{%"
    :back "%}"
    :include-front t
    :include-back t)))

(mmm-add-classes
 '((my-django-var
    :submode python
    :face mmm-output-submode-face
    :front "{{"
    :back "}}"
    :include-front t
    :include-back t)))

(mmm-add-mode-ext-class nil "\\.tmpl\\'" 'embedded-css)
(mmm-add-mode-ext-class nil "\\.tmpl\\'" 'my-django-var)
(mmm-add-mode-ext-class nil "\\.tmpl\\'" 'my-django-expr)
(mmm-add-mode-ext-class nil "\\.tmpl\\'" 'html-js)

;; Use different colors for different sub-modes.
(setq mmm-submode-decoration-level 2)
;; Make the code submode a little more readable.
(set-face-background 'mmm-code-submode-face "#EEEEFF")

Some thoughts on when to return a List<> object and when to return an array in C#.

The analogy: strings

Many times in C#, a function that needs to build a string from constituent parts will use a StringBuilder internally, and then on the last line of the function call ToString() on the StringBuilder object. This is because strings in C# are immutable. That is, once they are set then can never be changed. Code that looks like it is changing a string in place is usually actually allocating a new string to hold the new value. Obviously, for a function that builds a string incrementally by constantly appending to it, this can lead to lots of reallocation, slowing down performance.

Here's an example of how it's normally done:

    public string GimmeString(int num)
    {
        StringBuilder sb = new StringBuilder("Initiating countdown: ");

        for (int i = num; i > 0; i--)
        {
            sb.Append(string.Format("{0}...", i));
        }
        sb.Append("Blastoff!\n");

        return sb.ToString();
    }
    

This is a pretty obvious design pattern. It's rare you would want to return the StringBuilder object itself, so almost always it is converted to a string before returning. The problem gets trickier, however, when we try to use a similar design pattern for arrays.

Arrays

Arrays in C# are not immutable, at least not in the way strings are. It could be said they are "partially immutable" (my own made-up term). You can swap out elements in the array as much as you like, but the array's length is fixed. Therefore, if you need to build up an array from constituent parts, it makes sense to use a List<> object for doing the work.

The analogy then becomes, in SAT terms:

StringBuilder : string :: List<> : array

The question then becomes whether or not to call ToArray() on the List<> before returning it. In this case, the answer's not as obvious. Let's examine the pros and cons of each approach:

Reasons to call ToArray()

• If the returned value is not meant to be modified, returning it as an array makes that fact a bit clearer.
• If the caller is expected to perform many non-sequential accesses to the data, there can be a performance benefit to an array over a List<>.
• If you know you will need to pass the returned value to a third-party function that expects an array.
• Compatibility with calling functions that need to work with .NET version 1 or 1.1. These versions don't have the List<> type (or any generic types, for that matter).

Reasons not to call ToArray()

• If the caller ever does need to add or remove elements, a List<> is absolutely required.
• The performance benefits are not necessarily guaranteed, especially if the caller is accessing the data in a sequential fashion. There is also the additional step of converting from List<> to array, which takes processing time.
• The caller can always convert the list to an array themselves.

ToArray() or not ToArray()? That is the question.

Based on these points, it seems to make the most sense as a general rule to simply return the List<> object directly, rather than converting it to an array before returning. Let me know if you disagree.

Here's an example:

    // A contrived example. Similar to Python's "range" function, but only
    // supports positive step.
    public List<int> GimmeInts(int start, int end, int step)
    {
        List<int> ret = new List<int>();

        for (int i = start; i < end; i += step)
        {
            ret.Add(i);
        }
        // Here you could have:
        // return ret.ToArray();
        return ret;
    }
    

Somebody posted a reply to my last post about Django's autoescaping mechanism. (They were too cowardly to post on my site, so they posted it at reddit.com.) The person said something like, "you shouldn't trust yourself to remember to escape your own variables." Oh, heaven forbid I trust myself to be a good programmer! That really got me thinking about the recent trends towards designing frameworks, APIs, even languages for mediocre programmers. We are sacrificing speed, simplicity and efficiency to make common bugs less common, trying to design away the mistakes inexpensive and poorly-trained computer scientists make.

Now, of course when I say "recent trends," I should acknowledge that this type of thinking has been around for decades. It was first truly popularized with the introduction of Java. Some people forget to free memory, so add garbage collection. Some people forget to bounds-check arrays, so make that automatic. Ooh, pointers are scary! Let's get rid of them. We can't allow our outsourced foreign coders direct access to memory!

Good training, along with working for nearly a decade as an embedded software engineer, has taught me good programming habits. I've learned to be conscious of memory leaks, to always check return values, to program defensively, to bounds-check. I've created software for shipping products in such low-level and "unprotected" languages as C++, C, and even Assembly. I've written production code within less than the memory space required for a Java byte-code interpreter. And of course, I'm not alone in this. There is a large subset of software developers who had to learn to program carefully, due to constraints out of their control. These types of good programming habits carry over into whatever platform or language is used.

I feel a lot of the new safety-net style approaches are simply enabling poor programmers to work on increasingly sophisticated projects. To get back to the example from my last post, Django is a wonderful tool. You can program a sophisticated database-centered multi-user web application without even knowing how to spell SQL. Django's recent addition of autoescaping, and more importantly, the enabling of autoescaping globally by default, is yet another example of API-design for the lowest common denominator. (I should note that I love Django. It saves me writing a lot of redundant code and provides a lot of things for free that I would otherwise need to write from scratch, so I don't mean to pick on Django here. It just happened to be the catalyst for this discussion.)

It's not all bad

I know I'm starting to sound like an old curmudgeon. "In my day, we didn't have variables, we just had to carry around rocks to count!" I'm not that old, really. And I'm certainly not advocating we go back to the days before garbage collection and bounds checking. Especially given the potential security ramifications of memory-management bugs, these things are especially important. I just want to urge caution before binding developers in a straitjacket. Rather than trying to design away all potential bugs at the level of the language or API, emphasize and facilitate good programming and testing practices. I've never once bought a For Dummies book, and I never will. Please don't force me to use a For Dummies application framework.