TextMate, Emacs and META indent-region

[cross-posted from the Desert Moon blog.]

I haven't used GNU Emacs very much since switching to TextMate in 2005. One Emacs feature which I really miss in TextMate is indent-region. It lets you take an entire region of code, whatever its language, whatever its mix of tabs and spaces and indentation widths, and re-format it using your preferred indentation style.

But wait! Emacs has a batch mode, and you can drive it from TextMate. Many thanks to Gragusa's Things for showing the way.

The post on Gragusa's Things is specific to R code, but I'm more interested in re-formatting C and C++ code. Here's my first cut at a general TextMate Bundle to re-format code regardless of the source language:

#!/usr/local/bin/python2.6
"""
Use Emacs to re-indent regions of the current buffer.
Inspired by 
http://gragusa.wordpress.com/2007/11/11/textmate-emacs-like-indentation-for-r-files/
"""
import tempfile
import os
import sys
import subprocess

# Use the same filename extension so Emacs will know which
# mode to use. 
ext = os.path.splitext(os.environ["TM_FILEPATH"])[-1]
outf = tempfile.NamedTemporaryFile(suffix=ext, delete=False)
pathname = outf.name

outf.write(os.environ["TM_SELECTED_TEXT"])
outf.close()

args = [
    "emacs", "-batch", pathname, 
    # Assume no emacs-startup.el
    "--eval", "(setq indent-tabs-mode nil)",
    "--eval", '(c-set-style "java")',
    "--eval", "(setq c-basic-offset 4)",
    "--eval", "(indent-region (point-min) (point-max) nil)", 
    "-f", "save-buffer"]
p = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if p.returncode:
    print(err)

inf = open(pathname, "r")
sys.stdout.write(inf.read())
inf.close()

os.remove(pathname)

NB:

1. Due to the use of the delete=False keyword argument to tempfile.NamedTemporaryFile, this command bundle requires Python 2.6+.
2. TextMate on OS X 10.5 won't, by default, have /usr/local/bin in its path; hence the pathetic shebang.

Anyway, install this as a new TextMate command bundle, assign a Key Equivalent such as ⌘-Shift-R, and enjoy.

Creating an 'hg ignore' extension

[cross-posted from the Desert Moon blog.]

I often wish Mercurial had an 'hg ignore' command similar to 'bzr ignore'. Turns out it's pretty easy to add one:

#!/usr/bin/env python
"""Ignore pathnames and patterns"""

import os

def ignore(ui, repo, *pathnames):
    """Ignore the given pathnames and patterns."""
    outf = open(os.path.join(repo.root, ".hgignore"), "a")
    for p in pathnames:
        outf.write(p + "\n")
    outf.close()
    return

cmdtable = {
    'ignore': (ignore, [], "hg ignore pathname [pathname]"),
}

To use this, save it to a file such as ${HOME}/platform/independent/lib/hg/ignore.py. Then add the extension to your ${HOME}/.hgrc:

[extensions]
~/platform/independent/lib/hg/ignore.py

Running TileCache within a Django Application

Punchline

Here is how to serve TileCache tile images from within a Django application.

from TileCache.Service import Service

_service = Service(...)

def get_tile(request):
    global _service

    format, image = _service.dispatchRequest(
        request.GET, request.path, request.method, 
        request.get_host())
    result = HttpResponse(str(image), mimetype=format)
    return result

Scenario

You're building a low-traffic Django-based GIS application, and you need to serve your own map layers. You're using TileCache to improve your application's performance. But installation and configuration are hassles.

• All of your servers must run with the right user and group IDs, so the Django app can expire the tile cache when necessary.
• Your Django app needs to understand the structure of the tile cache, so it can remove the correct tile images when the underlying data changes.
• Etc.

This would all be much easier if you could serve TileCache requests from within your Django application. They're both Python-based; why not?

The TileCache code base includes sample code that shows how to run TileCache as a CGI or a FastCGI service. I couldn't find any sample code for running TileCache within a Django application, but it was easy to convert the cgiHandler code for use with Django's HttpRequest objects.

Installation Prerequisites

In order for TileCache to generate its own tiles, instead of delegating to a separate mapserver instance, you must already have compiled and installed mapserver's Python mapscript bindings. For instructions on compiling the bindings see the mapscript/python/README file in the mapserver source distribution.

Configuring TileCache

import os
thisdir = os.path.abspath(os.path.dirname(__file__))
def relpath(p):
    return os.path.abspath(os.path.join(thisdir, p))

from TileCache.Service import Service
import TileCache.Layers.MapServer as MS

# Create the service 'singleton'.
_mapfile = relpath("../mapserv/data/mapfile.map")

_service = Service(
  _cache,  # See "Cache Invalidation", below
  { 
    "basic": MS.MapServer(
        "basic", _mapfile, layers="basic", debug=False),
  }
) 

Handling Tile Requests

This is the sweet part. It's derived from the cgiHandler() example in the TileCache source code, but Django's HttpRequest class makes the implementation very simple:

def get_tile(request):
    global _service

    format, image = _service.dispatchRequest(
        request.GET, request.path, request.method, 
        request.get_host())
    result = HttpResponse(str(image), mimetype=format)
    return result

What About Feature Info Requests?

I don't know much about the required web API of a WMS server, but it appears as if the same URL must serve both tiles and feature info requests; the type of request is determined by the Request querystring parameter.

Django's dispatch system is based on URL pathnames; I'm not aware of any way to dispatch based on query string parameters. So you'll need to either configure your web server (e.g. Apache) to rewrite WMS requests to distinct URLs provided by your Django app, or you'll need to do some dispatch within your Django app.

Suppose you opt for the latter. Then your urls.py might look something like this:

    ...
    url(r'^wms/$', 'world.views.wms', name='wms'),
    ...

and in world/views.py you might have this:

def wms(request):
    if request.GET.get("request") == "GetFeatureInfo":
        return get_feature_info(request)
    return get_tile(request)

Cache Invalidation

For my web app, several of the tile layers are derived from a Django model which is updated via the admin interface. Whenever the model changes, the tile cache for the corresponding layer(s) needs to be invalidated, so the images can be regenerated.

The TileCache Cache interface doesn't provide for invalidation. Since I'm using a filesystem-based cache, I subclassed TileCache.Caches.Disk to create a Disk cache which does support invalidation.

import shutil
from TileCache.Caches.Disk import Disk

class InvalidatingDisk(Disk):
    """A Disk cache which can invalidate its contents, 
       layer by layer."""
    def invalidate(self, layerName=None):
        if self.basedir:
            pathname = self.basedir
            if layerName is not None:
                pathname = os.path.join(self.basedir, 
                                        layerName)
            shutil.rmtree(pathname, ignore_errors=True)

Introducing Google Chrome Frame

Introducing Google Chrome Frame:

"With Google Chrome Frame, developers can now take advantage of the latest open web technologies, even in Internet Explorer. From a faster Javascript engine, to support for current web technologies like HTML5's offline capabilities and <canvas>, to modern CSS/Layout handling, Google Chrome Frame enables these features within IE with no additional coding or testing for different browser versions.

To start using Google Chrome Frame, all developers need to do is to add a single tag:

<meta equiv="X-UA-Compatible" content="chrome=1">
"

I guess that's good news. Makes you wonder why the target audience wouldn't just install Google Chrome. But I suppose this lets people continue to use IE while using modern web facilites on sites which require them.

TR: China Wind Energy Potential, HVDC

(Just taking notes, trying to understand what HVDC is, what it has to do with variable power sources such as wind, and why it makes buried transmission lines convenient.)

Technology Review: China's Potent Wind Potential:

"The major grid upgrades already under way in China are making extensive use of continental-scale high-voltage direct-current (HVDC) lines, which remain the stuff of supergrid blueprints in Europe and the United States. 'They are leading the world in implementing long-distance transmission schemes,' says Bjarne Andersen, director of U.K.-based consultancy Andersen Power Electronic Solutions and an expert in the ultra-efficient HVDC technology."

Technology Review: Europe Backs Supergrids:

"This summer [2008], for example, a negotiator appointed by the EC convinced France to accept a new transmission connection with Spain, breaking a 15-year impasse over expanding power exchanges between the countries. Use of high-voltage DC (HVDC) technology will enable planners to bury the new line and thereby overcome local opposition to conventional overhead AC transmission lines."

Wikipedia explains how HVDC can have low power losses:

"Power in a circuit is proportional to the current, but the power lost as heat in the wires is proportional to the square of the current. However, power is also proportional to voltage, so for a given power level, higher voltage can be traded off for lower current. Thus, the higher the voltage, the lower the power loss."

"The advantage of HVDC is the ability to transmit large amounts of power over long distances with lower capital costs and with lower losses than AC."

Lots of interesting stuff in the Wikipedia article. This excerpt seems to explain the connection between HVDC and variable power sources:

"Because HVDC allows power transmission between unsynchronised AC distribution systems, it can help increase system stability, by preventing cascading failures from propagating from one part of a wider power transmission grid to another. Changes in load that would cause portions of an AC network to become unsynchronized and separate would not similarly affect a DC link, and the power flow through the DC link would tend to stabilize the AC network. The magnitude and direction of power flow through a DC link can be directly commanded, and changed as needed to support the AC networks at either end of the DC link. This has caused many power system operators to contemplate wider use of HVDC technology for its stability benefits alone."

ActionScript: TileList, deleting, scrolling backwards

I have a Flex app which shows users a TiledList of chemical structure depictions. Since it can be a large list, it's lazy-loaded from the server as the user scrolls through the list.

Users can delete items from the list, with undo. In order to do this with reasonable performance, once the app has received confirmation from the server that an item has been deleted, it clears out the single deleted item from its local lazy-list.

To undo the deletion locally, the Flex app fills the correct lazy-list entry with an ItemPendingError; that error gets thrown as soon as the TileList tries to retrieve the item.

All of this works okay when the row containing the undeleted item is already visible. On the other hand, if the user has scrolled away from the row where the undeleted item will reappear, then when (s)he scrolls back the TileList simply empties out that item and all of the successive items in the row. Ugly!

Workaround

When the item is undeleted, immediately try to retrieve it via getItemAt(itemIndex). Catch the resulting ItemPendingError and register an ItemResponder. When the ItemResponder's result or fault method is called, tell the TileList to invalidateList(). If the undeleted item actually contains a value, the TileList will repaint correctly -- no more unsightly gaps.

import mx.collections.errors.ItemPendingError;
import mx.collections.ItemResponder;
[...]
try {
    structures.getItemAt(offset);
} catch (e:ItemPendingError) {
    e.addResponder(
    new ItemResponder(
        function(result:Object, token:Object = null):void {
            tilelist.invalidateList();
        },
        function(error:Object, token:Object = null):void {
            tilelist.invalidateList();
        }));
}

I like Mike -- General Michael Collins, That Is

This year's John Glenn Lecture Series featured Sen. Glenn, Chris Kraft and the three Apollo 11 astronauts. Michael Collins was as smart, funny and humble as in "When We Left Earth." His talk starts roughly 55 minutes in.

Apologizing for the lecture-unfriendly layout of the IMAX theater, which he helped approve:

"I'm down here in the bottom of a black hole about to be sucked in by gravity..."

After putting up this picture, which he took as the LEM began its descent to the lunar surface:

"I like that photo, it's my favorite one. You see in the little thing there are 3 billion people, and then in the big thing there are two people..."

About the glistening blue earth in the background:

"Serene it is not. Fragile it is. The world population when we flew to the moon was 3 billion people. Today it's over six and headed for eight, so the experts say. In my view this growth is not wise, healthy or sustainable[...]

"Our economic models are all predicated on growth. They require it. Grow or die, or maybe both: the dead zone created by the runoff from the Mississippi into the Gulf of Mexico is now larger than the State of New Jersey, and still growing...

"We need a new economic paradigm that somehow can produce prosperity without this kind of growth."

The video: http://www.youtube.com/watch?v=w9fCPhspOCQ

Sen. Lamar Alexander on Nuclear Energy

On July 13th Sen. Lamar Alexander held a press conference to propose a low-cost, clean energy plan centered on nuclear energy. I still haven't digested the whole proposal, but it's an interesting read.

C-SPAN has video. Senator Alexander's website has the proposal in PDF format.

One bullet item from the press conference really resonated:

"We want an America in which we are not creating “energy sprawl” by occupying vast tracts of farmlands, deserts, and mountaintops with energy installations that ruin scenic landscapes. The Great American Outdoors is a revered part of the American character. We have spent a century preserving it. We do not want to destroy the environment in the name of saving the environment."

Amen to that! "Energy installations" can be beautiful...



But I'd hate to see the Taos valley scarred over with wind turbines.

Anyway, just now the most cost effective way to address the electricity needs of the U.S. seems to be to reduce demand, by improving energy efficiency. Going forward, since our population is projected to grow by 44% by 2050, we'll probably still need to increase electricity production.

If we're willing to change U.S. policy on re-processing spent nuclear fuel, Sen. Alexander's proposal could work. France provides an existence proof.

---

Life Shore Gits Tedious

I was hoping to find numbers on total electricity consumption by country, to compare France's production capacity to our own projected needs. Instead I found this Wikipedia entry, which describes the currently-decreasing per capita electricity consumption of the U.S.; notes that the U.S. still consumes considerably more electricity per capita than countries such as Germany; discusses various ways of measuring national energy efficiency (e.g. energy intensity); describes the relationship between population growth and electricity consumption; and so forth.

Why does the reading list never get shorter? :)

Unresponsive console.app on OS X 10.5

Recently, when I opened console.app and tried to view either Console Messages or All Messages, cpu usage spiked and console.app became unresponsive. Activity Monitor showed aslmanager using up all of the cpu.

The following discussion thread helped solve the problem. The final required step seems to have been to remove the entire /private/var/log/asl/ directory before restarting syslogd.

Apple - Support - Discussions - ASLMANAGER hogging CPU, resisting fix? ...

Update: It looks like aslmanager first appeared in OS X 10.5.6. It also looks like the asl facility is Apple's replacement for syslogd, created to make it easier to quickly search system logs. From the asl(3) man page: "This API permits clients to create queries and search the message data store for matching messages."

I really like New Mexico...

... but every once in awhile I wish I was back in Dayton.

Apollo astronauts relive experiences at ceremony

Bravo Bill Gates

In the same vein as yesterday's half-baked post, Bill Gates has helped make freely viewable (if not downloadable) a lecture series by Richard Feynman.

From CNET:

"Gates first saw the series of lectures 20 years ago on vacation and dreamed of being able to make them broadly available. [...] Tapping his colleagues in Redmond to create interactive software to accompany the videos, Gates is making the collection available free from the Microsoft Research Web site."

The name of the site?
Project Tuva. Nice touch.


The site doesn't seem to load in Safari 4 w. Silverlight 2, but Firefox 3.5 / Mac works fine.

Lots of offsite backups

[behold, another half-baked post]

The Register says that NASA will on Thursday release 'greatly improved' footage from the Apollo 11 landing. They speculate that this footage is derived from original tapes of the landing, which in 2006 NASA admitted having lost.

I hope NASA makes the new video freely available for download. If they do, they'll get thousands (millions?) of offsite backups for free, hosted by history buffs around the world. And they won't need to worry so much about losing the originals again.

The Library of Congress has already done something similar with the nation's library, e.g. by posting images to Flickr.

Granted, backups are useless if you can't restore them. It should be easy to put out a call for well-known documents such as the lunar landing videos. But LoC has all kinds of documents ranging from famous to obscure, and retrieving them by broadcasting a call to volunteers would be dicey at best.

So it's interesting to see that LoC is launching a pilot program "to test the use of cloud technologies to enable perpetual access to digital content."

Canada and Japan blocking climate-change deal, Sir David King warns - Times Online

Canada and Japan blocking climate-change deal, Sir David King warns - Times Online:

"Governments previously were able to hide behind the US's intransigence on climate change, he said, but the pro-climate policies being launched by the Obama administration means this is no longer possible. 'The time has come for people to reveal their cards,' he told delegates."

via @TomRaferty by way of @timoreilly.

The Benefits of a Classical Education

Tim O'Reilly has posted yet another article full of thought-provoking nuggets, in which he answers interview questions for a special report:

The Benefits of a Classical Education

The article muses on ways in which capitalism can be altruistic rather than greedy; bumps up my respect for West Virginia's Robert Byrd, whom I often see as a detriment to Congress; and includes incisive quotes from Alexander the Great and Mark Twain ("While history doesn't repeat itself, it does rhyme"). All this in response to the first interview question.

A Supreme Leader Loses His Aura - NYTimes.com

If only because of the Times's reputation in recent years, I have to wonder how much of this report is real. Still, a compelling read.

Op-Ed Columnist - A Supreme Leader Loses His Aura as Iranians Flock to the Streets - NYTimes.com

@terrycojones has posted a link to the Wikipedia article on Iran's 1953 coup. Among other things I hadn't known that British Petroleum had its origins in the Anglo-Persian Oil Company. http://bit.ly/11q8Oq

Opponents blast Northwest Quadrant housing project

Santa Fe's northwest quadrant housing project has all of the smells of the malling of Beavercreek, Ohio. City staff will just keep asking for approval until they get the answer they want.

Opponents blast Northwest Quadrant housing project:

"Other residents questioned [...] a plan to pump sewage uphill and other facets.

The housing project on city-owned land would be concentrated on about 122 acres of the 540-acre Northwest Quadrant. The proposal calls for construction of more than 750 housing units, including single-family homes and multi-family complexes that would rise up to three stories tall [emph. added] as well as up to 110,000 [square] feet of mixed-use development that could include commercial uses. "

<Incoherent Pre-coffee Ramblings>

Up to three stories tall... on top of a ridge line. There goes the neighborhood.

Would there be no value in turning this land into public space?

When I first moved to Santa Fe ten years ago, I could walk to the top of the ridge above my apartment and take in a view that encompassed Los Alamos, the Sangre de Cristos, and the Sandias more than fifty miles to the south. The view is still available, and it's on land which city staff wants to bury under multi-story housing.

These days I live "in the hole" of Casa Solana, just to the south of the proposed development. The targeted land is still the best place, for miles around, to watch the fog of a morning storm turn into ragged, fast-moving tufts of cloud.

Of course, when I first moved here the open area was also filled with old mattresses, broken beer bottles and old engine blocks. Human nature is everywhere the same.

Perhaps awesome views are of value mainly to those who have lived too long amid urban sprawl. Even city planners, who must know that scenery is one of the reasons people visit northern New Mexico, believe they will gain more from taxes on developed land than from natural beauty.

"You can't eat scenery." — Victor, "Local Hero"

</Incoherent Pre-coffee Ramblings>

Palm's Big Opportunity

Via Macintouch:

"An iPhone app developer's world is lonely...

Three parties are involved: the developers (us), Apple, and the customers. For the most part, Apple stands between us and our customers[...] we can't issue refunds, we can only issue a few promo copies, we can't collect upgrade revenue, we can't respond to App Store reviews, we can't provide installation support, and we can't release updates to address customers' issues in a reasonable amount of time. We can't even tell them when the next update will be available, because we honestly don't know. [...] Our customers, like us, are mostly in the dark with this process, and we can't do much to help them.

For the most part, it's just us and Apple in the room.

And Apple's a brick wall.
"

In large part, Palm has based webOS on open standards. The Palm pre user experience is reported to be very good. Sprint (tethering) and AT&T (rug-yanking over data plans) both stink.

So will Palm be able to draw developers, and customers, to its platform by running a less authoritarian app store than Apple's? Will it even try to do so? Here's hoping...

We are too many

Sciam examines relationships between environmental degradation and population. Still haven't digested it all. This looks like the punchline:

Population and Sustainability: Can We Avoid Limiting the Number of People?: Scientific American:

"...the evidence suggests that what women want—and have always wanted—is not so much to have more children as to have more for a smaller number of children they can reliably raise to healthy adulthood. Women left to their own devices, contraceptive or otherwise, would collectively ‘control’ population while acting on their own intentions."

The TSA - back in bounds?

AOPA Online: TSA lessens security restrictions on transient pilots:

"According to the new directive, transient pilots who fly into commercial-service airports no longer need to get an airport badge or background check. However, they must remain close to their aircraft, leaving it only to walk to and from the fixed-base operator, service provider, or airport exit. The TSA also has said that it will make provisions for self-fueling operations and grant allowances for emergency situations."

Score one for AOPA.

...Mentally awake...

Brain Power - At Card Table, Clues to a Lucid Old Age - Series - NYTimes.com:

"Interacting with people regularly, even strangers, uses easily as much brain power as doing puzzles, and it wouldn't surprise me if this is what it's all about."

Heck, I have a hard time just remembering names during introductions...