Why Vim is great (Exhibit 2: Vim is scriptable)

One great feature of Vim is that it is scriptable, which allows pretty much infinite customizability. I already blogged about a plugin I wrote for speeding up C++ coding, but there are many other far more useful plugins out there. The point is, if you're a programmer and you want to speed up something you're doing often, you can easily do it yourself if an appropriate plugin doesn't already exist or you can't find it. My example above is obviously covered by some excellent plugins, but they are much more heavyweight than what I wanted to do.

Another example where an existing script (probably) doesn't exist and that I encountered recently was reformatting some context free grammars encoded in LaTeX. I was helping organize all the exam problems from the last several years of our compiler design course into a problem book to give out to students for practice. There were probably more than 50 grammars in these problems, written in several different styles of marking nonterminals, listing rules etc. For example, these were three of the styles we had originally:

S \(\to\) aAbBa | bBaAb
A \(\to\) AaBb | Ba | a
B \(\to\) Ab | b

S \(\to\) A a A b
A \(\to\) A b
A \(\to\) d B
A \(\to\) f
B \(\to\) f

S \(\to\) AB | dC     A \(\to\) aCB | \(\varepsilon\) 
B \(\to\) eS | bC     C \(\to\) c | aB

We obviously wanted to have a uniform style for all these grammars, so I wrote a short script that reads in a grammar in any of the original styles and reformats it. Obviously, you could write a program in any language to do that, but since these grammars were part of a large document, you'd have to delimit them somehow for the external program or copy/paste them around. If you think about it, the problem was an exact fit for an editor because it is part of editing a document. Also, it's not really possible to do this with a macro that doesn't itself contain some code. In any case, writing the script took probably less than 20 minutes; far less than it would have taken me to manually convert all the grammars, not to mention far less error prone.

So how do you write these scripts? Vim has a buit-in scripting language (commonly referred to just as Vimscript) which is a dynamic language that feels sort of like Python, but has somewhat clunky syntax. I'm not going to talk about the language very much here because it's covered well elsewhere, for example in a series of IBM DeveloperWorks articles, and in Vim's help by typing :help script.

The other thing to note is that you can actually script Vim using several other (better) languages, namely Python, Ruby, Perl, Scheme and Tcl. You can find great info on these in Vim's help under python, ruby, perl, mzscheme and tcl, respectively. I've only tried the Python interface myself and found it more enjoyable than using Vimscript. However, there are some unfortunate caveats. First, you'll need Vim compiled with the appropriate flag set to enable each interface (for example +python or +python3). This in itself motivates people to write plugins in Vimscript since it is the only language guaranteed to be supported (I read somewhere that Python was supposedly included in all the binaries after Vim 7.0, but I haven't been able to verify that right now). Furthermore, debugging these external scripts is a lot harder as the error messages you get aren't very helpful.

In any case, scriptability is a great feature that opens up a lot of opportunities for increased productivity, and that's what we're all after :). Happy Vimming!

Why Vim is great (Exhibit 1: macros across multiple buffers)

I'll give a little bit of an introduction to my Vim experience in the next paragraph and an introduction to a real life problem that I solved using this Vim "trick" recently in the paragraph after that. You can skip to paragraph four if you don't care about that and just want to see the "trick" (it should be more or less self-standing).

I've been using Vim on a daily basis for about four years now, and I'm really happy that I took the time to learn to use it and stuck it out when it was rough. There used to be a class in the first semester at my college that had a lab in Vi basics, but it wasn't really taught in a meaningful way but rather left the students to "try it out" which resulted in people trying to type something and "getting stuck" in some mode without knowing how to do anything. At the end, everyone (including me) hated it, thought it was an old editor that nobody should ever use. Later, I tried getting into it a few times and finally made it stick. I have by no means mastered it, but I feel pretty comfortable using it and I hope I can show you a few tricks that are not too basic and well known (I'm hoping to make a series of posts out of this).

I recently started maintaining a site (pro bono :) that was abandoned around 2007. The whole thing is mostly HTML only with a few PHP scripts here and there. Unfortunately, it is structured in a way that requires a lot of duplicated effort since most files are referenced in several places. Since the updates are not very frequent (about one a week) and there is a lot of existing content that is important, I decided (for now) not to port it to a CMS and just stick to the old system. I've since developed a few scripts and tools that make the update process pretty simple. The central item of the site are trip reports that include some text and a link to a picture gallery. We now use Picasa to host the pictures. Every report includes a link to the appropriate album (or several) at the bottom. This contains a clickable thumbnail of the album cover and a clickable album name below it. Both these links open the album in the same window (no target attribute). I got complaints that once you start browsing through the gallery, there's no easy way to get back to the main site. I didn't notice this because I have a habit of always using right click->open in new tab on all links. Unfortunately, Picasa doesn't allow you to add a link to your album (from searching around, it seems like a very requested feature). I thought my best option was to simply add target="_blank" to every gallery link so that it would open in a separate window. Now, since I'd uploaded two years worth of content (a lot of reports), there were a lot of links to fix.

So on to Exhibit 1: I have a ton of files that all contain two links to an album hosted on Picasa and I want them to start opening in a new window, i.e. need to add target="_blank" (there are other ways to fix this, but they all require at least some change in the link HTML). I remembered seeing a video of a similar problem a few months ago and how it could be solved with Vim. I downloaded all the files into a directory and opened them all with vim *. Then I recorded a macro to register a (with qa). The goal was to somehow identify the gallery link. Fortunately, this turned out to be really easy since both these links start with <a href="http://picasaweb. So I did a simple substitute command typing

:%s#<a \zs\zehref="http://picasaweb#target="_blank" #g

The only nontrivial part of this command are \zs and \ze which basically define where the substitute text (the target attribute) will be inserted.

And now the "magic" part. After applying this command, I typed :wn to store the changes and move to the next buffer and pressed q to stop recording. At this point, I looked at the number of remaining buffers and applied the macro that many times (50@a). The screen started blinking as Vim was crunching away and after a second or so stopped with the message "Cannot go beyond last file" which is expected due to the :wn command on the last file.

And that was it. The whole thing took me about a minute to do. Sure, I could have written a script that did the same thing but it would have probably taken a few more minutes, and, after all, Vim was made for these sort of things. I tried searching for the video that taught me this (i.e. that mentioned it - once you hear about it, it is very obvious) to give due credit, and managed to find it here (it's by Derek Wyatt). Till next time, happy Vimming :).

Vim C++ macro-like expansion script

Hi again!

Today I'll release a Vim script I wrote a few weeks ago (and have been planing to write for a long time). What it does is enable fast typing of idiomatic things like for loops, STL container declarations etc. The philosophy behind using a script like this is in line with the general philosophy of Vim, which is to make editing as fast as possible (fewest keystrokes to achieve a result). Also, writing idiomatic code is known to be error prone (ever typed ++i in an inner loop instead of ++j?) and is definitely tedious.

Another reason for writing this script are TopCoder competitions. Unlike most algorithm competitions, at TopCoder speed is a big factor. You only have 75 mintes to solve 3 problems (of increasing difficulty) and your submissions are timed from the moment you open the problem statement. Also, there is no partial credit for problems. After the coding phase, there's a short intermission and then a 15 minute challenge phase where other coders get a chance to view your source code and construct legal inputs that will break it (thus getting additional points for themselves and bringing your point total for that problem to 0). Finally, if your code survives the challenge phase, it has to pass all the automated system tests (some of which are "random", and some of which are specifically constructed to trip up wrong approaches). Some people at TopCoder use macros to achieve pretty much the same effect as this script. The main problem I have with this option is that it makes the code look ugly (sort of obfuscated). This makes it unnecesarilly hard for other people to challenge (although that's probably not the main intention) and, more importantly, it's bad style. Of course, using this script (or the macro method) probably won't increase your TopCoder rating; TopCoder algorithm competitions are primarily about quicky inventing (or applying) the correct algorithm, and the implementing it correctly and with reasonable efficiency. Still, being able to type code faster is always a good thing :).

So let me finally give you a short tour of what exaclty the script does (for the full list, see the comment on top of the file). The current functionality can be seperated into two groups: loop/block expansion and containers and type expansions.

Very often, you'll want to loop through an STL container, or repeat something n times. To do this, you'll write something like this:

for (int i=0; i<(int)my_vec.size(); ++i) {
  // per-item code goes here
}

Note that in "real" code you might use size_t or, better yet, vector<int>::size_type. Also notice the int cast on the container size. Comparing a signed to an unsigned value is not entirely safe in C++ (The int gets converted to an unsigned (assuming vector<int>::size_type is size_t and size_t is a typedef for unsigned, as is common on PCs) in what is called the usual arithmetic conversions. This can cause problems if the value of the int is negative, in which case, on a 2s complement machine, becomes a positive number and you get the "wrong answer".). Still, in idiomatic code like this, using a cast is an OK option IMO.

Anyway, to write the above loop with this script, you'd write:

@f i #my_vec

and press <F2> (I'll talk about the keyboard mappings in a sec). Vim will then expand the line to the above code (without the // per-item code goes here comment, of course :) and (if you were in insert mode) position the cursor at the "right" place (one indent level inside the block) so you can continue hacking.

If you want to loop to a value (or variable), and not trough a container, you'd write something like this:

@f i n

and get

for (int i=0; i<n; ++i) {

}

You probably guessed it, but the general syntax is:

@f index_var_name upper_limit

Where the upper limit can be preceded by a # to produce the size of a container.

If you want the loop condition to be less than or equal, prefix the upper limit with an equals sign like:

@f index_var_name =upper_limit


There's also a version that lets you specify a non-zero lower limit

@f index_var_name lower_limit upper_limit


and versions for looping downward

@fd index_var_name [lower_limit] upper_limit

(the square brackets indicate that the lower_limit is optional).

Finally, there are similar expressions for while and do-while loops, as well as for if branches:


@w expression
@d expression
@i expression


Finally, there is also support for for loops with iterators through containers. The syntax for this is (I'll explain what <container_expression> and <expanded_container_expression> are in the next paragraph):

@iter iter_name <container_expression> container_name

which generates

for (<expanded_container_expression>::iterator iter_name=container_name.begin(); iter_name!=container_name.end(); ++iter_name) {

}

There's also a @iterc version that generates a ::const_iterator iteration.

The second part of the functionality is container and type expansions. Basically, for:

@vi

(and <F2>) you get:

vector<int>

This works in all parts of the line (except in double quotes and comments), unlike the loop/block expansions that only work on the start of lines (modulo whitespace characters). I won't list all the supported expansions, but all the vector, set, map and pair that I tend to use frequently are supported. A few examples:

@vvi
@sd
@msi
@pllll

expands to:

vector< vector<int> >
set<double>
map<string, int>
pair<long long, long long>

Keyboard mappings are defined at the end of the script (so you can easily change it, if you like).

nmap <F2> :call BudaCppExpand()<Enter>
imap <F2> <Esc>:call BudaCppExpand()<Enter>A


I have some further ideas that I'm going to add in the short term. Since this post has gotten very long, I won't discuss them here except to say that you can read about them in the comment on top of the script :).

One important thing in Vim is making a habit out of things (so you don't have to think about whether or not you're going to press 'i' or 'a' or 'A' etc.). Since this script is pretty new I still haven't really gotten into the habit of using it as much as I'd like, but I'm trying :).

Without further ado, you can get the script here. To use it, you can put it into the ftplugins Vim directory.

If you find the script useful, have any suggestions about improvements and possible further additions or find a bug, please let me know!


Edit:
While writing this post, I noticed that it might be useful (and in line with the style of the script) to be able to expand things after } else to form if/else blocks properly which wasn't possible with the version I posted. I fixed that now so you should be able to do things like

if (a > b) {
  // do something
} else @i a+b+c > 42

and get it expanded into

if (a > b) {
  // do something
} else if (a+b+c > 42) {
  
}