VimIntegration

From Freehackers
Jump to: navigation, search

by Bluebird

Introduction

Vim is one of the most powerful editor for software development. Its portability and stability is a key to that. Vim regularly beats every other editor for software on polls in free software related websites. So, Vim is great, no doubt about it.

But there is something not so great about vim: it is very standalone. It is difficult to integrate vim into development environements. Vim has its own editor window, its own shortcuts, its own everything. There are graphical interfaces but they do not make vim more integrated. For many years, this was not a problem. Many developers were (and still are) running vim in one terminal, and make/gcc in another one. But, with advances of desktop environements like Gnome or KDE, and Integrated Development Environement like Eclipse, KDevelop, Anjuta, NetBeans and more on the other side, the use of advanced graphical integration of Vim has become something desirable.

The goal of this page is to maintain information about the successful and failed attempts of Vim integration and share technical knowledge about it.

Vim Integration : the challenge

When talking about Vim integration, there are usually two different aspects:

  • feature integration: the goal is to get the advantages of both the Vim feature set and the

integrated software feature set. It does not necessarily means merging the Vim window and the window of the integrated software.

  • window integration: this is to have the vim window become completely part of the integrating

software. No need to switch windows all the time, to switch between the integrating software and Vim.

Both these integration poses challenges. Integration at the feature set is the most desirable item. If it works, one can probably live with separate Vim and IDE windows.

Feature integration

To get feature integration, one need to get a bidirectional communication channel between Vim and IDE. The IDE must be able to open windows in Vim, set the cursor, set the text, even send key strokes. And the IDE must be notified if when the user opens or closes windows, when he moves the cursors, when he presses some key shortcuts.

Possibilities for implementation of this communication channels are:

  • vim server mode: works in the direction IDE to Vim, but not in the other direction.
  • proprietary mode: modify Vim to introduce some proprietary communication channel. Not very convenient because we restrict to a modified vim.
  • socket mode: Vim features a socket communcation channel, as part of the netbeans integration

features.

Window integration

On X11/Linux, out of process window integration is not too difficult. If you use the Gtk Vim implementation, Gtk features what is called a Gtk Socket, which is in practice a way to embed a Gtk window inside one another. This works out of process and takes care of forwarding the Gtk events to the embedded window. The method is limited to Gtk on X11/Linux. In theory, Gtk uses the XEmbed protocol which is also implemented by Qt. So, one could integrate a Gtk window inside a Qt application using Gtk Socket and XEmbed, but I haven't seen real-world examples of this.

If we don't talk about out of process but in-process integration (meaning, Vim runs inside the same process as the integrating entity), we run into a big problem. Vim has been designed for terminal input: one keycode is received, handled, and display is updated. The architecture of Vim is not really suited to the notion of event loop and asynchronous window refresh events, as generated by most GUI toolkits. To bridge that architecture gap is not simple and may create CPU intensive solutions (because active polling is needed).

For other platform than Linux/X11/Gtk , I haven't heard of any window integration features.

One nice trick is to use a terminal emulator, and then run Vim inside the terminal emulator. This is not as beautiful as a full graphical integration but it's a very convenient way to embed the Vim window.

History of Vim Integration

There are many attemps at Vim integration into IDE. I was part of 3 of them, which explains why they are much more detailed than the others. Ironically, the one that succeeded are not mine :-) You'll find a short description here of the different projects and their status.

KVim KPart

Website: ???

Status: dead.

KVim was a straight port of the Gtk client of vim in KDE/Qt. It ran reasonably well. We had problems though that sometimes, it would take 100% CPU. That's because Vim does not use the event loop of Qt, it does not really have the concept of event loop. So to make vim refresh correclty, we sometimes had to actively poll the Qt event loop for painting events. Nasty. But it worked not so bad.

And then came KPart. KPart is the destkop integration technology of KDE. It allows in-process embedding of software components, to make them usable accross the whole KDE desktop. We tried to make KVim fit into a KPart. But it was far a difficult task. The problems are that because KPart is in-process, it runs within the event loop of Qt. The trick we used with KVim was showing its limits within a KPart. We had problems of Vim taking 100% CPU, or KPart components without Vim inside.

We also wanted window integration. We used the XEmbed protocol, which was quite young and not that well working on the Qt side. There were race conditions, so that sometimes, the window of Vim would not get embedded properly but would be displayed separately.

Another problem was upstream modification of Vim. We sent one patch to Bram Molnaar, which was about 20 lines, which just added one command-line argument for Vim, for the XEmbed protocol. It took about one year and half for the patch to get accetped, eventhough the patch would change nothing to any of the existing vim code. The patch said that if a specific argument was passed to Vim, Vim would display its Window Identifier on standard output. If it took one year and half to get such a small change into Vim, how long would it take to get something real ?

Discouraged by the non-cooperative attitude of Bram and by the non progress of KVim KPart, and of the big mess that the vim sources are (they are written in a C of a previous era; we are probably too young to appreciate it), we decided to drop that project. To start a new one ! Let me introduce: Yzis.

Yzis

Status: dead.

Yzis was born of all the frustration of the KVim KPart. We decided to write a brand-new Vim, from scratch. It would be easy to integrate that new editor into any kind of IDE. Yzis would be primarly written in C++ and QtCore, be portable accross Windows, Linux and MacOs, feature an interface in Qt, ncurses and more. There were many nice things in Yzis: lua scripting, kate syntax highlighting, many non-regression tests.

In the end, the story looks like KVim. We went quite far, we had something usable from the user point of view: an editor with syntax highlighting, multiple windows on multiple buffers, many vim key bindings and a scripting language. But then it became too tedious to maintain. The wrong design decision and the quantity of work to make it evolve to a true Vim clone became overwhelming. The project was just too ambitious.

Mikmak, the main developer eventually left and after a failed attempt to revive the project in 2006-2007, I also left. The codebase was just too complicated, and doing a Vim clone from scrach was simply too difficult, especially in 3 different OS with 2 different frontends.

Yzis is now dying slowly but surely.

The interest generated by Yzis were quite high. The demand for Vim integrated into an IDE, or for Vim as a KDE KPart is still very strong. Since Yzis is dead, a few contributors to Yzis started their own project.

Ollie

Website: http://ollie.thrawn01.org/index.php/Main_Page

Status (may 2008): just started.

The core of Ollie will be written in LISP. One of the frontend of Ollie will be Qt/C++. The project is started from scratch.

In my opinion, Ollie is repeating the mistake of Yzis: too ambitious. Developing something like Vim is just too far away, especially with GUI abstraction. I wish their authors good luck but frankly, I will be surprised if it ever reaches the level of completeness of Yzis. On the other hand, I would have said the same thing when Mathias Ettrich started KDE: doom to failure, too ambitious. So, don't take my pessimistic view of the world.

Vim keybindings in Kate

No website.

Status (january 2009): working, and released with KDE 3.2

The idea is to use the well establish editor Kate and add Input Modes to make it behave like a Vim Clone.

Erlend Hamberg worked on it. The vi mode is now integrated directly in kate and was released along with KDE 3.2 . The functionality is still limited but is working nicely.

Vim Wrapper

Website: http://www.freehackers.org/VimWrapper

Status (may 2008): Just started.

During Yzis's fall, there were discussion on what to do next. One guy suggested to restart the KVim way: work with Vim to get into KDE. He also suggested to use the Netbeans interface stuff.

So, I had a deep look at the Netbeans integration in Vim. Either it was not there when we started KVim or we overlooked here, but the Netbeans integration in Vim provides mostly everything that we need for a good integration of Vim with other IDE.

Netbeans is a misnomer. This is the name of the first IDE in which Vim was integrated. Netbeans stuff in Vim is a socket protocol, with a generic communication interface that allows to receive events from Vim when something happens (new buffer opened, buffer closed) or to get and set data in Vim (cursor, text content, ...). There is nothing that really depends on Netbeans or java in this.

The socket interface makes Vim run in out-of-process fashion, while the other end of the socket can integrate nicely with any existing event loop.

This ability to use Vim is very much unknown to the outside world, so I started this Vim Wrapper project, where I want to collect information on how to use the Netbeans interface in Vim in other IDE, and provide several example information on how to do this. This is how I come to write this page.

PIDA

Website: http://pida.co.uk/

Status (may 2008): Successful and active.

PIDA is an IDE written in Python and PyGtk. It works mostly on Linux even if Windows and MacOs-X ports are available. PIDA integrates two editor engine. The first one is scite (http://www.scite.org scite) and the second one is Vim.

The Vim integration is composed of Vim window embedding using Gtk sockets and a protocol to communicate with Vim based on X-Atoms and the netbeans interface. Basically, instead of using a socket, X-Atoms are used to send and receive data from Vim.

The integration is successful, but I don't think it is portable accross desktops or window environements. The Gtk socket trick is nice (it uses XEmbed protocol, just like the original KVim) but will not work on non X11 environements. I am not even sure it works for a wrapper not written in Gtk. The X-Atom stuff is very specific to X11.

So, this integration of Vim is a success and shows that it is possible, but it does not easily transfer to other wrapping technology.

Clewn

Website: http://clewn.sourceforge.net/

Status (may 2008): Successful and active.

Clewn integrates gdb and Vim together. You can run gdb commands, and get Vim display breakpoints, current execution line, ...

Clewn is written in C and uses the Netbeans interface of Vim for all it work.


AGIDE

Website: http://www.a-a-p.org/

Status: Successful and dead.

AGIDE is an IDE written in python, for AAP, which is a replacement of Make started by Bram Molnaar, the author of Vim. AAP and AGIDE development have stopped nowadays, but AGIDE used the Netbeans interface of Vim to successfully control Vim and integrate it into an IDE.

Netbeans External Editor

Website: http://externaleditor.netbeans.org/

Status: dead.

This project is dead nowadays, but it seems to be the project that triggered the integration of remote control through a socket prorotol into Vim. The plugins to Netbeans has not survived history but the interface written was generic enough that many projects are using is successfully now.


VimPlugin (Eclipse Integration)

Website: http://vimplugin.sourceforge.net/wiki/pmwiki.php

Status (may 2008): Successful and active.

VimPlugin integrates Vim into Eclipse. It took an original appoach: use a terminal emulator to integrate the vim "window". The communication with Vim is based the Netbeans interface, with an implementation in Java.

As of may 2008, the project seems very active.


EClim (Eclipse Integration)

Website: http://eclim.sourceforge.net/

Status (may 2008): Successful and active.

Eclim took an original approach to Eclipse integration. Since it was so hard to integrate Vim into Eclipse, they integrated Eclipse into Vim. EClim comes a set of Vim plugins and java bridges to Eclipse, so that you can do what you would normally do in Eclipse directly inside Vim !


Vim Socket Protocol: Netbeans

It is completely possible to control Vim from an external entity. If you type :help netbeans in Vim, you will get some information on how to do that. Basically, if you pass special argumentss to Vim (-netbeans), Vim will open a socket connection to a given host:port and then uses the socket connection:

  • to receive "orders" that can: open and close documents, modify text, move cursor, show popups, highlight lines, ...
  • to send "events": new buffer opened, buffer was closed, text was deleted, shortcut key was pressed.

This gives almost complete control over Vim, using one of the most portable and flexible interface: a socket connection.

The protocol is text based and once you work your way around the little oddities, it works very well.

Some patches to enhance the netbeans interface here