Parnassus

Introduction: IDE plugins

IDE plugins are a mysterious topic.  With little official documentation, most material is scattered across a variety of blogs and websites, often out of date and referring to pre-Galileo IDEs.  But the documentation is improving and there are a number of great websites (try David G Hoyle’s – the best website on the the ‘Open Tools API’ I have seen) delving into the interfaces that OTAPI makes available.

If you want to write a normal IDE plugin, such as a dockable window containing your own material, something that interacts with the projects or project groups, adds or removes text to source code in the editor, implements a debug visualizer, etc, then the above collection of links plus some Googling should get you going.

But there are some things that the IDE’s API simply does not expose, things that advanced plugins need to do – like painting on the code editor.  There is no painting access or interface in the published API.  So how do open-source (like GExperts or CnPack) or commercial (like Castalia) plugins do it?

I can’t answer how commercial plugins work, since I don’t know, though I suspect very similarly to the technique presented here.  But I can tell you a technique I figured out with the help of looking at how CnPack achieved it to solve a few issues.  It works well, and seems stable, reliable, and fast.

This is the first of a series of posts on advanced Delphi IDE plugin functionality.  There are several goals: now I’ve found out how to do some of these trickier things, I want to share the information.  Second, I am writing a family of IDE plugins – things to fix parts of the IDE that after many years of usage I wish worked differently, or to add functionality that doesn’t exist.  And while I want to sell these plugins for a (small to trivial!) cost – I hope you, Delphi readers, will find them as useful as I will – I don’t want to hide the information I learned about how to build them.  Quite the opposite: I’d love to see a healthy plugin ecosystem develop around Delphi and Appmethod, rather than the small number of open source and commercial plugins that currently exist.

The following assumes you have a rough idea of how to create an IDE plugin – ie how to create a BPL that it loaded by the IDE.  But even if you don’t, you can read on anyway :)

Contents

• Introduction, and a few quick notes on IDE plugins (above)
• What is TEditControl?
• Painting unsuccessfully: by hooking windows and messages
• Painting successfully: patching IDE methods at runtime

By the end of this article, if you already know how to write an IDE plugin, you should be able to make your plugin paint on the code editor in an event that occurs each time a line is painted.  Completing the functionality, such as drawing in the right spot for each line, handling the gutter area, handling folded code etc, will be discussed in future articles in this series.

What is TEditControl?

Delphi/C++Builder is written mostly in Delphi, and so uses VCL menus, actions, and controls, an Application object, etc.  You can see this when writing standard plugins, where you use a descendant of TForm (eg a TDockableForm) to make a dockable window in the IDE, add images to the IDE’s image list, add menu items, etc.

This extends to the code editor itself, an instance of TEditControl, an internal Embarcadero-only descendant of TWinControl.  Here you can see it in Spy++:

An instance of TEditControl seen inside the IDE.

There is a single instance of the edit control shared by all source code tabs in the main IDE window – the tabs are more like a TTabControl than a TPageControl.  However, you can have multiple edit windows, each with an edit control, even showing the same source code.

The relationship between TEditControls and the documented OTAPI interfaces, such as IOTAEditView, can get complicated and will be addressed in another article.

But how do you find this control and paint on it?

Painting unsuccessfully: by hooking messages

I tried a number of techniques that didn’t work and I won’t delve into great detail.  My rough approach was to hook the window messages or events related to painting, and also to scrolling or other events in order to update internal information in my own plugins.

For example, I tried setting the control’s WindowProc to my own, in order to catch WM_PAINT messages – none were caught.  You could also try finding an OnPaint event or similar.  I also tried catching WM_HSCROLL and WM_VSCROLL messages via WindowProc, installing an application OnMessage handler (note if you try this approach that it uses a multicast event dispatcher, so don’t assign your OnMessage event to Application.OnMessage directly), and installing a message hook, and none of those methods caught those messages either.  I don’t know why: it’s possible that I was hooking the wrong control (despite it being the edit control) or that the messages are handled somewhere else.  I don’t know the internal structure of how the IDE interacts with the edit controls.

I would recommend you do not go down this road, since I had no success with this or several variations of it.  (That’s one short sentence that describes quite a bit of spent time.)

To do any of these, you also need to find instances of the edit control itself, wherever it is (or they are) in the IDE, another minor complication.  But it turns out that purely for painting, you don’t need to know about any specific instance of the edit control.  Instead…

Painting successfully: patching IDE methods at runtime

One method of implementing painting I had read about online was hooking or patching methods of the IDE classes.  This was referred to obliquely, as ‘you can patch the methods’, with no details about how.  So figuring out how was my next approach.  There are two steps: how to patch any arbitrary method, and then finding the right method(s) to patch.

How to patch an arbitrary method at runtime

A method has an address: a location in memory at which it starts.  Virtual methods or thunked (eg most Windows API) methods jump to this from another location in a variety of ways.  To patch a specific method, you need to find its true location (following the virtual call or the thunk) and then overwrite the first few bytes of the method itself with a call to jump to your replacement.  If your replacement does everything you want, that’s all you need to do, but if you want to call the original implementation as well then you need to replace the bytes you overwrote with their original contents and then call the original method by its original address.

Luckily there are several libraries out there for doing this at runtime in Delphi.

Important note: Please see an update to this section in Part 2 of this series. The below library is the wrong library to use to patch at runtime – it works, but it is incompatible with other plugins also trying to hook the same methods. Instead, follow the method described in Part 2.

The patching code I use is a variant of Chau Chee Yang’s code.  (You could probably also use another, such as the Delphi Detours Library.)  Using it, you can patch a method with code similar to:

  FOriginal := TCodeRedirect.GetActualAddr(…address of a method…);
  FHooked := TCodeRedirect.Create(@FOriginal, @ReplacementMethod);

GetActualAddr is a method that checks for an indirect jump.  I have not investigated the details of how well this handles both virtual methods and WinAPI-style thunked methods – in fact while I know the theory of how both are implemented I’m hazy on the exact details since I’ve never had to examine them closely.  This code works as-is for the purposes of the patching done here.

My version of the above code includes a bugfix for the Disable method: the code on the linked page attempts to write to memory to patch back the method, but it fails because it does not change the protection of the memory first allowing it to be written to. It also does not re-protect it once written, or flush the instruction cache afterwards.  A simple modification based on Enable gives:

procedure TCodeRedirect.Disable;
var
  OldProtect: Cardinal;
  P: pointer;
  n: NativeUInt;
begin
  if FInjectRec.Jump <> 0 then begin
    // David M – based on Enable()
    P := GetActualAddr(FSourceProc);
    if VirtualProtect(P, SizeOf(TInjectRec), PAGE_EXECUTE_READWRITE, OldProtect) then begin
      WriteProcessMemory(GetCurrentProcess, GetActualAddr(FSourceProc), @FInjectRec, SizeOf(FInjectRec), n);
      VirtualProtect(P, SizeOf(TInjectRec), OldProtect, @OldProtect);
      FlushInstructionCache(GetCurrentProcess, P, SizeOf(TInjectRec));
    end;
  end;
end;

Finding the right method to patch

You can now patch a method on a Delphi class live at runtime.  But which method?

At this point, I started searching through the methods in coreideX.bpl looking for appropriate ones to patch.  It looked like being a long search with many false starts and crashes based on incorrect method prototypes.  But I realised that there are open-source projects which implement editor painting already and they may well use this or another method, and so I looked at one to see how they achieved it: CnPack, which I had already downloaded based on David Heffernan’s suggestion for how to keep track of code folding – a topic I will return to in a later article.

Note: I have to thank the authors of CnPack for their code, which helped me solve some problems with my own.  It took me quite some time to figure this out, and I gratefully used their code for reference and help.

CnPack does indeed patch IDE methods – in fact it patches a lot of IDE methods, many more than just this one.  They patch one in particular, though: not a generic paint method for the control, but a method PaintLine – which looked perfect, since the purpose of painting on the edit control is (probably) to paint on specific lines of code, not to paint ‘in general’.

Remember, patching the TEditControl.PaintLine method means that whenever any instance of TEditControl tries to paint a line it calls your implementation instead of its own.  You don’t need to know about specific instantiations, just the class, because you are writing a replacement method for the class.  Your method will then call into the original patched-out code, in order to draw normally, as well as performing whatever painting it wants to do.

Note: this is completely unsupported by Embarcadero (or me.)  Their internal code can change at any time, even in the one version of the IDE.  Nevertheless, this specific method seems to be stable since at least Delphi 2010.  CnPack has a number of different method prototypes for different versions of the IDE, so this has changed in the past and may in the future.

The method in question is Editorcontrol.TCustomEditControl.PaintLine.  It resides in coreide*.bpl, such as coreide160.bpl, which is always loaded into the IDE.

PaintLine seen in Dependency Viewer

In the above screenshot, you can see PaintLine in Dependency Viewer with a “mangled” name.  Once demangled via tdump, it is (given in C++ format):

__fastcall Editorcontrol::TCustomEditControl::PaintLine(Ek::TPaintContext&, int, int, int)

We don’t know the structure of TPaintContext (which would be very useful) nor the meanings of those three integer parameters (yet; actually CnPack has deciphered two of them.)  Also not shown is the implicit Self parameter.  But this is enough to create a stub.

Hooking PaintLine and using it to paint

We already have a prototype for the method, but in the form of an object method not a standalone procedure.  To hook, we need to define a compatible method with the extra Self parameter:

To be completely clear, this is not an object method but a plain, non-OO procedure.

And to get it to be called, hook the IDE’s method:

Note: FCoreIDEHandle is the module handle of coreide*.bpl, loaded into the current process.  Finding this is left as an exercise for the reader, and there are variety of ways.  This method may get you started.

Note: Another exercise for the reader: call back into your own painting class to call your own code, rather than remain in procedural code only.

Note: Since Delphi is written in Delphi, you can find out quite a lot of information when debugging the IDE in a second instance of the IDE – one example is inspecting the Self object (the edit control) above if you break in your method.

There is a handy VCL class to represent a canvas on a TWinControl: TControlCanvas.  Create and use one:

• It always draws at (0, 0), not at the line that is supposedly being painted.  Since this is called for every line, the above ‘Hello world’ text is actually drawn over itself in the same palce many times, once for each line the code editor paints.  We need to extract the line information and paint at the correct offset down for each line, and correct offset right to handle the code gutter
• It doesn’t have any idea what unit or text it is actually drawing over
• It doesn’t have any idea what lines of text are visible
  • …and when it does, that is going to have to include handling code folding

Why am I investigating these topics?

I’m glad you asked :)

I’ve used Delphi since it was Turbo Pascal, first as a teenager/student and then over the past decade in action as my day-to-day IDE – including C++ Builder.  There are things about the IDE I wish worked differently: mistakes I keep making because I expect certain behaviour, even though I should know by now it works subtly differently.  There are also features I wish it had, and feature tweaks that would just make it a bit ‘slicker’, something it needs in order to compare to IDEs like Visual Studio which have a very polished UI.  Polish matters in an application you use all day, both visually and in behaviour.

Preview of an alpha, pre-release version of Parnassus Bookmarks

It will be available soon, and if you are interested in beta-testing please contact me by email or by commenting below.