Graphical Syntax View for Xtext

I demoed it so often that I almost forgot blogging about it: In Xtext 2.0 you can now visualize the syntax of your DSLs grammar in a live railroad diagram view. The view supports navigation between the editor and the view and you can export your diagrams to a file. We even have a special representation for unordered groups. Watch this screencast to see it in action:



The syntax graph is implemented in plain Draw2d using a custom layout algorithm. It is technically not related to the generic graphical view I presented in my previous blogpost.

Using Xbase to Define a Generic Graphical View

While textual representations such as code are perfectly suited to process the details of an object model, a graphical view can be very helpful to display the relationships between objects.

Most graphics frameworks in the Eclipse ecosystem provide graphical editors. Usually, the semantic model is mapped to its graphical representation using a hard wired transformation, be it in code (GEF, Graphiti) or in models and a code generator (GMF, the upcoming Spray project). There are two issues here: First, as opposed to a graphical view, a graphical editor requires the mapping to be bidirectional. That constrains the transformation a lot, e.g. requires having the same structure on the graphical as on the semantic side. Second, hard wiring the mapping introduces extra compilation/generation/deployment turnarounds.

I've created a prototype of a graphical view that is configured using two textual DSLs: A mapping DSL that declaratively describes the transformation from semantic objects to graphical elements, and a styling DSL to modify the graphical representation. The graphical view interprets the mapping and the stylesheet on a given input model, giving immediate feedback on changes of these live models.

Both DSLs are implemented in Xtext 2.0 and make heavy use of the new Xbase expression language library. As Xbase uses Java's typesystem, we can refer to any POJO as an input model. Xbase's rich semantics allow comfortably navigating the models references, and its interpreter can be customized easily to fit our needs. The graphics are implemented using plain GEF and Zest layouts.

The following screencast gives you a short demo. (Sorry for the poor video quality, it gets better if you got to full screen mode)

Rename Refactoring in Xtext 2.0

The upcoming Eclipse Indigo version of Xtext ships with experimental support for generic rename refactoring. It allows you to rename model elements and automatically fix all links to these elements. This blog entry describes the main components behind this new feature. The following screencast shows refactoring for a domainmodel language:

Enabling Refactoring

To enable refactoring support for your language, you have to add the RefactorElementNameFragment in the fragment section of the MWE workflow of your language, e.g.

// rename refactoring
fragment = refactoring.RefactorElementNameFragment {}

The fragment has an additional flag useJdtRefactoring which can be used to delegate to JDT's refactorings for languages using the JVM types compiling to Java (i.e. the domain model example or Xtend). Usually users are fine with the defaults.

After running the workflow, you will have working refactoring support - at least if you have stuck to the defaults with regard to naming, cross-referencing. and indexing. Give it a try. If it doesn't work yet you have to adapt the infrastructure a bit more.

Customizing

The most likely component you want to customize is the IRenameStrategy. This component defines how the declaration of the target element is performed. It has two major responsibilities:

• Apply and revert the declaration change on the semantic model (methods applyDeclarationChange and revertDeclarationChange))
  
  The default is to look for an EAttribute 'name' on the target object and set its value using EMFs reflective API.


• Create the LTK Change objects of the declaration change. These changes will be aggregated, checked for overlaps, presented to you in the preview and finally executed if you apply the refactoring.
  
  The default is to use the ILocationInFileProvider to locate the text range representing the name and create a ReplaceEdit for it.

As the IRenameStrategy is a stateful object, you have to bind a custom IRenameStrategy.Factory instead of the strategy itself to integrate your changes.

The other component you might want to customize is the IDependentElementsCalculator. Dependent elements are those elements whose qualified name changes when the target element is renamed. E.g. when you rename a Java class the qualified names of its inner classes change, too, thus references to these have to be updated. This calculation is performed by the IDependentElementsCalculator. By default, all elements contained in the target element are added. This matches Xtext's default strategy of qualified name computation.

The Internals

The following describe the steps taken in the refactoring and the responsible components

Refactoring UI

In terms of UI, the refactoring of Xtext looks pretty much the same as in JDT. When the user has triggered the refactoring - by a keystroke or using a context menu entry - the editor goes into a linked editing mode allowing to edit all occurrences of the element in the current document at once. Pressing the keys again or using the popup menu, the user switched to a dialog wizard or a preview wizard. Thanks a lot to Holger who did the major work on the UI.

The UI parts pass an element of type IRenameElementContext to the non-UI components, holding information on the target element, its EClass, the current editor etc.

Non-UI

The non-UI part and thereby the actual refactoring is mainly managed by the RenameElementProcessor.

As a first step, it loads the model declaring the element to be renamed into a new ResourceSet that is configured with respect to the declaring language. This allows to easily abandon all changes as well as avoiding concurrency issues. Some basic checks are performed like 'is the element available' or 'is the resource writeable' and the IRenameStrategy is initialized.

Then we calculate the dependent elements.

After that the URIs of the elements that are affected by the refactoring are calculated. Those URIs may change, e.g. if you use a name-based fragment strategy. So we calculate the original URIs, apply the change and recalculate the new URIs. The result is a Map<URI, URI>. This task is the responsibility of the IRenamedElementTracker.
In the end, the changes are reverted to leave the resource in the same state as before.

The final step is to handle the cross-references to the renamed elements. We have to consider resource internal cross-references as well as the ones from other resources. The latter can be retrieved from Xtext's index. Note that if your external cross-references are not indexed, they won't be found and consequently won't be updated by the refactoring.

As references can come from any other language, we use a ReferenceUpdaterDispatcher that searches for local and indexed references and dispatches to the IReferenceUpdater of the referring language. We have two implementations of this interface: The DefaultReferenceUpdater works with Xtext resources and uses the language's serializer to update the reference text. OTOH, the EmfResourceReferenceUpdater works for all EMF based languages and re-serializes the whole resource after applying the semantic change.

Rename Participants

Finally, one refactoring can trigger another. E.g. when renaming a rule in an Xtext grammar, the returned EClass should be renamed, too. For these cases, you can register a RenameParticipant by the common means of LTK. If the target of the participant is Xtext based, you can use a AbstractProcessorBasedRenameParticipant in order to reuse lots of the already described classes.

EclipseCon 2011 Slides

EclipseCon 2011 was once again a cool conference, even though I really needed a week of holiday to recover from the bad weather, the jet-lag and the general lack of sleep :-) This is why it took me some time to upload the slides of our presentations. But better late than never, here they are.

First, Sebastian and me presented What's Cooking in Xtext 2.0. The room was almost full, and when I asked a vast majority of the audience had already had contact with Xtext. To me that shows that Xtext has really become mainstream, a really good feeling. Here are the slides:

What's Cooking in Xtext 2.0

On Thursday, Sven, Sebastian and me gave the tutorial on Pragmatic DSL Design with Xtext. There were so many participants, that we had to switch rooms with Stephan Herrmann's tutorial on Object Teams. Thanks for that, Stephan! And here are our slides:

Pragmatic DSL Design with Xtext, Xbase and Xtend 2

Did I mention EclipseCon?

I just realized that I haven't blogged on this year's EclipseCon yet. Time to do that. You might have guessed it:


As a committer to several Eclipse projects, I am really looking forward to meeting "the family". It's always been big fun and I guess it will be the same this year. But the main reason for going there is of course:


Together with Sebastian I will give a talk on What's cooking in Xtext 2.0 on Tuesday at 2pm. We're eager to show you all the new features of Xtext we've been working on during the last months, especially the expression language library Xbase and the code generation langauge Xtend.

You should not miss our tutorial
Pragmatic DSL Design with Xtext, Xbase and Xtend 2 on Thursday at 10:30am. Sven will join us to give you a walk through tutorial on a modern way of creating DSLs. We just had another rehearsal on this one and I promise you it will be a real burner.


All of that wouldn't have been possible without the support from the company we're working for: itemis. Into the bargain we're bronze sponsors for this years conference.
It is great to work for a company that really lives open source.

JDK Source Code on MacOSX

After installing the latest update of Java for MacOSX Snow Leopard - must have been in March 2010 if my file system is not cheating on me - I missed the source code for the JDK classes. I remembered I had to install the Java Developer Package from the Apple developer center, but I couldn't find any. Consequently, I learned to live without the sources and forgot about the issue.

Yesterday I stumbled across a big performance issue in the upcoming Xtend2 compiler that only happened on my machine. Profiling yielded that it was because I have no local sources (another strange story).

So I decided to look for the developer package once again et voilà: Apple had been so kind to release it in the meantime. BTW, the timestamp says October 20th :-o

So here's what I did to get the MacOSX Java sources back (it's on your own risk, I disclaim any responsibility if you should break or machine)

1) Get an Apple Developper Connection account (it's free, but you have to register).
2) Login choose Java and download the Java for Mac OS X Developer Package matching your OS and Java version.
3) Install the DMG.

The install location of the JDK is somehow weird - e.g. you cannot directly open it in Eclipse's "Choose JDK location" dialog. To make it the default, I found it easiest to retarget the symbolic link pointing to the default JRE. You have to be super user in this case.

4) Open a Terminal and enter

cd /System/Library/Frameworks/JavaVM.framework/Versions
sudo rm CurrentJDK
sudo ln -s /Library/Java/JavaVirtualMachines/1.6.0_22-b04-307.jdk/Contents CurrentJDK

If Eclipse uses the default MacOSX JDK, you should now have sources.

Learn how to create a language for Android development using Eclipse Modeling

If you are lucky, you have already registered for
Eclipse Summit Europe 2010 in Ludwigsburg, Germany. If not, I have sad news for you: It is sold out. Sorry.



If you're there, I hope you registered for our tutorial Creating a Language for Android Apps with Eclipse Modeling. A joint team from Obeo (Mikael Barbero and Stephane Begaudeau)
and itemis (Holger Schill, Dennis Hübner, and me) will introduce you to the basics of the Android SDK and show how EMF, Xtext, and Acceleo can be leveraged to simplify the development of Android Apps.
Bring your laptops, it's going to be a hands-on experience.

If you like this approach, learn how it works on larger scale in the talk Models To Go: How We Built a DSL for Mobile Apps With Tools From the Eclipse Modeling Project
by Peter Friese and Heiko Behrens who implemented the official ESE App for the iPhone.

Curious about what we have in mind for the future of Xtext? Then you must see Live Coding: Building a UI styling language for E4 with Xtext. Sven Efftinge and Sebastian Zarnekow will demonstrate Xbase, an Xtext based expression language bringing behaviour to your DSLs.
Sven will also talk about Dependency Injection which is the key to extensibility and an easy programming model, not only in Xtext.

Itemis will give two more presentations: Andreas Unger and Axel Terfloth from our embedded team will teach you on Developing Embedded Software with Eclipse, State Machines and Block Diagrams.
Nirmal Sasidharan will present Eclipse - an IDE for Organisational Change. I am sure you'll like these, too!

I am particularly looking forward to the keynotes, especially the one by Jeff Norris from NASA who already rocked this year's EclipseCon.

See you in Ludwigsburg!