Eclipse Con Europe 2011

The slides of my talk A Fresh Look at Graphical Editing at Eclipse Con Europe 2011 are online now. If you're interested in the code it's on github, but remember it's a prototype - no tests so far, no support. Unfortunately, there was no cake being served at 5pm, so the pie fight had to be cancelled.

As I have been asked this a couple of times at the booth: I am convined it makes no sense turning the presented view framework into a graphical editing framework. I am not entirely opposed to graphical editors but keep in mind that

1. You're going to need much more than graphical editing frameworks provide (projection, navigation, browsing, ...) I'd estimate a factor of 10-20 in the effort of building a graphical tool as opposed to a textual tool.
   
2. The bi-directional mapping from the semantic model to the graphical one is source of a lot of problems. Not all of them can be hidden from the users.
   
3. The structure of the semantic model must match the diagram model. This is not even true for the simplest models like UML2 class diagrams. You should be in a position to change the semantic metamodel to have a perfect structural match.
   

Would have been interested to discuss the last point with Bran Selic, who claimed in his keynote that UML2 is cool but the tools suck...

Eclipse Con Europe 2011 was an excellent event as always. I really enjoyed meeting all the nice people from the community. Thanks a lot for the organization and especially for the delicious food.

But remember: After the conference is before the conference. The CfP for Eclipse Con 2012 in Reston, Virgina, is open. Hope to see all of you there.

Join My Pie Fight Session at EclipseCon Europe 2011

Competition is hard at EclipseCon: Everybody has to make sure to get the most participants into his talk. As Sven already announced a quiz for the same timeslot as mine I've had to figure out something even more exciting.

So here's my idea for my session:

I'll begin the session ranting on every Eclipse project that's somehow related to graphics. I'll shamelessly reveal all of their weaknesses. Everybody involved in one of these projects will be so offended that they start to get personal, first against me then against each other. When emotions boil up, I expect people to self-organize in teams, like The Knights of GMF Runtime or The Zest'o'Maniacs. My session is at 5pm so I expect a good supply of cake as in the years before. I am starting the fight by throwing the first cake, e.g. on the GMF Tooling Ultras. They try to answer but unluckily hit a member of the Spray Front, who's already in the midst of a quarrel with a Graphiti Warrior. Thinking it was a GEF Cyclope who threw the cake... A real pie fight breaks loose.

Hehehe (demonic laughter)

Well... thinking again... the cake at EclipseCon is usually quite yummy. It would be a pity to waste it in such a way. Maybe I should come up with something more peaceful.

OK, I see, I have to make that differently: I am going to point to some general issues with graphical editing. I'll weigh the cost against the usefullness and show, why an interpreted graphical view is so much simpler to implement and to use than a graphical editor. Second part will be a live demo of a prototype for such a framework.

Maybe I should bring a helmet, just in case someone did not notice my change of plan.

Git and Bugzilla patches

I am one of these unteachable conservatives that sometimes prefer the shell over IDE. Using git is not an exception here. One of the things I've always been missing was to apply a patch that is attached to a bugzilla entry directly, i.e. by referring to its URL without downloading it to a file. Fiddling arround a bit, I found a solution that I think is elegant.

You'll need wget for this to work. On my Mac I use the one from Mac Ports.

Add the following line to your .profile (or the config file of whatever shell you use)

git config --global alias.applyurl '!wget -qO- $1 | git apply -'

You should now have a new git command applyurl. Now open the bug with the patch you want to apply in your browser and copy the link of the patch. Finally you can on the command line, you can type

git applyurl [paste the url here]

That's it.

Extending Xbase

The new expression language library Xbase allows to integrate expressions in your own Xtext 2.0 DSLs. Xbase offers a grammar, a large runtime library and a compiler as well as an interpreter to execute these expressions. Thus, Xbase opens the field for a much wider class of DSLs and annihilates prior advantages of internal DSLs over external DSLs based on Xtext.

In a series of internal workshops on Xbase, I've often encountered one question: Can we extend the expressions of Xbase? In this blogpost, I am going to show you how to do that. It is based on the domain model example shipped with Xtext 2.0, which you can instantiate by choosing File > New > Example... > Xtext > Xtext Domain-Model Example.

The Grammar

By default, Xbase only knows integer literals. In this example we are adding decimal literals.

If you want to add a new concept to Xbase expressions syntactically, you have to hook into the right rule of Xbase. As the name suggests a literal can be added in the Xbase rule XLiteral. So in the Dominamodel.xtext you have to add

import "http://www.eclipse.org/xtext/xbase/Xbase"
...
XLiteral returns XExpression:
  XClosure |
  XBooleanLiteral |
  XIntLiteral |
  XNullLiteral |
  XStringLiteral |
  XTypeLiteral |
  DecimalLiteral; // our new alternative

DecimalLiteral:
  intPart=INT '.' decimalPart=INT;

As the EPackage of the domainmodel language is generated this will introduce a new EClass DecimalLiteral as soon as you regenerate the language infrastructure.

Type Provider

For the sake of simplicity, our decimal literal will be mapped to java.lang.Double. We have to adapt the type provider accordingly

public DomainmodelTypeProvider {
  @Inject
  private TypeReferences typeReferences;  
  ...
  protected JvmTypeReference _type(DecimalLiteral literal, 
                                   boolean rawType) {
    return typeReferences.getTypeForName(Double.class, 
                                         literal);
  }
}

Compiler

The compiler has to understand what to do with decimal literals. We have to add a method that appends the concrete syntax of the Java double literal for a DecimalLiteral

public class DomainmodelCompiler {
  ...  
  protected void _toJavaExpression(DecimalLiteral expr,
                                   IAppendable b) {
    b.append(expr.getIntPart() + "." 
      + expr.getDecimalPart());
  }

  protected void _toJavaStatement(DecimalLiteral expr, 
                                  IAppendable b, 
                                  boolean isReferenced) {
    generateComment(expr, b, isReferenced);
  }
}

We can now use decimal literals in your expressions and they are correctly converted to java.lang.Doubles. The following domain model should now compile:

entity DecimalLiteralTest {
  op approxPi() : double {
    3.1415926535
  }
}  

Note that autoboxing/unboxing works in Xbase the same as in Java. Still we do not have any operations for decimal literals. The next section shows how to add them.

Operators

Operators in Xbase are implemented as Java methods following a naming convention. For example, the operator + will be mapped to a call to a method named operator_plus(). There are several places where Xbase tries to find this method

1. in the current type as operator_plus(OperandType1, OperandType2)


2. in the class of the first operand, as OperandType1.operator_plus(OperandType2)


3. in the extension scope as operator_plus(OperandType1, OperandType2)

The first alternative is ruled out, as we want to provide global operations. The second does not apply as we cannot change the final class java.lang.Double, so we're eventually dealing with alternative 3.

Let's write a new extension library class holding the operations as static methods. These methods need to be available at runtime, too, as the generated Java code will call them. That's why it is a good idea to put such extension classes into a separate plug-in org.eclipse.xtext.example.domainmodel.lib.

package org.eclipse.xtext.example.domainmodel.lib;
public class DecimalExtensions {
  public static Double operator_plus(Double x, 
                                     Double y) {
    return x + y;
  }
  public static Double operator_minus(Double x, 
                                      Double y) {
    return x - y;
  }
  public static Double operator_multiply(Double x, 
                                         Double y) {
    return x * y;
  }
  public static Double operator_divide(Double x,
                                       Double y) {
    return x / y;
  }
}

Globally available extension classes and literal classes have to be registered in the
StaticMethodsFeatureForTypeProvider. We override it

package org.eclipse.xtext.example.domainmodel.scoping;
...
public class DomainmodelStaticMethodsProvider 
    extends StaticMethodsFeatureForTypeProvider {
  @Override
  protected Iterable getVisibleTypesContainingStaticMethods
    (JvmTypeReference reference) {
    Iterable resultFromSuper = 
      super.getVisibleTypesContainingStaticMethods
        (reference);
    if (reference != null && reference.getType() != null
        && "java.lang.Double"
          .equals(reference.getType().getIdentifier())) {
      return Iterables.concat(Collections.singletonList
("org.eclipse.xtext.example.domainmodel.lib.DecimalExtensions"),
          resultFromSuper);
    }
    return resultFromSuper;
  } 
}

and bind our customized implementation

public class DomainmodelRuntimeModule 
  extends AbstractDomainmodelRuntimeModule {
...
  public Class 
      bindStaticMethodsFeatureForTypeProvider() {
    return DomainmodelStaticMethodsProvider.class;
  }
}

If you add the plug-in org.eclipse.xtext.example.domainmodel.lib to the classpath of the project in the runtime workspace, the following domain model should compile:

entity DecimalLiteralTest2 {
  op circleArea(double radius) : double {
    3.1415926535 * radius * radius
  }
}  

Graphical Syntax View for Xtext

I demoed it so often that I almost forgot blogging about it: In Xtext 2.0 you can now visualize the syntax of your DSLs grammar in a live railroad diagram view. The view supports navigation between the editor and the view and you can export your diagrams to a file. We even have a special representation for unordered groups. Watch this screencast to see it in action:



The syntax graph is implemented in plain Draw2d using a custom layout algorithm. It is technically not related to the generic graphical view I presented in my previous blogpost.

Using Xbase to Define a Generic Graphical View

While textual representations such as code are perfectly suited to process the details of an object model, a graphical view can be very helpful to display the relationships between objects.

Most graphics frameworks in the Eclipse ecosystem provide graphical editors. Usually, the semantic model is mapped to its graphical representation using a hard wired transformation, be it in code (GEF, Graphiti) or in models and a code generator (GMF, the upcoming Spray project). There are two issues here: First, as opposed to a graphical view, a graphical editor requires the mapping to be bidirectional. That constrains the transformation a lot, e.g. requires having the same structure on the graphical as on the semantic side. Second, hard wiring the mapping introduces extra compilation/generation/deployment turnarounds.

I've created a prototype of a graphical view that is configured using two textual DSLs: A mapping DSL that declaratively describes the transformation from semantic objects to graphical elements, and a styling DSL to modify the graphical representation. The graphical view interprets the mapping and the stylesheet on a given input model, giving immediate feedback on changes of these live models.

Both DSLs are implemented in Xtext 2.0 and make heavy use of the new Xbase expression language library. As Xbase uses Java's typesystem, we can refer to any POJO as an input model. Xbase's rich semantics allow comfortably navigating the models references, and its interpreter can be customized easily to fit our needs. The graphics are implemented using plain GEF and Zest layouts.

The following screencast gives you a short demo. (Sorry for the poor video quality, it gets better if you got to full screen mode)

Rename Refactoring in Xtext 2.0

The upcoming Eclipse Indigo version of Xtext ships with experimental support for generic rename refactoring. It allows you to rename model elements and automatically fix all links to these elements. This blog entry describes the main components behind this new feature. The following screencast shows refactoring for a domainmodel language:

Enabling Refactoring

To enable refactoring support for your language, you have to add the RefactorElementNameFragment in the fragment section of the MWE workflow of your language, e.g.

// rename refactoring
fragment = refactoring.RefactorElementNameFragment {}

The fragment has an additional flag useJdtRefactoring which can be used to delegate to JDT's refactorings for languages using the JVM types compiling to Java (i.e. the domain model example or Xtend). Usually users are fine with the defaults.

After running the workflow, you will have working refactoring support - at least if you have stuck to the defaults with regard to naming, cross-referencing. and indexing. Give it a try. If it doesn't work yet you have to adapt the infrastructure a bit more.

Customizing

The most likely component you want to customize is the IRenameStrategy. This component defines how the declaration of the target element is performed. It has two major responsibilities:

• Apply and revert the declaration change on the semantic model (methods applyDeclarationChange and revertDeclarationChange))
  
  The default is to look for an EAttribute 'name' on the target object and set its value using EMFs reflective API.


• Create the LTK Change objects of the declaration change. These changes will be aggregated, checked for overlaps, presented to you in the preview and finally executed if you apply the refactoring.
  
  The default is to use the ILocationInFileProvider to locate the text range representing the name and create a ReplaceEdit for it.

As the IRenameStrategy is a stateful object, you have to bind a custom IRenameStrategy.Factory instead of the strategy itself to integrate your changes.

The other component you might want to customize is the IDependentElementsCalculator. Dependent elements are those elements whose qualified name changes when the target element is renamed. E.g. when you rename a Java class the qualified names of its inner classes change, too, thus references to these have to be updated. This calculation is performed by the IDependentElementsCalculator. By default, all elements contained in the target element are added. This matches Xtext's default strategy of qualified name computation.

The Internals

The following describe the steps taken in the refactoring and the responsible components

Refactoring UI

In terms of UI, the refactoring of Xtext looks pretty much the same as in JDT. When the user has triggered the refactoring - by a keystroke or using a context menu entry - the editor goes into a linked editing mode allowing to edit all occurrences of the element in the current document at once. Pressing the keys again or using the popup menu, the user switched to a dialog wizard or a preview wizard. Thanks a lot to Holger who did the major work on the UI.

The UI parts pass an element of type IRenameElementContext to the non-UI components, holding information on the target element, its EClass, the current editor etc.

Non-UI

The non-UI part and thereby the actual refactoring is mainly managed by the RenameElementProcessor.

As a first step, it loads the model declaring the element to be renamed into a new ResourceSet that is configured with respect to the declaring language. This allows to easily abandon all changes as well as avoiding concurrency issues. Some basic checks are performed like 'is the element available' or 'is the resource writeable' and the IRenameStrategy is initialized.

Then we calculate the dependent elements.

After that the URIs of the elements that are affected by the refactoring are calculated. Those URIs may change, e.g. if you use a name-based fragment strategy. So we calculate the original URIs, apply the change and recalculate the new URIs. The result is a Map<URI, URI>. This task is the responsibility of the IRenamedElementTracker.
In the end, the changes are reverted to leave the resource in the same state as before.

The final step is to handle the cross-references to the renamed elements. We have to consider resource internal cross-references as well as the ones from other resources. The latter can be retrieved from Xtext's index. Note that if your external cross-references are not indexed, they won't be found and consequently won't be updated by the refactoring.

As references can come from any other language, we use a ReferenceUpdaterDispatcher that searches for local and indexed references and dispatches to the IReferenceUpdater of the referring language. We have two implementations of this interface: The DefaultReferenceUpdater works with Xtext resources and uses the language's serializer to update the reference text. OTOH, the EmfResourceReferenceUpdater works for all EMF based languages and re-serializes the whole resource after applying the semantic change.

Rename Participants

Finally, one refactoring can trigger another. E.g. when renaming a rule in an Xtext grammar, the returned EClass should be renamed, too. For these cases, you can register a RenameParticipant by the common means of LTK. If the target of the participant is Xtext based, you can use a AbstractProcessorBasedRenameParticipant in order to reuse lots of the already described classes.

EclipseCon 2011 Slides

EclipseCon 2011 was once again a cool conference, even though I really needed a week of holiday to recover from the bad weather, the jet-lag and the general lack of sleep :-) This is why it took me some time to upload the slides of our presentations. But better late than never, here they are.

First, Sebastian and me presented What's Cooking in Xtext 2.0. The room was almost full, and when I asked a vast majority of the audience had already had contact with Xtext. To me that shows that Xtext has really become mainstream, a really good feeling. Here are the slides:

What's Cooking in Xtext 2.0

On Thursday, Sven, Sebastian and me gave the tutorial on Pragmatic DSL Design with Xtext. There were so many participants, that we had to switch rooms with Stephan Herrmann's tutorial on Object Teams. Thanks for that, Stephan! And here are our slides:

Pragmatic DSL Design with Xtext, Xbase and Xtend 2

Did I mention EclipseCon?

I just realized that I haven't blogged on this year's EclipseCon yet. Time to do that. You might have guessed it:


As a committer to several Eclipse projects, I am really looking forward to meeting "the family". It's always been big fun and I guess it will be the same this year. But the main reason for going there is of course:


Together with Sebastian I will give a talk on What's cooking in Xtext 2.0 on Tuesday at 2pm. We're eager to show you all the new features of Xtext we've been working on during the last months, especially the expression language library Xbase and the code generation langauge Xtend.

You should not miss our tutorial
Pragmatic DSL Design with Xtext, Xbase and Xtend 2 on Thursday at 10:30am. Sven will join us to give you a walk through tutorial on a modern way of creating DSLs. We just had another rehearsal on this one and I promise you it will be a real burner.


All of that wouldn't have been possible without the support from the company we're working for: itemis. Into the bargain we're bronze sponsors for this years conference.
It is great to work for a company that really lives open source.

JDK Source Code on MacOSX

After installing the latest update of Java for MacOSX Snow Leopard - must have been in March 2010 if my file system is not cheating on me - I missed the source code for the JDK classes. I remembered I had to install the Java Developer Package from the Apple developer center, but I couldn't find any. Consequently, I learned to live without the sources and forgot about the issue.

Yesterday I stumbled across a big performance issue in the upcoming Xtend2 compiler that only happened on my machine. Profiling yielded that it was because I have no local sources (another strange story).

So I decided to look for the developer package once again et voilà: Apple had been so kind to release it in the meantime. BTW, the timestamp says October 20th :-o

So here's what I did to get the MacOSX Java sources back (it's on your own risk, I disclaim any responsibility if you should break or machine)

1) Get an Apple Developper Connection account (it's free, but you have to register).
2) Login choose Java and download the Java for Mac OS X Developer Package matching your OS and Java version.
3) Install the DMG.

The install location of the JDK is somehow weird - e.g. you cannot directly open it in Eclipse's "Choose JDK location" dialog. To make it the default, I found it easiest to retarget the symbolic link pointing to the default JRE. You have to be super user in this case.

4) Open a Terminal and enter

cd /System/Library/Frameworks/JavaVM.framework/Versions
sudo rm CurrentJDK
sudo ln -s /Library/Java/JavaVirtualMachines/1.6.0_22-b04-307.jdk/Contents CurrentJDK

If Eclipse uses the default MacOSX JDK, you should now have sources.