Thursday, July 14, 2011

Extending Xbase

The new expression language library Xbase allows to integrate expressions in your own Xtext 2.0 DSLs. Xbase offers a grammar, a large runtime library and a compiler as well as an interpreter to execute these expressions. Thus, Xbase opens the field for a much wider class of DSLs and annihilates prior advantages of internal DSLs over external DSLs based on Xtext.

In a series of internal workshops on Xbase, I've often encountered one question: Can we extend the expressions of Xbase? In this blogpost, I am going to show you how to do that. It is based on the domain model example shipped with Xtext 2.0, which you can instantiate by choosing File > New > Example... > Xtext > Xtext Domain-Model Example.

The Grammar


By default, Xbase only knows integer literals. In this example we are adding decimal literals.

If you want to add a new concept to Xbase expressions syntactically, you have to hook into the right rule of Xbase. As the name suggests a literal can be added in the Xbase rule XLiteral. So in the Dominamodel.xtext you have to add

import "http://www.eclipse.org/xtext/xbase/Xbase"
...
XLiteral returns XExpression:
XClosure |
XBooleanLiteral |
XIntLiteral |
XNullLiteral |
XStringLiteral |
XTypeLiteral |
DecimalLiteral; // our new alternative

DecimalLiteral:
intPart=INT '.' decimalPart=INT;

As the EPackage of the domainmodel language is generated this will introduce a new EClass DecimalLiteral as soon as you regenerate the language infrastructure.

Type Provider


For the sake of simplicity, our decimal literal will be mapped to java.lang.Double. We have to adapt the type provider accordingly

public DomainmodelTypeProvider {
@Inject
private TypeReferences typeReferences;
...
protected JvmTypeReference _type(DecimalLiteral literal,
boolean rawType) {
return typeReferences.getTypeForName(Double.class,
literal);
}
}


Compiler


The compiler has to understand what to do with decimal literals. We have to add a method that appends the concrete syntax of the Java double literal for a DecimalLiteral

public class DomainmodelCompiler {
...
protected void _toJavaExpression(DecimalLiteral expr,
IAppendable b) {
b.append(expr.getIntPart() + "."
+ expr.getDecimalPart());
}

protected void _toJavaStatement(DecimalLiteral expr,
IAppendable b,
boolean isReferenced) {
generateComment(expr, b, isReferenced);
}
}

We can now use decimal literals in your expressions and they are correctly converted to java.lang.Doubles. The following domain model should now compile:

entity DecimalLiteralTest {
op approxPi() : double {
3.1415926535
}
}

Note that autoboxing/unboxing works in Xbase the same as in Java. Still we do not have any operations for decimal literals. The next section shows how to add them.

Operators


Operators in Xbase are implemented as Java methods following a naming convention. For example, the operator + will be mapped to a call to a method named operator_plus(). There are several places where Xbase tries to find this method

  1. in the current type as operator_plus(OperandType1, OperandType2)

  2. in the class of the first operand, as OperandType1.operator_plus(OperandType2)

  3. in the extension scope as operator_plus(OperandType1, OperandType2)


The first alternative is ruled out, as we want to provide global operations. The second does not apply as we cannot change the final class java.lang.Double, so we're eventually dealing with alternative 3.

Let's write a new extension library class holding the operations as static methods. These methods need to be available at runtime, too, as the generated Java code will call them. That's why it is a good idea to put such extension classes into a separate plug-in org.eclipse.xtext.example.domainmodel.lib.

package org.eclipse.xtext.example.domainmodel.lib;
public class DecimalExtensions {
public static Double operator_plus(Double x,
Double y) {
return x + y;
}
public static Double operator_minus(Double x,
Double y) {
return x - y;
}
public static Double operator_multiply(Double x,
Double y) {
return x * y;
}
public static Double operator_divide(Double x,
Double y) {
return x / y;
}
}

Globally available extension classes and literal classes have to be registered in the
StaticMethodsFeatureForTypeProvider. We override it

package org.eclipse.xtext.example.domainmodel.scoping;
...
public class DomainmodelStaticMethodsProvider
extends StaticMethodsFeatureForTypeProvider {
@Override
protected Iterable getVisibleTypesContainingStaticMethods
(JvmTypeReference reference) {
Iterable resultFromSuper =
super.getVisibleTypesContainingStaticMethods
(reference);
if (reference != null && reference.getType() != null
&& "java.lang.Double"
.equals(reference.getType().getIdentifier())) {
return Iterables.concat(Collections.singletonList
("org.eclipse.xtext.example.domainmodel.lib.DecimalExtensions"),
resultFromSuper);
}
return resultFromSuper;
}
}

and bind our customized implementation

public class DomainmodelRuntimeModule
extends AbstractDomainmodelRuntimeModule {
...
public Class
bindStaticMethodsFeatureForTypeProvider() {
return DomainmodelStaticMethodsProvider.class;
}
}

If you add the plug-in org.eclipse.xtext.example.domainmodel.lib to the classpath of the project in the runtime workspace, the following domain model should compile:

entity DecimalLiteralTest2 {
op circleArea(double radius) : double {
3.1415926535 * radius * radius
}
}

9 comments:

Sebastian Zarnekow said...

Hi Jan,

I'm afraid there is a typo: operation_plus should read operator_plus.

Karsten Thoms said...

Hi Jan,

thanks for this post. Now some things become clearer again :-)

~Karsten

Jan Köhnlein said...

@Sebastian: Thanks, I fixed it. Fortunately, the code snippets are correct.

Konstantin Komissarchik said...

http://en.wikipedia.org/wiki/XBase

May want to have a chat with Eclipse Legal regarding the naming...

Michael Vorburger said...

This is very very cool! Thanks a lot for the write-up.

Is there any way to avoid repeating
the whole XClosure ... XTypeLiteral, say something like (I'm just making the syntax up) XLiteral << DecimalLiteral;

betto said...

Is this still valid also in Xtext 2.1?

Peter said...

Hello,
Thanks for the post.
My problem is that i'm stucked on DomainmodelCompiler extends XbaseCompiler
Such a class remains useless unless it's not bound/injected somewere.

Could someone please tell me, what is to be done to activate it.
Thanks!

Norbert Noad said...

Please show how to use the new Domainmodel Classes (how to inject them?)
thank you, nn

Jan Köhnlein said...

Sorry, but this post refers to provisional API from Xbase 2.0 and
today we are going to release 2.4.

Lots of things have changed in the meantime, e.g. double literals are already included in Xbase and type computation will change significantly in 2.4.

The new way to add a new expression to Xbase is documented in the template example in the 7 languages.
The docs should be updated by tomorrow (make sure they refer to the TemplateTypeComputer). Please ask furhter questions in the Xtext Forum.