NeoJoin

NeoJoin is a declarative query language for view-based model-driven software development. It allows to easily create views based on one or more source models using a declarative SQL-like syntax. It supports multiple backends for the model-view transformation, one of which is based on triple graph grammars (TGGs) and supports bidirectional and incremental transformations between models and views.

Note: The frontend is currently not connected to the TGG backend, so an automatic transformation of the queries in NeoJoin syntax is only possible with the EMF backend. We plan to add support for this in the future.

Syntax Example

export reviewedrestaurants to "http://vitruv.tools/models/reviewedrestaurant"

import "http://vitruv.tools/models/restaurant"
import "http://vitruv.tools/models/reviewpage"

from Restaurant rest
join ReviewPage rev
    using name
where rev.reviews.length > 3
create ReviewedRestaurant {
    rest.name
    offeredDishes = rest.sells create Dish {
        it.name
        it.price
    }
    avgRating := rev.reviews.avg[ it.rating ]
    numReviews := rev.reviews.length
}

Features

• Generation of the view type (view meta-model) based on a query
• Transformation of models to derive an instance of a view based on a query
• Bidirectional and incremental transformations using the TGG backend
• Queries can select, join, filter and group classes from the source models
• Features in the view can be copied from the source models, renamed or calculated based on a custom expression
• Query conditions and feature definition using Xtend expressions
• VSCode plugin for syntax highlighting, code completion, and live visualization of the resulting view type
• Input and output meta-models as .ecore and instance-models as .xmi files

Usage

NeoJoin comes with two frontends and two backends. The first frontend is the VSCode plugin, which provides IDE support and view type visualization. The second frontend is the CLI, which provides commands for generating the view type, as well as executing the model-level transformations, currently only with our unidirectional, state-based EMF backend. The other backend, currently in development, offers bidirectional, incremental model-level transformations by generating triple graph grammars (TGGs).

The VSCode plugin and the CLI executable can be downloaded from the build results of the last successful nightly builds.

Requirements

• JDK 21+
• VSCode (for the VSCode plugin)
• Graphviz (for the view type visualization in the VSCode plugin)
• Neo4j (for the TGG backend)
• eMoflon::Neo installed in an Eclipse instance (for the TGG backend)

VSCode Plugin

Provides basic IDE support (syntax highlighting, code-completion, etc.) and a side-by-side live visualization of queries. Can be installed from the .vsix file via Extensions > 3-dot menu > Install from VSIX....

Commands

• Show Query Visualization - Show a live visualization of the queries in the currently active text editor.
• Generate View Type - Generate the view type resulting from the queries in the currently active text editor and save it as an Ecore file next to the query file.
• Restart Server - Restart the language server after configuration changes.

Settings

• Meta Model Search Path - Semicolon separted list of file URLs to search for source meta models.
• Debug - Enable debug logging.

Command Line Interface

Usage: neojoin [-hV] -m=MODEL-PATH [-g=OUTPUT] [-i=MODEL-PATH -t=OUTPUT] QUERY

      QUERY                Path to the query file.

  -h, --help               Show this help message and exit.
  -m, --meta-model-path=MODEL-PATH
                           Model path (see below) to find referenced
                             meta-models (.ecore).
  -V, --version            Print version information and exit.

Generate the meta-model:
  -g, --generate=OUTPUT    Generate the meta-model and write it to the given
                             output file or directory.

Transform the input models:
  -i, --instance-model-path=MODEL-PATH
                           Model path (see below) to find instance models (.xmi).
  -t, --transform=OUTPUT   Transform the input models based on the query and
                             write the result to the given output file or
                             directory.

Model Path
  A semicolon separated list of paths to search for models
  used in the options --meta-model-path and --instance-model-path.
  Examples:
  - Linux: /path/to/directory;/path/to/file.ecore
  - Windows: C:\path\to\directory;C:/path/to/file.ecore

Note: QUERY and OUTPUTs need to be specified with regular paths, e.g., /x/y/z or ./x/y.

Transformation of Operators to Triple Graph Grammars

1. Build view from operators directly: var view = new View(); view.addQuery(...);
2. Resolve interdependent queries (see documentation in Query): var queries = view.resolveQueries()
3. Transform queries to TGG rules: var rules = query.toRules();
4. Build triple grammar from TGG rules: var grammar = new TripleGrammar(name, ..., rules);
5. Optional: generate eMoflon::Neo project: new Scaffolding(...).create(...);

You can use the convenience function API.generateProjectForView(...) for steps 2 to 5.

Running the Generated Model-View Transformations

Prerequisites:

• Eclipse with eMoflon::Neo set up
• Neo4j Desktop version 4.4.X

Make sure the Neo4j database is running and the Neo4j username/password is correctly configured in Eclipse. Additional information about how to set up Neo4j and eMoflon::Neo is available here.

The generated eMoflon::Neo project can be used for various model checking and transformation scenarios. The following steps describe how to execute the forward and sync transformations with an initial source model:

1. Open the generated project folder with Eclipse
   • The src/*.msl files contain the generated TGG project and (meta)models
   • The src/*Runner.java files can be executed to sync the source and target models with Neo4j
   • For each source model, a src/*ForwardRunner.java is available to generate the target model and establish correspondences
2. Right-click the desired src/*ForwardRunner.java > Run As > Java Application
   • This will clear the Neo4j database and run the ForwardRunner for this model
3. Run match (source {enamespace: "YOUR_SOURCE_MODEL_NAME"}), (target {enamespace: "Target"}) return source, target in the Neo4j Browser
   • This will output the graph with the source and target elements as well as the correspondences
4. Update the source and/or target model with Neo4j Clauses
   • Make sure the property _cr_: true is added to created nodes and edges
   • Make sure the correct enamespace: "YOUR_SOURCE_MODEL_NAME" property is added to created nodes
   • Make sure the property _de_: true is added to "deleted" nodes and edges
5. Right-click the desired src/*SyncRunner.java > Run As > Java Application
   • This will run the SyncRunner for this model
   • Afterward, the source and target should be consistent again
6. Optional: Dump the models using the script
   • Make sure the source and target model names are correctly configured

Repository Structure

Project

Directory	Content
docs	Project documentation
lang	Prototype implementation (see below)
scripts	Helper scripts for the project
scripts/dump	Script for exporting models from a Neo4j graph database
vscode-plugin	VSCode plugin for NeoJoin language support and visualization

Prototype Implementation

Module / Package	Description
backend-emf	EMF-based transformation engine to derive instances of the view based on the query
backend-tgg	eMoflon::Neo-based transformation engine for bidirectional transformation between models and views
backend-tgg/driver	Auxiliary logic for generating eMoflon::Neo Eclipse projects
backend-tgg/operators	Primary operators for creating TGGs
backend-tgg/transpiler	The core logic and operators for creating TGGs from operators
frontend/cli	CLI interface to execute meta-model generation and instance model transformation (currently EMF-only)
frontend/ide	IDE support via language server protocol (LSP)
frontend/language	Main language implementation and meta-model generator
model	Abstract query representation (AQR)

Build

Requirements

• JDK 21+
• Node.js 20 (+ NPM)

Maven Projects

cd lang

# either: re-generate and compile the code
./mvnw clean compile
# or: skip the generation, compile the code and run the tests
./mvnw test -P skip-code-generation
# or: package everything to JAR files
./mvnw verify

We use SonarQube to analyze our Maven projects.

VSCode Plugin

• Open the vscode-plugin folder in VSCode (top-level in a workspace)
• Run npm install to install dependencies
• Go to Run and Debug > Select launch configuration Launch Client > Press Start Debugging
  • The language server needs a few seconds to start up, so code completion / analysis will not work right at the start.
  • The plugin requires that the .jar files have been generated and are at their default location.
  • If you cannot find the launch configuration Launch Client, ensure that you have opened VSCode with the vscode-plugin folder as your workspace.
• If you get the error message Activating extension 'vitruv-tools.neojoin' failed: Cannot find module check the build task for potential problems: Bottom Panel > Select tab Terminal > Select task watch (on the right)

Notes

• Xtext heavily uses generated classes which means that opening this repository in a Java IDE after cloning will look like a christmas tree. Run maven compile to generate all missing classes. To improve compile times afterwards you can skip re-generation of classes by activating the maven profile skip-workflow.

Used Technology

• EMF as modeling framework
• Xtext as DSL framework
• Xbase as embedded expression language
• jte as templating engine
• eMoflon::Neo as TGG engine
• Neo4j as graph database