Adapt document to IFile when unable to retrieve file from buffer manager

[raghucssit]

Some framework like Xtext does not connect their document to buffer manager. So LSPEclipseUtils returns null for URI and IFile in such cases.
So we can try to adapt the IDocument to URI and IFile.

see #1500

[Copilot]

Pull request overview

This pull request adds support for Eclipse's adapter framework to LSPEclipseUtils.getFile() and LSPEclipseUtils.toUri(IDocument) methods to handle documents from frameworks like Xtext that don't connect their documents to Eclipse's buffer manager. The change enables GitHub Copilot and other LSP-based tools to work with custom document implementations.

Changes:

• Modified toUri(IDocument) to attempt adaptation to URI when buffer manager resolution fails
• Modified getFile(IDocument) to attempt adaptation to IFile when buffer manager resolution fails

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[rubenporras]

Can you explain a bit more about the problem? I think that if you are using Xtext with LSP4E, Xtext should only be part of the Language Server and it should not be present on the UI plugins.

[raghucssit]

Can you explain a bit more about the problem? I think that if you are using Xtext with LSP4E, Xtext should only be part of the Language Server and it should not be present on the UI plugins.

This is an General Purpose Utility feature we need. We should be able to adapt XtextDocument to URI or IPath or IFile via ITextFileBufferManager or IAdaptable interface.

Why we need this ?
Any client(including UI components) which has an instance of XtextDocument may not be dependent on Xtext instead they see IDocument. But client may need an IFile solely from IDocument. We can do it if it is IAdaptable or there is an AdapterFactory. Currently XtextDocument don't support both.

Here I already have PR to support adaptable feature in Xtext.
eclipse-xtext/xtext#3614

Real world use case:
Github copilot has an instance of IDocument from XtextEditor. They want to retrieve URI of the file underneath IDocument.
They use LSPEclipseUtils to retrieve URI. But this utility returns null.
Why ?
It depends on ITextFileBufferManager to get the URI. ITextFileBufferManager maintains a map of URI to document. The problem is XtextDocumentProvider does not connect the XtextDocument with ITextFileBufferManager so it returns null.

Because copilot cannot derive URI of the file. It does not support any auto complete for XtextEditor now. This fix will fix that problem.

So we can support this feature and help any clients who has an instance of XtextDocument and want either URI or IPath or IFile from it.

Once XtextDocument has the adaptable capability here in this PR we use that capability using Adapter.

[raghucssit]

FYI @iloveeclipse

[iloveeclipse]

@rubenporras : this PR helps Copilot plugin in Eclipse to "see" connection between current document in the opened Xtext based editor and the underlined IFile or Uri. Copilot uses LSP4E. Without that connection between document and file/uri, nothing works in Copilot with Xtext based text editors. "Chat" can't link the file in question to the context, "code predictions" and all other "AI" functionality simply doesn't work.

[iloveeclipse]

Note: currently, even if Xtext patch is not yet merged (see eclipse-xtext/xtext#3615), Eclipse based products can still work around that by contributing an adapter factory, but it would only work if LSP4E would support it via current PR.

[rubenporras]

Hi @iloveeclipse and @raghucssit,

while I understand that technically adding an adapt at the end of method would make this work for any adaptable I do not think it is the right place to do it.

LSP4E has been designed to work only with the generic editor (or more concrete with editors that use the a ITextFileBufferManager) and the takes the assumption at many places that this is the case, which the Xtext editor is not doing.

With that in mind, I do not think it is correct that Copilot uses this code outside of this context. They could as well wrap the toUri on his end and add the call there, or have their own toUri method.

[iloveeclipse]

With that in mind, I do not think it is correct that Copilot uses this code outside of this context. They could as well wrap the toUri on his end and add the call there, or have their own toUri method.

The problem is, that Copilot doesn't have any dependency to Xtext (it doesn't need it at all) so it can't just take "XtextDocument" and do something specific with that. They have lot of very generic code that works on top of IDokcument API & LSP4E and that works quite well.

LSP4E has been designed to work only with the generic editor (or more concrete with editors that use the a ITextFileBufferManager) and the takes the assumption at many places that this is the case, which the Xtext editor is not doing.

Could you point to possible problems? I believe Raghu didn't saw any after patching LSP4E locally.

[rubenporras]

I do not have time to look for possible problems. My main point is that LSP4E code does not need this code to work, it is rather CoPilot that needs to do the mapping, so why not add a similar code to the copilot codebase?

[iloveeclipse]

My main point is that LSP4E code does not need this code to work, it is rather CoPilot that needs to do the mapping, so why not add a similar code to the copilot codebase?

@raghucssit : could you please evaluate whether it will be possible?

[raghucssit]

My main point is that LSP4E code does not need this code to work, it is rather CoPilot that needs to do the mapping, so why not add a similar code to the copilot codebase?

@raghucssit : could you please evaluate whether it will be possible?

I moved document adaption logic to copilot. But this did not work as expected because document connection with lspe4e fails.

DocumentContentSynchronizer tries to derive URI using LSPEclipseUtils and throws NPE.

java.lang.NullPointerException
	at org.eclipse.lsp4e.DocumentContentSynchronizer.<init>(DocumentContentSynchronizer.java:91)
	at org.eclipse.lsp4e.LanguageServerWrapper.lambda$17(LanguageServerWrapper.java:880)
	at java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:718)
	at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

LSP4E don't support custom documents they don't register with Eclipse Buffer Manager..

@rubenporras Do we need to support this ? I have explianed one major usecase that is xtext.

[raghucssit]

Github copilot for eclipse tries to connect document and IFile using LanguageServerWrapper#connect(@Nullable IDocument document, IFile file). This API internally tries to instantiate DocumentContentSynchronizer. DocumentContentSynchronizer constructor throws NPE because it tries to derive URI using

lsp4e/org.eclipse.lsp4e/src/org/eclipse/lsp4e/LSPEclipseUtils.java

Line 404 in 0a47315

public static @Nullable URI toUri(@Nullable IDocument document) {

[rubenporras]

Hi @raghucssit ,

thanks for looking it up. If they actually call connect and not LSPEclipseUtils as utility I think this change is not enough. LSP4E really relies on editors to use the ITextFileBufferManager. See for example the class org.eclipse.lsp4e.LanguageServerWrapper.LSFileBufferListener.LSFileBufferListener() that is used by the wrapper. So for example, all the functionality provided by it would not work. I think there must be other side effects (just taking a look at the places in the codebase that use an ITextFileBufferManager).

I would expect that merging this PR will cause follow-up problems. From LSP4E point of view, the ideal solution would be that the Xtext Editor uses a file buffer manager, but I do not know how involved such change would be.

[iloveeclipse]

From LSP4E point of view, the ideal solution would be that the Xtext Editor uses a file buffer manager, but I do not know how involved such change would be.

We evaluated that, it is unfortunately not possible.
We will look into LSFileBufferListener code, thanks for the hint.

[rubenporras]

As a note, the LSFileBufferListener is not the only place. There are other places which rely on a file buffer mannager.

[iloveeclipse]

As a note, the LSFileBufferListener is not the only place. There are other places which rely on a file buffer mannager.

Let assume LSFileBufferListener part could be fixed by adding listener on the workspace and check for related IFile resource change events.

What other functionality in LSP4E would need to be touched to add proper support for Xtext editors in LSP4E?

[raghucssit]

I have added Fallback resource listener which listens to IFile resource changes.. Here the challenge is any file managed by buffer manager also triggers resource change notifications and both the listeners gets triggered. To avoid that I have used time stamp logic.

Buffer listener is used in places in LSP4E

1. LanguageServerWrapper - which i have patched here.
2. TypeHierarchyView - I did not patch because i did not get how to use it. In Eclipse TypeHierarchyView contributed by JDT always gets called. I don't know how to use this view.
   @rubenporras Do you know how to invoke TypeHierarchyView from LSP4E instead of JDT ?

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (6)

org.eclipse.lsp4e/src/org/eclipse/lsp4e/LSPEclipseUtils.java:1371

• getFile(IDocument) now falls back to Adapters.adapt(document, IFile.class, true), but this new code path isn’t covered by tests. Please add a unit test similar to the toUri(IDocument) case to verify that when the buffer manager can’t resolve a path, the adapter-provided IFile is returned (and that existing behavior still returns the workspace file when a buffer-backed path exists).

	public static @Nullable IFile getFile(@Nullable IDocument document) {
		IPath path = toPath(document);
		IFile file = getFile(path);
		//if we cannot determine file via buffer manager then try to adapt from document.
		if (file == null) {
			file = Adapters.adapt(document, IFile.class, true);
		}

org.eclipse.lsp4e/src/org/eclipse/lsp4e/LanguageServerWrapper.java:334

• PRE_DELETE / PRE_CLOSE events are not save operations, but this calls DocumentContentSynchronizer.documentAboutToBeSaved(), which can trigger willSaveWaitUntil/format-on-save edits. This can unexpectedly modify the document during a delete/close and send incorrect LSP lifecycle signals. Consider removing this call here, or only invoking it for actual save events (e.g., when you can reliably detect a save/CONTENT change).

						DocumentContentSynchronizer dcs = connectedDocuments.get(uri);
						if (dcs != null) {
							// Mirror buffer.stateChanging -> documentAboutToBeSaved
							try {
								dcs.documentAboutToBeSaved();
							} catch (Exception e) {
								LanguageServerPlugin.logError(e);
							}
						}

org.eclipse.lsp4e/src/org/eclipse/lsp4e/LanguageServerWrapper.java:376

• The workspace fallback path sends textDocument/didSave unconditionally on any IResourceDelta.CONTENT change. DocumentContentSynchronizer.documentSaved(...) explicitly checks the server's textDocumentSync.save capability and suppresses didSave when save == null; this code bypasses that and may spam servers that don't support save notifications. Please mirror the same capability check here (and ideally reuse the existing documentSaved logic where possible).

					// Content change (save/replace) -> attempt to notify documentSaved
					if (kind == IResourceDelta.CHANGED && (flags & IResourceDelta.CONTENT) != 0) {
						URI uri = LSPEclipseUtils.toUri(file);
						if (uri != null) {
							// If buffer listener recently handled this URI, skip to avoid duplicate handling
							Long last = bufferEventTimestamps.get(uri);
							if (last != null && System.currentTimeMillis() - last < BUFFER_EVENT_DEBOUNCE_MS) {
								return false; // skip this file
							}
							DocumentContentSynchronizer dcs = connectedDocuments.get(uri);
							if (dcs != null) {
								try {
									// Mirror buffer.dirtyStateChanged(..., false) -> documentSaved
									final var identifier = LSPEclipseUtils.toTextDocumentIdentifier(uri);
									final var params = new DidSaveTextDocumentParams(identifier, dcs.getDocument().get());
									// send didSave notification via wrapper to keep ordering
									LanguageServerWrapper.this.sendNotification(ls -> ls.getTextDocumentService().didSave(params));
								} catch (Exception e) {

org.eclipse.lsp4e/src/org/eclipse/lsp4e/LanguageServerWrapper.java:339

• In the PRE_DELETE/PRE_CLOSE branch, disconnectTextFileBuffer(uri) is called and then disconnect(uri) is called, but disconnect(uri) already calls disconnectTextFileBuffer(uri) when a document listener is present. This is redundant and makes the control flow harder to follow; consider letting disconnect(uri) handle the buffer cleanup and remove the extra call here.

						if (isConnectedTo(uri)) {
							LanguageServerPlugin.logInfo("Workspace PRE_DELETE/PRE_CLOSE disconnect for: " + uri); //$NON-NLS-1$
							disconnectTextFileBuffer(uri);
							disconnect(uri);
						}

org.eclipse.lsp4e/src/org/eclipse/lsp4e/LanguageServerWrapper.java:309

• The PR description focuses on adapting IDocument to URI/IFile, but this file also introduces a new workspace-level IResourceChangeListener with debouncing and custom didSave/disconnect behavior. If this additional listener is required for the adaptation feature to work (e.g., to get save/delete events without file buffers), it should be called out explicitly in the PR description (and ideally justified with the specific failure mode it addresses), since it changes lifecycle behavior at workspace scope.

	// Fallback workspace listener to catch resource-level events for files that are not backed by file buffers
	private final IResourceChangeListener resourceFallbackListener = new ResourceFallbackListener();

	private final class ResourceFallbackListener implements IResourceChangeListener {

org.eclipse.lsp4e/src/org/eclipse/lsp4e/LSPEclipseUtils.java:418

• toUri(IDocument) now falls back to Adapters.adapt(document, URI.class, true), but there is no corresponding test coverage to ensure adapter-based URI resolution works (and doesn’t regress the existing buffer-based behavior). Since this is a behavior change used by downstream integrations (e.g., Xtext/Copilot), please add a unit test that registers an IAdapterFactory for a custom IDocument and asserts LSPEclipseUtils.toUri(document) returns the adapted URI when no file buffer is present.

	public static @Nullable URI toUri(@Nullable IDocument document) {
		if(document == null) {
			return null;
		}
		ITextFileBuffer buffer = toBuffer(document);
		IPath path = toPath(buffer);
		IFile file = getFile(path);
		if (file != null) {
			return toUri(file);
		} else if(path != null) {
			return toUri(path.toFile());
		} else if (buffer != null && buffer.getFileStore() != null) {
			return buffer.getFileStore().toURI();
		}
		return Adapters.adapt(document, URI.class, true);

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[raghucssit]

@rubenporras I have fixed the copilot review comments.. Now the PR is ready for review..
ResourceListener seems to work fine.. Anyway this is a parallel solution to honor bufferlistener like contract and works soley on non eclipse buffer documents.
Please check if this approach sounds good.

[rubenporras]

Hi @FlorianKroiss ,

I did not (and will probably not) have time soon to review this PR properly. Would you have time to do the review?

[FlorianKroiss]

Hi @rubenporras, sorry for the late response.
I'm not (yet) familiar with the internals of LanguageServerWrapper, so reviewing this properly would more time than I have at the moment.

I can see why we would want to support Documents which live outside of the buffer manager, but at a glance, the proposed changes look like a makeshift solution.

[raghucssit]

Hi @rubenporras, sorry for the late response. I'm not (yet) familiar with the internals of LanguageServerWrapper, so reviewing this properly would more time than I have at the moment.

I can see why we would want to support Documents which live outside of the buffer manager, but at a glance, the proposed changes look like a makeshift solution.

AS per @rubenporras LSP4E is designed to work with documents managed by eclipse buffer manager. We have found a client(Xtext) which don't use eclipse buffer manger yet they have their documents edited on eclipse workbench.
If we don't support that use case in LSP4E then any client which don't uses buffer manger will not get language server benefit. which is the case currently.
Copilot simply rely on LSP4E and it's utils to handle connection by giving documents. For all documents it works fine but for XText documents it does not work by default.

The solution initially i have proposed was to just add adapter to Utils then we found buffer listener actions are missing for non buffered documents.. then only option we found was to add file change listener..
This is the only solution i can think of now.. Please feel free to share your thoughts i am happy to try them out and implement.

I really wish this change to be accepted into LSP4E because Xtext framework is used a lot in many companies. As i have pointed out already as copilot fully rely on LSP4E the copilot code suggestion support will be missed out for Xtext editors.

[trancexpress]

You can also use the below patch to test as Plugin Test. Apply the patch onto org.eclipse.lsp4e.test

Any reason not to add this test as part of the change?

[iloveeclipse]

Any reason not to add this test as part of the change?

Since there are no extra dependencies added (like Xtext &Co), it would be good to include in the PR.

[raghucssit]

You can also use the below patch to test as Plugin Test. Apply the patch onto org.eclipse.lsp4e.test

Any reason not to add this test as part of the change?

It is plugin-in test. I assume LSP4E has only pure junit tests. It may fail that is the reason.
I did not check this fully. I can check and confirm my assumption.

[raghucssit]

Fixed. All the review comments.
Added preference based fallback resource listener which is false by default.

[iloveeclipse]

Raghu, I see some code consistency issues: sometimes you use spaces, sometimes tabs for indentation, but project setting is tabs only.

Please fix.

Interestingly, github doesn't show that even if I've explicitly set "show whitespace" changes.

[iloveeclipse]

Few code readability issues below. Beside this, it seem to be OK.

[raghucssit]

Fixed all inconsistent formatting. Now all files in this change uses Tab as format spacing. Added useful comments where necessary.

@rubenporras Please check the changes.. If changes looks good then i will squash all the commits.

[rubenporras]

Could you take a look at my last comment? Then it is good to me.

[raghucssit]

@rubenporras Fixed your suggestion. I think this PR is good to merge.
The current change do not effect the existing users unless they enable the preference explicitly.

[iloveeclipse]

Raghu, I guess you want squash all 3 commits into one?

[raghucssit]

@rubenporras I have fixed minor inconsistency. Now the code looks good.