|BIO| |BLOG POSTS - BY DATE| |BLOG POSTS - BY TOPIC| |CONTACT| |WELCOME|




Complete guide to Robot Framework support in Visual Studio Code - Introduction


21 Jun 2022


[ visual-studio-code VSC language-server-protocol language-server LSP Robot-Framework robotframework testautomation testframeworks ]


Preface.

This is the first of a series of posts that will help (aspiring) Robot Framework developers to become proficient and productive on Microsoft’s Visual Studio Code (VSC). This IDE (Integrated Development Environment) is quite powerful, versatile and flexible and is, as such, a great alternative to other IDE’s.

In this introductory article we will get acquainted with a few fundamental concepts and technologies that underly VSC’s Robot Framework support.

Please note that, if you are not interested in how stuff works under the hood, you can skip this and wait for the publication of the next post in this series. That post will serve as a detailed installation guide.

If you decided to read on, then let’s not waste any more time and words and get started!

What's in this article?

  1. A new kid on the block.
  2. About language servers:
    1. What is a language server?
    2. What kind of services does it provide?
    3. To whom (or what) does it provide these services?
    4. How does it provide those services?
    5. A small peek under the hood.
  3. The Robot Framework language server.
  4. What's next?

A new kid on the block.

One of the rather unique characteristics of the Robot Framework is the comprehensiveness (one might even say: ‘vastness’) of its ecosystem. That ecosystem consists of a broad range of test libraries and tools that can be used to extend the framework with all sorts of capabilities and features. Quite often there are even multiple libraries and tools being offered for the same sort of capability or feature.

An example of the latter is the rather large number of IDE and source code editor plug-ins that are available to us:

An overview of RF editor/IDE plug-ins.
An overview of RF editor/IDE plug-ins.
Source: https://robotframework.org/?tab=tools#resources

Often, a person that is new to Robot Framework (or to the art of test automation) will choose the stand-alone, dedicated RIDE. This acronym stands for Robot (Framework) Integrated Development Environment. It is, indeed, a truly great editor if you are setting your first steps towards becoming a Robot Framework Developer. It sports all basic necessities, such as syntax highlighting, code completion and all kinds of refactoring functionality (such as keyword extraction).

However, as one becomes more experienced and is (consequently) joining more complex projects, the need for more advanced editor capabilities arises. That is when the plethora of editor/IDE plug-ins comes in handy. As you can gather from the above screen shot, there are plug-ins for a lot of current, popular IDE’s and editors to choose from.

At RoboCon 2021 another great addition to this already large collection of plug-ins was introduced by Fabio Zadrozny: the Robot Framework Language Server.

This tool adds Robot Framework support to not just one, but even two highly popular and powerful IDE’s, namely Microsoft’s Visual Studio Code (i.e. IntelliSense) and JetBrain’s PyCharm (i.e. IntelliJ).

Please note that, since this series is about Robot Framework and VSC, we will ignore PyCharm here as well as in the follow-up posts.

As we will see in the remainder of this post, the language server functions as integration ‘glue’ between Visual Studio Code on the one hand and the Robot Framework on the other.[1]

About language servers.

The Language Server is central to Robot Framework support in VSC. It is doing all of the magic. Thus it would be to our advantage to gain some in-depth knowledge about this component.

For starters: what is a language server?

Well, a language server is exactly that:

a software component that provides services pertaining to a specific programming language.

For instance, one that exposes Python-specific services. Or one that adds Java-specific services. Or one that holds services specific to the Robot Framework scripting language.

Okay ... but ... what kind of services does it provide?

Basically: services that aid a developer in delivering code more efficiently and less error prone.

That is, services such as:

Here are two examples (click to enlarge):

Example of code completion.
Example of code completion.
Example of code validation.
Example of code validation.

Right! And to whom (or what) does it provide these services?

A language server is, ultimately, created to assist and support developers that code in the specific language that the LS supports.

But it does so only indirectly, namely by lending its language specific intelligence to extend the capabilities of the types of tools that devs typically use to produce code: IDE’s and/or source code editors.

The IDE (or source code editor) acts as a client and, as such, consumes the services provided by the language server. As mentioned, it is the language server that actually contains the language-specific intelligence. It holds an intimate knowledge of the details of the language, such as its syntax. The language server ‘shares’ this knowledge with the client.

For instance, as a developer is typing a line of code in an IDE, the latter continually sends completion requests to the (language) server. To each request the server sends a fitting response to the client. Each response holds one or more suggestions, that the client typically presents to the developer in a list of some sort. The dev subsequently selects a suggestion. This saves her some typing and sometimes even some thinking.

Got it. Then how does it provide those services?

To answer that question, we’ll have to take a look at some history.

Before the arrival of language servers the job of adding support for a new programming language to an IDE was quite tedious and time-consuming. Support for a specific language had to be implemented in the form of a plug-in (i.e. extension) for an IDE. However, each IDE had its own (extension) API and it was, consequently, quite laborious to adapt the code of an existing extension to port it over to another IDE. Therefore, effectively, a unique language extension had to be build for any IDE that was to support the language in question. Either by an IDE vendor, the developers of a programming language or some third party that was somehow interested enough to undertake the effort.

Example: Python language support by way of proprietary-API extensions.
Example: Python language support by way of proprietary-API extensions.

The concept of a ‘language server’ was originally devised within Microsoft. Their goal was to decouple programming language support (on the one hand) and IDEs (on the other hand), so as to be able to develop and distribute them independently from each other.

For this they conceived of an JSON-RPC based protocol: The Language Server Protocol (LSP). For an IDE to support a specific language, for instance Python, it had to provide a client component that would be proprietary, but also LSP-compliant. Because of the latter it would then be able to communicate with any LSP-compliant language server component, for instance a Python LS.

In this manner one and the same language server can, with very little effort, be re-used with different development tools, effectively creating a general purpose service. This service and the IDE/client run in different processes which can communicate using the language server protocol over JSON-RPC:

Example: Python language support with a general-purpose Python language server.
Example: Python language support with a general-purpose Python language server.

This will stimulate developers of programming languages to develop their own language servers, because it is now very easy to reuse the same code base to add support to other IDE’s as well.[2]

Example: The same language server used with multiple, different IDE's.
Example: The same language server used with multiple, different IDE's.

At the same time, this will save the IDE vendor the effort. And even if a vendor will have (or wants) to add support for a specific language, that will be much easier because of the simplified and standardized way of communication.

Winners everywhere.

Super duper! Can I have a small, extra peek under the hood?

Sure!

Let’s first take a closer look at JSON-RPC and subsequently go over the LSP.

JSON-RPC

JSON-RPC is a Remote Procedure Call (RPC) protocol that uses JSON to define and structure data that is to be exchanged through specific types of messages between application nodes in a distributed environment.

For this goal JSON-RPC defines three message patterns: ‘request’, ‘response’ and ‘notification’. Each type of message has a very simple JSON structure. For instance, a request will be in the form of a JSON object that holds four items: two strings, one structured value (which may be an object or an array) and a value that can be either a string, a number or null. For example:

{"jsonrpc": "2.0", "method": "add", "params": {"first_nr": 5, "second_nr": 7}, "id": 1}

The first item specifies the JSON-RPC version. The second item holds the name of the function that is to be called. The third item is optional and contains possible arguments that need to be passed into the called function. The fourth item is the id of the transaction that is taking place (the server response will contain the same id).

Since it is an RPC protocol, JSON-RPC has been designed to call functions (procedures, methods) irrespective of whether such a function is local or remote. To the caller the location of the called function is completely transparent. Similarly, the protocol does not specify the transport method for messages: it is transport-agnostic. Thus we may use TCP, HTTP or other means of transportation.

LSP

As was just mentioned, the Language Server Protocol builds on top of JSON-RPC. Accordingly, it ‘inherits’ the message types and patterns that were just described. The LSP adds HTTP-like headers (currently only ‘Content-Length’ and ‘Content-Type’) to the messages. As you can see, all three JSON-RPC message types are being utilized in the following example:

Example of communication between an IDE client and a language server.
Example of communication between an IDE client and a language server.
Source: https://microsoft.github.io/language-server-protocol/overviews/lsp/overview/

Accordingly, a request may look something like the following (source):

Content-Length: ...\r\n
\r\n
{
	"jsonrpc": "2.0",
	"id" : 1,
	"method": "textDocument/definition",
	"params": {
		"textDocument": {
			"uri": "file:///p%3A/mseng/VSCode/Playgrounds/cpp/use.cpp"
		},
		"position": {
			"line": 3,
			"character": 12
		}
	}
}

As you can see, the structure of the message conforms to the format specified by JSON-RPC.

The ‘method’ string item is used to convey the specific language server feature that the client wants to employ. In this case it concerns a “Go to Definition” request, in which the client asks for the position of a symbol’s definition. The ‘params’ object specifies the parameters relevant to such a request: the document and the position (both line and character) of the symbol in question.

Also note the header part of the message that precedes the content part and that is terminated on line two (using “\r\n”, which is also used to separate the individual header fields).

Next to specifying the messages, the LSP describes various other aspects of the communication between the language client and the language server. In particular, it specifies the following:

Server lifecycle: The client is in charge of lifecycle management: such as initialization, shutdown and exit. Part of the lifecycle is also a handshake-like exchange of capabilities between client and server after initialization has been completed.
Document synchronization: Synchronization of the contents of a text document. The contents is managed by the client and it is up to the client to notify the server of events, such as changes to the document or the saving of a document.
Language features: The actual language specific intelligence as described above, such as code completion and go-to-declaration.
Others: Workspace features and Window features

Indeed, the LSP specification is rather voluminous, particularly when compared to the JSON-RPC specification. If interested you can read more about the specification here.

The Robot Framework language server.

The Language Server Protocol was originally developed by Microsoft.

However, in cooperation with a couple of other companies they turned it into an open standard. That means that anyone can create a language server based on the protocol specification.

As was mentioned earlier, Fabio Zadrozny did exactly that: at the request of RoboCorp (who funded the project), he developed a language server and subsequently used it to implement Robot Framework support for both PyCharm and VSC in the form of two extensions.[3]

The Robot Framework Language Server and the VSC extension were then presented by Fabio during RoboCon 2021.

At the moment of writing, the latest version of the Robot Framework Language Server is 0.48.2. You can always check out the change log to see what is the most recent version and which features were added to it.

What's next?

In the remainder of this series of articles we will go into the details of installing, setting up and configuring Visual Studio Code and the RF-LS extension for VSC. Additionally, we will discover and explore all kinds of helpful features of the Robot Framework VSC plug-in as well as of Visual Studio Code itself!

If you don’t want to wait for the next article, you may find the following online resources to be helpful starting points for your own journey:

Robot Framework Language Server (VSC extension):

VSC itself:

Of course, there are many, many more resources. You may find them by submitting some basic search queries to your favorite search engine.

Okay, now that we have some background, we can finally go to work!

Therefore, the next entry will feature a complete and detailed installation guide of the entire development stack.

Hope to see you next time around!





[1] As a side note: there are a couple of alternatives, such as RobotCode, which is also a great tool. However, the Robot Framework Language Server appears, by all means, to be the most future-proof solution for adding Robot Framework support to Visual Studio Code. But history may prove me wrong.😉 (back)

[2] The protocol (naturally) does not specify how a language server is to be integrated into a specific IDE. That depends on the (and thus: differs per) IDE. The LS will have to be added to an IDE in the form of a compatible extension/plug-in (which will 'wrap' the LS). So to some extent there still remains an effort in 'porting' a language server from one IDE to another. (back)

[3] The VSC plug-in is distributed as a .vsix package file (e.g. 'robocorp.robotframework-lsp-0.48.2.vsix') and the PyCharm extension as a .zip archive file (e.g. 'robotframework-intellij-0.48.2.zip'). As a VSIX file adheres to the OPC (Open Packaging Convention) standard, both file types can be opened with any tool that can handle ZIP files. (back)

Comments


Please join the discussion: place a comment on this page.

The comment will be immediately visible in this comment section.

Please note a Github account is required to comment.