--- title: "Developer Diaries: How We Built a Better Browser-Based IDE with Monaco | CoderPad" description: "Our team is rolling out our new CoderPad Monaco editor. It was a fun challenge to implement and I'd love to share how we did it!" canonical: "https://coderpad.io/blog/development/developer-diaries-how-we-built-coderpad-monaco-ide/" last_modified: "2023-06-05 14:35:47" --- [Back to blog](https://coderpad.io/blog/) # Developer Diaries: How We Built a Better Browser-Based IDE with Monaco March 2, 2022 8 Minute Read By Corbin Crutchley [Development](https://coderpad.io/blog/development/) [Product Updates](https://coderpad.io/blog/product-updates/) Integrated development environments (IDEs) are probably the most important tool we use as software developers and engineers. Using another tool, like a notepad or word processor, would be like trying to hammer a nail with your fist. That’s why our team just spent a good chunk of time working to [improve the code editor](https://coderpad.io/blog/product-updates/introducing-coderpad-monaco/) in the pads that software engineering candidates use to complete technical assessments – but it was anything but an easy feat. To create the best in-browser IDE experience, we leveraged much of the work from Microsoft’s immensely popular Visual Studio Code (VSCode) IDE, but with a CoderPad twist… VSCode runs inside of an environment called [Electron](https://www.electronjs.org/), which allows a web environment to bind to native APIs, like`filesystem`. Therefore their core code editor, called Monaco, can run on the web. VSCode uses Monaco as its editor, which we now do too! However, Monaco has a few limitations that we had to overcome when building an in-browser IDE. These limitations are in place because VSCode has access to native APIs, while Monaco on the web does not. Let’s look at a few of these limitations, and what we did to solve them. ## Learn how to run front-end developer interviews that don’t suck [Read our guide](https://coderpad.io/blog/interviewing/5-tips-for-interviewing-frontend/) ## Parsing, Lexing, and ASTs – Oh My! Before we talk about the limitations, we need to quickly understand how your code is processed in order to generate compiled instructions. First, it’s important to realize that when you type text into the computer it’s just that: plain text. It’s stored the same way that a`.txt`file is stored. Many compilers will even allow you to compile code directly from a`.txt`file. Because of this, we need a way for the compiler (even for runtime languages without an ahead-of-time compiler like JavaScript and Python) to have a better understanding of the context it handles. We do this by creating an Abstract Syntax Tree (AST). This AST allows your computer to read your code like an instruction book. Each section of this instruction book contains all the information a computer needs to transform your original source code into machine-readable code such as assembly. Your editor is also able to use this AST to provide various functions that enhance the editing experience. The AST is constructed by two tools: A lexer and a parser. ## Lexer A lexer takes plain text and converts it into a list of “tokens.” These tokens are a small`allowlist`of values that can be better understood by the computer. For example: ``` const magicNumber = 185;Code language: JavaScript (javascript) ``` Might be turned into a list of: ``` [CONSTANT_KEYWORD, IDENTIFIER, ASSIGN, INTEGER, SEMICOL]Code language: JavaScript (javascript) ``` by the lexer. Each section of the string that comprises the code is converted to smaller sections called tokens. For example, “const” would be a “CONST_KEYWORD” token. These tokens are defined by a “grammar” that declares which characters should and should not be matched to a token in a language’s source code. It’s important to note that a tokenizer does not yet actually understand what your code is trying to do. This means that if we added an extraneous equals sign into our code, it would not yet throw an error, but instead tokenize just fine. When a duplicate token is added, say with two assignment equals signs, it won’t throw an error at this stage. Instead, the lexer will tokenize them both without a problem. While the lexer is important, it only tells half the story behind a language’s syntax because of its inability to actually understand the language. This is where the parser comes into play. ## Parser Once the lexer gathers a list of tokens, it’s time for the parser to assemble an Abstract Syntax Tree (AST) from those tokens. The AST takes the tokens and adds in further context to fully understand what your code is attempting to say. E.g., our`magicNumber`example might be turned into a structure similar to the following: An AST is a tree-like structure that’s defined in a pattern explained more below. This shows us that we’re trying to: 1. `const`assign a`VariableDeclaration` 2. with an`Identifier`of`magicNumber`, and 3. an`init`ial value of`185`, a`NumericLiteral` This parser is the step where syntax validation occurs. If we attempted to add back the second equals sign, we might get the following error: > Uncaught SyntaxError: Unexpected token ‘=’ This error is telling us that while the lexer was able to identify the symbol, the parser did not expect said token where it was present. > Want to learn more about syntax parsing and how your computer takes source code and converts it into a running program? [I wrote an article to help explain this more in depth!](https://unicorn-utterances.com/posts/how-computers-speak#lexer) ## Syntax Highlighting Using this parser-assembled AST, we’re now able to easily add in syntax highlighting to our editor by running a tokenizer over our code and assigning each unique token a different color, right? This is true, but points towards some problems with Monaco’s default tokenizer. See, [VSCode itself utilizes grammar files called “tmLanguage” files to tokenize your code](https://code.visualstudio.com/api/language-extensions/syntax-highlight-guide#tokenization). This tokenizer is borrowed from [TextMate](https://macromates.com/). However, parsing these files requires a C library called [Oniguruma](https://github.com/kkos/oniguruma) to parse and execute these files. Because this library does not run in a pure JavaScript environment – which was a requirement for Monaco historically due to browser support – Monaco ships out-of-the-box with a different grammar language. This alternative syntax parser is called [Monarch](https://microsoft.github.io/monaco-editor/monarch.html), and unfortunately, does not yield as high-quality highlighting results as TextMate. Take a look for yourself: Monarch is displaying a majority of the code as plain text as opposed to TextMate’s rendering, which shows much more information. This impacts screen-readers as well! Notice how most of the semantic value is lost in Monarch. TextMate, on the other hand, is able to showcase that`console`is an object with a method of`log`. Luckily for us, newer browsers ship with Web Assembly supported out-of-the-box, which means that we can compile Oniguruma from C to WASM and ship that with our IDE. More specifically, we’re using ` [vscode-oniguruma](https://github.com/microsoft/vscode-oniguruma) `, which is made by Microsoft. But this introduced yet another problem: Integrating TextMate into Monaco typically requires hacks of some kind, and introduces some flaws into the experience. To work around this, we actually [forked the`monaco-editor`package](https://www.npmjs.com/package/@codingame/monaco-editor) to make it easier to integrate with ` [vscode-textmate](https://github.com/microsoft/vscode-textmate) `, the very package VSCode uses to interpret TextMate packages. ## Language Services Now that we have basic syntax highlighting explained, let’s walk through how we were able to add in error displaying. [VSCode utilizes something called a Language Service Provider](https://code.visualstudio.com/api/language-extensions/language-server-extension-guide) (LSP). When a user makes changes to a file, that file is then persisted to the filesystem. Then, whenever the user has an interaction that requires more metadata than the IDE currently knows about a file’s syntax, it makes a request to this service in order to get said metadata. This works because the LSP is smarter than the AST the editor has on hand, thanks to additional processing done on the parsed code. For example, say the user wants to see the TypeScript definition for a variable they see. When they hover over the variable, it sends a request to the LSP server, including a bit of information to fulfill the request. Every time the user makes an edit to a file, the LSP server handles parsing the relevant files and uses its programmed logic to generate metadata. Then, when a user makes a request for this metadata, say, with a hover, an RPC connection returns the required metadata to show to the user about their requested TypeScript type information. The language service provider parses the file every time after it’s edited. Then, once data is requested, it’s able to read from that parsed information to pass to the client. This service is also used to report errors, warnings, and other information that an IDE might usually surface to a developer. It also enhances autocomplete by providing the required metadata to make intelligent suggestions. We’re also using an LSP, although we’ve had to do things slightly differently from VSCode. VSCode has the advantage of running on a user’s computer. Locally, the language server is able to run using native programming languages. However, since we’re doing this in the browser, we don’t have the ability to do this for all of [our 30+ supported programming languages](https://coderpad.io/languages/) without our bundle size exploding. To solve this problem, we set up LSP cloud servers to handle requests as if the user was running the editor on their computer. ## Extended Syntax Support These language services aren’t just helpful for error displays; they also allow us to enhance the syntax highlighting provided using “semantic highlighting” For example, this is what syntax highlighting looks like without semantic highlighting: Without semantic highlighting classes and methods are often not displayed properly. While this is what it looks like with semantic highlighting: Classes are now displayed with the proper highlighting based on usage. Notice how the parameters that are passed in are highlighted the same color. Likewise,`Range`and`Position`are both acknowledged as classes, while`getFoldingRanges`is presented as a function, even when used as a parameter. This is all possible because the language service provides more metadata than is available to the AST. It’s able to intelligently parse out this context in order to make the editing experience that much better. This feature is [even utilized in VSCode](https://code.visualstudio.com/api/language-extensions/semantic-highlight-guide), and [well documented in the LSP specification](https://microsoft.github.io/language-server-protocol/specifications/specification-current/#textDocument_semanticTokens). Because we already utilize LSP, as mentioned earlier, we get this out-of-the-box with minimal additional lift. ## Additional Features While we touched on a lot of features that developers expect, it’s far from a complete list. We also wanted to enable experiences like: - Different themes (currently light and dark modes, with more to come!) - VIM & emacs modes - Keybindings and user configurations (coming soon) - Language-specific behaviors (such as COBOL magic tabs) To make all of these experiences possible, [we’ve added a wrapper package around Monaco — which we creatively call`monaco-editor-wrapper`](https://www.npmjs.com/package/@codingame/monaco-editor-wrapper). Our Monaco editor wrapper contains the Monaco editor alongside language extensions and additional features. ## Conclusion Hopefully, this has been an interesting insight into the nuts and bolts of our new Monaco editor. We hope you enjoyed reading about how we crafted the new experience as much as we did building it–we loved the challenge! If so, [check out our careers page](https://coderpad.io/careers/) for open engineering roles, and come work with us on fun problems like this. ## Subscribe for more CoderPad insights ## Trending Articles Previous slide Next slide [Product Updates ## Your Question Bank Is Leaking. Here’s the Fix. Every hiring team eventually hits the same wall: TA own raising the bar, don’t have the technical knowledge required to write questions that test for it. So teams are improvising, a shared question bank, a busy engineer’s spare 20 minutes, and last year’s assessment recycled one more time. It feels fine butt isn’t. Recycled and […] Read the post](https://coderpad.io/blog/product-updates/your-question-bank-is-leaking-heres-the-fix/) [Product Updates ## 2026 Product Recap: Technical Hiring Has Changed. We Built for What Comes Next. AI has fundamentally changed how engineers work. It’s not a tool candidates might use anymore. It’s embedded in how code gets written, reviewed, and shipped every day. In many teams, the majority of code is already AI-assisted—and growing. Which leads to a clear conclusion for hiring teams: Trying to remove AI from the interview process […] Read the post](https://coderpad.io/blog/product-updates/2026-product-recap/) [Product Updates ## Top 14 CoderPad Features of 2024 – Features That Were Too Good to Miss 2024 has been an exciting year for technical hiring, and we’ve been honored to be part of your journey. Whether you’re a recruiter working hard to attract the right talent, an interviewer crafting thoughtful assessments, or a candidate showing your best work in sometimes stressful situations, every step of the process matters. This year, our […] Read the post](https://coderpad.io/blog/product-updates/top-14-coderpad-features-2024/) [Product Updates ## CoderPad June releases You can now add a custom time frame for candidates to complete tests, give candidates the option to take a test in Chinese, export your pad and test usage stats... Read the post](https://coderpad.io/blog/product-updates/coderpad-june-releases/) [Product Updates ## CoderPad April Releases Learn what's new with CoderPad for April. Read the post](https://coderpad.io/blog/product-updates/coderpad-april-releases/) [Product Updates ## CoderPad March Releases Webcam proctoring, custom question insights, and more for March. Read the post](https://coderpad.io/blog/product-updates/coderpad-march-releases/) [Product Updates ## CoderPad February Releases New product features for February! Read the post](https://coderpad.io/blog/product-updates/coderpad-february-releases/) [Announcements ## The 2023 CoderPad year in review Please this countdown of the top features we think will help you make hiring and interviewing more real, fast and fair in 2024. Read the post](https://coderpad.io/blog/announcements/the-2023-coderpad-year-in-review/) [Product Updates ## CoderPad November Releases Find out what's new this month in CoderPad Screen & CoderPad Interview Read the post](https://coderpad.io/blog/product-updates/coderpad-november-releases/) [Announcements ## CoderPad October Releases It's been a busy October at CoderPad! Read the post](https://coderpad.io/blog/product-updates/coderpad-october-releases/) [Product Updates ## CoderPad September Releases Learn what's new in CoderPad this month. Read the post](https://coderpad.io/blog/product-updates/coderpad-september-releases/) [Product Updates ## CoderPad August Releases Learn more about the great things coming your way. Read the post](https://coderpad.io/blog/product-updates/coderpad-august-releases/) ## Hire with confidence. See why engineering teams trust CoderPad to evaluate developers with realistic, AI-ready hiring workflows. [Request a demo](https://coderpad.io/sales/)