Author: admin

Categories
Uncategorized

Setting up Coveralls for your Ember Addons

In this tutorial, we will see how to setup automated code coverage metrics collection for your Ember addons using Coveralls and Github Actions.

Why Code coverage?

In simple terms, code coverage is the fraction or percentage of code paths executed by some test or test suite of a program. It is generally measured by a tool which executes the test and logs the lines of code, the test “touches” while running. At its most basic, every conditional statement creates a “branch” defining two unique code paths, and theoretically, both “branches” of each condition must be executed by the test suite in order for the developer to be certain that the code works correctly in each condition.

Code coverage is often used as a metric to determine the effectiveness of Unit tests. Low coverage typically means that developers are not writing adequate unit tests. This signifies that there are many code paths in the application which may possibly behave incorrectly.

ember-cli-code-coverage

This addon provisions to gather Code coverage for your Ember apps and addons using Istanbul. Install this addon into your Ember app or addon with ember install;

The latest version published on npm is 0.4.2. This might be a very old version and not Octane compatible, you might run into some errors while running the tests. But there is a latest one in Github release with the tag v1.0.0-beta.9. If you are running into any problems with the npm package , you can directly install the 1.x beta version using yarn or npm.

Or using yarn

Coverage will only be generated when an environment variable is true (by default COVERAGE) and running your test command like normal.

Coveralls

Coveralls is a language-agnostic and CI-agnostic web service to help you track your code coverage over time, and ensure that all your new code is fully covered.

Add your github repo for the app/addon to Coveralls using Add Repos option in the website.

After adding the repo, Coveralls will provide you with a repo token, which we need to identify the repo with Coveralls. You need to add a Github Secret with the value of this token in your repo. This will be later used in our Github actions to set up the COVERALLS_REPO_TOKEN environment variable before sending coverage information to Coveralls.

Install the coveralls node package.

Configure your test script to include coverage information.

Add a separate coveralls script to upload the coverage information to Coveralls service.

This will upload the coverage statistics generated while running tests in a folder called coverage in a file known as lcov.info

Generating coverage statistics in CI

If you want to generate coverage information and upload it to Coveralls while running your tests in CI whenever you push your code to the repo or whenever a pull request is raised, you can go for some automated CI setup using Travis or Github Actions. In this tutorial we are going to look at how we can achieve this using Github Actions.

Github actions for Coveralls

This GitHub Action posts your test suite’s LCOV coverage data to coveralls.io for analysis, change tracking, and notifications. You don’t need to add the repo to Coveralls first, it will be created when receiving the post.

When running on pull_request events, a comment will be added to the PR with details about how coverage will be affected if merged.

Create a new job for your Github Actions or add it part of the existing one.

If you want to see the above setup in action, please take a look at my ember-chance addon repo.

Optional Goodies: README Badge

And finally if you want to add the coverage statistics as a badge in your README, you can do so by adding the following snippet at the top of your README file, which will show how much the coverge % is for your addon.

Please ensure to replace the above snippet with the appropriate values for the placeholders like <github-user-name> and <repo-name>

Recap

  1. Install ember-cli-code-coverage
  2. Install coveralls from npm
  3. Modify the test script in package.json to include COVERAGE=true
  4. Add a new script for coveralls to upload the coverage info
  5. Add your repo to Coveralls.io and get the repo_token
  6. Add a Github secret with the value of the repo_token in the repository
  7. Setup automated coverage collection in CI using Coveralls Github Actions

References:

Categories
Articles Javascript

Snowpack – How it works?

Snowpack is a post-install tool. It runs after npm install, and it essentially exists to convert your npm packages (in your “nodemodules/” directory) into JS files that run in the browser without a bundler (written to a “webmodules/” directory).

Creating a sample app

In this tutorial we are going to create a simple demo app which is going to make use of a library called finance. We are going to create an app to calculate simple interest from principal, rate and time. This application has got no fancy user interface or something. It just calls the method from the library by giving parameters and print the result in the console.

First let’s setup the application by creating the necessary files.

index.html

This is how our index.html file will look like. As you can see, there’s nothing much in the page apart from the script tag at the bottom. This script tag includes a file called app.js from the src folder of the app and having the attribute type set as module. This means that app.js is an ES module that can be used directly in the page. We will see what goes inside the app.js file later.

snowpack-demo/package.json

This is how our package.json will look like for the demo app. The important thing to note here is the dependency of finance package.

Creating a dummy package inside nodemodules

And now for this example, we are not going to actually install any npm package. We are going to create custom packages on the fly within the nodemodules folder. That’s how nodemodules work, at the end of the day all the packages are just a folder of files with a package manifest, in other words a package.json. Hence in order to create a new package all you need is two things: one package.json and the source file.

For the finance package we were talking about earlier, we are going to create the package in the same way like below.

package: finance

And the package finance will contain two functions: one for calculating simple interest from principal, rate and time and the other one including the principal. We will be using only the simpleInterest function for our demo. The other one just exists for the sake of it.

finance/package.json

The package.json for the finance module is a normal package.json manifest with one exception. We are adding the module field to point out the ESM version of this module by telling where it resides. Since we have already written the package using ES import and export statements, the value of this field is the same as the main field which points to index.js

package: math

Now it’s time to take a look at the math package. It’s simple library exposing primitive operations like add, multiple, divide, etc., and it follows Common JS export system. The reason it is using CMD is to demonstrate the capabilities of Snowpack in handing Common JS modules. Snowpack can also bundle Common JS modules but which are actually the internal dependencies of your parent packages.

math / index.js

The below are the contents of the math library.

Now the dependency tree of our demo app looks like this.

Now install the dependencies using npm and then run snowpack.

Snowpack will read the dependencies from the package.json and start bundling them. Each individual dependency is built with all its dependent packages are flattened into a single file. As you can see below the finance and math packages is bundled into a single file in the new webmodules directory called finance.js. And this is the file which we will be consuming in our demo app.

Now if you inspect the finance.js file in your webmodules folder.

Now we can see how we can use the finance.js from the webmodules folder in our app.js

Peer Dependencies

Now what about peer dependencies? Snowpack is very well equipped for handling peer dependencies in your applications also. It will properly bundle your dependencies by putting the commonly used code such as peer dependencies inside a common folder so that the packages which are consuming these can easily access the same without redundancy.

The dependency tree of our app is very simple we have only two packages finance which is depending on math. Let’s introduce a new package called bmi which will expose methods for calculating bmi (body mass index). The bmi package is also depending on math package for its calculation. Hence our math package now becomes a peer dependency for finance and bmi.

We are going to follow the same steps for creating the bmi package just as we did for finance.

package/bmi

Now add the same package to the dependencies list for the demo app in package.json

The dependency tree of our demo will now look like this:

Now install the dependencies using npm and then run snowpack.

You can optionally add “snowpack” as a “prepare” script to your package.json and npm/yarn will automatically run it after every new dependency install. This is recommended so that new dependencies are automatically included in your webmodules/ directory immediately.

After the installing and running snowpack the bundled files inside webmodules directory will be three Javascript files. One for the bmi package, one for the finance package and we now have common directory which contains the common code in the file named index-093dfa0c.js used by both the packages, which is actually the math package code.

If you inspect the contents of the file in the webmodules folder you can see yourself that Snowpack changed both the bmi and finance package to import from the common math package bundled.

This is how the bundled bmi package will look like now.

And this is how the bundled finance package will look like.

And if you are curious what goes inside the common index file, as I mentioned previously it just contains the code of the math package.

Now we can import the bmi package into our demo application from the webmodules folder like below:

Production Builds

Snowpack is not only optimized for development environment but also for production builds. You can create compressed or minified version of your dependencies for using in production environments and deploying with Snowpack. It also generates source maps when you are bundling for production. All you need to do is to pass the –optimize flag while running Snowpack.

Tree Shaking

Snowpack helps you to remove any unused code from your dependencies (when “Automatic Mode” is enabled via the –include flag). In order for Tree shaking to work properly we need to have ESM compatible versions for all your packages. Since our math package is based on Common JS module system, we need to have a separate ESM version for the same like below.

It is actually quite easy all you have to do is to convert them using export syntax for each methods which are exported from the math package instead of using the module.exports

And you also need to make some changes with the package.json of the math package by exposing the ESM version using the module field.

Now if you run Snowpack again with the –include flag with the app.js file.

You will have your math package properly tree-shaked or the unused subtract methods from the package will be removed since it is not used by any of the dependencies.

That’s all from this tutorial. Hope you have a better idea now about how Snowpack bundles your dependencies in your applications from the above examples. Please let me know for any issues or feedback about the article in the comments.

Cover Image by Chris Biron on Unsplash

Categories
Articles Javascript

JARVIS – Write me a Codemod

In this article, we are going to take a look at a tool called JARVIS which will profoundly transform the way how you write codemods.

This is a continuation post for AST Finder which is a tool to generate an api to find AST nodes automatically from source code.

If you want to know more about codemods, their building blocks and how they work, please check out this detailed post about codemods.

What?

JARVIS is actually a playground for writing codemods, where in you can specify the input and output code, choose the appropriate transformation you want to apply and the codemod is automatically generated for you.

JARVIS currently supports Javascript and Glimmer.js templates. For Javascript, you can use parsers like recast and babel. For Glimmer.js templates which is a more advanced form of Handlebars templates used by Ember.js framework, it is using the ember-template-recast parser to parse and transform the code.

It gives a playground to write code transformations just like AST Explorer but with a more fine-grained control and flexibility to write the transformations because it automatically figures out a lot of things for you like, how you can build new nodes, how you can find nodes from the AST, how to replace, remove or insert nodes into the tree, which visitor function you need to use for writing transformations.

Why?

If the goal is to enhance software development throughput, you can either:

• get faster people to do the work,
• get more people to do the work, or
• automate the work.

― Chad Fowler in The Pragmatic Programmer

We already have a very good tool called “ast-explorer” for visualizing abstract syntax trees or ASTs and writing codemods. Why we need a new tool then? Because AST explorer is only for exploring your ASTs, it doesn’t tell how to find or build AST nodes. You have to figure out a lot of things before you can start writing a codemod.

I spent a lot of time figuring out the jscodeshift apis, by looking at their documentation, learning from existing codemods, searching in google, and so on. The learning curve is kind of steep. And my colleagues have to tread down the same path. Whenever we tell any new developer in our team to learn about codemods, they face the same challenges we had in learning about them. Hence I wanted to make their lives easier and approach, learning about codemods a painless task. And that’s why JARVIS is created.

The first task in writing codemods is to find the appropriate nodes in the AST and work on them. And we definitely need a tool which makes it easy to find nodes. The problem is, there is no proper documentation on finding AST nodes using the jscodeshift api. All you have to do is learn from other awesome-codemods out there and sift through the code and find out how you can query nodes.

And the second thing, when you are replacing nodes, you need to know how you can build new nodes. While writing codemods, most of the time we will be creating new nodes. JARVIS can take care of these things for you so that you can focus on building or tweaking the transformation rather than wasting most of your precious time in finding the reference or documentation for building and finding AST nodes.

Once you are done with the transformation, you can simply copy the codemod from JARVIS, put it in a file, let’s say for example transform.js and run it through with jscodeshift.

How?

JARVIS uses two npm packages underneath. One for building AST nodes from code and the other one for finding AST nodes from code. It is pretty much a combination of both AST-Builder and AST-Finder working together to automatically generate code transforms.

First you need to specify the input code in the top-left panel (please see the annotated screenshot above for reference), then you choose the appropriate AST Node operation below. You can choose to remove the node, replace , insert the new node before or after the existing node.

Second, if you are replacing code you can enter the destination code how it is going to change in the bottom-left panel. Then JARVIS will automatically generate a boilerplate code for your transformation in the top-right panel. You can tweak the transformation according to your requirements and verify the transformation is working properly or not by looking at the output in the bottom-right panel.

Let’s consider a simple example in JS. Let’s say you want to replace all your foo() calls with foo.bar() in your code. The first task here is you need to identify the type of AST node you want to manipulate, in our case replace. This is an ExpressionStatement with an expression of type CallExpression. Whereas, the expected output node type is a MemberExpression. Hence you need to understand the syntax and semantics of finding a CallExpression and create a MemberExpression AST node, which will be a very difficult endeavor for someone who has just starting with codemods.

What JARVIS does here is, it figures out the type of the input node and the output node, the node manipulation operation you want to do with, and then automatically generates a codemod boilerplate for the same.

The codemod for the above transformation will look something like this:

To know more about how JARVIS works, you can take a look at the source code here, it is built in Ember.js.

And if you are a dark-theme fan, JARVIS also allows to switch your editor themes to dark mode. Please use the Report issues link at the footer if you want to report any issues or feedback, you can tell us how we can improve the tool and what additional languages / parsers we need to support.

Stay tuned or subscribe to the newsletter at the bottom to know more about the exciting tools we are building around ASTs and Codemods.

References

Categories
Articles Javascript

AST Finder – Finding AST nodes from code

In this article, we are going to take a look at a tool called ‘AST Finder’ which will significantly improve the developer experience for writing codemods.

This is a continuation post for AST Builder which is a tool to generate AST nodes automatically from source code.

If you want to know more about codemods, their building blocks and how they work, please check out this detailed post about codemods.

What?

AST Finder is actually a playground for finding AST nodes using source code. Because ASTs play a big role in writing codemods, this tool will assist the developers to a great extent in writing codemods. Because codemods actually do AST-to-AST transformation on your source code, and that’s one of the primary reasons, why codemods are more resilient in making effective code transformations.

Why?

Never memorize something that you can look up.

― Albert Einstein

We already have an well-established and battle-tested tool called “ast-explorer” for visualizing abstract syntax trees or ASTs. Why we need a new tool then? Because AST explorer is only for exploring your ASTs, it doesn’t tell how to find AST nodes.

The first task in writing codemods is to find the appropriate nodes in the AST and work on them. And we definitely need a tool which makes it easy to find nodes. The problem is, there is no proper documentation on finding AST nodes using the jscodeshift api. All you have to do is learn from other codemods out there and sift through the code and find out how you can query nodes.

Now let’s say you want to replace a CallExpression, foo.bar() with a new one like foo(). The AST representation for the above two expressions will be like:

I have omitted a lot of information in the above code for clarity and readability purposes. It only contains the relevant information for the actual CallExpression AST node. If you want to explore the full tree structure of the AST, you can check it in ast-explorer.

As you can see from the above two AST nodes, the only difference between the two is the callee object which is a simple Identifier in foo() and a MemberExpression in foo.bar(). Usually with codemods, we will be replacing the original expression with the new one. Hence here, we will be replacing the original CallExpression with a new one like this.

In order to replace the old CallExpression with a new one, first we need to find the existing CallExpression. From the above codemod you can see we are querying the AST using jscodeshift api like this:

If you try to find the above CallExpression within ast-explorer transform editor, you will be having a tough time if you are doing it for the first-time. Because you are not very familiar with the find api in the first place, and you don’t know the right order and type of parameters you need to supply to correctly find the AST node. And don’t forget the typos and punctuation errors you make while typing the code.

There are also some subtle nuances with the jscodeshift api which beginners won’t know for example, the api j.callExpression is a constructor for building CallExpression nodes, whereas j.CallExpression is an instance of the type CallExpression which is basically used to find nodes of the type CallExpression.

This is where AST Finder comes into the picture, it is acting as a reference guide for find apis to easily query your AST nodes. Just input the code in the input editor (see the image above to identify the input editor which is always at the top left pane in the layout) you will get the find api automatically generated for you without any mistakes. So if you input foo.bar() into the AST Finder, it will give you something like:

Now, you can simply copy the query from AST Finder and use it in your codemods. How cool is that?

How?

AST Finder uses ‘ast-node-finder‘ an npm package underneath, which provides the apis for finding AST nodes through jscodeshift. All the api’s takes an object as a parameter and returns the find api in string format, which you can use with jscodeshift to query new nodes. The object which is passed to the api as a parameter is actually a node in the AST generated by the respective parser. When you feed the node to the ast-node-finder api, you get back the jscodeshift api to find that node.

This allows developers to easily and effectively find AST nodes from source code. All you have to do is just enter or copy paste the source code into the input editor and you can see the jscodeshift api automatically generated for you in the output editor.

The above snippet will generate some thing like this:

You can also use the AST Finder to visualize your AST on the top right pane without all the noise and clutter of meta information. We deliberately filter out the loc nodes from the AST and also the tokens, since we feel it is not of much use for working with codemods. To dig deep into the finder you can take a look at the source code here, it is built in Ember.js.

And if you are a dark-theme fan, AST Finder also allows to switch your editor themes to dark mode. Please use the Report issues link at the footer if you want to report any issues or feedback, you can tell us how we can improve the tool and what additional languages we need to support.

Stay tuned or subscribe to the newsletter at the bottom to know more about the exciting tools we are building around ASTs and Codemods.

References