Final Blog Post

Introduction

This is my final blog post for my Software Engineering class. In this post I briefly describe Plato, I give the results of my experiment, and I analyze what I learned about the open source code for my project.

First of all, JavaScript is a scripting language that has grown tremendously in popularity within the past few years. RedMonk [1] now ranks it as #2 on its list of most popular languages on GitHub. JavaScript’s popularity is in large part due to its ease for amateur programmers. The lax syntax and higher ordered nature of the language make it approachable to even novices who have never programmed before. However, when advanced developers seek to use JavaScript, they soon find that JavaScript is lacking the source code analytics tools and debugging frameworks many are accustomed to with languages like Java and C++. Plato seeks to remedy this problem.

Plato is a JavaScript static source code analysis and complexity visualization tool. Say that five times fast. Plato accepts any JavaScript project (the entire project) or file as an input and spits out a beautiful visualization of various complexity metrics such as cyclomatic complexity and maintainability scores. I will cover how Plato does this in my Podcast. For now, it is enough to know that Plato is a tool desperately needed by JavaScripters.

Results

To analyze the tool, I ran Plato on 10 open source projects available through GitHub (the links are at the bottom). The projects are divided in to two groups. The first group exclusively contains five very popular projects with years of development time. These projects typically have many contributing developers and a strong user base in the JavaScript community. The second group of projects contains five less popular, though interesting, projects with significantly less development time and number of developers. My goal in this experiment is to explore the complexity differences between the two groups.

Each project report developed by Plato houses the following information (taken from https://github.com/es-analysis/plato):

Total/Average Lines of code – The total number of lines of code displayed next to the average number of lines of code per file in the project.

Average Maintainability – A value between 0 and 100 that represents the relative ease of maintaining the code. A higher value represents easier maintainability.

Maintainability – The maintainability score of each source file.

Estimated Errors in Implementation – Maurice Halstead’s estimated number of errors using Halstead complexity measures.

Lint Errors – The number of errors and warnings generated by another JavaScript source code analysis tool: JSLint.

To run Plato on a given project, I first installed node.js. From the node.js command prompt, I executed the following command line instruction for each project: “plato –r –d “reportName” project” where –r recursively searches the “project” directory and –d initiates the report to be named “reportName.”

Group 1 Projects

Group 1 consists of BootStrap, d3, Gmaps, Moment, and Node.

BootStrap

Bootstrap1

Bootstrap2

BootStrap is a popular framework used for web development. Surprising to me, the BootStrap project is only 9518 source code lines long. The low number of lines of code and limited number of files means it is quicker to load on web browsers. It also has an average maintainability of 70.2. This is a respectable number as you will soon see.

D3

d31 d32

D3 stands for data driven documents. It is a popular library for creating aesthetically pleasing data visualizations. It has nearly 30,000 source lines of code and it received the highest Average Maintainability score at 77.89

Gmaps

gmaps1 gmaps2

Gmaps.js developers describe their application as “google maps api with less pain and more fun.” They are correct. I personally have used gmaps.js to create a web-based bus tracking application for my undergraduate school. It is a great tool. It is interesting to note that it has a maintainability of only 67.9. This may be due to the hack nature of the library, by essentially wrapping everything from the google maps api in to even higher order functions.

Moment

moment1 moment2

Moment is a date library useful for anything you would ever want to do with date objects. I am curious as to why Moment needs nearly 50,000 source lines of code. That probably hurts its maintainability score quite a bit.

Node

Node is a runtime platform for “easily building fast, scalable network applications.” It is great for just about anything web based, especially data-heavy and real time applications.

Unfortunately, due to performance instability, Plato was unable to display the report for Node. Node has a whopping 291,621 source lines of code with an average of only 93 lines of code per file. That means there are about 3100 source files in the node project. That’s a lot of development time for a JavaScript library! One interesting point to note is that Node achieved the second highest Average Maintainability score of all the big projects with a score of 75.95.

Group 2 Projects

Group 2 consists of Full Screen Mario, Ghost, ngProgress, One Page Scroll, and rainyday.

Full Screen Mario

fsmario1 fsmario2

Full Screen Mario is an “HTML5 remake of the original Super Mario Brothers, expanded for modern browsing.” Full Screen Mario is a blast to play. It is well designed and was created by just a handful of developers in less than a year. Unfortunately, as of October 31st, the project no longer runs on fullscreenmario.com due to Nintendo Copyright Infringement. Regardless, the project is downloadable and playable through their GitHub (link at the bottom). Full Screen Mario sets the bar low for all the projects with the lowest Average Maintainability value – a score of only 53.85.

Ghost

ghost1 ghost2

Ghost is a simple to use open source blogging platform. More information can be found at http://ghost.org. Ghost reaches 32,500 source lines of code but manages to score an Average Maintainability number of 72.53.

ngProgress

ng1 ng2

ngProgress is a “slim, site-wide progressbar for AngularJS.” Perhaps not surprisingly, this is the least popular project of both groups. It is also the smallest by a long shot, at only 153 source lines of code.

One Page Scroll

onepage1 onepage2

One Page Scroll is a plugin used to create “an Apple-like one page scroll website.” It does just that. With only one contributor, this project is the smallest on the list. This project does not reach 1000 source lines of code but manages create the third lowest Average Maintainability score of all projects with a score of 58.53.

rainyday

rainy1 rainy2

rainyday is a JavaScript library that “makes use of the HTML5 canvas to render an animation of raindrops falling on a glass surface.” Go to http://maroslaw.github.io/rainyday.js/ to see a cool demo. rainyday scores the second lowest  of all projects in the Average Maintainability metric with a score of 53.33.

Conclusions

First of all, I believe Plato is a great interactive tool. It visualizes complexity scores unlike any other static source code analysis tool, and is one of only a handful of useful source code analysis tools available to JavaScript developers. I drew many conclusions about both the tool and the open source code projects I used.

As far as each group goes, I certainly found the whole of Group 2 projects easier to interact with on the visualization. Smaller projects meant better performance when using Plato. The performance issue on Plato severely hindered my ability to gain any useful information on Node. As a whole, I noticed that Group 1 averaged a much higher Average Maintainability score with an average of: 70.5. Group 2 averaged an Average Maintainability score of 61.7. I learned that there is a strong positive correlation between the number of source lines of code, the Estimated Errors in Implementation, and Lint Errors. I could not determine any correlation between Average Maintainability and the other metrics. Some projects had a high average maintainability with a low number of source lines of code, whereas a project like d3 was one of the biggest projects and scored the highest Average Maintainability.

One of the drawbacks of Plato is that the metrics used to score the complexity values are not transparent. They are not easily understandable. One interesting anomaly is the ngProgress project. I’m unsure how much more “maintainable” a 153 source line of code project could possibly be. The user is left to wonder what it would take for a score above 80, let alone 100!

Another drawback of Plato is the overall lack of insight. Aside from easily seeing a link between source lines of code and errors, not much else useful comes of Plato. The Average Maintainability score is a nice ego boost if your project scores higher than a friends, but aside from really low values in this department, my guess is that no developer really cares about this score.

Overall, I believe that with more development time, Plato could provide greater insight in to the problems of a project. A potential feature idea would be a suggested change for each Lint Error. Another useful feature would be helpful hints to reduce the Estimated Errors in Implementation score and raise the Average Maintainability score. More transparency of these metrics would be highly beneficial to a developer seeking to improve upon his or her code.

References

[1] http://redmonk.com/sogrady/2013/07/25/language-rankings-6-13/

All project links can be found here: http://adamgillfillan.com/2013/10/21/list-of-open-source-projects/

Leave a Reply

Your email address will not be published.