- Published on
How I evaluate new tools
- Authors
- Name
- Seb
Whenever facing a new challenge for implementing a new system or functionality, I try to stick to the mottos don't re-invent the wheel and use the right tool for the job - I attempt to find the right libraries or software that can be used as building blocks for solving the problem at hand.
You want to parse JSON? Use a JSON-parsing library! You want to validate some data input? Use a validation library! You want to write structured logs? Use a proper logging library and a dedicated log shipping tool! You want to handle complex user authentication in your microservice stack? Use a user authentication service!
Sometimes it can be tempting to write a small quick solution yourself. However, even for the most simple of tasks, the complexity of the problem can be easily underestimated. Can you handle all edge cases, have you thought about the security aspects of your code? In the end, you might end up writing your own library which shadows the functionality of existing, mature and battle-tested libraries.
It's 2022 and Open Source is thriving. Let's make use of it!
My checklist when selecting a library or tool
Often, there's many competing libraries and tools available. How do I choose which one to use?
Use Open Source
I will always try to go for Open Source solutions. Besides the obvious benefit of being free, Open Source libraries and tools are often more secure (as many eyes have and can check them), better maintained (more hands on deck), and if something really does not work, you can always fix it yourself as a last resort.
I will often just naively google for
github <problem to solve>
; Examples beinggithub java json parser
orgithub python data validation
...Check the project's license
Does the library or tool you choose permit commercial use?
Check the project's health
There's a couple of things you can check on the project's GitHub page. These checks are not silver bullets, but they can give us an idea about the state of the project.
- How many GitHub stars does it have? Are many people watching and using it?
- How many open GitHub issues does it have? Are there a lot of unaddressed issues?
- When was the last commit? Is it actively maintained?
- How many maintainers does it have? Is it just one or two people doing everything? If they disappear, the project might stall.
Check the project's documentation
Does the project have a good documentation? Can you understand the Get Started section? If it's a platform, does it support Docker, so we can easily try it out locally, or later host it ourselves?
Another matter of importance is the maturity of the software: Are we past alpha/beta yet? Do we have a
v1.0.0
? Are there many breaking changes between versions? We definitely do not want to use immature libraries, as it might make our lives harder in the future.Check the project's security
You can search security vulnerability databases like the USA NVD CVE database or CVE details or just type
<project> vulnerabilities
or<project> cve
into google to find vulnerabilities for your candidate.Does our candidate have a lot of issues in the past? What's the severity? Are there maybe even open issues with high severity?
We have to be confident in the security of the software that we chose.
Check the solution's past user experience
After going through our list of checks, we might have a candidate that we are confident in. But what do other users think about our software?
As a big fan of Hackernews, I like to do a cheeky google search for
hackernews <project>
and read the comments of users on the threads about the project.With such an extensive user base of experienced and proficient members on Hackernews, it will serve us well to read their feedback on our target. Our potential candidate can look great on paper, but users may still find it hard to work with, buggy, or might even suggest a superior alternative!
Summary
Don't jump on the first solution you find, try to evaluate carefully. Once decided and implemented, the cost of maintenance or changing to another solution can be significant.