People who know me know I'm a bit of a nut for
Node.JS, the JavaScript Application Framework. Over the past several years, I've been writing packages to make developing applications easier. The one that can have immediate utility to virtually all Node developers is
SN-Props (fomerly node-props.) On the surface, it looks like a simple package that reads a JSON configuration file from the command-line and passes it to the program shortly after launch. It certainly does do that, but it's even a little more capable, so I figured it would be useful to describe what I was trying to build and the problem I was trying to solve.
In the old days the preferred method of changing the behavior of a program from "Development Mode" to "Production Mode" was via the NODE_ENV environment variable. Before you launched the program, you set this variable to "production" or "development." Inside the program, you would check the environment variable and change your settings appropriately. This led to code that looked like this:
var listen_port;
var listen_addr;
var db_addr;
var db_pass;
if( "production" === process.env.NODE_ENV ) {
listen_addr = "0.0.0.0";
listen_port = 80;
db_addr = "prod-db.example.com";
db_pass = "DJl87sJXX01";
} else {
listen_addr = "127.0.0.1";
listen_port = 8080;
db_addr = "localhost";
db_pass = "password";
}
The biggest problem I have with code like this is it makes Node applications brittle. You wind up hard-coding things like database addresses & passwords into the app. If your operations staff needs to change the IP address or password of a machine, you have to track down a developer. And worse -- you wind up committing your database passwords to your source repo.
So the next thing we tried was to read a different JSON configuration file based on the environment variable. That file would contain all your favorite settings in a separate file. During development, you would use the file "dev.json" and in production you would look for "prod.json":
prod.json:
{
"listen_addr":"0.0.0.0",
"listen_port": 80,
"db_addr": "prod-db.example.com",
"db_pass": "DJl87sJXX01"
}
dev.json:
{
"listen_addr":"127.0.0.1",
"listen_port": 8080,
"db_addr": "localhost",
"db_pass": "password"
}
This is MUCH better. You no longer have to hard-code config options into the source and your ops staff doesn't have to worry about inadvertently borking production code if they need to change production settings.
At about this time, I started working with "redundant arrays of indepensive web apps." A typical deployment for me would be to have six copies of an app server running simultaneously. And through the magic of virtualization, we would sometimes add an extra two or three servers to meet high-demand periods.
After trying to ensure that
exactly the right version of a configuration file made it onto any given server, I realized I wanted was to store my config JSON files on an internal web server. Instead of reading a file on the local file-system for config settings, I just queried the web server. This worked pretty well: I only had to configure one or two files on a single web server to change the behavior of the entire cluster. Yes, I could have used SCP to copy config files, but it was a bit of a pain to program our deploy system to distribute the right file to newly started instances and time it so we pushed the file out before the app started but after the openssh server started. No. It was much better to have the application itself
pull it's config data instead of having a central server
push the config to new machines.
I was very happy with this solution until we split our app into shards. All of the sudden we needed to have a lot more per-machine config data. Our initial idea was to load up the config server with a bunch of different files, one per machine and make the apps smart enough to pull the right file. This turned out to be annoying due to the large number of config settings that were common to all machines. If you wanted to change a global setting, you had to change the setting in each of the per-machine config files.
The answer was to split our config settings into per-app and per-machine settings. The per machine settings included IP addresses of proxies & sometimes databases. The per app settings included things like tables, usernames and passwords for databases and the like. And after writing a few apps that made two explicit HTTP(S) queries after starting up, it made sense to move that behavior into it's own package.
And that is how the SN-Props was born (though we called it node-props back then.) And this is how I use it today. I put per-machine settings in a local file and per-app or global settings on an internal web server. The per-machine settings file would likely look like this:
{
"listen": {
"port": 9001,
"host": "127.0.0.1"
},
"telemetry": "http://west-coast.telemetry.sm5.us/"
}
while the global settings file would look like this:
{
"appname": "Cookr",
"tagline": "Crowdfunding innovative recipes since 2014",
"database": {
"host": "cookr.db.sm5.us",
"user": "cookr",
"password": "DJl87sJXX01",
"collection": "cookr_main"
}
}
To start the service, I don't monkey with environment variables, i just create three versions of the various settings files: one for development, another for integration testing and a third for production. When i want to start the app, I list the URLs of the config files on the command line:
/opt/node/bin/node cookr.js --config file:///etc/permachine.json --config http://deploy.int.sm5.us/cookr.json
The
SN-Props package is licensed with under a MIT License, so you can use it as well. Here's a code sample of how easy it is to use:
require( 'sn-props' ).read( function( properties ) {
// Do something with the properties here. The object passed to this
// function has the contents of all config files merged into it.
} );
