Thursday, March 12, 2015

getput : trivial key/value pair persistence tool in node/express/fs_db and js

One thing I've confirmed while learning grails is that for many tasks I'm more comfortable doing the "heavy lifting" in the browser. The primary limitation is persistence; localstorage is fun, but not reliable for longterm things, and various users can't share information.

I thought a simple key/value store would be empowering for hackmatch-style products, so I wrote one in node.js, using the popular express library and fs_db, a trivial key/value store that writes to the filesystem. (Mostly this whole thing is a convenience wrapper for fs_db)

The package.json for that was
  "name": "getput",
  "version": "1.0.0",
  "description": "trivial persistent keyval GET and PUT",
  "main": "index.js",
  "dependencies": {
    "body-parser": "^1.12.0",
    "express": "4.x",
    "fs_db": "0.0.4"

(So if you make a directory, put that package.json in it, and run npm install, it should handle all the dependencies)

I also made two subfolders: public/ (to store some static files to be served up by express) and db/ (for use by fs_db)

My main app was in index.js:
var DB = require( "fs_db" );
var express = require('express');
var bodyParser  = require('body-parser');
var app = express();

// Create a new store, writing data beneath db/.
var store = new DB( "db" );

app.put('/db/:id', function(req, res){
  var id =;
  var data = req.body.value;
  store.set( id,data );

app.get('/db/:id', function(req, res){
  var id =; 
  store.get( id, function( error, value ) {

app.use(express.static(__dirname + '/public')); 
console.log('Listening on port 3000');

Once that was in place, I created the js client, getput.js in public/:
function getput(pUrl){
    var url = pUrl;
    if(url.slice(-1) != '/') url += '/'; //ensure url ends with /
    return {
        get : function(key,callback){
            var newcallback = function(data){

                data=(data!="")?JSON.parse(data):null; //"" is unfound
                if(callback) callback(JSON.parse(data));
        getraw : function(key,callback){
            $.get(url+encodeURIComponent(key), function(data){
                if(callback) callback(data);
        put : function(key,val,callback){
        putraw : function(key,val,callback){
                url: url+encodeURIComponent(key),
                data: JSON.stringify({"value":val}),
                type: 'PUT',
                contentType: 'application/json',
                success: function() {
                    if(callback) callback();
This has a dependency on JQuery, for the Ajax. (The one stumbling block I had in this was properly lining up the contentType and the correct express bodyParser (express recently changed how it handles body parsing, so many online examples are out of date) and I didn't feel like refrustrating myself by making it work with an Ajax wrapper replacement.)

4 functions are exposed, get()/put() (which assumes you're storing JSON) and getraw()/putraw() (which doesn't). You initialize it with the main url express is listening to. (Hmm- I probably should figure the correct way of having node take an argument from the commandline)

Also in public/ I put this simple tester index.html page:
<!doctype html>
<meta charset="utf-8">
<title>getput tester</title>
<script src=""></script>
<script src="getput.js"></script>
    var getrputr = getput("/db");
    function put(){
    function get(){

key: <input id="key"> val: <input id="val">
<button onclick="put();">put</button>
key: <input id="getkey"> 
<button onclick="get();">get</button>
<span id="getval"></span>

It demonstrates getraw()/putraw().

I run the whole shebang with node or nodemon and bam!
clearly, this is why I wish to be a designer/developer hybrid

So a few gotchas still around, and stopping me from elevating this to a proper project until my free cycles return to a sane levels... the jQuery dependency, I should probably throw in a test or two, and then I think there might a value size limit for fs_db that I should document or code up.

No comments:

Post a Comment