JSON round trip with Node.js

One of the first things you need to do, if you’re serious about writing a RIA with a JavaScript backend, is be able to quickly send messages to and from the server. JSON is obviously the best format for JavaScript-to-JavaScript communication. So, I set up a simple example of a node.js server that can both send and receive JSON objects via AJAX, and cache them in memory on the server. The full code of the example is out on github:

I’m going to pluck out the juicy bits right here, though, and explain them.

Client To Server

The first thing you need to do is be able to POST a JSON object. This is easy enough with jQuery:

function put(id, data, callback) {
 $.ajax('' + id + '/', {
 type: 'POST',
 data: JSON.stringify(data),
 contentType: 'text/json',
 success: function() { if ( callback ) callback(true); },
 error : function() { if ( callback ) callback(false); }

Note that the body of the POST is not URL encoded, like that of a POSTed form: that’s verbose and wasteful, and gets us nothing since we’d have to decode it on the server anyway. Note also that I’m using JSON.stringify. This is in the ECMA-262 standard, built into modern browsers, and Douglas Crockford has written a JSON compatibility library for legacy browsers.

The next step is to receive that message on the server. Inside of a HTTP response handler:

http.createServer(function(request, response) {
 if ( request.method === 'POST' ) {
 // the body of the POST is JSON payload.
 var data = '';
 request.addListener('data', function(chunk) { data += chunk; });
 request.addListener('end', function() {
 store[id] = JSON.parse(data);
 response.writeHead(200, {'content-type': 'text/plain' });

The request is emitting multiple “data” events, each with a piece of the JSON string: we have to accumulate all of these into one string. When all data is received, the “end” event is emitted, and we can proceed to parse the now-complete JSON string. In this case our handling consists only of tucking away the deserialized object in the store. Afterwards, we return a empty document with a “200 OK” status.

I should probably do error handling on the JSON.parse as it’s likely to throw an exception, but I forgot. Typical error handling looks like this:

try {
 store[id] = JSON.parse(data);
} catch ( e ) {
 response.writeHead(500, {'content-type': 'text/plain' });
 response.write('ERROR:' + e);

Server To Client

This is very simple. On the server, we just have to get the object out of the store, serialize it, and write it out.

if ( request.method === 'GET' ) {
 // exact id lookup.
 if ( id in store ) {
 response.writeHead(200, {'content-type': 'text/json' });
 response.write( JSON.stringify(store[id]) );
 } else {
 response.writeHead(404, {'content-type': 'text/plain' });
 response.write('no data for ' + id);

Note that I’m using the mime type text/json. The official MIME type is application/json, but I’ve had trouble with frameworks treating that as unencoded binary data. You should probably use the standard, though, unless you have a good reason.

jQuery supports JSON data right out of the box, so there’s barely anything for us to do on the client:

function get(id, callback) {
 $.ajax('' + id + '/', {
 type: 'GET',
 dataType: 'json',
 success: function(data) { if ( callback ) callback(data); },
 error : function() { if ( callback ) callback(null); }


It’s easy to send JSON from the client to the server, and even easier to get it from the server to the client. There are no no mismatched data types, no parsing or serialization algorithms, just two environments that speak the same language communicating in a minimal (but not trivial) subset of that language. Can you see why I’m so excited about this stuff?

Reference : http://oranlooney.com/json-round-trip/

How to Use the Table Storage Service from Node.js

This guide shows you how to perform common scenarios using the Windows Azure Table storage service. The samples are written written using the Node.js API. The scenarios covered include creating and deleting a table, inserting and querying entities in a table. For more information on tables, see the Next Steps section.

Table of Contents

What is the Table Service?
Create a Windows Azure Storage Account
Create a Node.js Application
Configure your Application to Access Storage
Setup a Windows Azure Storage Connection
How To: Create a Table
How To: Add an Entity to a Table
How To: Update an Entity
How to: Change a Group of Entities
How to: Query for an Entity
How to: Query a Set of Entities
How To: Query a Subset of Entity Properties
How To: Delete an Entity
How To: Delete a Table
Next Steps

What is the Table Service?

The Windows Azure Table storage service stores large amounts of structured data. The service accepts authenticated calls from inside and outside the Windows Azure cloud. Windows Azure tables are ideal for storing structured, non-relational data. Common uses of Table services include:

  • Storing a huge amount of structured data (many TB) that is automatically scaled to meet throughput demands
  • Storing datasets that don’t require complex joins, foreign keys, or stored procedures and can be denormalized for fast access
  • Quickly querying data such as user profiles using a clustered index

You can use the Table service to store and query huge sets of structured, non-relational data, and your tables scale when volume increases.


The Table service contains the following components:


  • URL format: Code addresses tables in an account using this address format:
    http://<storage account>.table.core.windows.net/<table>

    You can address Azure tables directly using this address with the OData protocol. For more information, see OData.org

  • Storage Account: All access to Windows Azure Storage is done through a storage account. The total size of blob, table, and queue contents in a storage account cannot exceed 100TB.
  • Table: A table is an unlimited collection of entities. Tables don’t enforce a schema on entities, which means a single table can contain entities that have different sets of properties. An account can contain many tables.
  • Entity: An entity is a set of properties, similar to a database row. An entity can be up to 1MB in size.
  • Properties: A property is a name-value pair. Each entity can include up to 252 properties to store data. Each entity also has three system properties that specify a partition key, a row key, and a timestamp. Entities with the same partition key can be queried more quickly, and inserted/updated in atomic operations. An entity’s row key is its unique identifier within a partition.

Continue reading

Node.js + MongoDB = Love: Guest Post from MongoLab

Node.js with the popular document-oriented MongoDB make for a deeply powerful and robust application platform. Or in other words, they rock.

(Note: This blog post was contributed by Ben Wen of MongoLab – a Joyent Partner and provider of MongoDB hosting, support and analytics)

Pair Joyent Cloud’s hosted node.js SmartMachine Appliance with MongoLab’s hosted MongoDB and the integration becomes downright operatic. Angels sing. Trumpets blare. Grey storm thunderheads of object-relational-mapping haze part. Revealed are golden rays of low-impedance JSON object storage and query. All in the fertile green valley of asynchronous JavaScript on the unflappable, cool bedrock of Joyent’s SmartMachine hosting platform. Songbirds tweet. Life is good. Metaphors strain.

More prosaically, the high performance asynchronous design of node.js and the tunable latency/consistency of MongoDB mean a high throughput application can be assembled in a compressed timeframe and with standard tools you probably have laying around the home. Since MongoLab runs managed hosted MongoDB instances on Joyent’s Cloud near a node.js SmartMachine, you get world-class operation of both environments.

Below, we’ll take a quick spin setting up a MongoLab database and a no.de account. We’ll build a minimalistic Web Server that can do some data inserts and queries and display it through a gratuitous 3D guestbook demo.

For the impatient

  1. Sign up at mongolab.com and create a MongoLab database on Joyent Cloud and note database name,hostname, port, database username/password
  2. Sign up at no.de and start a SmartMachine
  3. git clone git://github.com/mongolab/demo-node-01.git
  4. Modify config.js with database credentials and connection info from Step 1.
  5. git commit -a -m "updated config"
    git remote add mongolabdemo <your no.de machine>.no.de
    git push mongolabdemo master
  6. Point your WebGL capable browser to <your no.de machine>.no.de and enjoy.

For the really impatient

  1. Go to nodejs.mongolab.com with your WebGL compatible browser

What is MongoDB?

First a quick word about MongoDB for the newly initiated. MongoDB is a non-relational database system that emphasizes horizontal scale across multiple servers, tunable consistency, and high performance. Being a document-database, it uses JSON notation to describe data and sports a rich query language with indexes to enhance query speed. It also has a map-reduce framework for more intense data analysis and transformation. There is growing adoption of MongoDB for large stores of documents like in a Content Management System or in data analytics, for feature-rich Web 2.0 sites and games, and for persistent stores for mobile applications. Its code is open source licensed under the Gnu AGPL v3.0 and is commercially licensed from its author, 10Gen. Large corporations and smaller outfits are using MongoDB in production today. New users, you are in good company.

Continue reading

Building node.js on Cygwin (Windows)

Building node.js on Cygwin (Windows)

NOTE: v0.2.6 and v0.3.1 build on Cygwin. The versions between v0.3.1 and v0.4.0 do not. Since v0.4.0 node builds again on Cygwin. Since v0.4.3 node requires gcc4-g++ on Cygwin.

This tutorial will guide you through setting up the latest stable version of node.js on Cygwin. You don’t need to have a working Cygwin install.

  1. Grab and install Cygwin.
  2. Using setup.exe from Cygwin (1), install the following packages required to compile node.js:
    • devel → gcc4-g++ [Builds v0.4.2 and earlier use gcc-g++]
    • devel → git
    • devel → make
    • devel → pkg-config
    • devel → zlib-devel
    • libs → openssl-devel
    • net → openssl
    • python → python

    You may also want to install the following: * editors → vim or editors → nano for (4) below * web → curl if you wish to install npm, node’s package manager

    You can use the search box at the top-left to locate packages quickly.

  3. It’s time to clone the Git repository and build node.js. Start a new Cygwin shell (bash, zsh, etc.), openStart → Cygwin → Cygwin Bash Shell. Run the following commands:
    $ cd ~ $ git clone git://github.com/joyent/node.git $ cd node $ git fetch --all # if the above fails complaining --all is not recognised, try: git fetch origin $ git tag $ git checkout [latest stable tag from previous command, e.g., v0.2.5] $ ./configure # Pre-emptively create out/default just in case https://github.com/joyent/node/issues/1734 is not fixed $ mkdir -p out/default $ make $ make install 

    It is recommended you checkout a stable tag since most of the time building master fails. If you receive an error at any of the above steps, look further down for possible solutions.

  4. Set up Domain Name Resolution (DNS)Cygwin internally uses Windows for DNS queries. node.js uses the c-ares library that relies on /etc/resolv.conf. Cygwin ships with an empty /etc/resolv.conf. In order to enabled networking from your scripts, add these IPs to the file (Google Public DNS):
    $ vim /etc/resolv.conf nameserver nameserver 

    For Vim newbies: use i to enter insert mode, <Esc>:wq to exit insert mode, enter the command window, write and quit. If you are uncomfortable with Vim, use nano /etc/resolv.conf instead.

    Note even if /etc/resolv.conf is correctly set to Google DNS, ECONNREFUSED will happen on .listen() if Windows is not connected to the Internet and you attempt to listen at localhost or the machine name (ie. joeslaptop etc.) Only workaround for offline Windows (such as a development VM) to successfully .listen() at the local host is to use explicitly, neither localhost nor the machine name will currently (0.4.5) resolve when offline.

    If you cannot ping the Google DNS IPs listed above (because of a firewall, for instance) you will have to use different IPs. Open a DOS prompt and run “ipconfig /all”, which will list the DNS servers that Windows is currently using. Try using those IPs (prefixed with “namesever “) in the resolve.conf instead.

Build Problems

python.exe: Can’t Open File ‘waf-light’:

C:\Program Files\Python26\python.exe: can't open file '/home/stan/node/tools/waf-light': [Errno 2] No such file or directory

This is not an issue with node.js. You are using the Windows version of python in Cygwin. It doesn’t know how to process Cygwin style path names. You need to remove the Windows version (or make sure is not in the PATH) and install the Cygwin version of Python using setup.exe.

# Update PATH before running ./configure for node.js
export PATH=/usr/bin:$PATH

Unable to Remap to Same Address as Parent

fatal error – unable to remap \?\C:\cygwin\lib\python2.6\lib-dynload\time.dll to same address as parent: 0×360000 != 0×3E0000

This is not an issue with node.js either. Install base → rebase using setup.exe first then close all Cygwin instances. Start dash or ash(located in the bin directory under Cygwin’s installation) and run:

$ /bin/rebaseall -v

It should finish with no errors. If instead the above results in an error like:

rebaseall:'/cygdrive/c/Users/ADMINI~1/AppData/Local/Temp' is not writable

Open up a Cygwin shell and run:

$ chmod 777 ~/AppData/Local/Temp

Close your shell window and repeat the steps above.

Once you are done, restart your PC. Remember to close all open Cygwin shells before using rebaseall.

If you still get similar messages during configure, try following the suggestions found here:http://code.google.com/p/chromium/wiki/CygwinDllRemappingFailure

../deps/v8/tools/jsmin.py SyntaxError: invalid syntax

Cygwin uses a different way of handling symlinks than a regular Unix system. If you cloned node.js using msysGit it’s likely you’ll end up here as well. Open up jsmin.py in an editor – you will see the actual path it needs to be symlinked to. To address this, start a new Cygwin shell andcd to the cloned node.js directory. Run:

# from the node.js cloned repository directory, run:
$ cd tools
$ ln -fs `cat jsmin.py`
$ cd ..

Re-run ./configure.

Exception: supported architectures are arm, ia32, x64 but NOT ‘x86’.

Cygwin is returning the wrong CPU architecture (usually uname from minGw gets in the way). Use the dest-cpu flag with the value of ia32:

$ ./configure --dest-cpu=ia32

Build failed: -> task failed (err #2): {task: libv8.a SConstruct -> libv8.a}

This comes about when $SHELL has a windows based path instead of a unix based one. E.g., C:\bin\bash instead of of /bin/bash. So, try,

$ export SHELL=/bin/bash

and re-run make.

You may also experience this problem when trying to build v0.4.0, try updating your version of gcc (v8 in v0.4.0 requires gcc-4.x to compile).

Build failed: -> src/eio/eio.c:1014: error: storage size ofbuf’ isn’t known

See https://github.com/joyent/node/issues/1483

cannot find config.h

You have to remove the old build files.

$ git clean -f -d -x

and re-run make.

cannot find a c compiler OR an error message that mentions rebase (cannot rebase)

You might be on Windows 7. Open up c:\cygwin\bin\ash.exe` then type

$ /bin/rebaseall

If the rebase fails (as on Win7 x64), you may need to edit the rebase script

Go to at line# 110 in /bin/rebaseall file and add: -e ‘/\/sys-root\/mingw\/bin/d’

Ex. in Cygwin 1.7.9 (current as of writing this)

  sed -e '/cygwin1\.dll$/d' -e '/cyglsa.*\.dll$/d' -e 's/^/\//' >"$TmpFile"


  sed -e '/\/sys-root\/mingw\/bin/d' -e '/cygwin1\.dll$/d' -e '/cyglsa.*\.dll$/d' -e 's/^/\//' >"$TmpFile"

Reference : https://github.com/joyent/node/wiki/Building-node.js-on-Cygwin-(Windows)

Ajax IM — Instant Messaging framework

What is Ajax IM?

Ajax IM (“Ajax Instant Messenger”) is a browser-centric instant messaging framework. It uses AJAX to create a real-time (or near real-time) IM environment that can be used in conjunction with existing community and commercial software, or simply as a stand-alone product.

Ajax IM has been truly rebuilt from the bot­tom up, imple­ment­ing the lat­est libraries, tech­niques, and shini­ness includ­ing jQuery, local ses­sion stor­age (using HTML5 or Flash), Node.js, and much more.


Let’s get you set up, shall we?

By default, there are two choices for installation: thePHP-based or standalone servers. While thestandalone server is recommended for scaling and extensibility, it requires you to be on a VPS or dedicated server. On the other hand, the PHP-basedserver does not have this requirement, though it is recommended that you use a server other than Apache in this case

Step 1 — Extract

After downloading the latest version, extract the archive into its own folder. Upload all included files to your server.

Step 2 — Configure and Install

Change the permissions for config.php andjs/im.js to 755 (writable). Additionally, if you plan on using a library that uses the standalone Node.js server (NodeJS or NodeJS Guests), set the permissions for node/config.js to 755 as well. The install script will need to alter these files during installation (unless you handle installation manually).

In your browser, navigate to install.php in the folder where you uploaded Ajax IM (e.g.,http://yoursite.com/ajaxim/install.php). The installer will guide you through the steps and configuration options for setting up Ajax IM, the database (if necessary), and the standalone server (if necessary).

Note: After completing installation, deleteinstall.php.

config.php     →   0755 (rwxrw-rw--)
 ↳ im.js       →   0755 (rwxrw-rw--)
 ↳ config.js   →   0755 (rwxrw-rw--)

Step 3 — Implement

To use Ajax IM on your website, you’ll only need to manually include the im.load.js script in your pages. The script will automatically test for and include any of the dependencies and the main Ajax IM library.

If you’re using the included MySQL database engine and the Default or NodeJS server libraries, you should read either the Quick Start with PHP or Quick Start with Node.js guides.

<script type="text/javascript"

Step 4 — Finished

That’s it! There really isn’t anything more to the installation.

For more information about integrating Ajax IM with your database, take a look at the documentation; also be sure to check out the add-ons (coming soon) page for alternative server scripts, database engines, and modifications.

Reference : http://ajaxim.com/installation/



WebMatrix and node.js: The easiest way to get started with node on Windows

Tomasz Janczuk and Steve Sanderson are having entirely too much fun. I posted just two weeks ago on Installing and Running node.js applications within IIS on Windows – Are you mad? when Tomasz and the team got node.js running rather nicely under IIS. Now they’ve got a nice little development environment in WebMatrix.


Continue reading

Installing and Running node.js applications within IIS on Windows ?

First, what’s node.js?

If you’re not familiar with node.js, it’s a new web programming toolkit that everyone’s talking about. It’s the one that makes you feel not hip if you don’t know what it is. Like Ruby on Rails was a few years back. Folks called it “Node” and it’s basically server-side JavaScript. The idea is that if you are doing a bunch of JavaScript on the client and you do JavaScript all day, why not do some JavaScript on the server also. One less thing to learn, I suppose.

If you are an ASP.NET programmer, you can think of node.js as being like an IHttpHandler written in JavaScript. For now, it’s pretty low-level. It’s NOT an HttpHandler, but I’m using an analogy here, OK? Here’s a lovely article by Brett McLaughlin that goes into more detail about Node.js and what it is. His subtitle is “Node isn’t always the solution, but it does solve some important problems” and that’s just exactly it.


Continue reading