<-- Previous Next -->

Coding with Spidermonkey Part VI - Sockets

Now that we have a decent, working, implementation of file objectes, I say it's time to move on to a topic I like: Sockets. I dunno why, but I just like sockets. So, lets make some objectes to deal with socket connections. We could probably make various types of socket objectes, but for the purposes of this tutorial, and to not make my head hurt as much, we will just create one single socket object.

I am going to develop a socket classes based on what fuctionallity I need in order to create a script which is able to open a socket to a webserver, make a request for a page and download that page to disk with a random file name. I have all the disk IO stuff I need right now from the previous tutorials, now I need the socket IO.

While I develop this, I will think a little about convinence features and how to make using the socket object easier. For instance, making the socket constructor new Socket() create the most common type of socket (streaming TCP/IP socket) and making reading and writing to the socket easy.

Lets start with the constructor function, Socket(). It will accept three parameters, just like the C socket() call. The three parameters can either be all specified, or all ignored however. If all parameters are left out, the constructor will default to streaming TCP/IP socket, the kind used in most internet applications. If only one or two parameters are specified, an exception will be thrown.

Socket constructor function definition


static JSBool Socket_ctor(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval){
    SocketInformation *info=NULL;
    if (argc < 1){
        /*If no parameters, we assume the most common type of socket which is: */
        /*socket(PF_INET, SOCK_STREAM, SOL_TCP) */
        info = Socket_createInfo();
        if (info == NULL){
            char *msg = errnomsg(errno);
            jsval exception = printf_exception(cx, "Cannot create socket descriptor: %s", msg);
            free(msg);
            JS_SetPendingException(cx, exception);
            return JS_FALSE;
        }

        info->socketDescriptor=socket(PF_INET, SOCK_STREAM, SOL_TCP);
        if (info->socketDescriptor == INVALID_SOCKET){
            char *msg = errnomsg(errno);
            jsval exception = printf_exception(cx, "Cannot create socket descriptor: %s", msg);
            free(msg);
            JS_SetPendingException(cx, exception);
            return JS_FALSE;
        }

        JS_SetPrivate(cx, obj, info);
        return JS_TRUE;
    }
    else if (argc == 3){
        int domain, type, proto;

        info = Socket_createInfo();
        if (info == NULL){
            char *msg = errnomsg(errno);
            jsval exception = printf_exception(cx, "Cannot create socket descriptor: %s", msg);
            free(msg);
            JS_SetPendingException(cx, exception);
            return JS_FALSE;
        }

        domain=JSVAL_TO_INT(argv[0]);
        type = JSVAL_TO_INT(argv[1]);
        proto =JSVAL_TO_INT(argv[2]);

        info->socketDescriptor=socket(domain, type, proto);
        if (info->socketDescriptor == INVALID_SOCKET){
            char *msg = errnomsg(errno);
            jsval exception = printf_exception(cx, "Cannot create socket descriptor: %s", msg);
            free(msg);
            JS_SetPendingException(cx, exception);
            return JS_FALSE;
        }
        JS_SetPrivate(cx, obj, info);

        return JS_TRUE;
    }
    else {
        jsval exception = printf_exception(cx, "Incorrect parameters for Socket()");
        JS_SetPendingException(cx, exception);
        return JS_FALSE;
    }
}

Next, we need to create a destructor function that will close the socket if necessary. This is to make sure proper clean up is done when we exit the script, incase people forget to call the close method, which we will create a little later to allow explicit cleanup operations. This is accomplished using the finalize member of the JSClass datatype. The function we define for the finalize operation is has a prototype of type JSFinalizeOp which is defined as:

JSFinalizeOp


void JSFinalizeOp(JSContext *cx, 
JSObject *obj);

In our desstructor we will check to see if a SocketInfo structure is associated with the object, which really should always be true. Then we will check to see if a socket is connected and if memory has been allocated for the addr member and perform the aproiate cleanup operations.

Destructor functions


static void Socket_dtor(JSContext *cx, JSObject *obj){
    SocketInformation *info = JS_GetPrivate(cx, obj);
    if (info != NULL){
		if (info->addr != NULL){
			free(info->addr);
		}
		if (info->socketDescriptor != INVALID_SOCKET){
			close(info->socketDescriptor);
	        info->socketDescriptor=INVALID_SOCKET;
		}
		free(info);
    }
}

Now that we have resource management in place, and a simple definition of a socket class, we need to define a few static properties before we can do much else. These properties are the equivlent of the constants used in C to define parameters, such as PF_INET, SOL_TCP, etc. We will define these as static properties of the Socket object so they will be accessed as: Socket.PF_INET. To do this define the properties using the JSPropertySpec array. This is an array of structures that define the property name, and ID value, and the getter and setter of property. In our case, these properties are read only so no setter function needs to be defined. The getter function will be called whenever the property is accessed in the script and needs to provide a mapping between the properties assigned ID value and the real value you want the script to have. The getter is defined on a per-property basis so you can either use one function to resolve all properties, or use separate functions for each property, or do something in the middle of that.

Property code


#define JSSOCK_SOL_TCP 1
#define JSSOCK_PF_INET 2
#define JSSOCK_SOCK_STREAM 3

static JSPropertySpec Socket_static_properties[] = {
    {"SOL_TCP", JSSOCK_SOL_TCP, JSPROP_ENUMERATE|JSPROP_READONLY|JSPROP_PERMANENT, Socket_getProperty, NULL},
    {"PF_INET", JSSOCK_PF_INET, JSPROP_ENUMERATE|JSPROP_READONLY|JSPROP_PERMANENT, Socket_getProperty, NULL},
    {"SOCK_STREAM", JSSOCK_SOCK_STREAM, JSPROP_ENUMERATE|JSPROP_READONLY|JSPROP_PERMANENT, Socket_getProperty, NULL},
    {NULL}
};

static JSBool Socket_getProperty(JSContext *cx, JSObject *obj, jsval id, jsval *ret){
    int idNum = JSVAL_TO_INT(id);
    switch(idNum){
		case JSSOCK_PF_INET:
			*ret = INT_TO_JSVAL(PF_INET);
			break;
		case JSSOCK_SOCK_STREAM:
			*ret = INT_TO_JSVAL(SOCK_STREAM);
			break;
		case JSSOCK_SOL_TCP:
			*ret = INT_TO_JSVAL(SOL_TCP);
			break;
        default:
            *ret = INT_TO_JSVAL(idNum);
    }
    return JS_TRUE;
}

To add the properties to the object, you specify the JSPropertySpec array we defined as either the ps or static_ps parameters of the JS_InitClass function. As I said before, I want these to be static properties so I pass the array to the static_ps parameter.

With resource management now in place, and the proper constants defined we can start to make sockets useful. Lets start off with a connect method which will connect the socket to some resource. This is basically going to wrap the C connect call. Since technically the socket may not be a TCP/IP socket, we need to provide the ability to connect for other protocols. We will do this by having an object as the argument which specifies the connection details. We do not need to create anything special for the object, we will just use the standard javascript object format. Our connect function will simply look for specific properties on that object to get details from. Following the C standard, the following members of the object will be recgonized:

family - The address family being used.
details - Details of the connection, what is defined depends on the value of family as defined below.
- PF_INET
  - port - The port number to connect to (when family=AF_INET)
  - address - The hostname or IP address to connect to (when family=AF_INET)

First off, in our connect function we will grab the first parameter and convert it to a JSObject * value using JS_ValueToObject(JSContext *cx, jsval value, JSObject * obj). Once we have the object, we can check the properties that were set on it using the JS_GetProperty( JSContext * cx, JSObject * obj, const char * property, jsval *value). This will return the value of the property into the value parameter, or set it to be JSVAL_VOID if the property is not set. Remember that point: the function does return true if the property is not set, and it sets the value to JSVAL_VOID (undefined).

Socket_connect


static JSBool Socket_connect(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval){
	SocketInformation *info = JS_GetPrivate(cx, obj);
	if (argc == 1){
		/*They passed an object specifying how to connect.*/
		JSObject *obj;
		if (JS_ValueToObject(cx, argv[0], &obj) == JS_TRUE){
			int family;
			JSObject *details;
			jsval propVal;


			JS_GetProperty(cx, obj, "details", &propVal);
			JS_ValueToObject(cx, propVal, &details);

			JS_GetProperty(cx, obj, "family", &propVal);
            if (propVal == JSVAL_VOID){
					jsval exception = printf_exception(cx, "family property is required.");
					JS_SetPendingException(cx, exception);
					return JS_FALSE;
            }
			family = JSVAL_TO_INT(propVal);

			if (family == JSSOCK_PF_INET){
				char *address;
				int port;
				struct hostent *hostent;

				JS_GetProperty(cx, details, "address", &propVal);
				address = JS_GetStringBytes(JS_ValueToString(cx, propVal));

				JS_GetProperty(cx, details, "port", &propVal);
				port = JSVAL_TO_INT(propVal);

				
				/*We have our connection details.  Now we need to resolve address if required */
				hostent = gethostbyname(address);
				if (hostent == NULL){
					jsval exception = printf_exception(cx, "Cannot resolve host name '%s'", address);
					JS_SetPendingException(cx, exception);
					return JS_FALSE;
				}
				else {
					struct in_addr *addr = (struct in_addr*)hostent->h_addr_list[0];
					struct sockaddr_in *conAddr = malloc(sizeof(*conAddr));

					conAddr->sin_family = PF_INET;
					conAddr->sin_port = htons(port);
					conAddr->sin_addr = *addr;

					info->addr = (struct sockaddr*)conAddr;

					if (connect(info->socketDescriptor, info->addr, sizeof(*(info->addr))) == 0){
						info->isConnected = 1;
						return JS_TRUE;
					}
					else {
						char *msg = errnomsg(errno);
						jsval exception = printf_exception(cx, "Cannot connect to '%s:%d': %s", address, port, msg);
						free(msg);
						JS_SetPendingException(cx, exception);
						return JS_FALSE;
					}
				}
			}
            else {
                jsval exception = printf_exception(cx, "Protocol family unsupported.");
                JS_SetPendingException(cx, exception);
                return JS_FALSE;
            }
		}
		else {
			jsval exception = printf_exception(cx, "Expected object as first parameter");
			JS_SetPendingException(cx, exception);
			return JS_FALSE;
		}
	}
	else {
		jsval exception = printf_exception(cx, "Unknown parameter");
		JS_SetPendingException(cx, exception);
		return JS_FALSE;
	}
}

Next up, we need to provide some read and write fuctionality for this socket. These functions are really pretty simple, and not much different from the file IO functions for reading and writing. I chose not to do any kind of buffering on the read data to keep things simple. I will take care of that in javascript when I make use of the socket class. No explination is really necessary for these functions if you have been following the rest of my tutorials, so here is the code:

Socket_read and Socket_write



static JSBool Socket_read(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval){
	SocketInformation *info = JS_GetPrivate(cx, obj);
	if (info != NULL){
		if (info->socketDescriptor != INVALID_SOCKET && info->isConnected == 1){
			JSString *socketData;
			char *buff = JS_malloc(cx, 1024);
			int bytesRead = 0;
			memset(buff, 0, 1024);

			bytesRead = read(info->socketDescriptor, buff, 1024);
			
			socketData = JS_NewString(cx, buff, bytesRead);
			*rval = STRING_TO_JSVAL(socketData);
			return JS_TRUE;
		}
		else {
			jsval exception = printf_exception(cx, "Not Connected.");
			JS_SetPendingException(cx, exception);
			return JS_FALSE;
		}
	}
	else {
		jsval exception = printf_exception(cx, "No socket.");
		JS_SetPendingException(cx, exception);
		return JS_FALSE;
	}
}

static JSBool Socket_write(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval){
	SocketInformation *info = JS_GetPrivate(cx, obj);
	if (info != NULL){
		if (info->socketDescriptor != INVALID_SOCKET && info->isConnected == 1){
			if (argc == 1){
				JSString *dataToWrite;
				char *data;

				dataToWrite = JS_ValueToString(cx, argv[0]);
				data = JS_GetStringBytes(dataToWrite);

				write(info->socketDescriptor, data, strlen(data));
				return JS_TRUE;
			}
			else {
				jsval exception = printf_exception(cx, "Invalid parameters");
				JS_SetPendingException(cx, exception);
				return JS_FALSE;
			}
		}
		else {
			jsval exception = printf_exception(cx, "Not Connected.");
			JS_SetPendingException(cx, exception);
			return JS_FALSE;
		}
	}
	else {
		jsval exception = printf_exception(cx, "No socket.");
		JS_SetPendingException(cx, exception);
		return JS_FALSE;
	}
}

That's it. Our socket class is done. Lets test it out with a simple javascript which makes a HTTP request to my site and downloads the home page and dumps it to the console. Once we are sure it is working we can hop into creating a HTTP object in javascript that makes use of this class and the file classes to provide file downloads from websites.

test.js


//Lets test some sockets.
try {
	var socket = new Socket();
	Console.writeln("Socket object created successfully.");

	//Check the constants
	Console.writeln("SOL_TCP = " + Socket.SOL_TCP);

	//Connect to my website
	//socket.connect({});
	socket.connect({family:Socket.PF_INET, details:{port:80,address:"aoeex.com"}});

	//Make a request
	socket.write("GET / HTTP/1.0\r\nHost: aoeex.com\r\n\r\n");

	//Read the response
	while ((data=socket.read()).length > 0){
		//Read the banner
		Console.writeln(data);
	}

	//Close up shop
	socket.close();
}
catch (e){
	Console.writeln ("\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~");
	Console.writeln("Caught exception. ");
	Console.writeln(e);
}

To get the entire source package to try this code out yourself, download the Revision 1 (r1) package below. Feel free to experiment with this as much as you want, or just move on with the rest of the tutorial where we work solely within the javascript to create a HTTP request object. This object will allow easy access to website files and provide function to either read into a variable, or download directly to disk.

Pulling it all together

So, now that we have file functions, we have directory functions, and we have socket functions lets make an app to download a given URL. This will be like a basic form of wget written in javascript. When executed it will prompt for a URL, and also a save path. A special save path of '-' will be understood to mean the console. If '-' is used, we will download the code and echo it out, other wise we will download to disk using the specified path.

Since the console stuff we wrote before does not allow access to any command line arguments, we have to do prompts. In a future tutorial I may look at how to provide variables to a script such as to allow the useage of command line arguments.

wget.js



function HTTPRequest(url){
	this.urlParts = this.parseUrl(url);
	this.socket = null;
	this.buffer = '';
	this.socketIsEOF=false;
}

HTTPRequest.prototype.parseUrl = function(url){
	var parts = {};

	//All I need is the domain and the path.
	var pos = url.indexOf('://');
	if (pos != -1){
		url = url.substring(pos+3);
	}

	pos = url.indexOf('/');
	if (pos != -1){
		parts.domain = url.substring(0, pos);
		parts.path = url.substring(pos);
	}
	else {
		parts.domain = url;
		parts.path = '/';
	}


	pos = parts.domain.indexOf(':');
	if (pos != -1){
		parts.port = parseInt(parts.domain.substring(pos+1));
		parts.domain = parts.domain.substring(0, pos);
	}
	else {
		parts.port = 80;
	}
	return parts;
};

HTTPRequest.prototype.open = function(method, postData){
	this.socket = new Socket();
	this.socket.connect({family:Socket.PF_INET, details:{address:this.urlParts.domain, port:this.urlParts.port}});

	var headers = method+' '+this.urlParts.path+' HTTP/1.0';
	headers += '\r\nHost: '+this.urlParts.domain;
	headers += '\r\nConnection: close';

	if (method.toUpperCase() == 'POST'){
		headers += '\r\nContent-length: '+postData.length;
		headers += '\r\nContent-type: application/x-www-formdata';
		headers += '\r\n\r\n';
		headers += postData;
	}
	else {
		headers += '\r\n\r\n';
	}

	this.socket.write(headers);
};

HTTPRequest.prototype.downloadToFile = function(file){
	var fileInfo = new File(file);
	var doneHeaders = false;
	if (!fileInfo.exists())
	{
		fileInfo.create();
	}
	var file = new FileStream(file);
	
	file.open();
	while ((data=this.read()).length > 0){
		if (!doneHeaders && data.trim() == ''){
			doneHeaders = true;
		}
		else if (doneHeaders){
			file.write(data);
		}
	}

	file.close();
}

HTTPRequest.prototype.download = function(){
	var data = '';
	var buffer = '';
	var doneHeaders = false;

	while ((buffer = this.read()).length > 0){
		if (!doneHeaders && buffer.trim() == ''){
			doneHeaders = true;
		}
		else if (doneHeaders){
			//Console.writeln('Read: '+buffer);
			data += buffer;
		}
	}
	return data;
}

HTTPRequest.prototype.read = function(){
	if (this.socketIsEOF == false){
		var buff = this.socket.read();
		if (buff.length > 0){
			this.buffer += buff;
		}
		else {
			this.socketIsEOF = true;
		}
	}

	var pos = this.buffer.indexOf('\r\n')+2;
	if (pos == 1){
		pos = this.buffer.indexOf('\r')+1;
		if (pos == 0){
			pos = this.buffer.indexOf('\n')+1;
			if (pos == 0){
				pos = this.buffer.length;
			}
		}
	}

	var data = this.buffer.substring(0, pos);
	this.buffer = this.buffer.substring(pos);

	return data;

};

HTTPRequest.prototype.close = function(){
	this.socket.close();
};


String.prototype.trim = function(){
	return this.replace(/^\s*/, '').replace(/\s*$/, '');
}

Console.writeln('Enter a URL to download: ');
var url = Console.read().trim();

Console.writeln('Enter a save path (- for stdout): ');
var file = Console.read().trim();

var request = new HTTPRequest(url);
request.open('GET');

if (file == '-'){
	Console.writeln(request.download());
}
else {
	request.downloadToFile(file);
}

request.close();

<-- Previous Next -->

Keith M.
E-mail: keithm -- aoeex + com (you can figure that out, right?)
http://www.aoeex.com/

Attachments

r1.tar.bz2 (.gz) -- An archive of the first revision of the code. This revision has a working socket object.
- console.c
- console.h
- directory.c
- directory.h
- directorystream.c
- directorystream.h
- engine.c
- file.c
- file.h
- filestream.c
- filestream.h
- js_util_functions.c
- js_util_functions.h
- Makefile
- socket.c
- socket.h
- stream.c
- stream.h
- test.js
r2.tar.bz2 (.gz) -- This revision has the same basic C code, but includes a wget.js file wich is a program to download a URL
- console.c
- console.h
- directory.c
- directory.h
- directorystream.c
- directorystream.h
- engine.c
- file.c
- file.h
- filestream.c
- filestream.h
- js_util_functions.c
- js_util_functions.h
- Makefile
- socket.c
- socket.h
- stream.c
- stream.h
- test.js
- wget.js

Name: mk1

Email:

Comment: Hi, actually none of the vi parts of your tutorials actually work here. always looks like this: --------------- ./engine JS Error: out of memory File: (null):0 Failed to create global object. ------------- Adjusting the values passed to JS_NewRuntime and JS_NewContext it actually does something, but nothing usefull: --------- ./engine test.js JS Error: TypeError: Console.writeln is not a function File: test.js:26 ------- plattform here is linux, js --version JavaScript-C 1.7 pre-release 3 2007-04-01 dpkg -l | grep -i libmozjs ii libmozjs-dev 1.8.1.4-2ubuntu5 Development files for the Mozilla SpiderMonk ii libmozjs0d 1.8.1.4-2ubuntu5 The Mozilla SpiderMonkey JavaScript library ii libmozjs0d-dbg 1.8.1.4-2ubuntu5 Development files for the Mozilla SpiderMonk

Name: mk1

Email:

Comment: on debian/ubuntu, change the makefile too CFLAGS=-DMOZILLA_1_8_BRANCH -DXP_UNIX -I/usr/include/mozjs -g -Wall -pedantic -Wno-long-long debian/ubuntu mozjs is compiled with MOZILLA_1_8_BRANCH# which changes the size of JSFunctionSpec

Coding with Spidermonkey Part VI - Sockets

Pulling it all together

Attachments

Leave a comment