Coding With SpiderMonkey
Part VI - Sockets
Now that we have a decent, working, implementation of file objects, I say it's time to move on to a topic I like: Sockets. I don't know why, but I just like sockets. So, lets make some objects to deal with socket connections. We could probably make various types of socket objects, but for the purposes of this tutorial, and to not make my head hurt as much, we will just create one single socket object.
I am going to develop a socket classes based on what functionality I need in order to create a script which is able to open a socket to a web server, make a request for a page and download that page to disk with a random file name. I have all the disk IO stuff I need right now from the previous tutorials, now I need the socket IO.
While I develop this, I will think a little about convenience features and how to make using the socket object easier. For instance, making the socket constructor new Socket() create the most common type of socket (streaming TCP/IP socket) and making reading and writing to the socket easy.
Let's start with the constructor function, Socket(). It will accept three parameters, just like the C socket() call. The three parameters can either be all specified, or all ignored, however. If all parameters are left out, the constructor will default to streaming TCP/IP socket, the kind used in most internet applications. If only one or two parameters are specified, an exception will be thrown.
static JSBool Socket_ctor(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval){
SocketInformation *info=NULL;
if (argc < 1){
/*If no parameters, we assume the most common type of socket which is: */
/*socket(PF_INET, SOCK_STREAM, SOL_TCP) */
info = Socket_createInfo();
if (info == NULL){
char *msg = errnomsg(errno);
jsval exception = printf_exception(cx, "Cannot create socket descriptor: %s", msg);
free(msg);
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
info->socketDescriptor=socket(PF_INET, SOCK_STREAM, SOL_TCP);
if (info->socketDescriptor == INVALID_SOCKET){
char *msg = errnomsg(errno);
jsval exception = printf_exception(cx, "Cannot create socket descriptor: %s", msg);
free(msg);
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
JS_SetPrivate(cx, obj, info);
return JS_TRUE;
}
else if (argc == 3){
int domain, type, proto;
info = Socket_createInfo();
if (info == NULL){
char *msg = errnomsg(errno);
jsval exception = printf_exception(cx, "Cannot create socket descriptor: %s", msg);
free(msg);
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
domain=JSVAL_TO_INT(argv[0]);
type = JSVAL_TO_INT(argv[1]);
proto =JSVAL_TO_INT(argv[2]);
info->socketDescriptor=socket(domain, type, proto);
if (info->socketDescriptor == INVALID_SOCKET){
char *msg = errnomsg(errno);
jsval exception = printf_exception(cx, "Cannot create socket descriptor: %s", msg);
free(msg);
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
JS_SetPrivate(cx, obj, info);
return JS_TRUE;
}
else {
jsval exception = printf_exception(cx, "Incorrect parameters for Socket()");
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
}
Next, we need to create a destructor function that will close the socket if necessary. This is to make sure proper clean up is done when we exit the script, in case people forget to call the close method, which we will create a little later to allow explicit cleanup operations. This is accomplished using the finalize member of the JSClass data type. The function we define for the finalize operation is has a prototype of type JSFinalizeOp which is defined as:
void JSFinalizeOp(JSContext *cx, JSObject *obj);
In our destructor we will check to see if a SocketInfo structure is associated with the object, which really should always be true. Then we will check to see if a socket is connected and if memory has been allocated for the addr member and perform the appropriate cleanup operations.
static void Socket_dtor(JSContext *cx, JSObject *obj){
SocketInformation *info = JS_GetPrivate(cx, obj);
if (info != NULL){
if (info->addr != NULL){
free(info->addr);
}
if (info->socketDescriptor != INVALID_SOCKET){
close(info->socketDescriptor);
info->socketDescriptor=INVALID_SOCKET;
}
free(info);
}
}
Now that we have resource management in place, and a simple definition of a socket class, we need to define a few static properties before we can do much else. These properties are the equivalent of the constants used in C to define parameters, such as PF_INET, SOL_TCP, etc. We will define these as static properties of the Socket object, so they will be accessed as: Socket.PF_INET. To do this, define the properties using the JSPropertySpec array. This is an array of structures that define the property name, and ID value, and the getter and setter of property. In our case, these properties are read only so no setter function needs to be defined. The getter function will be called whenever the property is accessed in the script and needs to provide a mapping between the properties assigned ID value and the real value you want the script to have. The getter is defined on a per-property basis, so you can either use one function to resolve all properties, or use separate functions for each property, or do something in the middle of that.
#define JSSOCK_SOL_TCP 1
#define JSSOCK_PF_INET 2
#define JSSOCK_SOCK_STREAM 3
static JSPropertySpec Socket_static_properties[] = {
{"SOL_TCP", JSSOCK_SOL_TCP, JSPROP_ENUMERATE|JSPROP_READONLY|JSPROP_PERMANENT, Socket_getProperty, NULL},
{"PF_INET", JSSOCK_PF_INET, JSPROP_ENUMERATE|JSPROP_READONLY|JSPROP_PERMANENT, Socket_getProperty, NULL},
{"SOCK_STREAM", JSSOCK_SOCK_STREAM, JSPROP_ENUMERATE|JSPROP_READONLY|JSPROP_PERMANENT, Socket_getProperty, NULL},
{NULL}
};
static JSBool Socket_getProperty(JSContext *cx, JSObject *obj, jsval id, jsval *ret){
int idNum = JSVAL_TO_INT(id);
switch(idNum){
case JSSOCK_PF_INET:
*ret = INT_TO_JSVAL(PF_INET);
break;
case JSSOCK_SOCK_STREAM:
*ret = INT_TO_JSVAL(SOCK_STREAM);
break;
case JSSOCK_SOL_TCP:
*ret = INT_TO_JSVAL(SOL_TCP);
break;
default:
*ret = INT_TO_JSVAL(idNum);
}
return JS_TRUE;
}
To add the properties to the object, you specify the JSPropertySpec array we defined as either the ps or static_ps parameters of the JS_InitClass function. As I said before, I want these to be static properties, so I pass the array to the static_ps parameter.
With resource management now in place, and the proper constants defined we can start to make sockets useful. Let's start off with a connect method which will connect the socket to some resource. This is basically going to wrap the C connect call. Since technically the socket may not be a TCP/IP socket, we need to provide the ability to connect for other protocols. We will do this by having an object as the argument which specifies the connection details. We do not need to create anything special for the object, we will just use the standard javascript object format. Our connect function will simply look for specific properties on that object to get details from. Following the C standard, the following members of the object will be recognized:
family- The address family being used.details- Details of the connection, what is defined depends on the value offamilyas defined below.PF_INETport- The port number to connect to (whenfamily=AF_INET)address- The hostname or IP address to connect to (whenfamily=AF_INET)
First off, in our connect function we will grab the first parameter and convert it to a JSObject * value using JS_ValueToObject. Once we have the object, we can check the properties that were set on it using the JS_GetProperty. This will return the value of the property into the value parameter, or set it to be JSVAL_VOID if the property is not set. Remember that point: the function does return true if the property is not set, and it sets the value to JSVAL_VOID (undefined).
static JSBool Socket_connect(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval){
SocketInformation *info = JS_GetPrivate(cx, obj);
if (argc == 1){
/*They passed an object specifying how to connect.*/
JSObject *obj;
if (JS_ValueToObject(cx, argv[0], &obj) == JS_TRUE){
int family;
JSObject *details;
jsval propVal;
JS_GetProperty(cx, obj, "details", &propVal);
JS_ValueToObject(cx, propVal, &details);
JS_GetProperty(cx, obj, "family", &propVal);
if (propVal == JSVAL_VOID){
jsval exception = printf_exception(cx, "family property is required.");
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
family = JSVAL_TO_INT(propVal);
if (family == JSSOCK_PF_INET){
char *address;
int port;
struct hostent *hostent;
JS_GetProperty(cx, details, "address", &propVal);
address = JS_GetStringBytes(JS_ValueToString(cx, propVal));
JS_GetProperty(cx, details, "port", &propVal);
port = JSVAL_TO_INT(propVal);
/*We have our connection details. Now we need to resolve address if required */
hostent = gethostbyname(address);
if (hostent == NULL){
jsval exception = printf_exception(cx, "Cannot resolve host name '%s'", address);
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
else {
struct in_addr *addr = (struct in_addr*)hostent->h_addr_list[0];
struct sockaddr_in *conAddr = malloc(sizeof(*conAddr));
conAddr->sin_family = PF_INET;
conAddr->sin_port = htons(port);
conAddr->sin_addr = *addr;
info->addr = (struct sockaddr*)conAddr;
if (connect(info->socketDescriptor, info->addr, sizeof(*(info->addr))) == 0){
info->isConnected = 1;
return JS_TRUE;
}
else {
char *msg = errnomsg(errno);
jsval exception = printf_exception(cx, "Cannot connect to '%s:%d': %s", address, port, msg);
free(msg);
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
}
}
else {
jsval exception = printf_exception(cx, "Protocol family unsupported.");
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
}
else {
jsval exception = printf_exception(cx, "Expected object as first parameter");
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
}
else {
jsval exception = printf_exception(cx, "Unknown parameter");
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
}
Next up, we need to provide some read and write functionality for this socket. These functions are really pretty simple, and not much different from the file IO functions for reading and writing. I chose not to do any kind of buffering on the read data to keep things simple. I will take care of that in javascript when I make use of the socket class. No explanation is really necessary for these functions if you have been following the rest of my tutorials, so here is the code:
static JSBool Socket_read(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval){
SocketInformation *info = JS_GetPrivate(cx, obj);
if (info != NULL){
if (info->socketDescriptor != INVALID_SOCKET && info->isConnected == 1){
JSString *socketData;
char *buff = JS_malloc(cx, 1024);
int bytesRead = 0;
memset(buff, 0, 1024);
bytesRead = read(info->socketDescriptor, buff, 1024);
socketData = JS_NewString(cx, buff, bytesRead);
*rval = STRING_TO_JSVAL(socketData);
return JS_TRUE;
}
else {
jsval exception = printf_exception(cx, "Not Connected.");
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
}
else {
jsval exception = printf_exception(cx, "No socket.");
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
}
static JSBool Socket_write(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval){
SocketInformation *info = JS_GetPrivate(cx, obj);
if (info != NULL){
if (info->socketDescriptor != INVALID_SOCKET && info->isConnected == 1){
if (argc == 1){
JSString *dataToWrite;
char *data;
dataToWrite = JS_ValueToString(cx, argv[0]);
data = JS_GetStringBytes(dataToWrite);
write(info->socketDescriptor, data, strlen(data));
return JS_TRUE;
}
else {
jsval exception = printf_exception(cx, "Invalid parameters");
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
}
else {
jsval exception = printf_exception(cx, "Not Connected.");
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
}
else {
jsval exception = printf_exception(cx, "No socket.");
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
}
That's it. Our socket class is done. Let's test it out with a simple javascript which makes an HTTP request to my site and downloads the home page and dumps it to the console. Once we are sure it is working we can hop into creating an HTTP object in javascript that makes use of this class and the file classes to provide file downloads from websites.
//Lets test some sockets.
try {
var socket = new Socket();
Console.writeln("Socket object created successfully.");
//Check the constants
Console.writeln("SOL_TCP = " + Socket.SOL_TCP);
//Connect to my website
//socket.connect({});
socket.connect({family:Socket.PF_INET, details:{port:80,address:"aoeex.com"}});
//Make a request
socket.write("GET / HTTP/1.0\r\nHost: aoeex.com\r\n\r\n");
//Read the response
while ((data=socket.read()).length > 0){
//Read the banner
Console.writeln(data);
}
//Close up shop
socket.close();
}
catch (e){
Console.writeln ("\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~");
Console.writeln("Caught exception. ");
Console.writeln(e);
}
To get the entire source package to try this code out yourself, download the Revision 1 (r1) package below. Feel free to experiment with this as much as you want, or just move on with the rest of the tutorial where we work solely within the javascript to create an HTTP request object. This object will allow easy access to website files and provide function to either read into a variable, or download directly to disk.
Pulling it all together
So, now that we have file functions, we have directory functions, and we have socket functions lets make an app to download a given URL. This will be like a basic form of wget written in javascript. When executed it will prompt for a URL, and also a save path. A special save path of '-' will be understood to mean the console. If '-' is used, we will download the code and echo it out, otherwise we will download to disk using the specified path.
Since the console stuff we wrote before does not allow access to any command line arguments, we have to do prompts. In a future tutorial I may look at how to provide variables to a script such as to allow the usage of command line arguments.
function HTTPRequest(url){
this.urlParts = this.parseUrl(url);
this.socket = null;
this.buffer = '';
this.socketIsEOF=false;
}
HTTPRequest.prototype.parseUrl = function(url){
var parts = {};
//All I need is the domain and the path.
var pos = url.indexOf('://');
if (pos != -1){
url = url.substring(pos+3);
}
pos = url.indexOf('/');
if (pos != -1){
parts.domain = url.substring(0, pos);
parts.path = url.substring(pos);
}
else {
parts.domain = url;
parts.path = '/';
}
pos = parts.domain.indexOf(':');
if (pos != -1){
parts.port = parseInt(parts.domain.substring(pos+1));
parts.domain = parts.domain.substring(0, pos);
}
else {
parts.port = 80;
}
return parts;
};
HTTPRequest.prototype.open = function(method, postData){
this.socket = new Socket();
this.socket.connect({family:Socket.PF_INET, details:{address:this.urlParts.domain, port:this.urlParts.port}});
var headers = method+' '+this.urlParts.path+' HTTP/1.0';
headers += '\r\nHost: '+this.urlParts.domain;
headers += '\r\nConnection: close';
if (method.toUpperCase() == 'POST'){
headers += '\r\nContent-length: '+postData.length;
headers += '\r\nContent-type: application/x-www-formdata';
headers += '\r\n\r\n';
headers += postData;
}
else {
headers += '\r\n\r\n';
}
this.socket.write(headers);
};
HTTPRequest.prototype.downloadToFile = function(file){
var fileInfo = new File(file);
var doneHeaders = false;
if (!fileInfo.exists()){
fileInfo.create();
}
var file = new FileStream(file);
file.open();
while ((data=this.read()).length > 0){
if (!doneHeaders && data.trim() == ''){
doneHeaders = true;
}
else if (doneHeaders){
file.write(data);
}
}
file.close();
};
HTTPRequest.prototype.download = function(){
var data = '';
var buffer = '';
var doneHeaders = false;
while ((buffer = this.read()).length > 0){
if (!doneHeaders && buffer.trim() == ''){
doneHeaders = true;
}
else if (doneHeaders){
//Console.writeln('Read: '+buffer);
data += buffer;
}
}
return data;
};
HTTPRequest.prototype.read = function(){
if (this.socketIsEOF == false){
var buff = this.socket.read();
if (buff.length > 0){
this.buffer += buff;
}
else {
this.socketIsEOF = true;
}
}
var pos = this.buffer.indexOf('\r\n')+2;
if (pos == 1){
pos = this.buffer.indexOf('\r')+1;
if (pos == 0){
pos = this.buffer.indexOf('\n')+1;
if (pos == 0){
pos = this.buffer.length;
}
}
}
var data = this.buffer.substring(0, pos);
this.buffer = this.buffer.substring(pos);
return data;
};
HTTPRequest.prototype.close = function(){
this.socket.close();
};
String.prototype.trim = function(){
return this.replace(/^\s*/, '').replace(/\s*$/, '');
};
Console.writeln('Enter a URL to download: ');
var url = Console.read().trim();
Console.writeln('Enter a save path (- for stdout): ');
var file = Console.read().trim();
var request = new HTTPRequest(url);
request.open('GET');
if (file == '-'){
Console.writeln(request.download());
}
else {
request.downloadToFile(file);
}
request.close();
Attachments
- r1.tar.bz2 -- An archive of the first revision of the code. This revision has a working socket object.
- r2.tar.bz2 -- This revision has the same basic C code, but includes a wget.js file which is a program to download a URL
