Coding With SpiderMonkey
Part V - File and Directory Management
In the last article I touched on a file object and allowed for basic file tests to see if something exists, and to create a file if it does not exist. In this article I will expand on this to allow reading/writing, and get other status information about files such as permissions, modification times, etc. I will also add some objects to deal with directories and allow you to create/delete directories, enumerate directory contents, and get directory information.
The first thing I must do is figure out how I want to design my new objects. I want to create something using a stream like design, as eventually I would like to add support for other streams such as sockets. It would be nice to have a common API between stream. What I have come up with is to have File and Directory objects which provide information access, and FileStream and DirectoryStream objects that handle reading and writing. The layout as I see it right now looks like this:
// File object
// Provides access to information about a file.
class File {
public:
//Get the file permissions
int getPermissions()
//Is the file readable.
bool isReadable()
//Is the file writable
bool isWritable()
//Is the file executable
bool isExecutable()
//Is the file a link
bool isLink()
//Is the file a pipe
bool isPipe()
//Is the file a socket
bool isSocket()
//Is this a directory
bool isDirectory()
//Get the last modification date
Date getLastModified()
//Get the last access time
Date getLastAccess()
//Get the creation time
Date getCreationTime()
//Get file size
int getSize()
//Truncate the file to a given length
void truncate(int size)
//Update last modification time to now
void touch()
}
// Stream object
// Abstract object that is intended to provide a base API for streams, be they
// a socket, a file, a directory, or whatever.
class Stream {
public:
//Read a line
string read()
//Read len bytes
string read(int len)
//Read a single character
string readChar()
//Seek to a stream position
void seek(int pos, int base_point)
//Get the number of bytes available for reading, if available.
int getBytesAvailable()
//Write data to stream
int write(string data)
//Write data to stream, only write len bytes
int write(string data, int len)
//Determine if the stream is open
bool isOpen()
//Determine if we are at the end of the stream
bool isEnd()
//Get the error code from the last operation.
int getLastError()
}
// FileStream object
// Provides access to a stream for reading and writing to files.
class FileStream extends Stream {
public:
}
// Directory object
// Allows creation/deletion of a directory. Information about a directory can
// be obtained from the file object.
class Directory extends File {
public:
//Get number of files and directories in the directory.
int getSize()
//Get number of files
int getTotalFiles()
//Get number of directories
int getTotalDirectories()
//Unsupported, throw exception.
void truncate(int size)
}
// DirectoryStream object
// Allows enumeration of directory contents.
class DirectoryStream extends FileStream {
public:
//Read a filename
string read()
//Read len number of filenames
string read(int len)
//Unsupported, throw exception.
string readChar()
//Unsupported, throw exception.
void seek(int pos, int base_point)
//Gets number of filenames available for reading.
int getBytesAvailable()
//Unsupported, throw exception.
int write(string data)
//Unsupported, throw exception.
int write(string data, int len)
}
I decided that directories are just a special type of file, and can be treated as one, with just a couple special modifications to the meaning of the methods. Also, a directory stream can be created where instead of reading bytes as you would from a file, you just read file names from the directory. For this article, I will be splitting these objects so that each object is defined in its own .c/.h file pair. The files will be compiled and linked together to create the final object.
File object
Just like before, I am not going to spend a lot of time talking details. Instead, I will just cover some interesting points discovered while developing the classes. My starting point for this project was to split all the objects into their own files. This proved to be rather easy to do. I simply copy and pasted most of the code into a separate file, and linked them at compile time. I declared in the header file the JS class structure as an extern variable in order to use it later from other files to create instances of my classes. I am not entirely sure if this is the correct process, but I think it should work, so it is the route I am taking.
Another possibility would be to declare a public function as a constructor which would be able to initialize my classes and return an object. This may be a better idea as I can ensure the class is constructed properly, and I don't have to have so many global extern variables. I may switch to this later if I decide too.
Once I had the classes separated out into their own files, I completed the File object by removing the previous static methods, and filling in the remaining methods. I had a little trouble on the getLastModified, getLastAccess, and getCreationTime methods as I did not know how to construct a JS Date object from within C. I spent a while looking around the SpiderMonkey source and found that the jsdate.c file has a js_NewDateObjectMsec function which I could use. Given that function, I simply included jsdate.h into my project, and made a call to that function using the time returned from the stat() call. Simple eh? Did not work quite as I expected. The JS Date object measures time in Milliseconds (hence the Msec) so I need to multiply the time returned by 1000. I did that, and still came up with the incorrect result because it would always overflow storage and I could not pass the results of the multiplication into the function. After about 10 minutes of looking at this problem, I realized it must be doing multiplication as an integer even though the destination was a double. I changed the constant 1000 to 1000.0, and suddenly it was working just fine. So, remember when constructing a Date object from the native C code in SpiderMonkey, include jsdate.h, call js_NewDateObjectMsec() and be sure to multiply the unix timestamps (measured in seconds) by 1000.0 (don't forget the .0!) to convert the time into milliseconds.
#include <jsdate.h>
//You must include the .0 on the constant 1000.0 so the calculation is done as a double, not as an integer.
#define SEC_TO_MSEC(sec) (sec*1000.0)
static JSBool File_getLastModified(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval){
struct stat fileInfo;
FileInformation *info = (FileInformation*)JS_GetPrivate(cx, obj);
if (stat(info->filename, &fileInfo) == 0){
JSObject *dateObj = js_NewDateObjectMsec(cx, SEC_TO_MSEC(fileInfo.st_mtime));
if (!dateObj){
jsval exception = printf_exception(cx, "Unable to construct date object to hold time.");
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
*rval = OBJECT_TO_JSVAL(dateObj);
return JS_TRUE;
}
else {
char *msg = staterrorstr(errno);
jsval exception = printf_exception(cx, "stat failed on file (%d) %s", errno, msg);
free(msg);
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
}
The above date problem is the only problem I had with creating the File object. Everything else is fairly standard C and just moving the data from C to javascript through a couple conversions. My next step is to build the Directory object by extending the File object. I'm not sure exactly how to do this, but I will try and figure it out, or else come up with a new solution. I will return to write up my success with this later. Until then, you may examine the code as it stands right now by looking at the files attached. I have included them as a zip file, and also provide individual links to them for your convenience. Please see the attachments section at the bottom of the page for links to the files.
Directory object
Alright, been a few hours. Stopped working on this for a while; I have finished now though. I did end up doing as I described above and taking the definitions of the classes out of the header files, and moved them into the source files as static variables. I provide a function to get the prototype object that is created when I register the class, and that is used when I want to create another classes based on an existing class. Have a look at the two functions for registering the Directory and File classes, and you can see how I did the job.
//In file.c
static JSObject *FilePrototype = NULL;
JSBool register_class_File(JSContext *cx){
JSObject *globalObj = JS_GetGlobalObject(cx);
/* Define the file object. */
FilePrototype = JS_InitClass(cx, globalObj, NULL, &js_File_class, File_ctor, 1, NULL, File_methods, NULL, NULL);
if (!FilePrototype){
return JS_FALSE;
}
return JS_TRUE;
}
JSObject *File_getPrototypeObject(){
return FilePrototype;
}
//In directory.c
JSBool register_class_Directory(JSContext *cx){
JSObject *globalObj = JS_GetGlobalObject(cx);
JSObject *proto=File_getPrototypeObject();
/* Define the file object. */
DirectoryPrototype = JS_InitClass(cx, globalObj, proto, &js_Directory_class, Directory_ctor, 1, NULL, Directory_methods, NULL, NULL);
if (!DirectoryPrototype){
return JS_FALSE;
}
return JS_TRUE;
}
That code creates the directory object with a prototype of the file object, so the Directory object essentially extends the File object, and all the File methods are available automatically. Now, there is one method we want to override, and two additions methods that we want to add to the directory object. This is accomplished by simply declaring them in the Directory_methods array.
//Only declare new or changed methods
static JSFunctionSpec Directory_methods[] = {
{"getSize", Directory_getSize, 0, 0, 0},
{"getTotalFiles", Directory_getTotalFiles, 0, 0 ,0},
{"getTotalDirectories", Directory_getTotalDirectories, 0, 0, 0},
{NULL}
};
From there, everything is the same. Just implement those methods as normal and everything will work out just fine. My next step will be to work on the FileStream object. I've decided that the Stream object talked about above would just be more of an interface than a base class. SpiderMonkey does not actually support interface's as far as I know, neither does Javascript. It is possible I believe to do something that looks like an interface though. I will work on that, however I will not have anything for a few days working. In the meantime, just as above, check out the attachments section to download the second revision of the code and go over it.
FileStream and Stream objects
I have just completed my work on the FileStream and Stream objects. These were a little more difficult to implement, and required more thought. I had to make a few adjustments as I went to allow for them to work properly. The Stream object was a little tricky to figure out as it needed to be more of an interface. It is actually more of an abstract class in the code right now I suppose one could say, since Javascript does not support interfaces in the OOP sense.
My original plan for the Stream object was to simply not give it a constructor, and just give it the methods. All the methods would simply throw an Unimplemented exception as none of them can be implemented on the abstract level. This did not work well for me, so I did end up giving the object a constructor, but the constructor function throws an Unimplemented exception as well so technically the object can never be constructed. This seems to work well enough for my purposes.
With the base class in place, I then started on FileStream, which would be an actual implementation of the Stream interface. While creating this object, the most difficult method was the read method due to the fact that I wanted to have read-by-line built in. If a person called the read() method without arguments it would simply read til the end of the line. This took me quite a while to get implemented in a good and hopefully bug-free manner. I have done some testing, and all looks good so far. The rest of the methods were fairly easy to implement. I did decide to move the isEnd and isOpen methods to the Stream definition file. Those two methods actually can be implemented on the abstract level as they are merely testing a couple flags in the StreamInformation structure.
//This function serves multiple purposes depending on the number of arguments passed in (argc)
static JSBool FileStream_read(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval){
FileStreamInformation *info=(FileStreamInformation*)JS_GetPrivate(cx, obj);
char *buffer=NULL;
char *bufferPtr = NULL;
char *bufferDst = NULL;
char *bufferReallocPtr = NULL;
int bufferFillLen=0;
int bufferSize=0;
JSString *returnValue=NULL;
JSBool foundEndOfLine=JS_FALSE;
if (argc == 0){
while (foundEndOfLine == JS_FALSE){
if (info->streamInfo.bufferLength == 0){
if (filestream_fill_buffer(info) == JS_FALSE){
JS_SetPendingException(cx, printf_exception(cx, "Unable to read from file."));
JS_free(cx, buffer);
return JS_FALSE;
}
else {
if (info->streamInfo.bufferLength == 0){
goto complete;
}
}
}
bufferPtr = info->streamInfo.buffer;
bufferSize = 0;
while (bufferSize < info->streamInfo.bufferLength && (*bufferPtr != '\n' && *bufferPtr != '\r')){
bufferPtr++;
bufferSize++;
}
if (bufferSize < info->streamInfo.bufferLength){
if (*bufferPtr == '\r' && *(bufferPtr+1) == '\n'){
bufferPtr++;
bufferSize++;
}
if (*bufferPtr == '\r' || *bufferPtr == '\n'){
foundEndOfLine = JS_TRUE;
bufferSize++;
}
}
bufferReallocPtr = JS_realloc(cx, buffer, sizeof(*buffer)*(bufferSize+bufferFillLen+1));
if (bufferReallocPtr == NULL){
JS_SetPendingException(cx, printf_exception(cx, "Unable to allocate memory."));
if (buffer != NULL){
JS_free(cx, buffer);
}
return JS_FALSE;
}
else {
buffer = bufferReallocPtr;
}
memcpy(buffer+bufferFillLen, info->streamInfo.buffer, bufferSize);
bufferFillLen += bufferSize;
buffer[bufferFillLen] = '\0';
if (bufferSize < info->streamInfo.bufferLength){
info->streamInfo.bufferLength = info->streamInfo.bufferLength - bufferSize;
bufferPtr = info->streamInfo.buffer+bufferSize;
bufferDst = info->streamInfo.buffer;
while (bufferDst < (info->streamInfo.buffer+info->streamInfo.bufferLength)){
*bufferDst = *bufferPtr;
bufferDst++;
bufferPtr++;
}
bufferReallocPtr = realloc(info->streamInfo.buffer, sizeof(*(info->streamInfo.buffer))*info->streamInfo.bufferLength);
if (bufferReallocPtr == NULL){
JS_SetPendingException(cx, printf_exception(cx, "Unable to allocate memory for read buffer."));
JS_free(cx, buffer);
return JS_FALSE;
}
else {
info->streamInfo.buffer = bufferReallocPtr;
}
}
else {
free(info->streamInfo.buffer);
info->streamInfo.buffer = NULL;
info->streamInfo.bufferLength = 0;
}
}
}
else {
if (JS_ValueToInt32(cx, argv[0], &bufferSize) == JS_TRUE){
buffer = JS_malloc(cx, bufferSize+1);
memset(buffer, 0, bufferSize+1);
while (bufferFillLen < bufferSize){
if (info->streamInfo.bufferLength == 0){
if (filestream_fill_buffer(info) == JS_FALSE){
JS_SetPendingException(cx, printf_exception(cx, "Unable to read from file."));
JS_free(cx, buffer);
return JS_FALSE;
}
else {
if (info->streamInfo.bufferLength == 0){
goto complete;
}
}
}
if (bufferFillLen+info->streamInfo.bufferLength > bufferSize){
int diff = bufferSize - bufferFillLen;
memcpy(buffer+bufferFillLen, info->streamInfo.buffer, diff);
bufferFillLen += diff;
info->streamInfo.bufferLength = info->streamInfo.bufferLength - diff;
bufferPtr = info->streamInfo.buffer+diff;
bufferDst = info->streamInfo.buffer;
while (bufferDst < (info->streamInfo.buffer+diff)){
*bufferDst = *bufferPtr;
bufferDst++;
bufferPtr++;
}
bufferReallocPtr = realloc(info->streamInfo.buffer, sizeof(*(info->streamInfo.buffer))*info->streamInfo.bufferLength);
if (bufferReallocPtr == NULL){
JS_SetPendingException(cx, printf_exception(cx, "Unable to allocate memory for read buffer."));
JS_free(cx, buffer);
return JS_FALSE;
}
else {
info->streamInfo.buffer = bufferReallocPtr;
}
}
else {
memcpy(buffer+bufferFillLen, info->streamInfo.buffer, info->streamInfo.bufferLength);
bufferFillLen += info->streamInfo.bufferLength;
free(info->streamInfo.buffer);
info->streamInfo.buffer = NULL;
info->streamInfo.bufferLength = 0;
}
}
}
else {
JS_SetPendingException(cx, printf_exception(cx, "Bad parameter passed to FileStream.read()"));
return JS_FALSE;
}
}
complete:
if (buffer != NULL){
returnValue = JS_NewString(cx, buffer, bufferFillLen);
*rval = STRING_TO_JSVAL(returnValue);
}
else {
buffer = JS_malloc(cx, 1);
buffer[0] = '\0';
returnValue = JS_NewString(cx, buffer, 0);
*rval = STRING_TO_JSVAL(returnValue);
}
return JS_TRUE;
}
With all those methods set, I realized I never included a way to actually open and close the stream for reading and writing. Thus, I had to add open() and close() methods to the stream. For now, I just have it open always in read/write mode with no options available. It may be worth looking into in the future, but this provides good flexibility with little effort required.
Lastly, I realized need to implement the seek() method, and proceeded to do that. The seek() method was easy to implement, but it's definition calls for the use of constants on the Stream object, specifically Stream.SEEK_SET, Stream.SEEK_CURRENT, Stream.SEEK_END (which just map directly to their C equivalents). This required setting up a JSPropertySpec array, and defining these objects on the Stream class using the static_ps argument to JS_InitClass. At first, I thought it would be enough to just provide the name and ID, then NULL the rest, but it turns out I was wrong!
The ID you set for a property does not equal its value. Which this makes sense as not all properties you will want to define. You have to define a property getter and setter method. This ID is then passed to the getter and setter method so that you can identify which property is being gotten or set in a switch statement (or some other way). So I set up this getter function for the properties, and left the setter null as I want them to be readonly properties. Much to my surprise, things still did not work.
After 10 minutes of debugging or so, I realized I need to use JSVAL_TO_INT() on the id that is passed into the getter before I could process the switch statement correctly, as without that I am getting values that don't match what I assigned to the properties. With this in place, everything works fine. I had hopped that these properties would be automatically inherited in the same manner that the functions are on the FileStream class, but that was not the case. I would have to re-define them there. Instead, I chose to leave them defined only in the Stream object. It works, still makes sense, and was the easy thing to do
static JSBool Stream_propertyGetter(JSContext *cx, JSObject *obj, jsval id, jsval *vp);
static JSPropertySpec Stream_static_properties[] = {
{"SEEK_SET", SEEK_SET, JSPROP_ENUMERATE|JSPROP_READONLY|JSPROP_PERMANENT, Stream_propertyGetter, NULL},
{"SEEK_CURRENT", SEEK_CUR, JSPROP_ENUMERATE|JSPROP_READONLY|JSPROP_PERMANENT, Stream_propertyGetter, NULL},
{"SEEK_END", SEEK_END, JSPROP_ENUMERATE|JSPROP_READONLY|JSPROP_PERMANENT, Stream_propertyGetter, NULL},
{NULL}
};
JSBool register_class_Stream(JSContext *cx){
JSObject *globalObj = JS_GetGlobalObject(cx);
/* Define the file object. */
StreamPrototype = JS_InitClass(cx, globalObj, NULL, &js_Stream_class, Stream_undefined, 0, NULL, Stream_methods, Stream_static_properties, NULL);
if (!StreamPrototype){
return JS_FALSE;
}
return JS_TRUE;
}
static JSBool Stream_propertyGetter(JSContext *cx, JSObject *obj, jsval id, jsval *vp){
int thevalues = SEEK_SET;
thevalues = SEEK_CUR;
thevalues = SEEK_END;
switch (JSVAL_TO_INT(id)){
case SEEK_SET:
*vp = INT_TO_JSVAL(SEEK_SET);
break;
case SEEK_CUR:
*vp = INT_TO_JSVAL(SEEK_CUR);
break;
case SEEK_END:
*vp = INT_TO_JSVAL(SEEK_END);
break;
default:
return JS_FALSE;
}
return JS_TRUE;
}
DirectoryStream
The directory stream was a pretty simple object to create. I changed my mind about it extending file stream, and instead chose to just have it extend from stream in the same way that FileStream does. The seek() method could probably be implemented, now that I have looked at the C functions available, but I chose to leave it unimplemented.
The only new part about this object was the read(int len) method. This method required returning an array of file names, which was accomplished using the JS_NewArrayObject() function. It is a pretty easy method to use. Basically you just build a C array of jsval structures and then pass that into JS_NewArrayObject(). You pass that to javascript as you would any object, using the OBJECT_TO_JSVAL() macro.
static JSBool DirectoryStream_read(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval){
DirectoryStreamInformation *info=(DirectoryStreamInformation*)JS_GetPrivate(cx, obj);
struct dirent *entry;
if (argc == 0){
//...
}
else {
int numberToRead = 0;
if (JS_ValueToInt32(cx, argv[0], &numberToRead) == JS_TRUE){
jsval *fileNameArray = NULL;
int totalNamesRead = 0;
errno=0;
while ((totalNamesRead < numberToRead) && (entry = readdir(info->handle)) != NULL){
JSString *fileName = NULL;
int len = strlen(entry->d_name);
char *copy = JS_malloc(cx, len+1);
jsval *reallocPtr = NULL;
if (copy == NULL){
JS_SetPendingException(cx, printf_exception(cx, "Unable to read directory entry."));
return JS_FALSE;
}
memset(copy, 0, len+1);
strncpy(copy, entry->d_name, len);
fileName = JS_NewString(cx, copy, len);
totalNamesRead++;
reallocPtr = JS_realloc(cx, fileNameArray, sizeof(*fileNameArray)*totalNamesRead);
if (reallocPtr == NULL){
JS_SetPendingException(cx, printf_exception(cx, "Unable to read directory entry."));
return JS_FALSE;
}
else {
fileNameArray = reallocPtr;
fileNameArray[totalNamesRead-1] = STRING_TO_JSVAL(fileName);
}
errno = 0;
}
if (totalNamesRead < numberToRead){
if (errno == 0){
if (totalNamesRead == 0){
*rval = BOOLEAN_TO_JSVAL(JS_FALSE);
info->streamInfo.eofFlag=JS_TRUE;
return JS_TRUE;
}
else {
*rval = OBJECT_TO_JSVAL(JS_NewArrayObject(cx, totalNamesRead, fileNameArray));
return JS_TRUE;
}
}
else {
char *msg = errnostr(errno);
jsval exception = printf_exception(cx, "Unable to read directory stream. (%d) %s", errno, msg);
free(msg);
JS_SetPendingException(cx, exception);
return JS_FALSE;
}
}
else {
*rval = OBJECT_TO_JSVAL(JS_NewArrayObject(cx, totalNamesRead, fileNameArray));
return JS_TRUE;
}
}
else {
JS_SetPendingException(cx, printf_exception(cx, "Unable to understand length parameter."));
return JS_FALSE;
}
}
}
That is all I have to say on this endeavor into SpiderMonkey. I may update this article with more details in the future if I get people asking me any questions. Until then, the files are below, and the main points have been hit. Have fun with your venture into SpiderMonkey. Email address is below if you have questions or comments.
Attachments
- r1.tar.bz2 -- An archive of the first revision of the code. This revision has a working File object, and the objects have been split into separate files for clarity.
- r2.tar.bz2 -- An archive of the second revision of the code. This revision has a working Directory and File object.
- r3.tar.bz2 -- An archive of the third revision of the code. This revision has a working Stream and FileStream object.
- r4.tar.bz2 -- An archive of the fourth revision of the code. This revision has a working DirectoryStream object.
