so a few moments ago, @ZauberExonar asked on twitter if anyone was aware of a JSON parser in the Linden Scripting Language (LSL). people unfamiliar with LSL or Second Life(tm) may want to skip this blog posting. in typical "software architect" fashion, i'm answering his question with several paragraphs of "why JSON and LSL don't mix" (and what you can do about it.)
a very brief intro to JSON
JSON, as we all know, is a transfer syntax that encodes messages in a way that looks remarkably like a JavaScript / ECMAScript object declaration. So... if you execute this JavaScript:
var foo = { success: false, error: "insufficient cheese error: recommend rebooting universe" };console.log( JSON.stringify( foo ) );
you would get a JSON blob that looks something like this:
{"success":false,"error":"insufficient cheese error: recommend rebooting universe"}
so far, so good, right? you start with a JavaScript object and you convert it into a string that represents the object using a familiar syntax. it's now ready to send over the network. the receiver can execute this code to deserialize the string version of the message into a real live object:
var bar = JSON.parse( "{\"success\":false,\"error\":\"insufficient cheese error: recommend rebooting universe\"}" );
and now the receiver has an object it can manipulate.
the cool thing about JSON as a transfer syntax is messages are serialized using encoding rules that are way easy to process in JavaScript (the programming language of the web.) so it's easy to understand why web-devs love the heck out of JSON.
XML, in comparison, is seemingly bloated. XML allows you to define your own tags, each with semantics specific to the message. this can be a good thing in some situations, but you have to add the smarts to your JavaScript application to grab specific bits of data out of the XML and construct objects to contain the data.
enter the virtual world
but if we want to consume a JSON service in the virtual world, things get a little complicated. in Second Life, the programming language of choice is LSL (Linden Scripting Language.) a lot of people love to hate on LSL. i'm not going to do that here. but i will say this: parsing JSON or XML in LSL is a pain in the ass.
perhaps the most irritating aspect of LSL in this regard is it's lack of associative arrays. whether you call them maps, dictionaries, objects or associative arrays, they all let you use a string as a key into a collection of items.
because there's no associative array in LSL, you can't pull off the JavaScript trick of creating a new object that's essentially identical to the JSON. no, you have to map the contents of the JSON string into a list (which is like an array) or global variables.
JSON lovers aren't alone... XML partisans have discovered they're in the same boat: you have to parse the message and reason about the semantics of each field while constructing a list or setting global variables.
easy parsing?
and another aspect of JSON and XML parsing in LSL that's sub-optimal: you have to write LSL to look at each character in the message string.
okay, this is not precisely true. if your message does not include escaped quotes or curly braces, you can use llParseString2List() to build a list that's MUCH easier to parse. the down side of this approach is it's not entirely easy to handle curly braces inside strings and honestly, the code to implement it frequently looks bizarre.
so what i've started to do is to use a LSL-friendly format i'm calling the "DSD Text Transfer Syntax." experienced readers will recognize DSD as the abstract type system i routinely threatened the VWRAP group with. DSD text looks like this:
:v:1
appkey:u:77c3dd7c-4639-4681-aa52-a7dd35e96fd8
:{:
success:0:
error:s:insufficient%20cheese%20error%3A%20reboot%20universe
errno:i:23
desc:l:http%3A//example.com/error_descriptions/cheese.html
:}:
messages are a collection of individual lines (separated by either a CR or a CRLF.) each line contains three fields separated by colon characters. the first field is a "name" field; the second is a "type tag" while the third is the data in question. note that not all lines have a name, and not all lines have a data field. also note that colon characters are verboten in the data field, so we encode strings and URIs that may contain them.
people familiar with the LLSD binary encoding will likely recognize the tag characters, they're directly ripped off from that spec. (i also added a version tag so parsers will be able to know when they receive a message they may have problems interpreting.)
so the way you parse this in LSL is to split a message into lines like so:
// assume the string 'message' contains the complete message text
list lines = llParseString2List( message, ["\n", "\r", "\n\r"], [] );
integer lineCount = llGetListLength( lines );
integer i;
for( i = 0; i < lineCount; i++ ) {
string currentLine = llList2String( lines, i );
list fields = llParseString2List( currentLine, [":"], [] );
string name = llList2String( fields, 0 );
string key = llList2String( fields, 1 );
string data = llList2String( fields, 2 );
// now process the name-key-data triple
}
what you do when you process the name, key and data values depends on your app. i find i'm frequently either stuffing them into global variables or constructing new lists with the values. to parse the message above, we do something like this:
// parse location info message. generates a list with the following members:
// 0 - integer - success (1 for success, 0 for failure)
// 1 - integer - errno (0 for success)
// 2 - string - error description ("" if call was successful )
// 3 - string - URL for more information ("" if call was successful)
// 4 - string - region name
// 5 - float - x location in region
// 6 - float - y location in region
// 7 - string - comment text
list parseLocationInfoMessage( message ) {
integer success = TRUE;
integer errno = 0;
string error = "";
string desc = "";
integer in_map = FALSE;
integer version;
string region = "";
float x = 0.0;
float y = 0.0;
string comment = "";
list lines = llParseString2List( message, ["\n", "\r", "\n\r"], [] );
integer lineCount = llGetListLength( lines );
integer i;
for( i = 0; i < lineCount; i++ ) {
string currentLine = llList2String( lines, i );
list fields = llParseString2List( currentLine, [":"], [] );
string name = llList2String( fields, 0 );
string key = llList2String( fields, 1 );
string data = llList2String( fields, 2 );
// now process the name-key-data triple
if( 'v' == key ) {
version = (integer) data;
if( 1 != version ) {
success = FALSE;
errno = -1;
error = "invalid message version";
break;
}
} else if( '{' == key ) {
in_map = TRUE;
} else if( '}' == key ) {
in_map = FALSE;
} else {
if( FALSE == in_map ) {
success = FALSE;
errno = -2;
error = "parsing error";
break;
}
if( 'success' == name ) {
if( '1' == key ) {
success = TRUE;
} else {
success = FALSE;
}
} else if( 'errno' == name ) {
errno = (integer) data;
} else if( 'error' == name ) {
error = llUnescapeURL( data );
} else if( 'desc' == name ) {
desc = llUnescapeURL( data );
} else if( 'region' == name ) {
region = llUnescapeURL( data );
} else if( 'x' == name ) {
x = (integer) data;
} else if( 'y' == name ) {
y = (integer) data;
} else if( 'comment' == name ) {
comment = llUnescapeURL( data );
}
}
}
return( [ success, errno, error, desc, region, x, y, comment ] );
}
some might argue this is no less complex than XML or JSON parsing in LSL, but for my money, it seems a little more straight-forward.
making services give you DSD Text
ideally you're in a position to control both the server and the client. if you are, then your server code gets to decide what type of encoding you send back to the client.
slatebureau.com uses the same API endpoints for requests, irrespective of where they come from. to determine what encoding to use, it looks for the Accept:, Content-Type: and S-SecondLife-Shard: headers in the request and uses this algorithm.
- if an Accept: header is present, and you can generate the mime type the client is requesting, use it. if you can't generate the encoding requested, send a 406 status code.
in other words, if you ask for a 'application/json' or 'application/dsd+json', i'm going to give you JSON. if you ask for 'text/plain' or 'application/dsd+text', i'm going to give you the text format described here.
- if there's no Accept: header but there is a Content-Type: header present in the request, send the response in the format of the content type. if you can't generate that encoding, don't freak out, continue to step 3.
so if there was a Content-Type: header in the request and there wasn't an Accept: header, i'm going to try to encode the response using the same type. if your request included a json blob with an 'application/json' Content-Type, i'll try to generate that as a response.
- send a DSD-Text formatted response.
so... anyway... i wrote a few DSD parsers in PHP and JavaScript. i'll try to dig them out and post them to github. -cheers!
