Using the JSON Conversion Functions
===================================

The JSON Parser package provides three conversion functions, a JSON Parser, a JSON 
Generator and a JSON Validator. Additionally there are also a rich set of JSON
Support Functions to assist C developers is using JSON Data easily with the minimum
of coding overhead. Those are described in another document.

This document presents basic ideas on how to use the JSON Conversion Functions
provided in this JSON package, and a brief introduction on using the available
D-List functions, to access that JSON data after conversion.


Background
----------

This JSON Parser package was written to provide not only a fully functional, fast
and accurate JSON Parser for C programmers, but also to provide the JSON data in a
form that is more generally useful to C developers. Most JSON parsers write the
parsed JSON data into arrays etc, and there are numerous issues as C as a language
does not natively support unbounded or variable sized data. However the D-List 
package for C does natively provide such support with a very easy to use, and rich,
set of available functions to access, modify, add to, manipulate, manage and copy 
this variable data. The D-List package is also extremely fast & efficient and a 
great way to manipulate varied data from different sources.

All functions described in this document that start with json_xxx are part of the 
JSON Parser package, and details about there usage and arguments, can be found in
the JSON Parser documentation. All functions that start with list_xxx are part of
the D-List package.

JSON is a data format focused on inter-system and inter-language interoperability. 
If you are unfamiliar with JSON grammar or syntax, please refer to ECMA-404 and 
RFC 8259 or 
	
	http://json.org 

before trying to use the JSON Support Functions.

Throughout this JSON Parser package, all JSON Objects and JSON Arrays are accessed
and manipulated using D-List list_objects, and all JSON value items are elements
within those list_objects. The list_objects are arranged hierarchically allowing
these data structures to represent the JSON syntax exactly.

If you are unfamiliar with D-List, then please consult the reference documentation.
To find that documentation, or to learn more about list_objects, elements, and the
rich set of D-List functions; please refer to the D-List documentation, which is 
available in the D-List distribution Header Files, or a set of Doxy files that are
included in that distribution, or online at 

	http://info.fwsentry.org/dlist/index.html


JSON data representation produced by the JSON Parser
====================================================

Simply put, 
	a JSON Object is mapped to a subordinate D-List list_object
	a JSON Object Member name/value pair is an element within that list_object 
	a JSON Value is an element inside the parent list_object, which could be a
		JSON Object or JSON Array
	a JSON Array is a subordinate D-List list_object which has an element for
		each JSON Value within the JSON Array, inside the parent element which
		is enclosed in a JSON object or JSON Array
	
JSON value types -
	JSON_string is a pointer to the original string in the original JSON buffer,
		it is now nul terminated and compacted to remve escape sequences etc. 
		The pointer points to the first char after the original double quote 
		character, and the last character is the character before the ending
		double quote.
	JSON_array is a pointer to a new list_object which will contain the 
		values, one element per array value. If the array in the JSON code
		was a single entry then that single entry is a single element in
		the new subordinate Array list_object.
	JSON_object is a pointer to a new list_object which will contain the 
		Object Member name:value pairs, one list_object element per JSON Object 
		Member. If this JSON Object in the JSON data block only enclosed a single
		entry then that single entry is a single element in the new subordinate 
		list_object. There are no shortcuts, the hierarchy is preserved even for
		single elements.
		If the JSON Object was empty " { } " then the subordinate list will
		have zero elements, but there will still be a list_object describing it.
	All other JSON value types are converted and stored directly in the list
		elements, be they JSON Object Members (name/value), or a value only
		element as would be included in an Array, this includes all value types
		numbers, boolean, null, etc.
	
The D-List list_objects that now represent the JSON object, array and value 
structures, can be used just like any other D-List list_object. They can be sorted,
managed, iterated, and manipulated as needed. All data pointers within the lists
point to portions of the original JSON string buffer that was parsed.

DO NOT release the JSON buffer, until after the D-List list_objects that describe 
it have been erased. All D-List list_objects used for parsing JSON objects are built
with the LIST_COPY_DATA attribute, so all list storage is handled internally by the
JSON parsing code, and the deletions use D-List functions to properly free all the
allocated regions.

This representation of a JSON object eliminates all white space, and compresses
strings to remove escape characters, etc.

jCard representations are handled in the same way, a vCardArray would be an
array object, a new list_object, and each of the elements in that array would
be elements in the new list. If the element describes a subordinate JSON Array
or Object, there would be pointer in the element, to that list_object.

Important. For every element in every list_object, root or subordinate, that is
returned by the JSON Parser, you must always check the JSON_value_type in each
element BEFORE attempting to access the element value. Inaccurate handling of 
element values in your code will lead to code or data corruption. Please refer
to the typedef json_element_t in json-parser.h for more information. Fields that
are not used in an element will always be 0x00000000. (See Note 1).

Example of Parser Output
========================

Using this simple JSON data block, for example:
{	"country" : "JP", 
	"language" : "en", 
	"names" : [ "Hayao", "Totoro", "Ponyo", "Nausica", "Kiki" ], 
	"films" : { "Ponyo" : 2008,
				"Princess Mononoke" : 1997,
				"Tonari no Totoro" : 1988,
				"How Do You Live?" : "In Production"},
	"Ghibli" : [	{ "film" : "Spirited Away", "release" : 2001, "time" : 125},
					{ "film" : "Howl's Moving Castle", "release" : 2004, "time" : 119},
					{ "film" : "Grave of the Fireflies", "release" : 1988, "time" : 89},
					{ } ],
	"Studio" : "Kabushiki gaisha Sutajio Jiburi"
	"Formal" : "株式会社スタジオジブリ"
}

would be parsed into a set of D-List list_objects such as:
	
								value_type	object_name	: value
								----------	-----------	  -----
(p)-->	root_list->	element_1->	JSON_string	"country"	: "JP"

					element_2->	JSON_string	"language"	: "en"

					element_3->	JSON_array	"names"		: value.object --> array_list_a	 -->(a)

					element_4->	JSON_object	"films"		: value.object --> object_list_2 -->(b)

					element_5->	JSON_array	"Ghibli"	: value.object --> array_list_b	 -->(c)

					element_6->	JSON_string	"Studio"	: "Kabushiki gaisha Sutajio Jiburi"

					element_6->	JSON_string	"Formal"	: "株式会社スタジオジブリ"
				(end of root_list)

		 (a)-->	array_list_a->	element_1->	JSON_string	NULL : "Hayao"
								element_2->	JSON_string	NULL : "Totoro"
								element_3->	JSON_string	NULL : "Ponyo"
								element_4->	JSON_string	NULL : "Nausica"
								element_5->	JSON_string	NULL : "Kiki"
							(end of array_list_a)

		 (b)-->	object_list_2->	element_1->	JSON_integer	"Ponyo"				: 2008
								element_2->	JSON_integer	"Princess Mononoke"	: 1997
								element_3->	JSON_integer	"Tonari no Totoro"	: 1988
								element_4->	JSON_string		"How Do You Live?"	: "In Production"
							(end of object_list_2)

		 (c)-->	array_list_b->	element_1->	JSON_object	NULL : value.object --> object_list_3 -->(d)
								element_2->	JSON_object	NULL : value.object --> object_list_4 -->(e)
								element_3->	JSON_object	NULL : value.object --> object_list_5 -->(f)
								element_4->	JSON_object	NULL : value.object --> object_list_6 -->(g)
							(end of array_list_b)

		 (d)-->	object_list_3->	element_1->	JSON_string		"film"		: "Spirited Away"
								element_2->	JSON_integer	"release"	: 2001
								element_3->	JSON_integer	"time"		: 125
							(end of object_list_3)

		 (e)-->	object_list_4->	element_1->	JSON_string		"film"		: "Howl's Moving Castle"
								element_2->	JSON_integer	"release"	: 2004
								element_3->	JSON_integer	"time"		: 119
							(end of object_list_4)

		 (f)-->	object_list_5->	element_1->	JSON_string		"film"		: "Grave of the Fireflies"
								element_2->	JSON_integer	"release"	: 1988
								element_3->	JSON_integer	"time"		: 89
							(end of object_list_5)

		 (g)-->	object_list_6->	<no elements>
							(end of object_list_6)

For each element encountered in any list_object, the value_type of that
element will tell you what type of element it is, and from there you know
that it is either a value, in which case the value is there in the element,
or it is a JSON Array or JSON Object and the pointer to the relevant
subordinate list_object is there in the element. 

So in the above example calling json_parse_data_block() with the above
JSON data block would return a single pointer (p) to the root_list, which
would have 6 elements in it, 3 of them with subordinate list pointers 
in them, and a the json_data_type of JSON_object, so you knew the root
list represented a JSON Object at the top level. In this case the 
subordinate lists would have 5, 4 and 4 elements in them respectively. 
(See Note 2).

These lists can then be retrieved, searched, sorted, and managed as you
would any other D-List list_object using the various D-List functions. It 
is easy to find out if a JSON array or JSON Object was empty by calling the
function list_size() or list_is_empty() on that subordinate list_object. Or
by using list iteration so any processing code is only used when there are
elements to retrieve. Please refer to D-List documentation for more 
information on how to use D-List list_objects and functions. However even
though D-List provides a broad range of options to access and manage data
elements, it is at heart very easy to use the core set of D-List functions.

So with the above JSON sample data as input, this code snippet can find
a particular film year, using the JSON Parser default search functions

	list_object *root_list;
	json_member_list_t results;
	json_element_t *element_ptr, *film_element_ptr;
	unsigned int json_data_type;
	char *the_json_data;
	
	the_json_data = read_the_json_data(filename);

	root_list  = json_parse_data_block(the_json_data, &json_data_type, NULL, NULL, NULL, NULL, FALSE);
	if (json_data_type == JSON_object) element_ptr = list_simple_search(root_list, "films");
	if ((element_ptr) && (element_ptr->value_type == JSON_object)) {
		film_element_ptr = list_simple_search(element_ptr->value.object, "Ponyo");
		if ((film_element_ptr) && (film_element_ptr->value_type == JSON_string)){
			fprintf(stdout, "The Movie %s was released in %Ld.\n", 
					film_element_ptr->object_name, 
					film_element_ptr->value.integer);
		} else {
			fprintf(stdout, "Movie Ponyo not found in list\n");
		}
	} else {
		/* do something if the JSON object element name "films" is missing. */
	}
		
Another option to retrieve the specific film element is to use the JSON
Support Functions provided in the Package. So for example:

	list_object *root_list;
	json_member_list_t results;
	unsigned int json_data_type;
	int ret;
	
	root_list  = json_parse_data_block(the_json_data, &json_data_type, NULL, NULL, NULL, NULL, FALSE);
	if (json_data_type == JSON_object) {
		ret = json_find_member_value(&results, root_list, "films", FALSE, "Ponyo", FALSE);
		if ((ret > 0) && (results.value_type == JSON_integer)){
			fprintf(stdout, "The Movie %s was released in %Ld.\n", 
					results.json_member->object_name, 
					results.json_member->value.integer);
		}
	} else {
		fprintf(stdout, "Movie Ponyo not found in list\n");
	}
	
If you know the exact layout of a given JSON Data block it will be more 
efficient for you to write code to navigate it directly, as in the first
example above. However if you are unaware of its layout, it is easier to
use the built JSON Support Functions which can take the hassle out of 
finding arbitrary data in unknown or arbitrary locations within that 
JSON data. (See Note 3)
	
The utility json-parser-test which is part of this JSON Parser package
has a function to output a parsed set of list_objects of a JSON data 
block to stdout, this is a good way to see how the JSON data block is 
formatted into D-List list_objects and elements.

For more information about the provided JSON Support Functions please refer
to the documentation regarding them.

You must become familiar with D-List to properly access the data returned
by the JSON Parser. D-List is at heart a very simple library to use, and
although it has a huge range of element manipulation options, you only 
need to become familiar with a few core functions to make good use of the
JSON data block. These would include the list iteration services,
	
	list_iteration_start()			to scroll through the elements of a list_object
	list_iteration_has_more()			...
	list_iteration_get_next()			...
	list_iteration_stop()				...
	list_get_index()				to get a specific element in a list_object, perhaps the first
	list_simple_search()			to locate a specific element without scrolling through them
	list_fast_sort()				to sort a given list, in JSON by member name or by value
	list_erase()					to remove the list when finished with it
	list_size()						to know how many elements it has
	list_is_empty()						...
	list_append()					if you wish to add things to a list_object
	list_create()					if you wish to add an Array or a JSON Object to a list_object

there are many other functions, but in general this is a good set to learn first
to be able to make good use of the JSON data returned by the JSON Parser. Examples
of using all these D-List functions on JSON data blocks are available in the
regression test code, which is provided as part of this JSON Package. The 
implementation of the majority of JSON Support Functions also makes extensive use
of these and other D-List functions.

JSON Path Representation
========================

Aside from storing, accessing and manipulating JSON data, this package also 
has functions to easily use JSON Dotted Notation Paths with that data, and
passes lists of found elements that match variable search criteria.

Internally both these paths, and search result lists. are represented by 
a struct called json_member_list_t, which also includes variable arrays of path 
segment data that are stored and transfered between callers. Lists of these
structs are passed back to callers of the search functions, and those users 
must use the basic D-List primitives to access those results, and/or path
information. (See Note 4)

These functions are designed, so results or paths output by one function, can 
be input to another without user changes, so they can be used in an opaque 
manner. However they also contain useful information for developers, especially
the parent list_object of a given search result, which can then be used to 
manage the contents of that Object or Array.

The example code in the tutorial mentioned above and in json-parser-test.c will
serve as useful primers on how to access and use this internal data to
efficiently navigate and modify JSON data.

The variable path segments are represented by the struct json_path_t stored 
inside the Members List struct. (See Note 5).



NOTES
=====

(1)		The typedef for the JSON Data Element layout is
			
		typedef struct {
			enum JSON_value_types value_type;			/* the value type of this name/value pair (from JSON_value_types) */
			char *member_name;							/* Member name for JSON_object Member elements, NULL for all other value types */
			union {
				list_object *object;					/* ptr to list_object for a JSON_object, or JSON_array, or NULL */
				char *string;							/* For JSON_string or JSON_float_str (if requested) */
				json_integer_t integer;					/* For integer (JSON_integer) */
				json_double_t double_number;				/* For non integer (JSON_double) user must deal with the conversion issues */
				boolean boolean_value;					/* For JSON_boolean either TRUE or FALSE */
			} value;
			boolean free_string_ptr;					/* TRUE, remove function will free() the string and member_name pointer, FALSE it will not */
		} json_element_t;

		The only valid JSON Value Types in elements are:
		
			JSON_object, JSON_array, JSON_null, JSON_boolean, JSON_integer,
			JSON_double, JSON_float_str, and JSON_string
			
		The only valid JSON Value Types provided by the JSON Parser or accepted
		by the JSON Generator are:

			JSON_object, JSON_array, and JSON_value
			
		In this context JSON_value just means all other possible value types
		except JSON_object and JSON_Array. Outside of this context JSON_value
		is not, and should never be, used at all.

(2)		No, i am not an anime fan, just trying to make the dry documentation a 
		little more entertaining.
			
(3)		This example uses the function json_find_member_value() which will return
		a single instance of the data being searched for. However the function
		does return a count of the number of searches matches it actually found.
		As an alternative method you could call the broader function
		json_find_members_value_string() which will return a list of all the
		matches found based on the same type of (Member Name : String Value)
		evaluation of the entire JSON data block.

(4)		The typedef for the Member List struct is

		typedef struct {
			list_object *json_owner_object;			/* parent list_object of this JSON Member element, use to locate element in D-List */
			element_reference *reference;			/* D-List element reference of this JSON Member element, use to manipulate element in D-List */
			json_element_t *json_member;			/* Pointer to the actual JSON Member Element, use to access the Member data */
			JSON_value_types value_type;			/* the Value type of the JSON Member value */
			unsigned int path_segments;				/* the number of elements in the path array (e.g. 4 in the example below)  */
			json_path_t path[];						/* the JSON path of the Member as recorded when found (e.g. bakery.product.[19]."curry pan") */
		} json_member_list_t;

(5)		The typedef for the JSON Path segments is

		typedef struct {
			JSON_value_types value_type;				/* the type of the link link in the JSON Object path, JSON_object or JSON_array */
			union {
				unsigned int array_index;
				char *member_name;						/* AKA keys in javascript land etc */
			} link;
		} json_path_t;
