Using the JSON Conversion Functions
===================================

The JSON Parser package provides three conversion functions, a JSON Parser, a JSON 
Generator and a JSON Validator. Additionally there are also a rich set of JSON
Support Functions to assist C developers is using JSON Data easily with the minimum
of coding overhead. Those are described in another document.

This document presents basic ideas on how to use the JSON Conversion Functions
provided in this JSON package, and a brief introduction on using the available
D-List functions, to access that JSON data after conversion.


Background
----------

This JSON Parser package was written to provide not only a fully functional, fast
and accurate JSON Parser for C programmers, but also to provide the JSON data in a
form that is more generally useful to C developers. Most JSON parsers write the
parsed JSON data into arrays etc, and there are numerous issues as C as a language
does not natively support unbounded or variable sized data. However the D-List 
package for C does natively provide such support with a very easy to use, and rich,
set of available functions to access, modify, add to, manipulate, manage and copy 
this variable data. The D-List package is also extremely fast & efficient and a 
great way to manipulate varied data from different sources.

All functions described in this document that start with json_xxx are part of the 
JSON Parser package, and details about there usage and arguments, can be found in
the JSON Parser documentation. All functions that start with list_xxx are part of
the D-List package.

JSON is a data format focused on inter-system and inter-language interoperability. 
If you are unfamiliar with JSON grammar or syntax, please refer to ECMA-404 and 
RFC 8259 or 
	
	http://json.org 

before trying to use the JSON Support Functions.

Throughout this JSON Parser package, all JSON Objects and JSON Arrays are accessed
and manipulated using D-List list_objects, and all JSON value items are elements
within those list_objects. The list_objects are arranged hierarchically allowing
these data structures to represent the JSON syntax exactly.

If you are unfamiliar with D-List, then please consult the reference documentation.
To find that documentation, or to learn more about list_objects, elements, and the
rich set of D-List functions; please refer to the D-List documentation, which is 
available in the D-List distribution Header Files, or a set of Doxy files that are
included in that distribution, or online at 

	http://info.fwsentry.org/dlist/index.html


JSON Conversion Functions
=========================

This JSON package provides 3 JSON data conversion functions for callers to use. 

A JSON Validator, that will take as input a character array with a JSON data block
in it, and run a linear validation of the JSON data, returning a boolean value if 
it passes or fails validation. Optional error reporting is output to stderr.

A JSON Parser function takes the same character array JSON data block, and uses 
recursive parsing to translate it to D-List list_object structures for usage in a
C language code module. 

A JSON Generator function takes a hierarchical D-List list_object structure and
converts it into a character array as JSON textural data block, that strictly 
conforms to RFC 8259.

Both the JSON Parser and JSON Generator use recursive parsing to translate the 
input data. The JSON Validator uses, by design, a liner byte by byte approach to 
validate every single byte of an unknown JSON data block.

JSON Parser
-----------

The single most likely used JSON Function is the Parser.

Call the JSON Parser and return a hierarchical list_object containing the
parsed contents of the JSON data block requested. The original JSON data 
block will be modified.

The JSON data block can be parsed in 1 or 2 passes. If the JSON data 
originates from an unknown or unreliable source, it would be wise to pass 
it through the validator valid_json_data_block() first. Once identified as 
valid it can be parsed with this function. Alternatively, if the JSON data 
being parsed comes from a known good and reliable source (for example ARIN)
then sending it to the parser in one pass is probably the right choice.

This function parses the contents of the JSON data block while performing 
syntax and data validation, building a set of D-List list_objects and 
elements that contains all the JSON data with all the whitespace, syntax 
and non-data removed.

The original JSON data block provided by the caller is heavily modified as 
part of the parsing process. If you want to retain the original JSON data 
block intact, you must provide a copy of it to this function.

The goal of the function is to parse the JSON data block creating as little 
as possible in additional resources, and providing the data contents in a 
form of D-List list_objects that is natively variable and non-determinate. 
Rather than attempting to crow-bar variable data objects into some form of 
static C language array. This makes the parsing code much simpler to write
and maintain, while the results of the JSON data parsing activity are much 
easier to handle, find, or modify by the caller.

For example it is now very easy to rearrange the original JSON data 
elements, or sort them by arbitrary keys etc, or perform arbitrary searches
for data or keys.

Upon successful return, a pointer to a list_object is returned. It is very 
important to realize that the data values (such as strings) pointed to from
within the elements of these hierarchical list_objects, are still inside the
original JSON data block that was provided to this function. That original 
JSON data block was extensively modified to allow the strings it contains to
be nul terminated etc. and left in-situ. It is very important that the root
list_object returned to the caller is properly disposed first, BEFORE any 
attempt is made to release the original JSON data block. 

So for example:
		big_lump = (char *)malloc(big_lump_to_fit_json_data_block);
		{... read file or grab data from socket to fill big_lump ...}
		if (json_valid_data_block(big_lump, NULL, NULL, FALSE) {
			json_object_ptr = json_parse_data_block(big_lump, &return_value_type, NULL, NULL, NULL, NULL, FALSE);
			if (json_object_ptr) {
				while (still_working) {
					do something useful with the returned list_objects
						...
				}
				list_erase(json_object_ptr);
				if (json_object_ptr) free(json_object_ptr);
			}
		}
		if (big_lump) free(big_lump);

The root list_object returned is self cleaning, all data is managed 
internally, and the single list_erase of the root object, will cause all
the element records, arrays, and sub-level list_objects to be deallocated 
and removed. No special JSON Parser calls are required to manage the root
list_object, all data processing can be performed with the standard D-List
function calls. If you wish to perform sorts, searches, etc. the default
JSON search and compare functions have already been attached to both the 
root, and all the subordinate list_objects. The attached search and compare 
functions have already been tailored to either JSON Object or JSON Array 
type lists.

Additionally upon return, the output argument json_data_type is set to tell
the caller what the top level list_object retuned describes. In JSON the
root of a data block can only have one item. It can be a single value, or 
an array or an object. Rather than have a list_object that only has a 
single element, that itself is a list_object pointing to an array, etc; the 
root level list_object alway describes the top level JSON data block, so if
that is a JSON Array, then the elements of the root list_object will be the
elements of that array. If it is a single JSON value there will be a single
element describing that value.

You may optionally select a replacement search or compare function for the 
list_objects to be built during the parsing operation. If none are provided, 
then in-built default functions are used. You can select from any of the
other available built-in function, described in the header file, or you may
write your own to suit your specific data processing needs. If you chose to
replace the default search and compare functions after the lists are created,
the process to add them to every list_object in the data hierarchy would be
very tedious. You can always override a specific search or compare function 
for a specific list_object as needed, during later processing of the parsing
results.

Please do not use list_set_remove_function() to remove or replace the 
original remove_function set by this json_parse_data_block() function call.
It would lead to significant memory leaks, and potentially accessing 
deallocated heap storage. Other than the remove function, you are free to 
use or manipulate the returned list_objects as needed.

The JSON Parser normally converts all value tokens found in the JSON data
block. However the basic C language has limited ability to properly 
represent high precision exponential numbers. As this code was designed 
to be highly portable, only requiring a minimum of C99 to compile and run,
it uses the base forms of the language for numbers. For some users either
the limited precision of these forms, or their inability to hold huge
numbers will be an issue. So there is an option to instruct the parser to
instead provide these exponent numbers as a string value instead, and 
allow the user to use whatever methods they want to convert or use those
numbers. Integers and numbers with no exponent are always converted.
Integers to C type long integer, and fractional numbers to C type
long double. It is beyond the scope of this parser to accurately deal
with very large numbers with the precision used by some professions.

You can use the json_element_t typedef, to properly access the list element 
data. The values of JSON_value_types are used to decode what type of data
or value each list element contains.

	json_data_block	a nul terminated string containing the JSON data block
					to parse. The contents of this character buffer will 
					be extensively modified, and will be unusable by the
					caller upon return. However as noted above, do not 
					release or free this buffer, until after the returned
					list_object has been erased.
	json_data_type	Upon successful return, this argument is modified to
					reflect what the top level of the JSON data block
					represents, the 3 possible options are:
						JSON_object		the list_object returned is a 
										parsed JSON object
						JSON_array		the list_object returned is a 
										parsed JSON array
						JSON_value		the list_object returned contains 
										a single JSON value element, which
										is one of the type in the list 
										JSON_value_types
	object_search_function
					a D-List search function to add to each list_object that
					represents a JSON Object, that is created by the JSON 
					parsing activity. Or NULL if the default search function
					default_search_function_object_name() is to be added. 
	object_compare_function
					a D-List compare function to add to each list_object 
					represents a JSON Object, that is created by the JSON 
					parsing activity. Or NULL if the default compare function
					default_compare_function_object_names() is to be added.
	array_search_function
					a D-List search function to add to each list_object that
					represents a JSON Array, that is created by the JSON 
					parsing activity. Or NULL if the default search function
					default_search_function_array_values() is to be added. 
	array_compare_function
					a D-List compare function to add to each list_object 
					represents a JSON Array, that is created by the JSON 
					parsing activity. Or NULL if the default compare function
					default_compare_function_array_values() is to be added.
	keep_exponent	FALSE - Exponent numbers are converted to long double
					TRUE  - Exponent numbers are retained as text strings
					
	returns 		Upon successful parsing of the JSON data block, a pointer
					to a root list_object is returned, which is the top of
					the hierarchical JSON data block.
					If there was an error found in the JSON data block, a NULL
					pointer is returned.

list_object *json_parse_data_block(	char *json_data_block, 
									unsigned int *json_data_type,
									element_search object_search_function,
									element_compare object_compare_function,
									element_search array_search_function,
									element_compare array_compare_function,
									const boolean keep_exponent);


JSON Validator
--------------

Call the JSON syntax validator and return a boolean value based on either
successful or failed validation.

The JSON data block is parsed in a linear pass, validating each character 
of the provided JSON data block, all tokens, values, whitespace and syntax 
are checked.

Checks are made for validity of syntax, contents, and the relative
positioning of all tokens, structural, literal, object, array and value.

The contents of the character buffer will be remain unmodified. Validation 
only scans the JSON data block and makes no changes. If JSON errors are 
found a pointer to the invalid token can optionally be returned.

This JSON Validation will stop when it encounters a JSON syntax or value 
error, it will not perform any further validation of the data block. 

If the caller does not surpress warnings, information and the location of
the JSON error are written to stderr. If the caller suppressed warnings and
did not know why the JSON object was rejected by the validator, call this
function again without suppressing warnings to see the specific error
messages involved.
	
	json_object		a nul terminated string containing the JSON object to 
					validate. 
	object_error	if a syntax or value error is found in the JSON data 
					block, a pointer the portion of the JSON data block in 
					error is returned in this argument. This argument can 
					be NULL in which case no pointer is returned upon 
					encountering a JSON error.
	line_number		if a syntax or value error is found in the JSON data 
					block, the line number of the JSON token in error is 
					returned in this argument. This argument can be NULL
					in which case no line number is returned if a JSON 
					error was detected.
	suppress_warnings
					FALSE 	DO not suppress JSON error warnings written to
							stderr
					TRUE	Suppress possible error warnings, no output is
							written.
					
	returns 	TRUE 	- Successful validation of the JSON object syntax,
							grammar and values, no language errors.
					
				FALSE	- There was an error found in the JSON data block,
							if provided as arguments, a pointer is returned 
							in syntax_error and an integer in line_number
							corresponding to the found error. If warnings
							were not suppressed an message detailing the
							JSON error was written to stderr.
							
boolean json_valid_data_block(	const char *json_object, 
								char **object_error, 
								unsigned long int *line_number,
								const boolean suppress_warnings);


JSON Generator
--------------

Using a JSON Object data tree, represented by a hierarchical D-List list_object
structure, create a textural version of that JSON data presented and return a 
character array with that JSON data in it, in strict JSON Grammar format.

This function is essentially the reverse of json_parse_data_block(). It
will walk through a D-List list_object structure containing the 
representation of a JSON data block, and create a textural version of
that JSON data ready to be stored or transmitted over a network. 

The input data must be in the correct form and conform to JSON grammar.
Using the D-List hierarchical structure, there is very little that can be
in error in that structure, so the generation function produces few error
messages. The only real data problems that can occur are simple mistakes
such as providing inaccurate JSON value types, or forgetting to provide a
name for JSON Object members. 

All textural strings that are created by the JSON Generator assume the bytes
are either ASCII or UTF-8. It does not enforce any encoding standards. All
string bytes are passed through without change, except those that are 
required by the relevant JSON standards to be escaped. This includes all 
control characters which are either escaped with a '\' or by hex code.

The JSON Generator will produce human readable JSON text, that is well 
structured and easy to read. If you take an existing JSON data block,
parse it with the JSON Parser, and then run that output list_object 
through the JSON Generator, you will get back almost identical text
with the exception of probably different formatting style, and most
\uxxxx hex escapes in the original will be reproduced as UTF-8 instead.

	json_data		A pointer to a root list_object for the JSON data structure
					the caller wishes to use to generate a JSON Text Data Block.
	json_data_type	The type of data found at the top level, either JSON_object,
					JSON_array or JSON_value. This is the same kind of value 
					type that would be returned if a call was made to the
					function parse_json_data_block()
	json_text_buffer 
					A pointer to a location to write the pointer to the data block
					created by the JSON Generator.

	returns		 0	All JSON elements processed correctly, no syntax / grammar errors.
				 1	JSON Object was empty, but processed correctly.
				-1	A JSON grammar or syntax error was found below the top level
				-2	An error was found in the JSON syntax at the top level
				-3	An argument error occurred
				-4	An error occurred during memory allocation, there is no
					output buffer available.
					
int json_generate_data_block(	list_object *json_data, 
								unsigned int json_data_type,
								char **json_text_buffer);





