1

I have a 500,000 line json file I'm trying to read, I tried out a few online examples like mkyong's and many others, but this format is a first encounter for me. Can anyone help me to get started?

I'm using json-simple library.

Any enlighten on this topic? I'll be very grateful

 {
"info": {
    "category": "url", 
    "started": "2013-11-07 03:44:32", 
    "ended": "2013-11-07 03:46:55", 
    "version": "0.6", 
    "duration": 143, 
    "id": 37
}, 
"signatures": [], 
"static": {}, 
"dropped": [
    {
        "yara": [], 
        "sha1": "6fd9eb6a42fd531988fba24137a2fe29ad992465", 
        "name": "tia[1].png", 
        "type": "PNG image data, 27 x 23, 8-bit/color RGB, non-interlaced", 
        "sha256": "75d6d26afb9506145d7f6472555855ef4305e0ef3e7cf4cb3d943c58123c7f74", 
        "crc32": "FDCAC92F", 
        "path": "/home/user/cuckoo/storage/analyses/37/files/2435944344/tia[1].png", 
        "ssdeep": null, 
        "size": 387, 
        "sha512": "b47ca17050ff4b6ddab848195c17b875454aafbec06d07bba126e553c9d32647f461adee9d1a75bbfffa08d6a8fc955429562794b123bebc9ec23dc89bdefcc5", 
        "md5": "ad07ee4cb98da073dda56ce7ceb88f5a"
    }
 ]
}

REPLY

I currently don't understand how i can get the array in [dropped], I did some of the easier example, but this is a first for me. It seems like there a hierarchy but I don't know how to access it

1
  • 3
    Do you need help understanding the JSON format itself, or how to parse it with json-simple? Commented Nov 23, 2013 at 14:39

3 Answers 3

3

Why not start with the json-simple decoding examples? If it's the size of the file you're worried about, then you should use the 'SAX' style example. This code would be a good example of how not to do it. See the other comment for better ideas.

JSONObject data= (JSONObject)JSONValue.parse(new InputStreamReader(JsonSimpleEx.class.getResourceAsStream("/test.json")) );
JSONArray dropped = (JSONArray)data.get("dropped");
JSONObject first = (JSONObject)dropped.get(0);
System.out.println( first.get("crc32"));
Sign up to request clarification or add additional context in comments.

2 Comments

Oh gosh... I just figure it out when I saw this, my code is quite similar to yours. I find that when dealing with such a large json file this gets quite messy at times. What's a good way of coding then? using the SAX creating a class file specific to the fields that I want to collect? Thanks for all this enlighten.
The better way to code it is to create plain old java objects (POJO's) with elements that match the structure of your json data. A good json library will simply create new instances of those objects and return them to you so that you can get to the data by using your Java objects. Since I don't see anything like that in the json-simple examples I'm going to assume it doesn't support that. Most json libraries support this feature. For example, in the Gson library it is called 'deserialization'.
1

You can use Gson - from Google. It is simple and concise.

You need to declare your Java POJO that represents your JSON data, like:

public class Info {
   private Category category;
   ...
}

Hope this help. Good luck.

1 Comment

Thanks :) currently looking at Gson, I'm using this json-simple lib, which I suspect is lacking a few good feature I need.
0

write and read this:

[
  {
    "id": 1,
    "firstname": "Bart",
    "lastname": "Simpson",
    "address": {
      "city": "Springfield",
      "country": "USA"
    }
  },
  {
    "id": 2,
    "firstname": "Homer",
    "lastname": "Simpson",
    "address": {
      "city": "Springfield",
      "country": "USA"
    }
  },
  {
    "id": 3,
    "firstname": "Mickey",
    "lastname": "Mouse",
    "address": {
      "city": "Orlando",
      "country": "USA"
    }
  }

PersonJson:

public class PersonJSON {
    private int id;
    private String firstname;
    private String lastname;
    private AddressJSON address;
    
    public PersonJSON(int id, String firstname, String lastname, AddressJSON address)
    {
        this.id = id;
        this.firstname = firstname;
        this.lastname = lastname;
        this.address = address;
    }

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    public String getFirstname() {
        return firstname;
    }

    public void setFirstname(String firstname) {
        this.firstname = firstname;
    }

    public String getLastname() {
        return lastname;
    }

    public void setLastname(String lastname) {
        this.lastname = lastname;
    }
    
    public AddressJSON getAddress() {
        return address;
    }

    public void setAddress(AddressJSON address) {
        this.address = address;
    }

    public String toString()
    {
        return "id: " + this.id + " - Firstname: " + this.firstname + " - Lastname: " + this.lastname + ", " + getAddress().toString();
    }
}

AddressJson:

import com.google.gson.annotations.SerializedName;

public class AddressJSON {
    //@SerializedName("Cidade")
    private String city;
    private String country;
    
    public AddressJSON(String city, String country)
    {
        this.city = city;
        this.country = country;
    }

    public String getCountry() {
        return country;
    }

    public void setCountry(String country) {
        this.country = country;
    }

    public String getCity() {
        return city;
    }

    public void setCity(String city) {
        this.city = city;
    }
    
    @Override
    public String toString(){
        return city + ", " + country;
    }
}

Main:

import java.io.FileReader;
import java.io.FileWriter;
import java.lang.reflect.Type;
import java.util.ArrayList;
import java.util.List;

import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.reflect.TypeToken;
import com.google.gson.stream.JsonReader;

public class PDJSON {

    private List<PersonJSON> people;

    public PDJSON() {
        people = new ArrayList<PersonJSON>();
    }

    public void saveData(String filePath) {
        //Gson gson = new Gson();
        Gson gson = new GsonBuilder().setPrettyPrinting().create();
        
        try(FileWriter fw = new FileWriter(filePath)){
            gson.toJson(people, fw);        
        }catch(Exception e)
        {
            e.printStackTrace();
        }
    }

    public List<PersonJSON> loadData(String filePath) {
        Gson gson = new Gson();
        JsonReader jsonReader = null;

        final Type CUS_LIST_TYPE = new TypeToken<List<PersonJSON>>() {}.getType();
        //or TypeToken.getParameterized(ArrayList.class, PersonJSON.class).getType();
        
        try{
            jsonReader = new JsonReader(new FileReader(filePath));
        }catch (Exception e) {
            e.printStackTrace();
        }

        return gson.fromJson(jsonReader, CUS_LIST_TYPE); 
    }

    public static void main(String[] args) {
        PDJSON pdj = new PDJSON();
        pdj.people.add(new PersonJSON(1,"Bart", "Simpson", new AddressJSON("Springfield", "USA")));
        pdj.people.add(new PersonJSON(2,"Homer", "Simpson", new AddressJSON("Springfield", "USA")));
        pdj.people.add(new PersonJSON(3,"Mickey", "Mouse", new AddressJSON("Orlando", "USA")));

        pdj.saveData("resources/listofpeople.json");

        List<PersonJSON> lp = pdj.loadData("resources/listofpeople.json"); 
        for(PersonJSON pj : lp)
        {
            System.out.println(pj.toString());
        }
    }
}

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.