The program for creating the database, I am trying to do that in python. I have a json file whose contents will be used to populate the db.
In this we need to parse a JSON file, create a relational database dbjson corresponding to the JSON file, and upload the data into the dbjson.
The following is the formatted json file:
{
"cursor": {
"prev": null,
"hasNext": false,
"next": "1542275432008325:1:0",
"hasPrev": false,
"total": null,
"id": "1542275432008325:1:0",
"more": false
},
"code": 0,
"response": [
{
"editableUntil": "2018-11-22T11:20:37",
"dislikes": 0,
"numReports": 0,
"likes": 1,
"message": "<p>A sinking pound, resignations by the bucketful railroading through a so-called agreement nobody wants. For the good of the country Mrs. May 'go and go now'.</p>",
"id": "4196155749",
"createdAt": "2018-11-15T11:20:37",
"author": {
"username": "mnaid-233509073ed3432027d48b1a83f5fbd2",
"about": "",
"name": "baggiebuoy",
"disable3rdPartyTrackers": true,
"isPowerContributor": false,
"joinedAt": "2017-05-03T10:40:52",
"profileUrl": "https://disqus.com/by/mnaid-233509073ed3432027d48b1a83f5fbd2/",
"url": "",
"location": "",
"isPrivate": false,
"signedUrl": "",
"isPrimary": true,
"isAnonymous": false,
"id": "250728493",
"avatar": {
"small": {
"permalink": "https://disqus.com/api/users/avatars/mnaid-233509073ed3432027d48b1a83f5fbd2.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar32.png"
},
"isCustom": false,
"permalink": "https://disqus.com/api/users/avatars/mnaid-233509073ed3432027d48b1a83f5fbd2.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar92.png",
"large": {
"permalink": "https://disqus.com/api/users/avatars/mnaid-233509073ed3432027d48b1a83f5fbd2.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar92.png"
}
}
},
"media": [],
"isSpam": false,
"isDeletedByAuthor": false,
"isDeleted": false,
"parent": null,
"isApproved": true,
"isFlagged": false,
"raw_message": "A sinking pound, resignations by the bucketful railroading through a so-called agreement nobody wants. For the good of the country Mrs. May 'go and go now'.",
"isHighlighted": false,
"canVote": false,
"thread": "7046384220",
"forum": "expressandstar",
"points": 1,
"moderationLabels": [],
"isEdited": true,
"sb": false
},
{
"editableUntil": "2018-11-22T10:37:59",
"dislikes": 0,
"numReports": 0,
"likes": 0,
"message": "<p>This could be heaven, or this could be hell.</p>",
"id": "4196048572",
"createdAt": "2018-11-15T10:37:59",
"author": {
"username": "mnaid-29e1c59be16c852670e3be302e8c303b",
"about": "",
"name": "Mordecai",
"disable3rdPartyTrackers": false,
"isPowerContributor": false,
"joinedAt": "2017-05-03T13:01:02",
"profileUrl": "https://disqus.com/by/mnaid-29e1c59be16c852670e3be302e8c303b/",
"url": "",
"location": "",
"isPrivate": false,
"signedUrl": "",
"isPrimary": true,
"isAnonymous": false,
"id": "250739494",
"avatar": {
"small": {
"permalink": "https://disqus.com/api/users/avatars/mnaid-29e1c59be16c852670e3be302e8c303b.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar32.png"
},
"isCustom": false,
"permalink": "https://disqus.com/api/users/avatars/mnaid-29e1c59be16c852670e3be302e8c303b.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar92.png",
"large": {
"permalink": "https://disqus.com/api/users/avatars/mnaid-29e1c59be16c852670e3be302e8c303b.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar92.png"
}
}
},
"media": [],
"isSpam": false,
"isDeletedByAuthor": false,
"isDeleted": false,
"parent": 4196013534,
"isApproved": true,
"isFlagged": false,
"raw_message": "This could be heaven, or this could be hell.",
"isHighlighted": false,
"canVote": false,
"thread": "7046384220",
"forum": "expressandstar",
"points": 0,
"moderationLabels": [],
"isEdited": false,
"sb": false
},
{
"editableUntil": "2018-11-22T10:36:50",
"dislikes": 0,
"numReports": 0,
"likes": 0,
"message": "<p>The crappest of crap deals.<br>Will never get through Parliament.</p><p>You are the weakest link May, goodbye.</p>",
"id": "4196044068",
"createdAt": "2018-11-15T10:36:50",
"author": {
"username": "mnaid-29e1c59be16c852670e3be302e8c303b",
"about": "",
"name": "Mordecai",
"disable3rdPartyTrackers": false,
"isPowerContributor": false,
"joinedAt": "2017-05-03T13:01:02",
"profileUrl": "https://disqus.com/by/mnaid-29e1c59be16c852670e3be302e8c303b/",
"url": "",
"location": "",
"isPrivate": false,
"signedUrl": "",
"isPrimary": true,
"isAnonymous": false,
"id": "250739494",
"avatar": {
"small": {
"permalink": "https://disqus.com/api/users/avatars/mnaid-29e1c59be16c852670e3be302e8c303b.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar32.png"
},
"isCustom": false,
"permalink": "https://disqus.com/api/users/avatars/mnaid-29e1c59be16c852670e3be302e8c303b.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar92.png",
"large": {
"permalink": "https://disqus.com/api/users/avatars/mnaid-29e1c59be16c852670e3be302e8c303b.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar92.png"
}
}
},
"media": [],
"isSpam": false,
"isDeletedByAuthor": false,
"isDeleted": false,
"parent": null,
"isApproved": true,
"isFlagged": false,
"raw_message": "The crappest of crap deals.\nWill never get through Parliament.\n\nYou are the weakest link May, goodbye.",
"isHighlighted": false,
"canVote": false,
"thread": "7046384220",
"forum": "expressandstar",
"points": 0,
"moderationLabels": [],
"isEdited": false,
"sb": false
},
{
"editableUntil": "2018-11-22T10:28:13",
"dislikes": 0,
"numReports": 0,
"likes": 0,
"message": "<p>We are all just prisoners here of our own device. 😁</p>",
"id": "4196013534",
"createdAt": "2018-11-15T10:28:13",
"author": {
"username": "mnaid-42ba513c42a0fd6558aa44b1de658140",
"about": "",
"name": "chaffwolf",
"disable3rdPartyTrackers": true,
"isPowerContributor": false,
"joinedAt": "2017-12-20T16:45:10",
"profileUrl": "https://disqus.com/by/mnaid-42ba513c42a0fd6558aa44b1de658140/",
"url": "",
"location": "",
"isPrivate": false,
"signedUrl": "",
"isPrimary": true,
"isAnonymous": false,
"id": "274657487",
"avatar": {
"small": {
"permalink": "https://disqus.com/api/users/avatars/mnaid-42ba513c42a0fd6558aa44b1de658140.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar32.png"
},
"isCustom": false,
"permalink": "https://disqus.com/api/users/avatars/mnaid-42ba513c42a0fd6558aa44b1de658140.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar92.png",
"large": {
"permalink": "https://disqus.com/api/users/avatars/mnaid-42ba513c42a0fd6558aa44b1de658140.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar92.png"
}
}
},
"media": [],
"isSpam": false,
"isDeletedByAuthor": false,
"isDeleted": false,
"parent": 4195981074,
"isApproved": true,
"isFlagged": false,
"raw_message": "We are all just prisoners here of our own device. 😁",
"isHighlighted": false,
"canVote": false,
"thread": "7046384220",
"forum": "expressandstar",
"points": 0,
"moderationLabels": [],
"isEdited": false,
"sb": false
},
{
"editableUntil": "2018-11-22T10:27:03",
"dislikes": 0,
"numReports": 0,
"likes": 0,
"message": "<p>Should be no deal end off.<br>Nobody could possibly be the winner so it's simple leave and that's it .<br>Their will always be wannabes who think they can do better in it for only their own powers not for the general country or the people of it .,</p>",
"id": "4196012501",
"createdAt": "2018-11-15T10:27:03",
"author": {
"username": "mnaid-90ecce8d5dad4396f681182cb470872c",
"about": "",
"name": "wanderer in eire",
"disable3rdPartyTrackers": true,
"isPowerContributor": false,
"joinedAt": "2017-05-12T20:20:43",
"profileUrl": "https://disqus.com/by/mnaid-90ecce8d5dad4396f681182cb470872c/",
"url": "",
"location": "",
"isPrivate": false,
"signedUrl": "",
"isPrimary": true,
"isAnonymous": false,
"id": "251694793",
"avatar": {
"small": {
"permalink": "https://disqus.com/api/users/avatars/mnaid-90ecce8d5dad4396f681182cb470872c.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar32.png"
},
"isCustom": false,
"permalink": "https://disqus.com/api/users/avatars/mnaid-90ecce8d5dad4396f681182cb470872c.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar92.png",
"large": {
"permalink": "https://disqus.com/api/users/avatars/mnaid-90ecce8d5dad4396f681182cb470872c.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar92.png"
}
}
},
"media": [],
"isSpam": false,
"isDeletedByAuthor": false,
"isDeleted": false,
"parent": null,
"isApproved": true,
"isFlagged": false,
"raw_message": "Should be no deal end off.\nNobody could possibly be the winner so it's simple leave and that's it .\nTheir will always be wannabes who think they can do better in it for only their own powers not for the general country or the people of it .,",
"isHighlighted": false,
"canVote": false,
"thread": "7046384220",
"forum": "expressandstar",
"points": 0,
"moderationLabels": [],
"isEdited": false,
"sb": false
},
{
"editableUntil": "2018-11-22T10:26:42",
"dislikes": 0,
"numReports": 0,
"likes": 0,
"message": "<p>Damned if she does, damned if she doesn't. All for a cause she does not believe in.<br>She will go down in history whatever happens. <br>The question is: who better to lead the country at such an important juncture??<br>Answers on a postcard!</p>",
"id": "4196012237",
"createdAt": "2018-11-15T10:26:42",
"author": {
"username": "mnaid-42ba513c42a0fd6558aa44b1de658140",
"about": "",
"name": "chaffwolf",
"disable3rdPartyTrackers": true,
"isPowerContributor": false,
"joinedAt": "2017-12-20T16:45:10",
"profileUrl": "https://disqus.com/by/mnaid-42ba513c42a0fd6558aa44b1de658140/",
"url": "",
"location": "",
"isPrivate": false,
"signedUrl": "",
"isPrimary": true,
"isAnonymous": false,
"id": "274657487",
"avatar": {
"small": {
"permalink": "https://disqus.com/api/users/avatars/mnaid-42ba513c42a0fd6558aa44b1de658140.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar32.png"
},
"isCustom": false,
"permalink": "https://disqus.com/api/users/avatars/mnaid-42ba513c42a0fd6558aa44b1de658140.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar92.png",
"large": {
"permalink": "https://disqus.com/api/users/avatars/mnaid-42ba513c42a0fd6558aa44b1de658140.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar92.png"
}
}
},
"media": [],
"isSpam": false,
"isDeletedByAuthor": false,
"isDeleted": false,
"parent": null,
"isApproved": true,
"isFlagged": false,
"raw_message": "Damned if she does, damned if she doesn't. All for a cause she does not believe in.\nShe will go down in history whatever happens. \nThe question is: who better to lead the country at such an important juncture??\nAnswers on a postcard!",
"isHighlighted": false,
"canVote": false,
"thread": "7046384220",
"forum": "expressandstar",
"points": 0,
"moderationLabels": [],
"isEdited": false,
"sb": false
},
{
"editableUntil": "2018-11-22T09:50:32",
"dislikes": 0,
"numReports": 0,
"likes": 0,
"message": "<p>Theresa May is using The Eagles Hotel California lyrics as the inspiration for her Brexit plan.</p><p>You can check out any time you like, but you can never leave.</p>",
"id": "4195981074",
"createdAt": "2018-11-15T09:50:32",
"author": {
"username": "mnaid-e82a88d937e60267fd2c866b01131ada",
"about": "",
"name": "Olly the cat",
"disable3rdPartyTrackers": false,
"isPowerContributor": false,
"joinedAt": "2017-05-03T10:27:23",
"profileUrl": "https://disqus.com/by/mnaid-e82a88d937e60267fd2c866b01131ada/",
"url": "",
"location": "",
"isPrivate": false,
"signedUrl": "",
"isPrimary": true,
"isAnonymous": false,
"id": "250727584",
"avatar": {
"small": {
"permalink": "https://disqus.com/api/users/avatars/mnaid-e82a88d937e60267fd2c866b01131ada.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar32.png"
},
"isCustom": false,
"permalink": "https://disqus.com/api/users/avatars/mnaid-e82a88d937e60267fd2c866b01131ada.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar92.png",
"large": {
"permalink": "https://disqus.com/api/users/avatars/mnaid-e82a88d937e60267fd2c866b01131ada.jpg",
"cache": "//a.disquscdn.com/1541535443/images/noavatar92.png"
}
}
},
"media": [],
"isSpam": false,
"isDeletedByAuthor": false,
"isDeleted": false,
"parent": null,
"isApproved": true,
"isFlagged": false,
"raw_message": "Theresa May is using The Eagles Hotel California lyrics as the inspiration for her Brexit plan.\n\nYou can check out any time you like, but you can never leave.",
"isHighlighted": false,
"canVote": false,
"thread": "7046384220",
"forum": "expressandstar",
"points": 0,
"moderationLabels": [],
"isEdited": false,
"sb": false
}
]
}
This data must be used to populate the database.
The following is the implementation I have worked out:
import csv, json, sys
if sys.argv[1] is not None and sys.argv[2] is not None:
fileInput = sys.argv[1]
fileOutput = sys.argv[2]
inputFile = open(fileInput)
outputFile = open(fileOutput, 'w')
data = json.load(inputFile)
inputFile.close()
output = csv.writer(outputFile)
output.writerow(data.keys())
#output.writerow(data[0].keys()) # header row
for row in data:
output.writerow(row.values())
I tried to parse data using json package in python but have not been able to successfully implement it.
